From owner-freebsd-net@freebsd.org Sun Jul 30 09:25:24 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CE4EDB2D58 for ; Sun, 30 Jul 2017 09:25:24 +0000 (UTC) (envelope-from amutu@amutu.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE6D36ADCC for ; Sun, 30 Jul 2017 09:25:23 +0000 (UTC) (envelope-from amutu@amutu.com) Received: by mail-oi0-x231.google.com with SMTP id x3so158352511oia.1 for ; Sun, 30 Jul 2017 02:25:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amutu-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tR/Alr1INvm040ybjOAuUOrkmsEPcZChaFHdgZC5LHA=; b=uWXo38YWiqMNhUDSMZOaBAiRV0jkE8w6NxVv0MTG3zCJPTWXoogvgHBkqwikjuvSJn HMhBxodHRukllludCiEbXxN7HyxP/9mZ1TOpaAAOh3GmPWFINBndZy4eBOeg5vIDqCb/ EqHcf7jjD4PcJUmZzoG12gnpdeymPLRDACubSnsSi/ohPBjmUE3wfSz4J2cYej9o/uYw x0Kxs3Djy3NxxOzg9oKBsWQe+W3yfpaNZhmF5xjprJvnRTcFOES0Iaev/qCCmdO0nExF a/iKWa9wtdBPw3MVW8y21vOJRkjJiYQEh1B1jFfgOBVYdRVzuO9necvUEAb9stRLtXPv bNzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tR/Alr1INvm040ybjOAuUOrkmsEPcZChaFHdgZC5LHA=; b=FD3d9wKbaM0J6dLO1TQTMH4llBNvoISfzse0DWA43ubNXIxquc298N5gQUi5/MFqZX Mr3LlnAyczkaiPugvgLVQiqsuxJ7aVBkJHfknJLKqToJTn4tOvehJc+/IZWX8IU34mul 6lemPPL0z4rmeS21qhnRhMIesuKEbGuN3pEakpoWT/XOj45XkXERHU0KYsAnfIzmfHcW MqAf1+T2DH0dUO5Fg8wBmOraX0moRWYeFsWKNOkR1ormRVJyOb3e3W4Vowxd/jNEEAog MpNUpURUmn/A3nrFUXX+/Q/O5VgfbgspjF8DHMx5GqVXN9IzpQ4oht7kF7zBRT5S3Vzy pUeA== X-Gm-Message-State: AIVw1121ic2tFYV1rsu1A1fCPtFbzfMYnSRCdSWUIoc3IRvHZxdru1R8 QpLIe0rze9Y720Bs X-Received: by 10.202.219.198 with SMTP id s189mr12912354oig.103.1501406722858; Sun, 30 Jul 2017 02:25:22 -0700 (PDT) Received: from mail-oi0-f53.google.com (mail-oi0-f53.google.com. [209.85.218.53]) by smtp.gmail.com with ESMTPSA id c8sm7717835oia.36.2017.07.30.02.25.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 30 Jul 2017 02:25:22 -0700 (PDT) Received: by mail-oi0-f53.google.com with SMTP id x3so158352343oia.1; Sun, 30 Jul 2017 02:25:21 -0700 (PDT) X-Received: by 10.202.198.199 with SMTP id w190mr10938188oif.93.1501406721760; Sun, 30 Jul 2017 02:25:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.128.197 with HTTP; Sun, 30 Jul 2017 02:25:01 -0700 (PDT) In-Reply-To: <8F4BB6E0-66A3-4367-BD86-DC29F2BA3C0A@pair.com> References: <8F4BB6E0-66A3-4367-BD86-DC29F2BA3C0A@pair.com> From: Jov Date: Sun, 30 Jul 2017 17:25:01 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: ACK Storm protection? To: Matt Riffle Cc: freebsd-security@freebsd.org, freebsd-net@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jul 2017 09:25:24 -0000 =E2=80=8B=E2=80=8B freebsd-net@ added. After google "ack storm freebsd" I find a very old SA: https://www.freebsd.org/security/advisories/FreeBSD-SA-98%3A07.rst.asc mentions: =E2=80=8B + * In the SYN-RECEIVED state, don't send an ACK unless the + * segment we received passes the SYN-RECEIVED ACK test. > + * If it fails send a RST. This breaks the loop in the > + * "LAND" DoS attack, and also prevents an ACK storm > + * between two listening ports that have been sent forged > + * SYN segments, each with the source address of the other. > + */ > + if (tp->t_state =3D=3D TCPS_SYN_RECEIVED && (tiflags & TH_ACK) && > + (SEQ_GT(tp->snd_una, ti->ti_ack) || > + SEQ_GT(ti->ti_ack, tp->snd_max)) ) > + goto dropwithreset;=E2=80=8B Not sure in the established state there also has ACK storm protection. 2017-07-22 2:57 GMT+08:00 Matt Riffle : > Hello, > > Starting on July 11, I=E2=80=99ve started to see an increasing number of = what > appear to be =E2=80=9CACK storms=E2=80=9D affecting a number of FreeBSD b= oxes I=E2=80=99m > administering. There are a few unsupported releases mixed in, but, this = is > also happening on boxes running 10.3-RELEASE-p3. > > In the cases we=E2=80=99re seeing, it begins with legitimate TCP traffic > requesting something over HTTP, but soon thereafter we get an out of wind= ow > packet and get in to a loop. If anybody is interested or especially if > they=E2=80=99ve experienced something similar, there are a few more detai= ls I could > share privately. > > Setting aside the cause, I=E2=80=99m interested in trying to mitigate the > problem. None of my Ubuntu boxes appear to be affected, I presume becaus= e > of these patches Google made to the kernel there: > > https://www.ietf.org/mail-archive/web/tcpm/current/msg09445.html < > https://www.ietf.org/mail-archive/web/tcpm/current/msg09445.html> > > Is there any equivalent protection for FreeBSD? In my own research I=E2= =80=99ve > been unable to find anything. In fact, beyond the message above you can= =E2=80=99t > find very much about ACK storms at all. > > Right now we=E2=80=99re mitigating with custom code that is sniffing pack= ets and > adding temporary firewall rules whenever it sees a loop start, and that= =E2=80=99s > working well enough, but, I=E2=80=99d prefer to handle it at a lower leve= l if > possible. > > Thanks, > > Matt R. > > > > > _______________________________________________ > freebsd-security@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-security > To unsubscribe, send any mail to "freebsd-security-unsubscribe@freebsd.or= g > " From owner-freebsd-net@freebsd.org Sun Jul 30 21:01:10 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98829DC220E for ; Sun, 30 Jul 2017 21:01:10 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8D06380AE7 for ; Sun, 30 Jul 2017 21:01:10 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v6UL01rb038712 for ; Sun, 30 Jul 2017 21:01:10 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201707302101.v6UL01rb038712@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-net@FreeBSD.org Subject: Problem reports for freebsd-net@FreeBSD.org that need special attention Date: Sun, 30 Jul 2017 21:01:10 +0000 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jul 2017 21:01:10 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 165622 | [ndis][panic][patch] Unregistered use of FPU in k In Progress | 206581 | bxe_ioctl_nvram handler is faulty New | 204438 | setsockopt() handling of kern.ipc.maxsockbuf limi New | 205592 | TCP processing in IPSec causes kernel panic New | 206053 | kqueue support code of netmap causes panic New | 213410 | [carp] service netif restart causes hang only whe New | 215874 | [patch] [icmp] [mbuf_tags] teach icmp_error() opt New | 217748 | sys/dev/ixgbe/if_ix.c: PVS-Studio: Assignment to New | 220076 | [patch] [panic] [netgraph] repeatable kernel pani Open | 173444 | socket: IPV6_USE_MIN_MTU and TCP is broken Open | 193452 | Dell PowerEdge 210 II -- Kernel panic bce (broadc Open | 194485 | Userland cannot add IPv6 prefix routes Open | 194515 | Fatal Trap 12 Kernel with vimage Open | 199136 | [if_tap] Added down_on_close sysctl variable to t Open | 202510 | [CARP] advertisements sourced from CARP IP cause Open | 206544 | sendmsg(2) (sendto(2) too?) can fail with EINVAL; Open | 211031 | [panic] in ng_uncallout when argument is NULL Open | 211962 | bxe driver queue soft hangs and flooding tx_soft_ 18 problems total for which you should take action. From owner-freebsd-net@freebsd.org Mon Jul 31 11:05:25 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF259DD03E3 for ; Mon, 31 Jul 2017 11:05:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AD96072B9C for ; Mon, 31 Jul 2017 11:05:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v6VB5PT8091585 for ; Mon, 31 Jul 2017 11:05:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220217] deadlock on enc and pf Date: Mon, 31 Jul 2017 11:05:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jul 2017 11:05:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220217 --- Comment #9 from commit-hook@freebsd.org --- A commit references this bug: Author: ae Date: Mon Jul 31 11:04:36 UTC 2017 New revision: 321779 URL: https://svnweb.freebsd.org/changeset/base/321779 Log: Add inpcb pointer to struct ipsec_ctx_data and pass it to the pfil hook from enc_hhook(). This should solve the problem when pf is used with if_enc(4) interface, and outbound packet with existing PCB checked by pf, and this leads to deadlock due to pf does its own PCB lookup and tries to take rlock when wlock is already held. Now we pass PCB pointer if it is known to the pfil hook, this helps to avoid extra PCB lookup and thus rlock acquiring is not needed. For inbound packets it is safe to pass NULL, because we do not held any PCB locks yet. PR: 220217 MFC after: 3 weeks Sponsored by: Yandex LLC Changes: head/sys/net/if_enc.c head/sys/net/if_enc.h head/sys/netipsec/ipsec.h head/sys/netipsec/ipsec_input.c head/sys/netipsec/ipsec_output.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Mon Jul 31 18:49:16 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2170ADB6AD7 for ; Mon, 31 Jul 2017 18:49:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0F515636BE for ; Mon, 31 Jul 2017 18:49:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v6VInCJG041033 for ; Mon, 31 Jul 2017 18:49:15 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220170] Can't load tcpmd5 kernel module due to missing ipsec_support Date: Mon, 31 Jul 2017 18:49:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: freebsd@gushi.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jul 2017 18:49:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220170 Dan Mahoney changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |freebsd@gushi.org --- Comment #3 from Dan Mahoney --- Ditto here. Wondering what the point of making this a kernel module was, if it requires= a custom kernel to load. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 08:21:14 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C485DC41C2 for ; Tue, 1 Aug 2017 08:21:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7AFB76A040 for ; Tue, 1 Aug 2017 08:21:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v718LE8E092696 for ; Tue, 1 Aug 2017 08:21:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221137] FreeBSD 11+ does not send ICMP redirects Date: Tue, 01 Aug 2017 08:21:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: emaste@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 08:21:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221137 Ed Maste changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org CC| |emaste@freebsd.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 09:45:54 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08A4ADCA99C for ; Tue, 1 Aug 2017 09:45:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA00F6D6F2 for ; Tue, 1 Aug 2017 09:45:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v719jrXf042581 for ; Tue, 1 Aug 2017 09:45:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220170] Add IPSEC_SUPPORT to GENERIC kernel (allowing tcpmd5 kernel module loading out of the box) Date: Tue, 01 Aug 2017 09:45:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: keywords version bug_status flagtypes.name cc short_desc bug_severity see_also Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 09:45:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220170 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |needs-patch Version|11.0-STABLE |CURRENT Status|New |Open Flags| |mfc-stable11? CC|re@FreeBSD.org |koobs@FreeBSD.org Summary|Can't load tcpmd5 kernel |Add IPSEC_SUPPORT to |module due to missing |GENERIC kernel (allowing |ipsec_support |tcpmd5 kernel module | |loading out of the box) Severity|Affects Only Me |Affects Some People See Also| |https://bugs.freebsd.org/bu | |gzilla/show_bug.cgi?id=3D1= 845 | |45 --- Comment #4 from Kubilay Kocak --- 11.1-RELEASE is out, pull re@ from CC Update scope of request to CURRENT with desirable MFC (to stable/11) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 10:50:17 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 111E2DCFFF6 for ; Tue, 1 Aug 2017 10:50:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3FBF71097 for ; Tue, 1 Aug 2017 10:50:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71AoGWe014945 for ; Tue, 1 Aug 2017 10:50:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221137] FreeBSD 11+ does not send ICMP redirects Date: Tue, 01 Aug 2017 10:50:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 10:50:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221137 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ae@FreeBSD.org --- Comment #1 from Andrey V. Elsukov --- You can add some security policy as a workaround, this disables the using of ip_tryforward(). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 10:56:07 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0E83DD02BE for ; Tue, 1 Aug 2017 10:56:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AF8EE715A1 for ; Tue, 1 Aug 2017 10:56:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71Au7C5030513 for ; Tue, 1 Aug 2017 10:56:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221137] FreeBSD 11+ does not send ICMP redirects Date: Tue, 01 Aug 2017 10:56:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: igorr@speechpro.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 10:56:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221137 --- Comment #2 from igorr@speechpro.com --- But it is good to have ICMP redirects without special tweaks. Everything worked in 9.X, and I have checked that 10.3 also sends ICMP redirects. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 11:09:14 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E589DD071F for ; Tue, 1 Aug 2017 11:09:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1CE1271C59 for ; Tue, 1 Aug 2017 11:09:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71B9D9V081909 for ; Tue, 1 Aug 2017 11:09:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221137] FreeBSD 11+ does not send ICMP redirects Date: Tue, 01 Aug 2017 11:09:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 11:09:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221137 --- Comment #3 from Andrey V. Elsukov --- (In reply to igorr from comment #2) > But it is good to have ICMP redirects without special tweaks. > Everything worked in 9.X, and I have checked that 10.3 also sends ICMP > redirects. Yes, it was broken with ip_tryforward introduction. But I don't like the solution with M_SKIP_FIREWALL. This flag can be used by pfil module and clearing it can break something. I prefer to reintroduce sysctl variable to disable tryforward. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 11:19:02 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67806DD0ECB for ; Tue, 1 Aug 2017 11:19:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 560EB72550 for ; Tue, 1 Aug 2017 11:19:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71BJ20P005016 for ; Tue, 1 Aug 2017 11:19:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221137] FreeBSD 11+ does not send ICMP redirects Date: Tue, 01 Aug 2017 11:19:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 11:19:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221137 Eugene Grosbein changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eugen@freebsd.org --- Comment #4 from Eugene Grosbein --- (In reply to Andrey V. Elsukov from comment #3) We already have sysctl net.inet.ip.redirect with default value of 1 that is broken now. We could change default to 0 (keep fastforwarding on by default) and disable fast path if net.inet.ip.redirect changed to 1. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 11:28:24 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E9D5DD12A6 for ; Tue, 1 Aug 2017 11:28:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D15972A41 for ; Tue, 1 Aug 2017 11:28:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71BSNDf028019 for ; Tue, 1 Aug 2017 11:28:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221137] FreeBSD 11+ does not send ICMP redirects Date: Tue, 01 Aug 2017 11:28:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: igorr@speechpro.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 11:28:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221137 --- Comment #5 from igorr@speechpro.com --- I have used M_SKIP_FIREWALL to avoid reintroducing packet into pfil_run_hooks(). As ICMP redirects are not very common, and if there are no much trouble with reprocessing packet in input firewall rules we can remove M_SKIP_FIREWALL f= rom my patch. Or use some different flag. Also I have not found any places referencing to M_SKIP_FIREWALL except ip_output.c. And it is called after ip_input(). So with my patch ip_tryforward() will be used most of the time. And in rare cases when ICMP redirect should be sent it will not be used. But I have nothing against using net.inet.ip.redirect as flag for disablinmg fast path and enabling ICMP redirects. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 13:44:55 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0533DAE164 for ; Tue, 1 Aug 2017 13:44:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BE3C27C33B for ; Tue, 1 Aug 2017 13:44:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71DitLH039090 for ; Tue, 1 Aug 2017 13:44:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220888] [panic][patch] Race between ngs_rcvdata() and soclose() Date: Tue, 01 Aug 2017 13:44:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: philip@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: philip@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 13:44:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220888 Philip Paeps changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |philip@FreeBSD.org Assignee|freebsd-net@FreeBSD.org |philip@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 15:33:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1AE4DB0850 for ; Tue, 1 Aug 2017 15:33:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9F6A67F8E4 for ; Tue, 1 Aug 2017 15:33:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71FXR7B066143 for ; Tue, 1 Aug 2017 15:33:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Tue, 01 Aug 2017 15:33:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 15:33:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 15:34:34 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB218DB0A5F for ; Tue, 1 Aug 2017 15:34:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 99AA77FB20 for ; Tue, 1 Aug 2017 15:34:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71FYYMN067802 for ; Tue, 1 Aug 2017 15:34:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221122] Attatching vxlan interfaces to a bridge stops all traffic on it Date: Tue, 01 Aug 2017 15:34:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 15:34:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221122 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 15:34:57 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 298CDDB0B22 for ; Tue, 1 Aug 2017 15:34:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 186537FC3F for ; Tue, 1 Aug 2017 15:34:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71FYuhs068685 for ; Tue, 1 Aug 2017 15:34:56 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221120] [panic] network stack when using FLOWTABLE Date: Tue, 01 Aug 2017 15:34:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 15:34:57 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221120 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-net@FreeBSD.org Summary|[panic] network stack |[panic] network stack when | |using FLOWTABLE --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 17:15:36 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7FC9EDB455D for ; Tue, 1 Aug 2017 17:15:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6892E18CE for ; Tue, 1 Aug 2017 17:15:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71HFabj055583 for ; Tue, 1 Aug 2017 17:15:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 220170] Add IPSEC_SUPPORT to GENERIC kernel (allowing tcpmd5 kernel module loading out of the box) Date: Tue, 01 Aug 2017 17:15:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: freebsd@gushi.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 17:15:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D220170 --- Comment #5 from Dan Mahoney --- Just so I understand, does this mean that there's a chance this will appear= in an 11.1-Pn patch release, or is this a "wait for 11.2 if there is one" sort= of case? I normally understand that we don't want kernel changes made within a release-cycle, but these are fairly minor changes (after all, full on optio= ns IPSEC is in -GENERIC) and it's pretty clear by the inclusion of the module = that this support was intended to be there. Failing that, might there be a possibility of breaking out the ipsec_support option to a loadable that applies to -GENERIC as well? Perhaps something t= hat could be compiled and loaded alongside base, even if not distributed with i= t? Updating to -STABLE on production systems is even more irritating than a cu= stom kernel. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 19:36:50 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E63BDB7AF7 for ; Tue, 1 Aug 2017 19:36:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CCB966628 for ; Tue, 1 Aug 2017 19:36:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71JamvC058026 for ; Tue, 1 Aug 2017 19:36:50 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Tue, 01 Aug 2017 19:36:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 19:36:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 Eugene Grosbein changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |eugen@freebsd.org --- Comment #2 from Eugene Grosbein --- Does not seem to be lagg(4) problem but rather ix(4) link instability probl= em. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 19:43:54 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F33DDDB7D75 for ; Tue, 1 Aug 2017 19:43:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1BE8669C7 for ; Tue, 1 Aug 2017 19:43:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71Jhr0U075835 for ; Tue, 1 Aug 2017 19:43:54 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221120] [panic] network stack when using FLOWTABLE Date: Tue, 01 Aug 2017 19:43:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 19:43:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221120 Eugene Grosbein changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed CC| |eugen@freebsd.org Resolution|--- |Overcome By Events --- Comment #5 from Eugene Grosbein --- Close this as FLOWTABLE deorbit sequence has already been initiated and it = is not present in the GENERIC for any supported release. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 20:39:49 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53C54DBCBC8 for ; Tue, 1 Aug 2017 20:39:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 41F4B682A6 for ; Tue, 1 Aug 2017 20:39:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71KdmqD023110 for ; Tue, 1 Aug 2017 20:39:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Tue, 01 Aug 2017 20:39:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vegeta@tuxpowered.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 20:39:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 vegeta@tuxpowered.net changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |vegeta@tuxpowered.net --- Comment #3 from vegeta@tuxpowered.net --- I have some Intel xl710 running failover lagg and I had an issue not totally unlike this one. One of ports in lagg (and always the same one, unless they were added in different order, then both would always work) would only send frames but never receive them, so the router would become master on carp on vlans on this lagg, never seeing carp advertisements of the primary router. That was on FreeBSD 11.0, though. I used ixgbe-2.5.15 from Intel instead of= the one coming with Kernel and the issue was gone. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 20:40:45 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2EE7DBCD2E for ; Tue, 1 Aug 2017 20:40:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82E57684BA for ; Tue, 1 Aug 2017 20:40:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71KejSH026254 for ; Tue, 1 Aug 2017 20:40:45 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Tue, 01 Aug 2017 20:40:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vegeta@tuxpowered.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 20:40:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 --- Comment #4 from vegeta@tuxpowered.net --- Sorry, I meant ixl-1.7.12 from Intel. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Tue Aug 1 21:02:41 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CFF6DDBD50C for ; Tue, 1 Aug 2017 21:02:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BEC076930A for ; Tue, 1 Aug 2017 21:02:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v71L2ftL094417 for ; Tue, 1 Aug 2017 21:02:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Tue, 01 Aug 2017 21:02:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Aug 2017 21:02:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 --- Comment #5 from Cassiano Peixoto --- Hi guys, It's a bug after commit r320897. I have the same issue running an ix interface in netmap mode after this com= mit. I had to downgrade ixgbe drivers to make it works again. Looks like a bug after driver update to 3.2.12-k. I'm copying the author of commit. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Aug 2 07:11:56 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA64ADC7DDE for ; Wed, 2 Aug 2017 07:11:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 96B287CC68 for ; Wed, 2 Aug 2017 07:11:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v727Bsif061659 for ; Wed, 2 Aug 2017 07:11:56 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 02 Aug 2017 07:11:55 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 07:11:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #128 from Eugene Grosbein --- (In reply to Harald Schmalzbauer from comment #127) Additionally, there were specific merges to release branch releng/11.1 befo= re release was tagged. So, 11.1-RELEASE does have those fixes. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Aug 2 07:57:46 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64E0DDC8C17 for ; Wed, 2 Aug 2017 07:57:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46FE07E048 for ; Wed, 2 Aug 2017 07:57:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v727viGF079121 for ; Wed, 2 Aug 2017 07:57:46 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 02 Aug 2017 07:57:45 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: bugzilla.freebsd@omnilan.de X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 07:57:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #129 from Harald Schmalzbauer --- (In reply to Eugene Grosbein from comment #128) Could you please tell us the revision? Hope I really missed something and y= ou are right, but if I e.g. watch the log of lib/libc/stdio/fclose.c, it wasn't touched since it has been copied from base/stable/11@r320475 to releng/11. kib@ did a MFS (r320763), but either I got all completely wrong and the referenced commits in my last post are unrelated&wrong, or they aren't in 1= 1.1 (which would explain why my local MFS doesn't fail). Btw. I locally MFS also your r320776 to 11.1.0 (r310888 MFC, usr.sbin/syslogd/syslogd.c: Retry to open an F_PIPE process when it dies unexpectedly). Please, let us document the 11.1-RELEASE status unambiguously here, I know = how tedious it is to waste time chasing contradictory facts :-) I'm happy if I'm proven wrong, but finally here's the information people can rely on. -harry --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Aug 2 08:26:12 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8083DC9A7B for ; Wed, 2 Aug 2017 08:26:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D4BFF7F52C for ; Wed, 2 Aug 2017 08:26:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v728QACp068087 for ; Wed, 2 Aug 2017 08:26:12 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Wed, 02 Aug 2017 08:26:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: flagtypes.name keywords bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 08:26:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Flags| |mfc-stable11? Keywords| |regression Status|New |Open --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Aug 2 08:29:21 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FFCADC9C60 for ; Wed, 2 Aug 2017 08:29:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E8FF7F7AB for ; Wed, 2 Aug 2017 08:29:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v728TKlE072613 for ; Wed, 2 Aug 2017 08:29:21 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221120] [panic] network stack when using FLOWTABLE Date: Wed, 02 Aug 2017 08:29:20 +0000 X-Bugzilla-Reason: AssignedTo CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 08:29:21 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221120 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-net@FreeBSD.org |eugen@freebsd.org CC| |freebsd-net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Aug 2 08:39:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 877E9DCA03A for ; Wed, 2 Aug 2017 08:39:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 760EA7FE84 for ; Wed, 2 Aug 2017 08:39:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v728dRNj097325 for ; Wed, 2 Aug 2017 08:39:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Wed, 02 Aug 2017 08:39:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kbowling@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 08:39:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 Kevin Bowling changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kbowling@freebsd.org --- Comment #6 from Kevin Bowling --- We're seeing this on cxgbe(4) as well, so I think it's related to if_lagg locking changes and the LACP init code. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Wed Aug 2 12:36:54 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3149ADCFF34 for ; Wed, 2 Aug 2017 12:36:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F9533F01 for ; Wed, 2 Aug 2017 12:36:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v72CarpE035481 for ; Wed, 2 Aug 2017 12:36:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 02 Aug 2017 12:36:53 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 12:36:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #130 from Eugene Grosbein --- (In reply to Harald Schmalzbauer from comment #129) 11.1 release branch releng/11.1 was created with r320486 from stable/11@r32= 0475 https://svnweb.freebsd.org/base?view=3Drevision&revision=3D320486 11.1-RELEASE was tagged with r321354, July 21: https://svnweb.freebsd.org/base?view=3Drevision&revision=3D321354 Biggest problem was one with syslog() and libc/syslog "cancel-safe" fix was merged to stable/11 with r320311 before creation of releng/11.1: https://svnweb.freebsd.org/base?view=3Drevision&revision=3D320311 - this ma= de mpd5 pretty stable with an exception of very rare case that were dealt with stdio fixes later. There were several "cancel-safe" commits to head's libc/stdio that stem from https://reviews.freebsd.org/D11246 - r320472 (main one) plus some fixes to = that fix: r320508 and r320509. They all were merged to stable/11 with r320942, J= uly 13: https://svnweb.freebsd.org/base?view=3Drevision&revision=3D320942 before creation of the release. I ran stable/11 that time and believed that future release would have it. But you are right, it hasn't. One should manually merge r320942 to the release and rebuild libc, or switc= h to stable/11, or just avoid very frequent access to mpd5 web console (disable = it altogether to make sure). --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Wed Aug 2 12:38:35 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70EADDD004A for ; Wed, 2 Aug 2017 12:38:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5955663438 for ; Wed, 2 Aug 2017 12:38:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v72CcYLX070237 for ; Wed, 2 Aug 2017 12:38:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 186114] net/mpd5 hangs after a certain number of users connect Date: Wed, 02 Aug 2017 12:38:34 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash, needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: eugen@freebsd.org X-Bugzilla-Flags: maintainer-feedback+ X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2017 12:38:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D186114 --- Comment #131 from Eugene Grosbein --- (In reply to Harald Schmalzbauer from comment #129) And syslogd(8) problem has no connection with this mpd5 problem. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-net@freebsd.org Thu Aug 3 18:04:14 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25D75DCB600 for ; Thu, 3 Aug 2017 18:04:14 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: from mail-qt0-f171.google.com (mail-qt0-f171.google.com [209.85.216.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E136764FD2 for ; Thu, 3 Aug 2017 18:04:13 +0000 (UTC) (envelope-from matt.joras@gmail.com) Received: by mail-qt0-f171.google.com with SMTP id p3so12187566qtg.2 for ; Thu, 03 Aug 2017 11:04:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding:content-language; bh=Yi93Mk50YsXe8aLGKhNuOSZTSu7C+Rt2/kJhXmhT4CU=; b=fcK7wLYwvpFvD2rOrwkhyjj3xIjIKfoGrTgmNsWgUr5Ao6wmQHL/bbGUlyEU/hlwYf j1vXaoioSKHHfAnOKUNoTQr7wfirZsnAWvHLZTCb5CihKNLwq/7aJxRf/il7IAZ79jVm CRs2uZ+WdBWHau+nAnhjUn3GGylq8C62CexhZTAcUyY69IdynTZhsZgHyq85+f5qUy0o 09dkNVOuQ7jFlOcp6B1J/6yeCjJ7KFts6G2XIvkWLHRv3lNYB7NqGM1FE+sHPQjhExY+ cxSvZGs0JQ9jEzIDktAoSeHm/kdafRJ8TlaXLGP+VO9ZYNgtDwkwrzzhlKIo+kdD9T16 lS1A== X-Gm-Message-State: AIVw1117G8xBkHuRYeckCdxKvcBvs69/MCQOtbeSCH71RFdBL5SAhSxB /rnd3ATgaF6A0bvhmtQ= X-Received: by 10.200.43.177 with SMTP id m46mr3216433qtm.189.1501783098849; Thu, 03 Aug 2017 10:58:18 -0700 (PDT) Received: from [192.168.2.122] (71-212-70-173.tukw.qwest.net. [71.212.70.173]) by smtp.gmail.com with ESMTPSA id h47sm8189962qte.67.2017.08.03.10.58.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Aug 2017 10:58:17 -0700 (PDT) To: freebsd-net@freebsd.org From: Matt Joras Subject: Request for comments/discussion on listen queue overflow logging Message-ID: Date: Thu, 3 Aug 2017 10:58:16 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2017 18:04:14 -0000 Hello, Some of you may have noticed a review I posed on Phabricator: https://reviews.freebsd.org/D11725. The general problem this is trying to solve is the lack of information about which application's listen queue has overflowed. As mentioned in the review, there is no good way to implement this where the message is currently logged, as it is at the generic socket layer. The approach I take in the review is to construct an additional more verbose log message (the TCP 4-tuple) at call point of sonewconn in the syncache code. I'd appreciate any feedback on the review or suggestions of alternative approaches. An approach I did think about, but discarded, was to add a protocol function to struct protosw, something like int pr_getsockdescr(struct socket *, char *, size_t) which would fill in a "human-readable" description of the socket based on whatever is appropriate. Thanks, Matt Joras From owner-freebsd-net@freebsd.org Thu Aug 3 20:50:19 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2767CDD2986 for ; Thu, 3 Aug 2017 20:50:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 168A86A5CA for ; Thu, 3 Aug 2017 20:50:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v73KoI2m006047 for ; Thu, 3 Aug 2017 20:50:18 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Thu, 03 Aug 2017 20:50:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: erj@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2017 20:50:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 --- Comment #7 from Eric Joyner --- Do we know who to assign this to, then? If it is actually a lagg problem and not a driver-specific one. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Thu Aug 3 21:01:23 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB139DD319B for ; Thu, 3 Aug 2017 21:01:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A8D216AD0B for ; Thu, 3 Aug 2017 21:01:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v73L1MpT093319 for ; Thu, 3 Aug 2017 21:01:23 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Thu, 03 Aug 2017 21:01:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: peixoto.cassiano@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2017 21:01:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 --- Comment #8 from Cassiano Peixoto --- (In reply to Eric Joyner from comment #7) So should i open a new PR related to driver update problem and netmap? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Aug 4 07:31:02 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80486DCACDE for ; Fri, 4 Aug 2017 07:31:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 66C4581B86 for ; Fri, 4 Aug 2017 07:31:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v747V2vG009219 for ; Fri, 4 Aug 2017 07:31:02 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221137] FreeBSD 11+ does not send ICMP redirects Date: Fri, 04 Aug 2017 07:31:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: igorr@speechpro.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 07:31:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221137 --- Comment #6 from igorr@speechpro.com --- Created attachment 185012 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D185012&action= =3Dedit New patch for rev 322037. Uses net.inet.ip.redirect as condition to use ip_tryforward() I have made new patch which uses sysctl net.inet.ip.redirect to decide when= ever use ip_tryforward(). Also it changes default value for net.inet.ip.redirect to 0. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Aug 4 10:45:48 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60080DD28ED for ; Fri, 4 Aug 2017 10:45:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 445003D45 for ; Fri, 4 Aug 2017 10:45:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v74AjmxO097958 for ; Fri, 4 Aug 2017 10:45:48 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Fri, 04 Aug 2017 10:45:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: konrad.kreciwilk@korbank.pl X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 10:45:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 --- Comment #9 from Konrad --- I set LAGG on igb0 and igb1 and seems to be ok: lagg0: flags=3D8843 metric 0 mtu 15= 00 =20=20=20=20=20=20=20 options=3D6403bb ether 00:1e:67:27:1d:5e inet 192.168.202.254 netmask 0xffffff00 broadcast 192.168.202.255=20 nd6 options=3D29 media: Ethernet autoselect status: active groups: lagg=20 laggproto lacp lagghash l2,l3,l4 laggport: igb0 flags=3D1c laggport: igb1 flags=3D1c lagg1: flags=3D8843 metric 0 mtu 15= 00 =20=20=20=20=20=20=20 options=3De400bb ether 0c:c4:7a:bd:65:58 inet 192.168.202.254 netmask 0xffffff00 broadcast 192.168.202.255=20 nd6 options=3D29 media: Ethernet autoselect status: active groups: lagg=20 laggproto lacp lagghash l2,l3,l4 laggport: ix0 flags=3D1c laggport: ix1 flags=3D1c but when ix's ports are configured as lagg1, all laggports work properly. W= hen I switch ix's ports as lagg0 and first in cloned_interfaces the problem appears: lagg0: flags=3D8843 metric 0 mtu 15= 00 =20=20=20=20=20=20=20 options=3De400bb ether 0c:c4:7a:bd:65:58 inet 192.168.202.254 netmask 0xffffff00 broadcast 192.168.202.255=20 nd6 options=3D29 media: Ethernet autoselect status: active groups: lagg=20 laggproto lacp lagghash l2,l3,l4 laggport: ix0 flags=3D1c laggport: ix1 flags=3D0<> lagg1: flags=3D8843 metric 0 mtu 15= 00 =20=20=20=20=20=20=20 options=3D6403bb ether 00:1e:67:27:1d:5e inet 192.168.202.254 netmask 0xffffff00 broadcast 192.168.202.255=20 nd6 options=3D29 media: Ethernet autoselect status: active groups: lagg=20 laggproto lacp lagghash l2,l3,l4 laggport: igb0 flags=3D1c laggport: igb1 flags=3D1c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Aug 4 11:26:19 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD3B0DD432E for ; Fri, 4 Aug 2017 11:26:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9BF12645A5 for ; Fri, 4 Aug 2017 11:26:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v74BQIb5021690 for ; Fri, 4 Aug 2017 11:26:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Fri, 04 Aug 2017 11:26:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 11:26:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 --- Comment #10 from Andrey V. Elsukov --- (In reply to Konrad from comment #9) > I set LAGG on igb0 and igb1 and seems to be ok: > but when ix's ports are configured as lagg1, all laggports work properly. > When I switch ix's ports as lagg0 and first in cloned_interfaces the > problem appears: Is it reproducible? Are you doing this in runtime manually without reboot? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Aug 4 11:49:13 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D79F4DD531C for ; Fri, 4 Aug 2017 11:49:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C5C6E650D2 for ; Fri, 4 Aug 2017 11:49:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v74BnCuK079429 for ; Fri, 4 Aug 2017 11:49:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Fri, 04 Aug 2017 11:49:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: konrad.kreciwilk@korbank.pl X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 11:49:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 --- Comment #11 from Konrad --- Yes, its reproducible. I change lagg configuration in rc.conf and reboot machine. if lagg0 will start with "ix1 flags=3D0" ix1 will have always "no carrier" = even if I remove from laggport. Manually removed ix1 and netif restart does no f= ix the problem (ix1 still has "no carrier"), only reboot. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Aug 4 16:25:22 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48000DBCFEC for ; Fri, 4 Aug 2017 16:25:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 370246EAF8 for ; Fri, 4 Aug 2017 16:25:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v74GPInp067027 for ; Fri, 4 Aug 2017 16:25:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-net@FreeBSD.org Subject: [Bug 221146] [LAGG] Problem with second laggport Date: Fri, 04 Aug 2017 16:25:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mjoras@freebsd.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-net@FreeBSD.org X-Bugzilla-Flags: mfc-stable11? X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 16:25:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D221146 Matt Joras changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mjoras@freebsd.org --- Comment #12 from Matt Joras --- Does this only happen with LACP or does it happen with any lagg type (e.g. failover)? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-net@freebsd.org Fri Aug 4 16:59:14 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AED5CDBE737 for ; Fri, 4 Aug 2017 16:59:14 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 444446FA42 for ; Fri, 4 Aug 2017 16:59:14 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id t201so24472862wmt.1 for ; Fri, 04 Aug 2017 09:59:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:subject:message-id:date:to :mime-version; bh=adi4m9KHHWK6C/yEZt3oLCU/xIKRdNScM8102LslH+A=; b=jOAsz4n8y3B8uLm687n4FIAphT2oTjcd9vHxEtsP4iU3f3onc6bYhlbHu9lCSEahQ8 fHrYbncSVoAv+kCnrRdQDf44xZZY1zB7FySd7PLt6cigX5ry0hsY9V8OmRAAqjvQSPbR fScdETtePDmEjf559V4/EpwaAHBupEEsdHLaj6buG7KzTnX7nyXwHYsbxISvFFN5h6LF um8f62l/B3JhXwc0ctKacDpkHh8TPo0fRB6uGUPF08DTVpghCpkPNmC+bYEqlZS3Ar41 gK5Yoyd5QJOgAf5BMAaZxxSCjeRywoVCvFhW1x/L/QaeCRfEgA3sKlAAbpb8rArHrEqg DTlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:subject :message-id:date:to:mime-version; bh=adi4m9KHHWK6C/yEZt3oLCU/xIKRdNScM8102LslH+A=; b=eIsdeBuZxWljEIOGVK7zv8esP4WJvpK9u7PI3jojkPdg3hVx8iWZYTvWfqjnWwxxBb 1u6G0VviJvMX002ZI4SB9ZgqifR08KAeO4TIdtkFYzhIb2LAvbRSzUGiQ4ipo/wG0sq9 QHpV+u4brBpWgEmMsvW02lX67kx46WbXBGtAXZZMeNaZZZ8uQJyo6dNgYmHczr32E6rS RMm03Sy3G65wfx6Z0tBrwRAuBtXWS4My4vz58ZeOfgAGvkzRPnThHgoNKlYHxCyzBtzG bvtVVn1kqnpdz6syJ5VYYGVuU6eDvx0E4m4RM/ofPauD+X8wAy6TBTyBvCCCfu+lrYld 6GbQ== X-Gm-Message-State: AHYfb5gx0IgK8CujdJhNE0a1di1mN8hLFv3zl5t+mVwy15CdnrwrTsju FeRsYM2WccnjnCk3swI= X-Received: by 10.28.33.85 with SMTP id h82mr1699272wmh.18.1501865952003; Fri, 04 Aug 2017 09:59:12 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id c141sm4217336wmd.44.2017.08.04.09.59.10 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 04 Aug 2017 09:59:11 -0700 (PDT) From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: mlx4en, timer irq @100%... Message-Id: Date: Fri, 4 Aug 2017 18:59:05 +0200 To: FreeBSD Net Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 16:59:14 -0000 Hello, Not sure this is the right list, but as it seems related to a mlx4en = device... # vmstat -i 1 (...) interrupt total rate cpu23:timer 1198 1127 # top -P ALL (...) CPU 23: 0.0% user, 0.0% nice, 0.0% system, 100% interrupt, 0.0% = idle # netstat -I mlxen0 -d -w 1 input mlxen0 output packets errs idrops bytes packets errs bytes colls = drops (and not output at all, same for mlxen1 !) # uname -sr FreeBSD 11.0-RELEASE-p9 So, as you can see, one of my CPUs is used at 100% by timer interrupts, since about 2 hours, and suddenly. Initiating network connections to this server is now slow. And what I found is that I can't use netstat on my 2 mlx4en devices = anymore (my monitoring tool is then no more fed). sysctl hw.mlxen0 is OK, no errors, and trafic counters grow slowly. What should I do ? How to investigate on this ? Thank you very much for your help & support, Best regards, Ben From owner-freebsd-net@freebsd.org Fri Aug 4 17:05:00 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 229E3DBEC98 for ; Fri, 4 Aug 2017 17:05:00 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFFE66FE30 for ; Fri, 4 Aug 2017 17:04:59 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 1A39026080B; Fri, 4 Aug 2017 19:04:51 +0200 (CEST) Subject: Re: mlx4en, timer irq @100%... To: Ben RUBSON , FreeBSD Net References: From: Hans Petter Selasky Message-ID: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> Date: Fri, 4 Aug 2017 19:02:38 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 17:05:00 -0000 On 08/04/17 18:59, Ben RUBSON wrote: > Hello, > > Not sure this is the right list, but as it seems related to a mlx4en device... > > # vmstat -i 1 > (...) > interrupt total rate > cpu23:timer 1198 1127 > > # top -P ALL > (...) > CPU 23: 0.0% user, 0.0% nice, 0.0% system, 100% interrupt, 0.0% idle > > # netstat -I mlxen0 -d -w 1 > input mlxen0 output > packets errs idrops bytes packets errs bytes colls drops > (and not output at all, same for mlxen1 !) > > # uname -sr > FreeBSD 11.0-RELEASE-p9 > > So, as you can see, one of my CPUs is used at 100% by timer interrupts, > since about 2 hours, and suddenly. > Initiating network connections to this server is now slow. > And what I found is that I can't use netstat on my 2 mlx4en devices anymore > (my monitoring tool is then no more fed). > > sysctl hw.mlxen0 is OK, no errors, and trafic counters grow slowly. > > What should I do ? > How to investigate on this ? > > Thank you very much for your help & support, > Hi, Try "procstat -ak". It should give an idea what is going on. What version of FreeBSD is this? Is this a regression issue? --HPS From owner-freebsd-net@freebsd.org Fri Aug 4 17:13:33 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35FD5DBF24C for ; Fri, 4 Aug 2017 17:13:33 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x22b.google.com (mail-wm0-x22b.google.com [IPv6:2a00:1450:400c:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5809970210 for ; Fri, 4 Aug 2017 17:13:32 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x22b.google.com with SMTP id t201so24891202wmt.0 for ; Fri, 04 Aug 2017 10:13:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:references:to:in-reply-to; bh=ui+KsMHwQDq92dln1PS1XYuEvAXLxom0u7PGJBfUcEs=; b=SU2wWl0wy9LTl6kc+zrCIZVV7ZW8bGykOT37zjK8dyt8f3K7VV26norWIBnYRwub1v RcWHt5p7ZmsS+tqPk7qKt0ioSYJugZKd5W0m7mDn9E6/HfnH0WA7jUUGOULJJlYjB2q/ ZbhI8cIaUhfy8rhe00xV25eZOgxo07MnOVAFAauoSS/IDq2kqe5BtcTJp5bjwsC5+vZH NpieNZq5VdvyhUWe3bz06EdRz0szKLIOhVlXGc8QM2gJXeTcg+RLB09Xgchmr3qzhBTO ehY+xl2tn6ISAK/BGBfH+Q7TlPxr4uck1iVLzElVSMDWqLs/5e8qm2AORWon/CihliES hxdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :references:to:in-reply-to; bh=ui+KsMHwQDq92dln1PS1XYuEvAXLxom0u7PGJBfUcEs=; b=lNdzJnklubdnXtjCp1t2+yW4uWer+arv2iPWXSA8FO/nQbC79p+rAqBurocZnfbytT UZ95OXinUNuSL1b50ZjK5VQPUvx0OvS3pvq87m+NNbhZ99p+nJ1/WKj0cvX5L/t5GQxN xgSb3O+p3aGzgnGFpCabqGDCb2lJP6pQCKFaUQVttnkkVXxfQQR6ZMqnCTE++CQ59ixf CwEnyc4xCuhjcJoNwhKzBY0SYwtO9D4NOe6qgdTKKhlNujWZolnXM0zgsKdi8PyVYW6p qWSxBrhrTkghPsfy4Wa9HdbbtJZVYXmQiJQppBrf7XbfmfzUifJRLfHku2ZIXvO4tlLd IkeQ== X-Gm-Message-State: AHYfb5jpCwvfDVUWqjYoc1IC/4LPR2NJv0meHR+UmYC7Mq1Y1Om2GU0m mn20HkRvqUVSPgFOV8U= X-Received: by 10.28.137.129 with SMTP id l123mr2147340wmd.101.1501866810498; Fri, 04 Aug 2017 10:13:30 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id y35sm4534921wrd.60.2017.08.04.10.13.29 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 04 Aug 2017 10:13:29 -0700 (PDT) From: Ben RUBSON Content-Type: multipart/mixed; boundary="Apple-Mail=_D37C05FA-4DF6-4726-9766-706C3E0D5472" Message-Id: <91DCB96E-4C08-44C5-94E7-E7C686DEFE5F@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mlx4en, timer irq @100%... Date: Fri, 4 Aug 2017 19:13:28 +0200 References: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> To: FreeBSD Net In-Reply-To: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 17:13:33 -0000 --Apple-Mail=_D37C05FA-4DF6-4726-9766-706C3E0D5472 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On 04 Aug 2017, at 19:02, Hans Petter Selasky wrote: >=20 > On 08/04/17 18:59, Ben RUBSON wrote: >> Hello, >> Not sure this is the right list, but as it seems related to a mlx4en = device... >> # vmstat -i 1 >> (...) >> interrupt total rate >> cpu23:timer 1198 1127 >> # top -P ALL >> (...) >> CPU 23: 0.0% user, 0.0% nice, 0.0% system, 100% interrupt, 0.0% = idle >> # netstat -I mlxen0 -d -w 1 >> input mlxen0 output >> packets errs idrops bytes packets errs bytes colls = drops >> (and not output at all, same for mlxen1 !) >> # uname -sr >> FreeBSD 11.0-RELEASE-p9 >> So, as you can see, one of my CPUs is used at 100% by timer = interrupts, >> since about 2 hours, and suddenly. >> Initiating network connections to this server is now slow. >> And what I found is that I can't use netstat on my 2 mlx4en devices = anymore >> (my monitoring tool is then no more fed). >> sysctl hw.mlxen0 is OK, no errors, and trafic counters grow slowly. >> What should I do ? >> How to investigate on this ? >> Thank you very much for your help & support, >=20 > Hi, >=20 > Try "procstat -ak". It should give an idea what is going on. >=20 > What version of FreeBSD is this? >=20 > Is this a regression issue? Hi HPS, and thank you for your answer, much appreciated ! procstat log attached. FreeBSD is 11.0-RELEASE-p9. A regression I'm not sure, as this server has more than 30 days of = uptime. 2 other identical servers with months of uptime without issue. Ben --Apple-Mail=_D37C05FA-4DF6-4726-9766-706C3E0D5472 Content-Disposition: attachment; filename=procstat.log Content-Type: application/octet-stream; name="procstat.log" Content-Transfer-Encoding: 7bit PID TID COMM TDNAME KSTACK 0 100000 kernel swapper mi_switch sleepq_timedwait _sleep swapper btext 0 100053 kernel thread taskq mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100055 kernel kqueue_ctx taskq mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100056 kernel aiod_kick taskq mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100062 kernel firmware taskq mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100065 kernel acpi_task_0 mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100066 kernel acpi_task_1 mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100067 kernel acpi_task_2 mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100073 kernel mps0 taskq mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100076 kernel mca taskq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100077 kernel system_taskq_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100078 kernel system_taskq_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100079 kernel system_taskq_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100080 kernel system_taskq_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100081 kernel system_taskq_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100082 kernel system_taskq_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100083 kernel system_taskq_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100084 kernel system_taskq_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100085 kernel system_taskq_8 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100086 kernel system_taskq_9 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100087 kernel system_taskq_10 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100088 kernel system_taskq_11 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100089 kernel system_taskq_12 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100090 kernel system_taskq_13 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100091 kernel system_taskq_14 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100092 kernel system_taskq_15 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100093 kernel system_taskq_16 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100094 kernel system_taskq_17 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100095 kernel system_taskq_18 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100096 kernel system_taskq_19 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100097 kernel system_taskq_20 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100098 kernel system_taskq_21 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100099 kernel system_taskq_22 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100100 kernel system_taskq_23 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100101 kernel dbu_evict mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100108 kernel CAM taskq mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100116 kernel mlx4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100122 kernel mlx4_en mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100123 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100124 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100125 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100126 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100127 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100128 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100129 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100130 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100131 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100132 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100133 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100134 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100135 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100136 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100137 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100138 kernel mlxen0 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100139 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100140 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100141 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100142 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100143 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100144 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100145 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100146 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100147 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100148 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100149 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100150 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100151 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100152 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100153 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100154 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100155 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100156 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100157 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100158 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100159 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100160 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100161 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100162 kernel mlxen0 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100163 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100164 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100165 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100166 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100167 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100168 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100169 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100170 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100171 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100172 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100173 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100174 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100175 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100176 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100177 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100178 kernel mlxen1 rx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100179 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100180 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100181 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100182 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100183 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100184 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100185 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100186 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100187 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100188 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100189 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100190 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100191 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100192 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100193 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100194 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100195 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100196 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100197 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100198 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100199 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100200 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100201 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100202 kernel mlxen1 tx cq mi_switch sleepq_wait msleep_spin_sbt taskqueue_thread_loop fork_exit fork_trampoline 0 100210 kernel if_config_tqg_0 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100211 kernel if_io_tqg_0 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100212 kernel if_io_tqg_1 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100213 kernel if_io_tqg_2 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100214 kernel if_io_tqg_3 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100215 kernel if_io_tqg_4 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100216 kernel if_io_tqg_5 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100217 kernel if_io_tqg_6 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100218 kernel if_io_tqg_7 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100219 kernel if_io_tqg_8 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100220 kernel if_io_tqg_9 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100221 kernel if_io_tqg_10 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100222 kernel if_io_tqg_11 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100223 kernel if_io_tqg_12 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100224 kernel if_io_tqg_13 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100225 kernel if_io_tqg_14 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100226 kernel if_io_tqg_15 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100227 kernel if_io_tqg_16 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100228 kernel if_io_tqg_17 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100229 kernel if_io_tqg_18 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100230 kernel if_io_tqg_19 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100231 kernel if_io_tqg_20 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100232 kernel if_io_tqg_21 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100233 kernel if_io_tqg_22 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100234 kernel if_io_tqg_23 mi_switch sleepq_wait msleep_spin_sbt gtaskqueue_thread_loop fork_exit fork_trampoline 0 100453 kernel cfiscsimt mi_switch sleepq_wait _cv_wait cfiscsi_maintenance_thread fork_exit fork_trampoline 0 100457 kernel cfiscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 100475 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100476 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100478 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100481 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100482 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100485 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100487 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100505 kernel zio_null_issue mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100506 kernel zio_null_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100507 kernel zio_read_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100508 kernel zio_read_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100509 kernel zio_read_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100510 kernel zio_read_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100511 kernel zio_read_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100512 kernel zio_read_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100513 kernel zio_read_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100514 kernel zio_read_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100515 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100516 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100517 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100518 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100519 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100520 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100521 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100522 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100523 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100524 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100525 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100526 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100527 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100528 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100529 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100530 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100531 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100532 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100533 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100534 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100535 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100536 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100537 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100538 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100539 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100540 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100541 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100542 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100543 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100544 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100545 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100546 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100547 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100548 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100549 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100550 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100551 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100552 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100553 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100554 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100555 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100556 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100557 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100558 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100559 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100560 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100561 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100562 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100563 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100564 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100565 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100566 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100567 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100568 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100569 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100570 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100571 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100572 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100573 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100574 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100575 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100576 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100577 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100578 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100579 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100580 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100581 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100582 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100583 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100584 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100585 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100586 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100587 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100588 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100589 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100590 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100591 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100592 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100593 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100594 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100595 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100596 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100597 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100598 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100599 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100600 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100601 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100602 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100603 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100604 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100605 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100606 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100607 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100608 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100609 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100610 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100611 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100612 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100613 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100614 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100615 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100616 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100617 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100618 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100619 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100620 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100621 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100622 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100623 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100624 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100625 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100626 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100627 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100628 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100629 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100630 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100631 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100632 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100633 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100634 kernel zio_write_intr_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100635 kernel zio_write_intr_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100636 kernel zio_write_intr_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100637 kernel zio_write_intr_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100638 kernel zio_write_intr_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100639 kernel zio_write_intr_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100640 kernel zio_write_intr_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100641 kernel zio_write_intr_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100642 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100643 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100644 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100645 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100646 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100647 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100648 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100649 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100650 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100651 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100652 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100653 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100654 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100655 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100656 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100657 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100658 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100659 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100660 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100661 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100662 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100663 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100664 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100665 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100666 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100667 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100668 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100669 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100670 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100671 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100672 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100673 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100674 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100675 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100676 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100677 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100678 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100679 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100680 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100681 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100682 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100683 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100684 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100685 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100686 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100687 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100688 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100689 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100690 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100691 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100692 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100693 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100694 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100695 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100696 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100697 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100698 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100699 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100700 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100701 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100702 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100703 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100704 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100705 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100706 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100707 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100708 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100709 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100710 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100711 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100712 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100713 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100714 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100715 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100716 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100717 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100718 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100719 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100720 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100721 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100722 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100723 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100724 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100725 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100726 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100727 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100728 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100729 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100730 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100731 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100732 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100733 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100734 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100735 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100736 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100737 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100738 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100739 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100740 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100741 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100742 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100743 kernel zio_free_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100744 kernel zio_claim_issue mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100745 kernel zio_claim_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100746 kernel zio_ioctl_issue mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100747 kernel zio_ioctl_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100749 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100750 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100751 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100752 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100753 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100754 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100755 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100756 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100757 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100758 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100759 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100760 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100761 kernel zfs_vn_rele_task mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 100808 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101022 kernel cblk0 taskq_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101023 kernel cblk0 taskq_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101024 kernel cblk0 taskq_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101025 kernel cblk0 taskq_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101026 kernel cblk0 taskq_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101027 kernel cblk0 taskq_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101028 kernel cblk0 taskq_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101029 kernel cblk0 taskq_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101030 kernel cblk0 taskq_8 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101031 kernel cblk0 taskq_9 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101032 kernel cblk0 taskq_10 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101033 kernel cblk0 taskq_11 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101034 kernel cblk0 taskq_12 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101035 kernel cblk0 taskq_13 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101036 kernel cblk1 taskq_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101037 kernel cblk1 taskq_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101038 kernel cblk1 taskq_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101039 kernel cblk1 taskq_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101040 kernel cblk1 taskq_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101041 kernel cblk1 taskq_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101042 kernel cblk1 taskq_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101043 kernel cblk1 taskq_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101044 kernel cblk1 taskq_8 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101045 kernel cblk1 taskq_9 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101046 kernel cblk1 taskq_10 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101047 kernel cblk1 taskq_11 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101048 kernel cblk1 taskq_12 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101049 kernel cblk1 taskq_13 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101050 kernel cblk2 taskq_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101051 kernel cblk2 taskq_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101052 kernel cblk2 taskq_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101053 kernel cblk2 taskq_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101054 kernel cblk2 taskq_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101055 kernel cblk2 taskq_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101056 kernel cblk2 taskq_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101057 kernel cblk2 taskq_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101058 kernel cblk2 taskq_8 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101059 kernel cblk2 taskq_9 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101060 kernel cblk2 taskq_10 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101061 kernel cblk2 taskq_11 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101062 kernel cblk2 taskq_12 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101063 kernel cblk2 taskq_13 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101064 kernel cblk3 taskq_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101065 kernel cblk3 taskq_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101066 kernel cblk3 taskq_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101067 kernel cblk3 taskq_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101068 kernel cblk3 taskq_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101069 kernel cblk3 taskq_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101070 kernel cblk3 taskq_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101071 kernel cblk3 taskq_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101072 kernel cblk3 taskq_8 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101073 kernel cblk3 taskq_9 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101074 kernel cblk3 taskq_10 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101075 kernel cblk3 taskq_11 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101076 kernel cblk3 taskq_12 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101077 kernel cblk3 taskq_13 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101078 kernel cblk4 taskq_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101079 kernel cblk4 taskq_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101080 kernel cblk4 taskq_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101081 kernel cblk4 taskq_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101082 kernel cblk4 taskq_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101083 kernel cblk4 taskq_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101084 kernel cblk4 taskq_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101085 kernel cblk4 taskq_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101086 kernel cblk4 taskq_8 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101087 kernel cblk4 taskq_9 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101088 kernel cblk4 taskq_10 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101089 kernel cblk4 taskq_11 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101090 kernel cblk4 taskq_12 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101091 kernel cblk4 taskq_13 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101092 kernel cblk5 taskq_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101093 kernel cblk5 taskq_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101094 kernel cblk5 taskq_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101095 kernel cblk5 taskq_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101096 kernel cblk5 taskq_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101097 kernel cblk5 taskq_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101098 kernel cblk5 taskq_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101099 kernel cblk5 taskq_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101100 kernel cblk5 taskq_8 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101101 kernel cblk5 taskq_9 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101102 kernel cblk5 taskq_10 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101103 kernel cblk5 taskq_11 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101104 kernel cblk5 taskq_12 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101105 kernel cblk5 taskq_13 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101106 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101107 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101108 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101109 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101110 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101111 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101112 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101113 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101114 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101115 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101116 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101117 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101118 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101119 kernel iscsimt mi_switch sleepq_wait _cv_wait iscsi_maintenance_thread fork_exit fork_trampoline 0 101127 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101128 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101130 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101131 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101132 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101133 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101136 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101137 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101138 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101139 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101140 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101141 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101142 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101143 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101144 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101145 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101146 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101147 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101148 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101149 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101150 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101151 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101152 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101153 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101154 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101155 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101534 kernel iscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 101535 kernel iscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 101795 kernel zio_null_issue mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101796 kernel zio_null_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101797 kernel zio_read_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101798 kernel zio_read_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101799 kernel zio_read_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101800 kernel zio_read_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101801 kernel zio_read_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101802 kernel zio_read_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101803 kernel zio_read_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101804 kernel zio_read_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101805 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101806 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101807 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101808 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101809 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101810 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101811 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101812 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101813 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101814 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101815 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101816 kernel zio_read_intr_0_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101817 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101818 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101819 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101820 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101821 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101822 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101823 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101824 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101825 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101826 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101827 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101828 kernel zio_read_intr_1_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101829 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101830 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101831 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101832 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101833 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101834 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101835 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101836 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101837 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101838 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101839 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101840 kernel zio_read_intr_2_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101841 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101842 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101843 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101844 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101845 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101846 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101847 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101848 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101849 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101850 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101851 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101852 kernel zio_read_intr_3_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101853 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101854 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101855 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101856 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101857 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101858 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101859 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101860 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101861 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101862 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101863 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101864 kernel zio_read_intr_4_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101865 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101866 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101867 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101868 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101869 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101870 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101871 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101872 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101873 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101874 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101875 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101876 kernel zio_read_intr_5_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101877 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101878 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101879 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101880 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101881 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101882 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101883 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101884 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101885 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101886 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101887 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101888 kernel zio_read_intr_6_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101889 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101890 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101891 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101892 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101893 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101894 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101895 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101896 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101897 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101898 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101899 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101900 kernel zio_read_intr_7_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101901 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101902 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101903 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101904 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101905 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101906 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101907 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101908 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101909 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101910 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101911 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101912 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101913 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101914 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101915 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101916 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101917 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101918 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101919 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101920 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101921 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101922 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101923 kernel zio_write_issue_ mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101924 kernel zio_write_intr_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101925 kernel zio_write_intr_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101926 kernel zio_write_intr_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101927 kernel zio_write_intr_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101928 kernel zio_write_intr_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101929 kernel zio_write_intr_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101930 kernel zio_write_intr_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101931 kernel zio_write_intr_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101932 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101933 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101934 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101935 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101936 kernel zio_write_intr_h mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101937 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101938 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101939 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101940 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101941 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101942 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101943 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101944 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101945 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101946 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101947 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101948 kernel zio_free_issue_0 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101949 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101950 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101951 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101952 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101953 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101954 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101955 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101956 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101957 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101958 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101959 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101960 kernel zio_free_issue_1 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101961 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101962 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101963 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101964 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101965 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101966 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101967 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101968 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101969 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101970 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101971 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101972 kernel zio_free_issue_2 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101973 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101974 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101975 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101976 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101977 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101978 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101979 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101980 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101981 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101982 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101983 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101984 kernel zio_free_issue_3 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101985 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101986 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101987 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101988 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101989 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101990 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101991 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101992 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101993 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101994 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101995 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101996 kernel zio_free_issue_4 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101997 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101998 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 101999 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102000 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102001 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102002 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102003 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102004 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102005 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102006 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102007 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102008 kernel zio_free_issue_5 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102009 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102010 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102011 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102012 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102013 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102014 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102015 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102016 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102017 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102018 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102019 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102020 kernel zio_free_issue_6 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102021 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102022 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102023 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102024 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102025 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102026 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102027 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102028 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102029 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102030 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102031 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102032 kernel zio_free_issue_7 mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102033 kernel zio_free_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102034 kernel zio_claim_issue mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102035 kernel zio_claim_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102036 kernel zio_ioctl_issue mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102037 kernel zio_ioctl_intr mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102039 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102040 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102041 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102042 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102043 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102044 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102045 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102046 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102047 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102048 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102049 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102050 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102051 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102052 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102053 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102054 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102055 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102056 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102057 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102058 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102059 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102060 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102061 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102062 kernel metaslab_group_t mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102063 kernel zfs_vn_rele_task mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102156 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102157 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102158 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102159 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102160 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102161 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102164 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102165 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102166 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102169 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102170 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102171 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102173 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102175 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102177 kernel zil_clean mi_switch sleepq_wait _sleep taskqueue_thread_loop fork_exit fork_trampoline 0 102184 kernel cfiscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 102185 kernel cfiscsimt mi_switch sleepq_wait _cv_wait cfiscsi_maintenance_thread fork_exit fork_trampoline 0 102186 kernel cfiscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 102187 kernel cfiscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 102188 kernel cfiscsimt mi_switch sleepq_wait _cv_wait cfiscsi_maintenance_thread fork_exit fork_trampoline 0 102189 kernel cfiscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 102190 kernel cfiscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 102191 kernel cfiscsimt mi_switch sleepq_wait _cv_wait cfiscsi_maintenance_thread fork_exit fork_trampoline 0 102192 kernel cfiscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 102193 kernel cfiscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 102194 kernel cfiscsimt mi_switch sleepq_wait _cv_wait cfiscsi_maintenance_thread fork_exit fork_trampoline 0 102195 kernel cfiscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 102196 kernel cfiscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 0 102199 kernel cfiscsimt mi_switch sleepq_wait _cv_wait cfiscsi_maintenance_thread fork_exit fork_trampoline 0 102200 kernel cfiscsitx mi_switch sleepq_wait _cv_wait icl_send_thread fork_exit fork_trampoline 0 102201 kernel cfiscsirx mi_switch sleepq_wait _cv_wait icl_receive_thread fork_exit fork_trampoline 1 100002 init - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_wait6 sys_wait4 amd64_syscall Xfast_syscall 2 100063 crypto - mi_switch sleepq_wait _sleep crypto_proc fork_exit fork_trampoline 3 100064 crypto returns - mi_switch sleepq_wait _sleep crypto_ret_proc fork_exit fork_trampoline 4 100068 cam doneq0 mi_switch sleepq_wait _sleep xpt_done_td fork_exit fork_trampoline 4 100069 cam doneq1 mi_switch sleepq_wait _sleep xpt_done_td fork_exit fork_trampoline 4 100070 cam doneq2 mi_switch sleepq_wait _sleep xpt_done_td fork_exit fork_trampoline 4 100071 cam doneq3 mi_switch sleepq_wait _sleep xpt_done_td fork_exit fork_trampoline 4 100072 cam doneq4 mi_switch sleepq_wait _sleep xpt_done_td fork_exit fork_trampoline 4 100109 cam scanner mi_switch sleepq_wait _sleep xpt_scanner_thread fork_exit fork_trampoline 5 100102 zfskern arc_reclaim_thre mi_switch sleepq_timedwait _cv_timedwait_sbt arc_reclaim_thread fork_exit fork_trampoline 5 100103 zfskern arc_user_evicts_ mi_switch sleepq_timedwait _cv_timedwait_sbt arc_user_evicts_thread fork_exit fork_trampoline 5 100104 zfskern arc_dnlc_evicts_ mi_switch sleepq_wait _cv_wait arc_dnlc_evicts_thread fork_exit fork_trampoline 5 100105 zfskern l2arc_feed_threa mi_switch sleepq_timedwait _cv_timedwait_sbt l2arc_feed_thread fork_exit fork_trampoline 5 100236 zfskern txg_thread_enter mi_switch sleepq_wait _cv_wait txg_quiesce_thread fork_exit fork_trampoline 5 100237 zfskern txg_thread_enter mi_switch sleepq_timedwait _cv_timedwait_sbt txg_sync_thread fork_exit fork_trampoline 5 100748 zfskern trim zroot-G1 mi_switch sleepq_timedwait _cv_timedwait_sbt trim_thread fork_exit fork_trampoline 5 100806 zfskern txg_thread_enter mi_switch sleepq_wait _cv_wait txg_quiesce_thread fork_exit fork_trampoline 5 100807 zfskern txg_thread_enter mi_switch sleepq_timedwait _cv_timedwait_sbt txg_sync_thread fork_exit fork_trampoline 5 102038 zfskern trim home mi_switch sleepq_timedwait _cv_timedwait_sbt trim_thread fork_exit fork_trampoline 6 100106 sctp_iterator - mi_switch sleepq_wait _sleep sctp_iterator_thread fork_exit fork_trampoline 7 100107 rand_harvestq - mi_switch sleepq_timedwait _sleep random_kthread fork_exit fork_trampoline 8 100110 soaiod1 - mi_switch sleepq_timedwait _sleep soaio_kproc_loop fork_exit fork_trampoline 9 100111 soaiod2 - mi_switch sleepq_timedwait _sleep soaio_kproc_loop fork_exit fork_trampoline 10 100001 audit - mi_switch sleepq_wait _cv_wait audit_worker fork_exit fork_trampoline 11 100003 idle idle: cpu0 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100004 idle idle: cpu1 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100005 idle idle: cpu2 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100006 idle idle: cpu3 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100007 idle idle: cpu4 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100008 idle idle: cpu5 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100009 idle idle: cpu6 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100010 idle idle: cpu7 mi_switch critical_exit sched_idletd fork_exit fork_trampoline 11 100011 idle idle: cpu8 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100012 idle idle: cpu9 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100013 idle idle: cpu10 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100014 idle idle: cpu11 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100015 idle idle: cpu12 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100016 idle idle: cpu13 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100017 idle idle: cpu14 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100018 idle idle: cpu15 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100019 idle idle: cpu16 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100020 idle idle: cpu17 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100021 idle idle: cpu18 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100022 idle idle: cpu19 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100023 idle idle: cpu20 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100024 idle idle: cpu21 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100025 idle idle: cpu22 acpi_cpu_idle cpu_idle_acpi cpu_idle sched_idletd fork_exit fork_trampoline 11 100026 idle idle: cpu23 mi_switch critical_exit sched_idletd fork_exit fork_trampoline 12 100027 intr swi1: netisr 0 mi_switch ithread_loop fork_exit fork_trampoline 12 100028 intr swi3: vm 12 100029 intr swi4: clock (0) tcp_tw_2msl_scan pfslowtimo softclock_call_cc softclock intr_event_execute_handlers ithread_loop fork_exit fork_trampoline 12 100030 intr swi4: clock (1) 12 100031 intr swi4: clock (2) 12 100032 intr swi4: clock (3) 12 100033 intr swi4: clock (4) 12 100034 intr swi4: clock (5) 12 100035 intr swi4: clock (6) 12 100036 intr swi4: clock (7) 12 100037 intr swi4: clock (8) 12 100038 intr swi4: clock (9) 12 100039 intr swi4: clock (10) 12 100040 intr swi4: clock (11) 12 100041 intr swi4: clock (12) 12 100042 intr swi4: clock (13) 12 100043 intr swi4: clock (14) 12 100044 intr swi4: clock (15) 12 100045 intr swi4: clock (16) 12 100046 intr swi4: clock (17) 12 100047 intr swi4: clock (18) 12 100048 intr swi4: clock (19) 12 100049 intr swi4: clock (20) 12 100050 intr swi4: clock (21) 12 100051 intr swi4: clock (22) 12 100052 intr swi4: clock (23) 12 100054 intr swi5: fast taskq 12 100057 intr swi6: task queue mi_switch ithread_loop fork_exit fork_trampoline 12 100058 intr swi6: Giant task mi_switch ithread_loop fork_exit fork_trampoline 12 100074 intr irq264: mps0 mi_switch ithread_loop fork_exit fork_trampoline 12 100075 intr swi0: uart uart 12 100118 intr irq265: mlx4_cor 12 100119 intr irq266: mlx4_cor 12 100120 intr irq267: mlx4_cor 12 100121 intr irq268: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100855 intr irq269: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100856 intr irq270: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100857 intr irq271: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100858 intr irq272: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100859 intr irq273: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100860 intr irq274: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100861 intr irq275: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100862 intr irq276: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100863 intr irq277: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100864 intr irq278: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100865 intr irq279: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100866 intr irq280: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100867 intr irq281: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100868 intr irq282: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100869 intr irq283: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100870 intr irq284: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100872 intr irq285: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100873 intr irq286: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100874 intr irq287: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100875 intr irq288: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100876 intr irq289: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100877 intr irq290: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100878 intr irq291: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100879 intr irq292: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100880 intr irq293: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100881 intr irq294: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100882 intr irq295: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100883 intr irq296: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100884 intr irq297: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100885 intr irq298: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100886 intr irq299: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 12 100887 intr irq300: mlx4_cor mi_switch ithread_loop fork_exit fork_trampoline 13 100059 geom g_event mi_switch sleepq_wait _sleep g_run_events fork_exit fork_trampoline 13 100060 geom g_up mi_switch sleepq_wait _sleep g_io_schedule_up g_up_procbody fork_exit fork_trampoline 13 100061 geom g_down mi_switch sleepq_wait _sleep g_io_schedule_down g_down_procbody fork_exit fork_trampoline 14 100112 soaiod3 - mi_switch sleepq_timedwait _sleep soaio_kproc_loop fork_exit fork_trampoline 15 100113 soaiod4 - mi_switch sleepq_timedwait _sleep soaio_kproc_loop fork_exit fork_trampoline 16 100114 enc_daemon0 - mi_switch sleepq_wait _sleep enc_daemon fork_exit fork_trampoline 17 100115 enc_daemon1 - mi_switch sleepq_wait _sleep enc_daemon fork_exit fork_trampoline 18 100117 g_mirror swap - mi_switch sleepq_wait _sleep g_mirror_worker fork_exit fork_trampoline 19 100203 pagedaemon - mi_switch sleepq_timedwait _sleep vm_pageout fork_exit fork_trampoline 19 100235 pagedaemon uma mi_switch sleepq_wait _sleep uma_reclaim_worker fork_exit fork_trampoline 20 100204 vmdaemon - mi_switch sleepq_wait _sleep vm_daemon fork_exit fork_trampoline 21 100205 pagezero - mi_switch sleepq_timedwait _sleep vm_pagezero fork_exit fork_trampoline 22 100206 bufdaemon - mi_switch sleepq_timedwait _sleep buf_daemon fork_exit fork_trampoline 23 100207 bufspacedaemon - mi_switch sleepq_timedwait _sleep bufspace_daemon fork_exit fork_trampoline 24 100208 syncer - mi_switch sleepq_timedwait _cv_timedwait_sbt sched_sync fork_exit fork_trampoline 25 100209 vnlru - mi_switch sleepq_timedwait _sleep vnlru_proc fork_exit fork_trampoline 120 100830 adjkerntz - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_sigsuspend sys_sigsuspend amd64_syscall Xfast_syscall 377 100816 devd - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 597 100910 syslogd - mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig seltdwait kern_select sys_select amd64_syscall Xfast_syscall 710 100844 iscsid - mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig iscsi_ioctl devfs_ioctl_f kern_ioctl sys_ioctl amd64_syscall Xfast_syscall 753 100851 ntpd - mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig seltdwait kern_select sys_select amd64_syscall Xfast_syscall 770 100914 smartd - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kern_nanosleep sys_nanosleep amd64_syscall Xfast_syscall 773 100871 rsync - mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig seltdwait kern_select sys_select amd64_syscall Xfast_syscall 780 100850 memcached - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kqueue_kevent kern_kevent sys_kevent amd64_syscall Xfast_syscall 780 100916 memcached - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kern_nanosleep sys_nanosleep amd64_syscall Xfast_syscall 780 100917 memcached - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kqueue_kevent kern_kevent sys_kevent amd64_syscall Xfast_syscall 780 100918 memcached - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kqueue_kevent kern_kevent sys_kevent amd64_syscall Xfast_syscall 780 100919 memcached - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kqueue_kevent kern_kevent sys_kevent amd64_syscall Xfast_syscall 780 100920 memcached - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kqueue_kevent kern_kevent sys_kevent amd64_syscall Xfast_syscall 780 100921 memcached - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep umtxq_sleep do_wait __umtx_op_wait_uint_private amd64_syscall Xfast_syscall 800 100812 sshd - mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig seltdwait kern_select sys_select amd64_syscall Xfast_syscall 822 100813 sendmail - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 825 100912 sendmail - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_sigsuspend sys_sigsuspend amd64_syscall Xfast_syscall 829 100907 cron - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kern_nanosleep sys_nanosleep amd64_syscall Xfast_syscall 831 100889 cron - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_wait6 sys_wait4 amd64_syscall Xfast_syscall 832 100927 cron - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_wait6 sys_wait4 amd64_syscall Xfast_syscall 835 100824 bash - mi_switch sleepq_wait _cv_wait zio_wait dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_freebsd_read VOP_READ_APV vn_read vn_io_fault1 vn_io_fault dofileread kern_readv sys_read amd64_syscall Xfast_syscall 836 100891 bash - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_wait6 sys_wait4 amd64_syscall Xfast_syscall 1554 100894 ctl work0 mi_switch sleepq_wait _sleep ctl_work_thread fork_exit fork_trampoline 1554 101015 ctl work1 mi_switch sleepq_wait _sleep ctl_work_thread fork_exit fork_trampoline 1554 101016 ctl work2 mi_switch sleepq_wait _sleep ctl_work_thread fork_exit fork_trampoline 1554 101017 ctl work3 mi_switch sleepq_wait _sleep ctl_work_thread fork_exit fork_trampoline 1554 101018 ctl work4 mi_switch sleepq_wait _sleep ctl_work_thread fork_exit fork_trampoline 1554 101019 ctl work5 mi_switch sleepq_wait _sleep ctl_work_thread fork_exit fork_trampoline 1554 101020 ctl lun mi_switch sleepq_wait _sleep ctl_lun_thread fork_exit fork_trampoline 1554 101021 ctl thresh mi_switch sleepq_timedwait _sleep pause_sbt ctl_thresh_thread fork_exit fork_trampoline 1556 101003 ctld - mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig seltdwait kern_select sys_select amd64_syscall Xfast_syscall 17469 100940 httpd - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_accept4 accept1 amd64_syscall Xfast_syscall 27489 102226 sshd - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 28968 100460 sftp-server - mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig seltdwait kern_select sys_select amd64_syscall Xfast_syscall 29301 102411 sleep - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kern_nanosleep sys_nanosleep amd64_syscall Xfast_syscall 29302 100845 procstat - stack_save_td_running sysctl_kern_proc_kstack sysctl_root_handler_locked sysctl_root userland_sysctl sys___sysctl amd64_syscall Xfast_syscall 43007 102245 sshd - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 43009 100929 bash - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_wait6 sys_wait4 amd64_syscall Xfast_syscall 77111 100375 python2.7 - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 77111 100479 python2.7 - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 77111 102218 python2.7 - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 77112 101123 python2.7 - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep pipe_read dofileread kern_readv sys_read amd64_syscall Xfast_syscall 77113 100903 python2.7 - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep pipe_read dofileread kern_readv sys_read amd64_syscall Xfast_syscall 77115 101007 top - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kern_nanosleep sys_nanosleep amd64_syscall Xfast_syscall 77116 102425 python2.7 - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep pipe_read dofileread kern_readv sys_read amd64_syscall Xfast_syscall 77117 102424 gstat - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kern_nanosleep sys_nanosleep amd64_syscall Xfast_syscall 77119 101473 bash - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep pipe_read dofileread kern_readv sys_read amd64_syscall Xfast_syscall 77130 100262 zpool - mi_switch sleepq_catch_signals sleepq_timedwait_sig _sleep kern_nanosleep sys_nanosleep amd64_syscall Xfast_syscall 77203 100499 bash - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_wait6 sys_wait4 amd64_syscall Xfast_syscall 77204 102325 tail - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kqueue_kevent kern_kevent sys_kevent amd64_syscall Xfast_syscall 95423 102251 httpd - mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt seltdwait kern_select sys_select amd64_syscall Xfast_syscall 95424 100895 perl - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep pipe_read dofileread kern_readv sys_read amd64_syscall Xfast_syscall 95425 100326 perl - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep pipe_read dofileread kern_readv sys_read amd64_syscall Xfast_syscall 95426 100980 perl - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep pipe_read dofileread kern_readv sys_read amd64_syscall Xfast_syscall 95427 102326 httpd - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_accept4 accept1 amd64_syscall Xfast_syscall 95428 102261 httpd - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_accept4 accept1 amd64_syscall Xfast_syscall 95429 102429 httpd - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_accept4 accept1 amd64_syscall Xfast_syscall 95430 100925 httpd - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_accept4 accept1 amd64_syscall Xfast_syscall 95431 102405 httpd - mi_switch sleepq_catch_signals sleepq_wait_sig _sleep kern_accept4 accept1 amd64_syscall Xfast_syscall --Apple-Mail=_D37C05FA-4DF6-4726-9766-706C3E0D5472 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_D37C05FA-4DF6-4726-9766-706C3E0D5472-- From owner-freebsd-net@freebsd.org Fri Aug 4 17:33:21 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A02A9DBFFE6 for ; Fri, 4 Aug 2017 17:33:21 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6951270AD6; Fri, 4 Aug 2017 17:33:21 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id AD0EC26080B; Fri, 4 Aug 2017 19:33:19 +0200 (CEST) Subject: Re: mlx4en, timer irq @100%... To: Ben RUBSON , FreeBSD Net , Gleb Smirnoff References: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> <91DCB96E-4C08-44C5-94E7-E7C686DEFE5F@gmail.com> From: Hans Petter Selasky Message-ID: Date: Fri, 4 Aug 2017 19:31:07 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <91DCB96E-4C08-44C5-94E7-E7C686DEFE5F@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 17:33:21 -0000 On 08/04/17 19:13, Ben RUBSON wrote: > 12 100029 intr swi4: clock (0) tcp_tw_2msl_scan pfslowtimo softclock_call_cc softclock intr_event_execute_handlers ithread_loop fork_exit fork_trampoline Hi, Can you "procstat -ak" a few times and grep for swi4. If the entry above does not disappear this is the culpit. Either the callout list is corrupted or there is an issue inside tcp_tw_2msl_scan(). I'm CC'ing Glebius hence he's been involved with timer issues in the kernel earlier. --HPS From owner-freebsd-net@freebsd.org Fri Aug 4 17:42:08 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3192CDC0489 for ; Fri, 4 Aug 2017 17:42:08 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6FD37102A; Fri, 4 Aug 2017 17:42:07 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id t201so25403098wmt.0; Fri, 04 Aug 2017 10:42:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=4Re2nXV+OGekkaMh341Q86ifJYbTQgOMCTvNlH9eEIY=; b=BpCZzMA98EssAbx8zWA3GDf3/6CkKdboWsu86n6fIeYmNoM/zeleiJLfHm5T0p6nW5 G1AcoDtV83Mv5ELztd1Z6MFgT6I/I+aBP5UFX6ezK5Okcvlx2P8MbmS5AYxjbXV3uV7/ jJLM2Cxxb2GHo3yHdCpV9tbs3zguVUr4oUFD56uX+Zwk0l6+S0hyCVKFGoyOpZJXI9i8 0dP9PG8TYcIQEASUyOqeUq1vbUP/BSSQ7fnEyBfDO2Y68feJ74yNg7ZySv8e5lvGgM+Y 8nT43+o6X6K2bKeIw2aUd8SriUiBEPyvbPaJHGAyeLGfg1bXz9NFGz6ec7vP9tyYJ1AD /QLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=4Re2nXV+OGekkaMh341Q86ifJYbTQgOMCTvNlH9eEIY=; b=mZu6n5uClbgAHqjKABt87IT9boR/5q9T7blyuPBcJZyHCfy7w7A0n2CG5bd1fWu67S yAepTrsvHkyA4il9vuaD90OhW1YspaWnc/i/zCpRkhBLv5IzgG9Vt8F/dMoB+4SkExX2 hlpan1AlpxmJEoo4HkG2xvIJWfFcKXuus//KdoTyfE01Z36Rc/UbrJw/fxogT9oZDfh3 My1rpt4/36aDs1XMELN32SkRvn8OP5DZbSFm6j5u1A1nXPeFFkE3neId2GkNDwucDMNn djAaKNdzbuUTSgsDTa3r/vZUAYun33ueS8t8PG4cuxTZmDRntBLqxB8Af9lPFX4fcxwt jkJw== X-Gm-Message-State: AHYfb5hloajywpUGhcThkxkdXNsDhJYo1JvBrTKFsLxwcVzfpcrQLLCX Xa3l88A20emz8eSPgjM= X-Received: by 10.28.25.6 with SMTP id 6mr1878232wmz.157.1501868525834; Fri, 04 Aug 2017 10:42:05 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id l25sm5097832wrb.44.2017.08.04.10.42.04 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 04 Aug 2017 10:42:05 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mlx4en, timer irq @100%... From: Ben RUBSON In-Reply-To: Date: Fri, 4 Aug 2017 19:42:03 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <4DF74CB8-23D2-4CCF-B699-5B86DAEA65E5@gmail.com> References: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> <91DCB96E-4C08-44C5-94E7-E7C686DEFE5F@gmail.com> To: FreeBSD Net X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 17:42:08 -0000 > On 04 Aug 2017, at 19:31, Hans Petter Selasky wrote: >=20 > On 08/04/17 19:13, Ben RUBSON wrote: >> 12 100029 intr swi4: clock (0) tcp_tw_2msl_scan = pfslowtimo softclock_call_cc softclock intr_event_execute_handlers = ithread_loop fork_exit fork_trampoline >=20 > Hi, >=20 > Can you "procstat -ak" a few times and grep for swi4. If the entry = above does not disappear this is the culpit. Either the callout list is = corrupted or there is an issue inside tcp_tw_2msl_scan(). Still here since my log catch 30 minutes ago, and sounds like it does = not want to disappear. > I'm CC'ing Glebius hence he's been involved with timer issues in the = kernel earlier. Feel free to ask me whatever you need to investigate on this ! I let this (production :/) server in this state to have a chance to get = interesting traces. Thank you, Ben From owner-freebsd-net@freebsd.org Fri Aug 4 17:47:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A347DDC094B for ; Fri, 4 Aug 2017 17:47:28 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [88.99.82.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A4DB71371 for ; Fri, 4 Aug 2017 17:47:27 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.129.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id EDB5C26080B; Fri, 4 Aug 2017 19:47:25 +0200 (CEST) Subject: Re: mlx4en, timer irq @100%... To: Ben RUBSON , FreeBSD Net References: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> <91DCB96E-4C08-44C5-94E7-E7C686DEFE5F@gmail.com> <4DF74CB8-23D2-4CCF-B699-5B86DAEA65E5@gmail.com> From: Hans Petter Selasky Message-ID: <473e881e-4b36-1538-2d92-7db37fa6f6a5@selasky.org> Date: Fri, 4 Aug 2017 19:45:13 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <4DF74CB8-23D2-4CCF-B699-5B86DAEA65E5@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 17:47:28 -0000 On 08/04/17 19:42, Ben RUBSON wrote: > >> On 04 Aug 2017, at 19:31, Hans Petter Selasky wrote: >> >> On 08/04/17 19:13, Ben RUBSON wrote: >>> 12 100029 intr swi4: clock (0) tcp_tw_2msl_scan pfslowtimo softclock_call_cc softclock intr_event_execute_handlers ithread_loop fork_exit fork_trampoline >> >> Hi, >> >> Can you "procstat -ak" a few times and grep for swi4. If the entry above does not disappear this is the culpit. Either the callout list is corrupted or there is an issue inside tcp_tw_2msl_scan(). > > Still here since my log catch 30 minutes ago, and sounds like it does not want to disappear. > >> I'm CC'ing Glebius hence he's been involved with timer issues in the kernel earlier. > > Feel free to ask me whatever you need to investigate on this ! > I let this (production :/) server in this state to have a chance to get interesting traces. > Hi, I guess we need to involve kgdb to get the full backtrace. Let's wait and see if anyone here knows how to do it right so the machine doesn't crash and the state is lost. --HPS From owner-freebsd-net@freebsd.org Fri Aug 4 17:53:10 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE83EDC0D73 for ; Fri, 4 Aug 2017 17:53:10 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FD9671715 for ; Fri, 4 Aug 2017 17:53:10 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x241.google.com with SMTP id y206so6453428wmd.5 for ; Fri, 04 Aug 2017 10:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=8Zc5tVKdKhUHHB3/3zdqpcWWkatr0ZBDjXVSjZ/rKu0=; b=UWFt6Rymh1g4xU61uPKtQRjMTYvUQPML72ICm+J9rJXUzuWIQthsSA2zTfZbpdFuwA YRWbeAOni52MWYWBqiJyCcZbOtUvsebczudyeQCE9weH2/YQVGd9Wd+MBm4d6HL9UaRS Z1byRUb/Ay1kqKkT6fqDG646KgnRjbFy5GrKMz0/wfmSacwJIGfWgm5tSBm2PrCDrU86 Q9XjOcuDFvttNI8G9zPc3fIedw+n3Yl74+z8nrYcHByBEv6JuCfB5G36iID++t41CsSG t1vxG7l4w0oJIRfivtsL83pG+omhLutI+S3kjFfn/ErIRw0slVJ0px94eqnHAiB6TDHo sZOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=8Zc5tVKdKhUHHB3/3zdqpcWWkatr0ZBDjXVSjZ/rKu0=; b=LP0qUy0m9egJKF1nlbaKORBtMU1SuX1mJhHUxEUANkQOywrzlRjnC61KI94LU3KSQs FKECUrLFkoTHKC4lFuyKJuFnAYRbyHYuPFKfgevEZZiZAzxH6em6jAEcyhA932UUiEta T84DcZBnLjDFEWdzUjGLpGP9wnH91Dx1kPtAZDNBJkkaJNHjsomCpTLT7YdmCE1UwIXA vrqi+pGd6j+1l0oZj+ZD+9rr9rb67ZqfN8FP6ex4xAfOMXa1g+eCNB2tl9lgsKq4vKzp 2iTGmrZbHZKsgPIHqpHuOt2srxXV75wt/N4r/SA2RwXfTBgpoACBRUU7EyqtkRDG6L0n y56w== X-Gm-Message-State: AHYfb5jmJRn2zjgbJmCugipkP+h1/FF1qSox5adylMpB85mbK4HTMMlg huPTPaMhJg3bTopAWfs= X-Received: by 10.28.2.8 with SMTP id 8mr1871483wmc.4.1501869188629; Fri, 04 Aug 2017 10:53:08 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id 53sm5871419wry.31.2017.08.04.10.53.07 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 04 Aug 2017 10:53:08 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mlx4en, timer irq @100%... From: Ben RUBSON In-Reply-To: <473e881e-4b36-1538-2d92-7db37fa6f6a5@selasky.org> Date: Fri, 4 Aug 2017 19:53:07 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <468D8DC1-2685-4C04-A830-03F8BEFA3408@gmail.com> References: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> <91DCB96E-4C08-44C5-94E7-E7C686DEFE5F@gmail.com> <4DF74CB8-23D2-4CCF-B699-5B86DAEA65E5@gmail.com> <473e881e-4b36-1538-2d92-7db37fa6f6a5@selasky.org> To: FreeBSD Net X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 17:53:10 -0000 > On 04 Aug 2017, at 19:45, Hans Petter Selasky wrote: >=20 > On 08/04/17 19:42, Ben RUBSON wrote: >>> On 04 Aug 2017, at 19:31, Hans Petter Selasky = wrote: >>>=20 >>> On 08/04/17 19:13, Ben RUBSON wrote: >>>> 12 100029 intr swi4: clock (0) tcp_tw_2msl_scan = pfslowtimo softclock_call_cc softclock intr_event_execute_handlers = ithread_loop fork_exit fork_trampoline >>>=20 >>> Hi, >>>=20 >>> Can you "procstat -ak" a few times and grep for swi4. If the entry = above does not disappear this is the culpit. Either the callout list is = corrupted or there is an issue inside tcp_tw_2msl_scan(). >> Still here since my log catch 30 minutes ago, and sounds like it does = not want to disappear. >>> I'm CC'ing Glebius hence he's been involved with timer issues in the = kernel earlier. >> Feel free to ask me whatever you need to investigate on this ! >> I let this (production :/) server in this state to have a chance to = get interesting traces. >=20 > Hi, >=20 > I guess we need to involve kgdb to get the full backtrace. Let's wait = and see if anyone here knows how to do it right so the machine doesn't = crash and the state is lost. I used to test kernel dump with "sysctl debug.kdb.panic=3D1" and it = worked correctly. However I'm not sure about debug.minidump, if we can have it enabled or = if it should be set to 0 in our case. Let's wait for others to be sure yes. Ben= From owner-freebsd-net@freebsd.org Fri Aug 4 19:09:11 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0DBACDC3F5D for ; Fri, 4 Aug 2017 19:09:11 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x232.google.com (mail-wr0-x232.google.com [IPv6:2a00:1450:400c:c0c::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C55A73D1B; Fri, 4 Aug 2017 19:09:10 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x232.google.com with SMTP id y43so19919018wrd.3; Fri, 04 Aug 2017 12:09:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=zqnlNfsuebDCpsr9nyyrdzWrkgKM0Ie7CusiPG7JafY=; b=pVTypQipJ14wxPs17HIQezM1tyGSRWp3PAihzGZCwYYYZbsOTwj4gASKmnEsp7LfNg DgG8+7RKtFyLt78dFBbwxWzGJZpE6OtbzG4DrbMJPS8/xX1HwpaONYBjQpmblJ+y4SYS 3iGsh1VENqa+2a2RK4H+TvPXo81BZH7zWPp9jDNrV0EzJwLYmzSjGaGPZ6td/yD/c8i3 ThuMVaheEY7w4OedSkI99dkn6QPedetq6rpMSPNoAgL/oykjs0RT6oUwtdfH4SB7Pr1x t83XcQcY890e6oGMCGfceC8h+V6LBVYfppgKwZTclHCL/2lyMRkC+myfK6kpNEPkdClF A6Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=zqnlNfsuebDCpsr9nyyrdzWrkgKM0Ie7CusiPG7JafY=; b=A3/3Xy/24B4qUpNzLboaKljUUHNDo50J3bFABGVcsV0hQkXCJXJet1ncz6yZHW1lZc 0o7Vi1IahdrGwbwjt+tzKpVAyz4YtvWX+mz7DLi0Um8LF5DnSNm9KKrQL1mDjuxnzP8Y WKjvmS4Sx/u6LvLLD3gcLHK3H2yIMlhQjou6XyGAWGrNTK1F+EyKX+re2DUUIrdT8A+s vOaIVZz+PfmXmuh2SIvR1K0BxxsAidFNNcqP2E9p+FJIo3NeiiunTJrrAayynLz5MEDQ +JHGSUE7RLTMAAQkDObCQFCAQBfuOIfFJD+sj69ZnDxhFeQwgfyItKS5GHukYSA7dXXi Hd5w== X-Gm-Message-State: AIVw111pMlSLyz4DHuAgx97PSkTJl/JH3a3L72icZAs8Uu8PgP2U/nzx 1T0BLV1oMcstqLUfg3I= X-Received: by 10.223.135.214 with SMTP id c22mr2795642wrc.263.1501873748059; Fri, 04 Aug 2017 12:09:08 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id x14sm8108280wmd.19.2017.08.04.12.09.06 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 04 Aug 2017 12:09:07 -0700 (PDT) Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mlx4en, timer irq @100%... From: Ben RUBSON In-Reply-To: <4DF74CB8-23D2-4CCF-B699-5B86DAEA65E5@gmail.com> Date: Fri, 4 Aug 2017 21:09:05 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <40602CEA-D417-4E5B-8C68-916958D49A0B@gmail.com> References: <72b9de84-5572-3737-b274-34d2c5cdf634@selasky.org> <91DCB96E-4C08-44C5-94E7-E7C686DEFE5F@gmail.com> <4DF74CB8-23D2-4CCF-B699-5B86DAEA65E5@gmail.com> To: FreeBSD Net X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2017 19:09:11 -0000 > On 04 Aug 2017, at 19:42, Ben RUBSON wrote: >=20 > Feel free to ask me whatever you need to investigate on this ! > I let this (production :/) server in this state to have a chance to = get interesting traces. Server no more in production, I moved service to the standby node. So we can do everything :) swi4 still burning ! What is strange is that according to my monitoring, it began exactly at = 15:00:00 UTC. Certainly a coincidence, as I can't find anything related. Ben