From owner-freebsd-stable@freebsd.org Sun Oct 9 08:28:49 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3448BC0761A for ; Sun, 9 Oct 2016 08:28:49 +0000 (UTC) (envelope-from chio1990@gmail.com) Received: from mail-pf0-x22b.google.com (mail-pf0-x22b.google.com [IPv6:2607:f8b0:400e:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 043437A1 for ; Sun, 9 Oct 2016 08:28:49 +0000 (UTC) (envelope-from chio1990@gmail.com) Received: by mail-pf0-x22b.google.com with SMTP id 190so41324632pfv.0 for ; Sun, 09 Oct 2016 01:28:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=WjhYiNavl7sfiHZPis6B5sjpN2RRXmxuq6//GJJhYbE=; b=NxpNwsp5UrNdvQBzLPJpCj/W6DOzJUgCad7EjTmmpItECmbSTdx+rVUXbPPpmpwsah TmabtSDjAx++kHafLa1ZEF41BtIQv5gaDTBAOF3OhgPp7stbVwHCpWE0WY76A+EAiCwV o6pu5ORAWkmQ4HYLM2B2aSTO4BOnH4ZTfr9NNqhXySwqtu40/q+iFsjKX51ojuz5wAZm qWzMNBqMJZIllNRAqsiP3kVeoTq9BdNDhJMwa7nTPma82ML+L5VJxNhQNlkaBPrd45tH WWCCLkUyvprCSLn73WtC4J8vKIplaSACP8HUAaxrbXNIDOgX0UNRIoQV1YaqZYlkEkr+ 3Hgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=WjhYiNavl7sfiHZPis6B5sjpN2RRXmxuq6//GJJhYbE=; b=VbZ+Ibro7ZAWTtpaFw7fScwhKfI+Ql7wF2GPgDdBjr0Y+/yjcjo/9/SewaxOYjpnDq VUyNOWToIfLitzB8P1LA8T8uEWgXMF5Y4e9XfllCQxr8DRlUr7hk2cbTApTaAQLd5IT2 uoSikalNP03fROovoNc4xORNhkwwAHCD5EJBQp+Ow6bx7eLMWCpUUgnSlvGkn6QXZ71J jwHh8ahM3bEQhddm3YQMA4/jPTtS43eM/CdYkYlqLS8AZyQ2dJG6B5oAqIBPDK59C3GG MyBx2X3xoZtAk8moh0Tw18XLWR+9f/Iq/rtAY2CdqmfdTumBcID49G16tHmx0XZr5apF XgzA== X-Gm-Message-State: AA6/9RmN7GLbcRZdmpY3kiylMMZ6HQn+ayoRl1UqXDQhDoYCshnqEAbr5vId2eW2wMM90Q== X-Received: by 10.98.14.207 with SMTP id 76mr52838857pfo.78.1476001728414; Sun, 09 Oct 2016 01:28:48 -0700 (PDT) Received: from kmatoMacBook-Pro.local ([221.234.47.40]) by smtp.gmail.com with ESMTPSA id c15sm25724621pfl.88.2016.10.09.01.28.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Oct 2016 01:28:47 -0700 (PDT) Subject: Re: FreeBSD 11.0-stable buildworld failured, maybe it's broken by r305866 To: freebsd-stable@freebsd.org References: <2ac28abf-3458-616f-b83d-12bf16d990e7@gmail.com> From: k simon Message-ID: Date: Sun, 9 Oct 2016 16:28:44 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Oct 2016 08:28:49 -0000 Ok, I solved it by copy those headers from /usr/src to /usr/include. Simon K. 20161009 > P.S. It's a quite old machine, it has not HPET device. > > > > Simon > 20161004 > > > 2016/10/4 20:40, k simon write: >> Hi,Lists, >> >> This is full source based "make buildworld" failure to r306669. >> >> >> clang -O2 -pipe -fno-omit-frame-pointer -march=core2 >> -I/usr/src/lib/libc/include -I/usr/src/lib/libc/../../include >> -I/usr/src/lib/libc/amd64 -DNLS -D__DBINTERFACE_PRIVATE >> -I/usr/src/lib/libc/../../contrib/gdtoa >> -I/usr/src/lib/libc/../../contrib/libc-vis -DINET6 >> -I/usr/obj/usr/src/lib/libc -I/usr/src/lib/libc/resolv -D_ACL_PRIVATE >> -DPOSIX_MISTAKE -I/usr/src/lib/libc/../libmd >> -I/usr/src/lib/libc/../../contrib/jemalloc/include -DMALLOC_PRODUCTION >> -I/usr/src/lib/libc/../../contrib/tzcode/stdtime >> -I/usr/src/lib/libc/stdtime -I/usr/src/lib/libc/locale -DBROKEN_DES >> -DPORTMAP -DDES_BUILTIN -I/usr/src/lib/libc/rpc -DYP -DNS_CACHING >> -DSYMBOL_VERSIONING -MD -MF.depend.__vdso_gettimeofday.o >> -MT__vdso_gettimeofday.o -std=gnu99 -fstack-protector-strong >> -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized >> -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int >> -Wno-unused-const-variable -Wno-tautological-compare -Wno-unused-value >> -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion >> -Wno-unused-local-typedef -Wno-switch -Wno-switch-enum >> -Wno-knr-promoted-parameter -Qunused-arguments -I/usr/src/lib/libutil >> -I/usr/src/lib/msun/amd64 -I/usr/src/lib/msun/x86 >> -I/usr/src/lib/msun/src -c /usr/src/lib/libc/sys/__vdso_gettimeofday.c >> -o __vdso_gettimeofday.o >> /usr/src/lib/libc/sys/__vdso_gettimeofday.c:43:27: error: too many >> arguments to function call, expected single argument 'vdso_th', have 2 >> arguments >> error = __vdso_gettc(th, &tc); >> ~~~~~~~~~~~~ ^~~ >> /usr/include/sys/vdso.h:65:1: note: '__vdso_gettc' declared here >> u_int __vdso_gettc(const struct vdso_timehands *vdso_th); >> ^ >> 1 error generated. >> *** Error code 1 >> >> Stop. >> make[4]: stopped in /usr/src/lib/libc >> *** Error code 1 >> >> >> >> Maybe it's broken by r305866. >> >> >> >> >> Simon >> 20161004 From owner-freebsd-stable@freebsd.org Sun Oct 9 09:01:23 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3765C0600B for ; Sun, 9 Oct 2016 09:01:23 +0000 (UTC) (envelope-from returnv@o.campanhasdigitais.we.bs) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8CC97D0E for ; Sun, 9 Oct 2016 09:01:23 +0000 (UTC) (envelope-from returnv@o.campanhasdigitais.we.bs) Received: by mailman.ysv.freebsd.org (Postfix) id 8BEA4C06008; Sun, 9 Oct 2016 09:01:23 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B8B2C06007 for ; Sun, 9 Oct 2016 09:01:23 +0000 (UTC) (envelope-from returnv@o.campanhasdigitais.we.bs) Received: from rev1.campanhasdigitais.we.bs (rev1.campanhasdigitais.we.bs [23.239.91.23]) by mx1.freebsd.org (Postfix) with ESMTP id 66B1FD0C for ; Sun, 9 Oct 2016 09:01:22 +0000 (UTC) (envelope-from returnv@o.campanhasdigitais.we.bs) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=default; d=campanhasdigitais.we.bs; h=To:Subject:Message-ID:Date:From:Reply-To:MIME-Version:List-Unsubscribe:Content-Type:Content-Transfer-Encoding; i=abuse@campanhasdigitais.we.bs; bh=A+q/AaMAxGNvTNJX/u3zzj8vXtY=; b=WsJ0UOHr42yLU2vvSphO/Q507TocJRQYUK19ktiAJkU6JAaZVifJOmz0eWxWzDwaILvP5qlv+jwJ wuyYcWJy3PtUGR4E0oSxQioDqb2ot0z031ylJlca0fjbncO4QUXl9y9NgPn+lKBGXMSboaoqYbxg +35oF/ByZjLl8rnGX84= To: stable@freebsd.org Subject: Prezado Sr.(a) - Feito sob medida para atender as suas necessidades. Message-ID: <8bc80771ee911b7decda3ef438d6fc58@x.campanhasdigitais.we.bs> Date: Sun, 09 Oct 2016 06:07:55 -0300 From: "=?UTF-8?B?U3VsQW3DqXJpY2EgU2VndXJvIFNhw7pkZQ==?=" <20sulamerica@e.campanhasdigitais.we.bs> Reply-To: comercialv@d.campanhasdigitais.we.bs MIME-Version: 1.0 X-Mailer-LID: 15,14 X-Mailer-RecptId: 3146075 X-Mailer-SID: 14 X-Mailer-Sent-By: 1 Content-Type: text/plain; format=flowed; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Oct 2016 09:01:23 -0000 Seu cliente de e-mail não pode ler este e-mail. Para visualizá-lo on-line, por favor, clique aqui: http://x.campanhasdigitais.we.bs/sistema/display.php?M=3146075&C=83e7773eab5fdbb34dea85fb43b4f7df&S=14&L=15&N=1 Para parar de receber nossos Emails:http://x.campanhasdigitais.we.bs/sistema/unsubscribe.php?M=3146075&C=83e7773eab5fdbb34dea85fb43b4f7df&L=15&N=14 From owner-freebsd-stable@freebsd.org Sun Oct 9 19:55:01 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91BF6C0B029 for ; Sun, 9 Oct 2016 19:55:01 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F322A9 for ; Sun, 9 Oct 2016 19:55:01 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id u99JsseZ078982 for ; Sun, 9 Oct 2016 15:54:59 -0400 (EDT) (envelope-from george+freebsd@m5p.com) To: FreeBSD Stable Mailing List From: George Mitchell Subject: Failing to upgrade from 10.1-RELEASE to 10.3-STABLE Message-ID: <5942b107-349b-4a97-7f26-e24ea09079bb@m5p.com> Date: Sun, 9 Oct 2016 15:54:54 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [10.100.0.247]); Sun, 09 Oct 2016 15:54:59 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Oct 2016 19:55:01 -0000 # freebsd-update -r 10.3-STABLE upgrade Looking up update.FreeBSD.org mirrors... 4 mirrors found. Fetching metadata signature for 10.1-RELEASE from update5.freebsd.org... done. Fetching metadata index... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/src world/base world/lib32 The following components of FreeBSD do not seem to be installed: world/doc world/games Does this look reasonable (y/n)? y Fetching metadata signature for 10.3-STABLE from update5.freebsd.org... failed. Fetching metadata signature for 10.3-STABLE from update4.freebsd.org... failed. Fetching metadata signature for 10.3-STABLE from update6.freebsd.org... failed. Fetching metadata signature for 10.3-STABLE from update3.freebsd.org... failed. No mirrors remaining, giving up. What am I doing wrong? (I get the same failure attempting to upgrade to 10.1-RELEASE and 10.2-RELEASE.) -- George From owner-freebsd-stable@freebsd.org Sun Oct 9 20:58:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDFBFC0701C for ; Sun, 9 Oct 2016 20:58:05 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (unknown [IPv6:2001:6a0:10f:1::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "plan-b.pwste.edu.pl", Issuer "plan-b.pwste.edu.pl" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D9E5DCB for ; Sun, 9 Oct 2016 20:58:03 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (zarychtam@localhost [127.0.0.1]) by plan-b.pwste.edu.pl (8.15.2/8.15.2) with ESMTPS id u99KvwIr008761 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sun, 9 Oct 2016 22:57:58 +0200 (CEST) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: (from zarychtam@localhost) by plan-b.pwste.edu.pl (8.15.2/8.15.2/Submit) id u99Kvwn5008759 for freebsd-stable@freebsd.org; Sun, 9 Oct 2016 22:57:58 +0200 (CEST) (envelope-from zarychtam) Date: Sun, 9 Oct 2016 22:57:58 +0200 From: Marek Zarychta To: freebsd-stable@freebsd.org Subject: possible regression on i386 Message-ID: <20161009205758.GB8329@plan-b.pwste.edu.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="rQ2U398070+RC21q" Content-Disposition: inline User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Oct 2016 20:58:06 -0000 --rQ2U398070+RC21q Content-Type: multipart/mixed; boundary="zx4FCpZtqtKETZ7O" Content-Disposition: inline --zx4FCpZtqtKETZ7O Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Dear Developers, I really appreciate your work for the project so it makes me really sorry to complain about the code, but probably commit r306905 breaks builds on i386 machines. I have been running 11.0-PRERELEASE on i386 machine for a few days. Upgrade from 9.3-STABLE through 10.3-STABLE went without an issue last week. Today I have tried to upgrade this system running on old Xeon without LM feature to latest version, but buildworld fails (see attached txt file). It is a quite old machine, where FreeBSD was installed 14 years ago, but regularly upgraded and always running supported branch. After reversion to r306777 world builds flawlessly. Best regards, --=20 Marek Zarychta --zx4FCpZtqtKETZ7O Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="r306905.bug.txt" --- all_subdir_usr.sbin --- --- all_subdir_usr.sbin/devctl --- --- .depend --- echo devctl.full: /usr/obj/usr/src/tmp/usr/lib/libc.a /usr/obj/usr/src/tmp/usr/lib/libdevctl.a >> .depend --- devctl.o --- clang -O2 -pipe -fno-strict-aliasing -march=pentium4 -g -MD -MF.depend.devctl.o -MTdevctl.o -std=gnu99 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c /usr/src/usr.sbin/devctl/devctl.c -o devctl.o --- all_subdir_lib --- --- test_archive_digest.o --- clang -O2 -pipe -fno-strict-aliasing -I/usr/src/lib/libarchive -I/usr/obj/usr/src/lib/libarchive/tests -I/usr/src/contrib/libarchive/libarchive -I/usr/src/contrib/libarchive/test_utils -DHAVE_LIBLZMA=1 -DHAVE_LZMA_H=1 -march=pentium4 -g -MD -MF.depend.libarchive_test.test_archive_digest.o -MTtest_archive_digest.o -std=gnu99 -fstack-protector-strong -Qunused-arguments -c /usr/src/contrib/libarchive/libarchive/test/test_archive_digest.c -o test_archive_digest.o --- all_subdir_tests --- --- mqtest2.debug --- objcopy --only-keep-debug mqtest2.full mqtest2.debug --- mqtest2 --- objcopy --strip-debug --add-gnu-debuglink=mqtest2.debug mqtest2.full mqtest2 --- mqtest3 --- (cd /usr/src/tests/sys/mqueue && DEPENDFILE=.depend.mqtest3 NO_SUBDIR=1 make -f /usr/src/tests/sys/mqueue/Makefile _RECURSING_PROGS=t PROG=mqtest3 ) --- all_subdir_usr.sbin --- --- all_subdir_usr.sbin/devinfo --- ===> usr.sbin/devinfo (all) --- all_subdir_tests --- --- .depend.mqtest3 --- echo mqtest3.full: /usr/obj/usr/src/tmp/usr/lib/libc.a /usr/obj/usr/src/tmp/usr/lib/librt.a >> .depend.mqtest3 --- mqtest3.o --- clang -O2 -pipe -fno-strict-aliasing -I/usr/src/tests -march=pentium4 -g -MD -MF.depend.mqtest3.mqtest3.o -MTmqtest3.o -std=gnu99 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c /usr/src/tests/sys/mqueue/mqtest3.c -o mqtest3.o /usr/src/tests/sys/mqueue/mqtest3.c:65:11: warning: implicit declaration of function 'mq_getfd_np' is invalid in C99 [-Wimplicit-function-declaration] FD_SET(mq_getfd_np(mq), &set); ^ --- all_subdir_usr.sbin --- --- .depend --- echo devinfo.full: /usr/obj/usr/src/tmp/usr/lib/libc.a /usr/obj/usr/src/tmp/usr/lib/libdevinfo.a >> .depend --- devinfo.o --- clang -O2 -pipe -fno-strict-aliasing -march=pentium4 -g -MD -MF.depend.devinfo.o -MTdevinfo.o -std=gnu99 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -c /usr/src/usr.sbin/devinfo/devinfo.c -o devinfo.o --- all_subdir_lib --- --- test_archive_getdate.o --- clang -O2 -pipe -fno-strict-aliasing -I/usr/src/lib/libarchive -I/usr/obj/usr/src/lib/libarchive/tests -I/usr/src/contrib/libarchive/libarchive -I/usr/src/contrib/libarchive/test_utils -DHAVE_LIBLZMA=1 -DHAVE_LZMA_H=1 -march=pentium4 -g -MD -MF.depend.libarchive_test.test_archive_getdate.o -MTtest_archive_getdate.o -std=gnu99 -fstack-protector-strong -Qunused-arguments -c /usr/src/contrib/libarchive/libarchive/test/test_archive_getdate.c -o test_archive_getdate.o --- all_subdir_tests --- 1 warning generated. --- mqtest3.full --- clang -O2 -pipe -fno-strict-aliasing -I/usr/src/tests -march=pentium4 -g -std=gnu99 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -o mqtest3.full mqtest3.o -lrt --- all_subdir_usr.sbin --- --- all_subdir_usr.sbin/devctl --- --- devctl.full --- clang -O2 -pipe -fno-strict-aliasing -march=pentium4 -g -std=gnu99 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Qunused-arguments -o devctl.full devctl.o -ldevctl --- all_subdir_tests --- mqtest3.o: In function `main': /usr/src/tests/sys/mqueue/mqtest3.c:98: undefined reference to `mq_getfd_np' /usr/src/tests/sys/mqueue/mqtest3.c:98: undefined reference to `mq_getfd_np' /usr/src/tests/sys/mqueue/mqtest3.c:99: undefined reference to `mq_getfd_np' /usr/src/tests/sys/mqueue/mqtest3.c:65: undefined reference to `mq_getfd_np' /usr/src/tests/sys/mqueue/mqtest3.c:65: undefined reference to `mq_getfd_np' mqtest3.o:/usr/src/tests/sys/mqueue/mqtest3.c:67: more undefined references to `mq_getfd_np' follow clang: error: linker command failed with exit code 1 (use -v to see invocation) *** [mqtest3.full] Error code 1 make[6]: stopped in /usr/src/tests/sys/mqueue 1 error make[6]: stopped in /usr/src/tests/sys/mqueue *** [mqtest3] Error code 2 make[5]: stopped in /usr/src/tests/sys/mqueue 1 error make[5]: stopped in /usr/src/tests/sys/mqueue *** [all_subdir_tests/sys/mqueue] Error code 2 make[4]: stopped in /usr/src/tests/sys 1 error make[4]: stopped in /usr/src/tests/sys *** [all_subdir_tests/sys] Error code 2 make[3]: stopped in /usr/src/tests 1 error make[3]: stopped in /usr/src/tests *** [all_subdir_tests] Error code 2 make[2]: stopped in /usr/src --- all_subdir_lib --- A failure has been detected in another branch of the parallel make make[6]: stopped in /usr/src/lib/libarchive/tests *** [libarchive_test] Error code 2 make[5]: stopped in /usr/src/lib/libarchive/tests 1 error make[5]: stopped in /usr/src/lib/libarchive/tests *** [all] Error code 2 make[4]: stopped in /usr/src/lib/libarchive 1 error make[4]: stopped in /usr/src/lib/libarchive *** [all_subdir_lib/libarchive] Error code 2 make[3]: stopped in /usr/src/lib 1 error make[3]: stopped in /usr/src/lib *** [all_subdir_lib] Error code 2 make[2]: stopped in /usr/src --- all_subdir_usr.sbin --- A failure has been detected in another branch of the parallel make --zx4FCpZtqtKETZ7O-- --rQ2U398070+RC21q Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAABCAAGBQJX+q9SAAoJEHWf7P/9Uo0s0O8IALi/jHRXO0p4glm+xwh62QEC m6fW6s2PXA0xV17w54vy8OUVlsfgV24a73SarBcEXvHOc4f0NCgPY8Q2pBcOEPgq +nqnp4c0n6qKpmbITyc8qigaV8XNisJF5kDXQoaJaAD3ED6dHxCFtg7YoR6We16R JnvzUAy9SFNybBr/ev/4QX3mPAGH2vOBdUWRV8QKzHPagttPHD0/VUGyWeoLy9nr KQkYI89+TDLUPb8MAJedEetRcW2jSLuTaXYptlSOKfRTyrE9mnFoDnGpFQCru4h5 1iPQqWJ2DsD6cXbMm455PzcJsdYKcfCJSE0DZ4vPYJey2ktO8rTftATSNzGrIs4= =C29D -----END PGP SIGNATURE----- --rQ2U398070+RC21q-- From owner-freebsd-stable@freebsd.org Sun Oct 9 22:56:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1DCEC0B414 for ; Sun, 9 Oct 2016 22:56:11 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7DC05E61 for ; Sun, 9 Oct 2016 22:56:11 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id u99Mu30S079727; Sun, 9 Oct 2016 18:56:08 -0400 (EDT) (envelope-from george+freebsd@m5p.com) Subject: Re: Failing to upgrade from 10.1-RELEASE to 10.3-STABLE To: Kurt Jaeger References: <5942b107-349b-4a97-7f26-e24ea09079bb@m5p.com> <20161009195746.GC4560@home.opsec.eu> Cc: FreeBSD Stable Mailing List From: George Mitchell Message-ID: Date: Sun, 9 Oct 2016 18:56:03 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161009195746.GC4560@home.opsec.eu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [10.100.0.247]); Sun, 09 Oct 2016 18:56:09 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Oct 2016 22:56:11 -0000 On 10/09/16 15:57, Kurt Jaeger wrote: > Hi! > >> What am I doing wrong? (I get the same failure attempting to upgrade >> to 10.1-RELEASE and 10.2-RELEASE.) -- George > > Ah, one thing: > > Please do update to the latest 10.1-REL patch level, first. > After upgrading to the latest 10.1-RELEASE: I can update to 10.3-RELEASE, and that will probably do for now. Should it have worked to update from 10.1-RELEASE to 10.3-STABLE directly? If I decide to upgrade from 10.3-RELEASE to 10.3-STABLE later on, should I expect that to work? -- George From owner-freebsd-stable@freebsd.org Sun Oct 9 23:31:52 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7EF6AC0BB4C for ; Sun, 9 Oct 2016 23:31:52 +0000 (UTC) (envelope-from jungleboogie0@gmail.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 470FBEDB for ; Sun, 9 Oct 2016 23:31:52 +0000 (UTC) (envelope-from jungleboogie0@gmail.com) Received: by mail-it0-x22b.google.com with SMTP id o19so64856415ito.1 for ; Sun, 09 Oct 2016 16:31:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+6C2pTVWzsLT67P+b5DvLjd56iAcTqoEDNEDD2KYwh4=; b=Qu0skcCiJ4qp4/2QI1byz3OXbilHwW1LrUzzcB6La9r3qs4d1jyFgbDelJZ9pjiXJA 8rjH+taeQ3g5plXZtsF+LG+DfSWeKl3cxhT/W9wzKNPCMzicrg5HzsOoTEbhqlmgkwK2 4uohdKPBIylGgDsYccbveaUM2veTTmyA+Z70XOCA3aJhxOURiAybZQKsIzWHynFv0j95 5XdbhWMXjPhRKSIuSu2xnT23/Tb3lBb+IlbsXxyqYJcgjVTLdFfle9pCe8qcTY4Hp7cd awzTPgl2jRk2xpLXmM5bFZ0nq5o/bQK65uVmQNdamqUJE3y1oxIPYkDvzsQ35IA0nNaZ /Pgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+6C2pTVWzsLT67P+b5DvLjd56iAcTqoEDNEDD2KYwh4=; b=dHmatb93DggPCoG9nIf+DpxGtGvt0Z4SJ88b3j9butbf1AblbgAP5P2b5GHbKKfTW+ A+P738k2WVjTPUv44T1m12qSRQ1eCI7JiJwb82n0Bz4iVjxRw86DvxtiCOjcG7P1UjCk 83PKeBUiMd6p+Roks49zZ3jKJqm8hTFvreT1L1cTmxboHNOgdBaiNrtDaTC2apyOlHSt 5iS/0vvCWm0hGjIxlJJXA/AImQkzNMR//XW6V2qKV2L7IClQ/5ogAg/TMQzO3aUPm1Vb /SwIN/MqYRzM0/OOsEJ+p+NKg+4pVl3r2JNUxuFeTQ/0kh9k4LHSf3KF+IFyawSLUwV6 FVvA== X-Gm-Message-State: AA6/9RkNFrA5Htgrsofils4z/hD00qIIVRxrOh7z+HWNVhtYYJDkBDXlsTjZoMfx0c7Ub1UbCmBss0d1hvluCA== X-Received: by 10.36.14.20 with SMTP id 20mr7319787ite.88.1476055911639; Sun, 09 Oct 2016 16:31:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.15.163 with HTTP; Sun, 9 Oct 2016 16:31:50 -0700 (PDT) Received: by 10.107.15.163 with HTTP; Sun, 9 Oct 2016 16:31:50 -0700 (PDT) In-Reply-To: References: <5942b107-349b-4a97-7f26-e24ea09079bb@m5p.com> <20161009195746.GC4560@home.opsec.eu> From: jungle Boogie Date: Sun, 9 Oct 2016 16:31:50 -0700 Message-ID: Subject: Re: Failing to upgrade from 10.1-RELEASE to 10.3-STABLE To: George Mitchell Cc: freebsd-stable , Kurt Jaeger Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Oct 2016 23:31:52 -0000 If I decide to upgrade from 10.3-RELEASE to 10.3-STABLE > later on, should I expect that to work? -- George > _ No. Freebsd-update is only for binary updates. Stable and head are where you build from source and therefore, freebsd-update doesn't work. From owner-freebsd-stable@freebsd.org Mon Oct 10 00:11:57 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01530C07504 for ; Mon, 10 Oct 2016 00:11:57 +0000 (UTC) (envelope-from enslay@gmail.com) Received: from mail-io0-x229.google.com (mail-io0-x229.google.com [IPv6:2607:f8b0:4001:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF5CFDB6 for ; Mon, 10 Oct 2016 00:11:56 +0000 (UTC) (envelope-from enslay@gmail.com) Received: by mail-io0-x229.google.com with SMTP id j37so96245501ioo.3 for ; Sun, 09 Oct 2016 17:11:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0EqZzAJByPowLmVQ9lgLUI1EUWaV5VzX+CIySlw/4yM=; b=nzIo9eE4u4zDystOZVYlqsik2dUq3YonkzzYI/f4Aa6eARs4s18ft+p8qWMcie0+H0 5oxUxCQo4wYD0WCKL689c6VkrGUz0KP/rVFFicxYOvyVA+zApx5gt0xoPFfHUQu6IJc+ wpWtC9UP8YboK6UWsDNs4G693lssWbiiXesKXv0+SiB01i5vIhsJoNpu1jfre6TDzU4Z zjsHx2ymfTAVv3xyxuNFmigbyBzhobmYCWp1riCVrik2TGWD+eErwApRKd7E/hPrMUxD 0+YUbjIEGmRjWlIfakvwREJC4I9ep8QPda0YS4zgC4xZfLCnuLxbypeQYgEyfqwrEYMI JEPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0EqZzAJByPowLmVQ9lgLUI1EUWaV5VzX+CIySlw/4yM=; b=WSpDgoTiPAvk0AkjKyNNoGQ0gFUU+RtrBNin8kpjUjnwcdOMLE8nD56tfkGFiZVPuX 0LiRgNXBt4pOJwysIFMYl68Iamz6eIgcGXSI//WUhs81AjHlc5zz2eTPQB5Ext9ct98S n+8CNvoHU7724Fz+toNgNgxt4/0ea5do+Vqak1V9tV36f1jRZux2r2UqRmzUbj6OQi6B mNKpVYzperGEez8Hv7XJ9ABtipOd6NGJNtd0wmTpSIRFgkq3z6IW3xPq1SsQGCUpHyML nnfItGy+q+60Ia2Ht53mVeepg9XkwOCVGhPlZ32Rb9X1fPUa5QQQzNdMgOxvdmMf5Vbk rzuA== X-Gm-Message-State: AA6/9Rk83YYh9iQzJyRIF/jjgm1R0GPPkpMCfVTF3woHbrgf+GlDMTQkwJMZVtUAhKpoqVLHmx2UBa5bMCAS1g== X-Received: by 10.107.152.74 with SMTP id a71mr34542291ioe.120.1476058316156; Sun, 09 Oct 2016 17:11:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.14.71 with HTTP; Sun, 9 Oct 2016 17:11:55 -0700 (PDT) In-Reply-To: <20161009205758.GB8329@plan-b.pwste.edu.pl> References: <20161009205758.GB8329@plan-b.pwste.edu.pl> From: Nathan Lay Date: Sun, 9 Oct 2016 20:11:55 -0400 Message-ID: Subject: Re: possible regression on i386 To: Marek Zarychta Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 00:11:57 -0000 If your problem is anything like mine was, buildworld is trying to link with /usr/lib/librt.so rather than the new one built during the buildworld build. As per a recent commit, the new librt will have the additional mq_getfd_np() symbol while the original /usr/lib/librt.so will not. That causes those unresolved reference errors for new code trying to use the mq_getfd_np() function. Try building and installing librt manually: cd /usr/src/lib/librt make && make install Then try buildworld again. Reference: https://svnweb.freebsd.org/base?view=revision&revision=306905 Best regards, Nathan Lay On Sun, Oct 9, 2016 at 4:57 PM, Marek Zarychta < zarychtam@plan-b.pwste.edu.pl> wrote: > Dear Developers, > > I really appreciate your work for the project so it makes me really > sorry to complain about the code, but probably commit r306905 breaks > builds on i386 machines. > I have been running 11.0-PRERELEASE on i386 machine for a few days. > Upgrade from 9.3-STABLE through 10.3-STABLE went without an issue last > week. Today I have tried to upgrade this system running on old Xeon > without LM feature to latest version, but buildworld fails (see attached > txt file). It is a quite old machine, where FreeBSD was installed 14 > years ago, but regularly upgraded and always running supported branch. > After reversion to r306777 world builds flawlessly. > > Best regards, > -- > Marek Zarychta > From owner-freebsd-stable@freebsd.org Mon Oct 10 05:07:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0F61C0B6AD; Mon, 10 Oct 2016 05:07:16 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8661A8FB; Mon, 10 Oct 2016 05:07:16 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from julian-mbp3.pixel8networks.com (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u9A576x1072849 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 9 Oct 2016 22:07:08 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: fix for use-after-free problem in 10.x To: Oliver Pinter References: <7b732876-8cc3-a638-7ff1-e664060d4907@freebsd.org> Cc: FreeBSD Stable , freebsd From: Julian Elischer Message-ID: <0f543bb5-468e-e559-1bd8-8e2cf3f8bbc3@freebsd.org> Date: Sun, 9 Oct 2016 22:07:01 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 05:07:16 -0000 On 8/10/2016 5:36 AM, Oliver Pinter wrote: > On 10/5/16, Julian Elischer wrote: >> In 11 and 12 the taskqueue code has been rewritten in this area but >> under 10 this bug still occurs. >> >> On our appliances this bug stops the system from mounting the ZFS >> root, so it is quite severe. >> Basically while the thread is sleeping during the ZFS mount of root >> (in the while loop), another thread can free the 'task' item it is >> checking in that while loop and it can be reused or filled with >> 'deadcode' etc., with the waiting code unaware of the change.. The fix >> is to refetch the item at the end of the queue each time around the loop. >> I don't really want to do the bigger change of MFCing the change in >> 11, as it is more extensive, though if someone else does, that's ok by >> me. (If it's ABI compatible) >> >> Any comments or suggestions? > Yes, please commit them. This patch fixes the ZFS + GELI + INVARIANTS > problem for us. > There is the FreeBSD PR about the issue: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209580 I committed a slightly better version to stable/10 should I ask for a merge to releng/10.3? > >> here's the fix in diff form: >> >> >> [robot@porridge /usr/src]$ p4 diff -du ... >> --- //depot/pbranches/jelischer/FreeBSD-PZ/10.3/sys/kern/subr_taskqueue.c >> 2016-09-27 09:14:59.000000000 -0700 >> +++ /usr/src/sys/kern/subr_taskqueue.c 2016-09-27 09:14:59.000000000 -0700 >> @@ -441,9 +441,10 @@ >> >> TQ_LOCK(queue); >> task = STAILQ_LAST(&queue->tq_queue, task, ta_link); >> - if (task != NULL) >> - while (task->ta_pending != 0) >> - TQ_SLEEP(queue, task, &queue->tq_mutex, PWAIT, "-", >> 0); >> + while (task != NULL && task->ta_pending != 0) { >> + TQ_SLEEP(queue, task, &queue->tq_mutex, PWAIT, "-", 0); >> + task = STAILQ_LAST(&queue->tq_queue, task, ta_link); >> + } >> taskqueue_drain_running(queue); >> KASSERT(STAILQ_EMPTY(&queue->tq_queue), >> ("taskqueue queue is not empty after draining")); >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >> From owner-freebsd-stable@freebsd.org Mon Oct 10 07:05:03 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D614AF72A5 for ; Mon, 10 Oct 2016 07:05:03 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B210E96A for ; Mon, 10 Oct 2016 07:05:02 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from liminal.local (unknown [IPv6:2001:8b0:151:1:1c1d:86a1:a200:b700]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id 51D35A74E for ; Mon, 10 Oct 2016 07:04:59 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/51D35A74E; dkim=none; dkim-atps=neutral Subject: Re: Failing to upgrade from 10.1-RELEASE to 10.3-STABLE To: freebsd-stable@freebsd.org References: <5942b107-349b-4a97-7f26-e24ea09079bb@m5p.com> <20161009195746.GC4560@home.opsec.eu> From: Matthew Seaman Message-ID: Date: Mon, 10 Oct 2016 08:04:58 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="qXQtVv2HgJlCiKIMBw8m0RcEn7PIUptLn" X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00,RDNS_NONE, SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on smtp.infracaninophile.co.uk X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 07:05:03 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qXQtVv2HgJlCiKIMBw8m0RcEn7PIUptLn Content-Type: multipart/mixed; boundary="g6GvrfaF0uUcTSJwPBT1oF8tR7I6IwuEh"; protected-headers="v1" From: Matthew Seaman To: freebsd-stable@freebsd.org Message-ID: Subject: Re: Failing to upgrade from 10.1-RELEASE to 10.3-STABLE References: <5942b107-349b-4a97-7f26-e24ea09079bb@m5p.com> <20161009195746.GC4560@home.opsec.eu> In-Reply-To: --g6GvrfaF0uUcTSJwPBT1oF8tR7I6IwuEh Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 09/10/2016 23:56, George Mitchell wrote: > On 10/09/16 15:57, Kurt Jaeger wrote: >> Hi! >> >>> What am I doing wrong? (I get the same failure attempting to upgrade= >>> to 10.1-RELEASE and 10.2-RELEASE.) -- George >> >> Ah, one thing: >> >> Please do update to the latest 10.1-REL patch level, first. >> > After upgrading to the latest 10.1-RELEASE: >=20 > I can update to 10.3-RELEASE, and that will probably do for now. > Should it have worked to update from 10.1-RELEASE to 10.3-STABLE > directly? If I decide to upgrade from 10.3-RELEASE to 10.3-STABLE > later on, should I expect that to work? -- George Most of the time you should be able to upgrade from any patch level of release to the latest on any supported release branch using freebsd-update(8). However there have been a number of occasions where changes to freebsd-update itself cause that not to work. This is one of those occasions. As you've discovered, the answer is to update to the latest patch level of the branch you're already on, which will pull in the necessary fixes to freebsd-update(8), and then you can upgrade to a more recent branch. As other people have noted, you can't use freebsd-update(8) to get to 10.3-STABLE -- for that, you need to build the OS from source. Cheers, Matthew --g6GvrfaF0uUcTSJwPBT1oF8tR7I6IwuEh-- --qXQtVv2HgJlCiKIMBw8m0RcEn7PIUptLn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQJ8BAEBCgBmBQJX+z2bXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkAThl0P/10Qb5cRO5wBGzdsQNvGI8W8 mT+DFuZ5xbIC0NSk55fxJar3UPhxjKo1w7mcbCvYSjKX2YuWWxwfjXMThoAkYPGO OKNcpZ1fd9T4r8MUsq1EQjMolGMTa4KK/SfI52ESdXW1oYXOleT9NeGG0wQJU7Qg ddAjazc5i1Fneukhp9P7n78XZK/ZoWjdjvInXCGJ+QqUow4lPRd9tIIhmyhXsiNB Un+a32WOKv7J18tIlVlDw2mUaBkPzKiUhZKo/R9qAfJ+bTFRGVxZ3HE00YB9OwRr d8k6SdQue63fgYyV9tcYpWSvN4odajwOF9mq91YhfxIAGYEgyIAqpVS1TSj1c7aH kHzKpuj5BDj71saBqG09yCKh9DqEJi0mujEMYuB9AaGkzT6mqmmNx31TlJ35arBe u9BYF+dax+H3r1RgcxQzj0wbHgF2RC+DS+mjHs260j/nxEbPruTEqtH6vUJLieeh 7LQ1+zkV8ugS7eiq4mrNnT2KAKmX3tRCyy8ZrWiC/P19EMPTbyDhBYwbmTnlKQKR ZsP/6ScViY/PUI4TR62MyeweMifZDRLITYUmNhuQSmdcDIqQutV3tHNzJlAzzUL+ BvYG9OD2P6tKlp9ZNIX9YMpg/sQJ4OSkqNNdKPsNDgJ/LOP9E6DBWEwwgv9jtCUa rBvhWr2UkfoJXM2TgIpp =Us2H -----END PGP SIGNATURE----- --qXQtVv2HgJlCiKIMBw8m0RcEn7PIUptLn-- From owner-freebsd-stable@freebsd.org Mon Oct 10 08:07:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29FE5C0731D for ; Mon, 10 Oct 2016 08:07:59 +0000 (UTC) (envelope-from sa.inbox@gmail.com) Received: from mail-oi0-x22e.google.com (mail-oi0-x22e.google.com [IPv6:2607:f8b0:4003:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ECBB614D for ; Mon, 10 Oct 2016 08:07:58 +0000 (UTC) (envelope-from sa.inbox@gmail.com) Received: by mail-oi0-x22e.google.com with SMTP id m72so119195028oik.3 for ; Mon, 10 Oct 2016 01:07:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=3RXa5gvdgQAqMeDJZPbFmtrY79JfKroxqVVuJ4F5WLw=; b=w+cICpvXt7+uuhDLwHCHHNcwofjDT/w14TZG1zs+M21y207XxoQNftMzp57uzq7tPB DW7xGEDaENoTV0GWWcPuPuZmtToK34rtye5WfFuZKObm0wqWxnNW++Y3GrwPb4joBUve Zj/F2pewgdjUj1MBNU5cm+eq9POGddq8OKRUYrTCUgkWAEPB2dPeP9pFLN9w8rbPhR+E duz4XJjO9c2E2YrmbkH+IpjwRxWDAgzWpfm0DII5QUqs9YNKQzPnxxnlClmIhG9u6SI9 v3mdMmmLipbH5nZiqIwFG+slQHsWcUSDXsUCxG6m2+MjMEZXRn8vI/CDIOBY5+n8JKAU sgsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=3RXa5gvdgQAqMeDJZPbFmtrY79JfKroxqVVuJ4F5WLw=; b=McCRHjZy3Ijs36yqrT4xrN6pmwWWSl5GUVV6t4N2ub2xDVYt1j/QpoM67mR6E1+Ray 9R3n76UvMbZlUjlNcjlUGeuBQdJz9whivdgkbU0SPov/8luVz3qRYPItQ5NMa0zpB7+a DJT/uTiB1a7+7KQfQLqYHrx+WUumdazkHXMiw719yCwGF7jLkMLCFY5pFeDQoUCGyfYP VK82ltigPWwC1JuuLEfxaSNMWii1zD3w0Kr6fysDEsy6OyHjMys4n45q3HWVbA4+KXpS g/3duunCtGRbB/VxuSk3lcK0S/1skQcX1iw+73G0HPLbU6xDcu7xZSga4oGMVg/8F1I8 KomA== X-Gm-Message-State: AA6/9RkZ3CKizBTP0Dt9woe3dv1PRpJzvOJUNXsvOU1vXfKhEQWdDzJ4xzem66zwe6HZgtiip27vymv4ipPMcw== X-Received: by 10.157.31.124 with SMTP id x57mr17177024otx.154.1476086877975; Mon, 10 Oct 2016 01:07:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.37.130 with HTTP; Mon, 10 Oct 2016 01:07:37 -0700 (PDT) From: Alexander Shitov Date: Mon, 10 Oct 2016 11:07:37 +0300 Message-ID: Subject: Do not upgrade Hyper-V instances from 10.x to 11.x yet To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 08:07:59 -0000 11.0-RELEASE has issues with Hyper-V compatibility. It does not recognizes and then detaches virtual disk. storvsc0: on vmbus0 (probe0:blkvsc0:0:0:0): storvsc scsi_status = 2 (probe0:blkvsc0:0:1:1): invalid LUN 1 ... (probe0:blkvsc0:0:0:1): invalid LUN 1 da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 da0: da0 at blkvsc0 bus 0 scbus1 target 0 lun 0 da0: detached Unfortunately bug affects possibility to upgrade existing instances and prevents new 11.0 installations in Hyper-V environment. PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212721 I guess that this might be fixed with an errata notice after the release announcement. --- Best Regards, Alexander From owner-freebsd-stable@freebsd.org Mon Oct 10 10:48:21 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71F41C0743B for ; Mon, 10 Oct 2016 10:48:21 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4A9F3218 for ; Mon, 10 Oct 2016 10:48:20 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id E9B132013E for ; Mon, 10 Oct 2016 06:48:19 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute5.internal (MEProxy); Mon, 10 Oct 2016 06:48:19 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=zyxst.net; h= content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=67B HASHwzNbmuOb6Fk//9tudQ84=; b=srTlQdAby8mFFcMEYpRi/NxY6HEhxr6UXCB 21GKUve/lvNdeUp/0pJwtpcWU43wX0fhHTQN9Z0jTn2hPsHEb6H0IzhFQb+oRwI+ jA1yRmfPnCtIJurt3gOajOXiQT6P2pXrkI9cxHZkXOq2aIbOvgQWvlUu3aB66T0n htY9arL0= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=67BHASHwzNbmuOb6Fk//9tudQ84=; b=X3Byf WeXdJzvj50cvXpbc7+gKsXAEUQMPxgfloEgWT6k0T0SXBCUTuYwrKai5aUnVUdiL 4ugiL3VA18KBIToKz2gYvyavpEpKhCEN8iYzCl54+412Y25yJRB1lw/Qy5jUHFA4 m7Hq3rFprTyzH52TI+z48jZn+6R5Ul/+B0zwjc= X-Sasl-enc: +eBD2kPeT8tI75xaXHZbRIQ2/3EJurkFRix292jTqZZP 1476096499 Received: from pumpkin.growveg.org (pumpkin.growveg.org [82.70.91.101]) by mail.messagingengine.com (Postfix) with ESMTPA id 7B9E6F29D9 for ; Mon, 10 Oct 2016 06:48:19 -0400 (EDT) To: freebsd-stable@freebsd.org From: tech-lists Subject: 11-current becoming 11-stable Message-ID: <24b1b61b-5b4d-ae98-11e0-7939c11d5a61@zyxst.net> Date: Mon, 10 Oct 2016 11:48:12 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 10:48:21 -0000 Hi, I have a desktop that has run 11-current for at least the last year. I periodically rebuild kernel and world to keep on top of things. As it's a desktop, there are lots of ports installed. These are installed via the older cd /port && make install clean way, rather than poudiere. Basically I'm asking that, when 11-stable becomes available (as far as I can see, it's still at PRERELEASE), is it worthwhile/advisable to recompile all my ports after doing the make world & make delete-old dance? thanks, -- J. From owner-freebsd-stable@freebsd.org Mon Oct 10 11:26:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1413C0B327 for ; Mon, 10 Oct 2016 11:26:35 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f66.google.com (mail-lf0-f66.google.com [209.85.215.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 62F43F65 for ; Mon, 10 Oct 2016 11:26:34 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f66.google.com with SMTP id p80so9498663lfp.1 for ; Mon, 10 Oct 2016 04:26:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=XrCGbOil6wNQ9qC18JMmhcCIxEHRAiMvA9AjtRBeojQ=; b=MjZEcrIrjsa8otFMMulGhaj+977DMpb6quPSYgc1Km5KCfVQYozVm6Rhc7Jqfx3pNb RjY2bYK5xcoa+r4PFBEhNrxywk1FJj61A64mPMiX/KJSm65YDmkSWZ9FVa37nbO0cKrS 84Uoj2axR+E8LkrpNi6IDmRbrVfiZJmQAfbyvYk8/jgxEbotClzYgLtU6MajB5FkEv+e Wy17d/WFJMJh9MGJedrM5zxMIn3qQAOMbd4MDRCzsA9UFInbJ0Z6i/Y6G0lpUgOnSfHn WwJmuOp7itqjrW3rbxzzCseCflZ+F5cvCbKK9WsDACNHOjGhbEYqlOFVnkVFthZKAPB2 TC9Q== X-Gm-Message-State: AA6/9RnP2l91WOb/KKuB2saWa8yjS7DOzVU7OFLpMC6i1UKnutkryN4rn5nhdQUbMNXUUA== X-Received: by 10.25.15.169 with SMTP id 41mr12323935lfp.19.1476098786694; Mon, 10 Oct 2016 04:26:26 -0700 (PDT) Received: from [10.100.64.17] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id f23sm623801lji.12.2016.10.10.04.26.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Oct 2016 04:26:25 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> Date: Mon, 10 Oct 2016 13:26:12 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161006111043.GH54003@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="OCKtAMIJv6mron2xficHhijhFvEF3eE3r" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 11:26:36 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --OCKtAMIJv6mron2xficHhijhFvEF3eE3r Content-Type: multipart/mixed; boundary="xUDws4JJQNipMCDbiexlEg21759fQGK8C"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20160921195155.GW2840@zxy.spb.ru> <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> In-Reply-To: <20161006111043.GH54003@zxy.spb.ru> --xUDws4JJQNipMCDbiexlEg21759fQGK8C Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, On 10/6/16 1:10 PM, Slawa Olhovchenkov wrote: > On Thu, Oct 06, 2016 at 09:28:06AM +0200, Julien Charbon wrote: >=20 >> 2. thread1: In tcp_close() the inp is marked with INP_DROPPED flag, t= he >> process continues and calls INP_WUNLOCK() here: >> >> https://github.com/freebsd/freebsd/blob/releng/11.0/sys/netinet/tcp_su= br.c#L1568 >=20 > Look also to sys/netinet/tcp_timewait.c:488 >=20 > And check other locks from r160549 You are right, and here the a fix proposal for this issue: Fix a double-free when an inp transitions to INP_TIMEWAIT state after having been dropped https://reviews.freebsd.org/D8211 It basically enforces in_pcbdrop() logic in tcp_input(): A INP_DROPPED inpcb should never be proceed further. Slawa, as you are the only one to reproduce this issue currently, could test this patch? (And remove the temporary patch I did provided to you before). I will wait for your tests results before pushing further. Thanks! diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c index c72f01f..37f27e0 100644 --- a/sys/netinet/tcp_input.c +++ b/sys/netinet/tcp_input.c @@ -921,6 +921,16 @@ findpcb: goto dropwithreset; } INP_WLOCK_ASSERT(inp); + /* + * While waiting for inp lock during the lookup, another thread + * can have droppedt the inpcb, in which case we need to loop ba= ck + * and try to find a new inpcb to deliver to. + */ + if (inp->inp_flags & INP_DROPPED) { + INP_WUNLOCK(inp); + inp =3D NULL; + goto findpcb; + } if ((inp->inp_flowtype =3D=3D M_HASHTYPE_NONE) && (M_HASHTYPE_GET(m) !=3D M_HASHTYPE_NONE) && ((inp->inp_socket =3D=3D NULL) || @@ -981,6 +991,10 @@ relocked: if (in_pcbrele_wlocked(inp)) { inp =3D NULL; goto findpcb; + } else if (inp->inp_flags & INP_DROPPED) = { + INP_WUNLOCK(inp); + inp =3D NULL; + goto findpcb; } } else ti_locked =3D TI_RLOCKED; @@ -1040,6 +1054,10 @@ relocked: if (in_pcbrele_wlocked(inp)) { inp =3D NULL; goto findpcb; + } else if (inp->inp_flags & INP_DROPPED) = { + INP_WUNLOCK(inp); + inp =3D NULL; + goto findpcb; } goto relocked; } else -- Julien --xUDws4JJQNipMCDbiexlEg21759fQGK8C-- --OCKtAMIJv6mron2xficHhijhFvEF3eE3r Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX+3rgAAoJEKVlQ5Je6dhxQ40H/0dYh5hPqNQX1r15Z0x1sE9q 9/Lh6Zn6cLM+cxH2Me5rKeVxmX28bpTIug00fbqk6CI0ZlRHS+R4/iP3w2yl40g1 FUGysS8Cvh3EErzsoKHNwscrbNI8DWLgftW0L+el+srGRcVupoHA12AIhMTNCxQ+ Y990PZKWmuOuxCNxkCbm+yadaQbaOsrGoI0uyEoLDovE/rHKr2ObrypFadrXxg64 VL9xegpLzXnVMBUc3b/FbGAyq33KZnAsqc1Thi7pXEm7Lk6rT/m5mq3XC5jcPt9r MIPV9/pNj2Dy7FCQV/K/714O/F8tpCWjtp69KWVB9tcQGVtmd5Fsnh2dMVBH47c= =x0Tb -----END PGP SIGNATURE----- --OCKtAMIJv6mron2xficHhijhFvEF3eE3r-- From owner-freebsd-stable@freebsd.org Mon Oct 10 11:30:48 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D52A5C0B5D5 for ; Mon, 10 Oct 2016 11:30:48 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 946EE699; Mon, 10 Oct 2016 11:30:48 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1btYn8-0007Mu-Hs; Mon, 10 Oct 2016 14:30:46 +0300 Date: Mon, 10 Oct 2016 14:30:46 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161010113046.GT54003@zxy.spb.ru> References: <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 11:30:48 -0000 On Mon, Oct 10, 2016 at 01:26:12PM +0200, Julien Charbon wrote: > > Hi, > > On 10/6/16 1:10 PM, Slawa Olhovchenkov wrote: > > On Thu, Oct 06, 2016 at 09:28:06AM +0200, Julien Charbon wrote: > > > >> 2. thread1: In tcp_close() the inp is marked with INP_DROPPED flag, the > >> process continues and calls INP_WUNLOCK() here: > >> > >> https://github.com/freebsd/freebsd/blob/releng/11.0/sys/netinet/tcp_subr.c#L1568 > > > > Look also to sys/netinet/tcp_timewait.c:488 > > > > And check other locks from r160549 > > You are right, and here the a fix proposal for this issue: > > Fix a double-free when an inp transitions to INP_TIMEWAIT state after > having been dropped > https://reviews.freebsd.org/D8211 > > It basically enforces in_pcbdrop() logic in tcp_input(): A INP_DROPPED > inpcb should never be proceed further. > > Slawa, as you are the only one to reproduce this issue currently, could > test this patch? (And remove the temporary patch I did provided to you > before). > > I will wait for your tests results before pushing further. OK, I am will try it tomorrow Thanks! > Thanks! > > diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c > index c72f01f..37f27e0 100644 > --- a/sys/netinet/tcp_input.c > +++ b/sys/netinet/tcp_input.c > @@ -921,6 +921,16 @@ findpcb: > goto dropwithreset; > } > INP_WLOCK_ASSERT(inp); > + /* > + * While waiting for inp lock during the lookup, another thread > + * can have droppedt the inpcb, in which case we need to loop back > + * and try to find a new inpcb to deliver to. > + */ > + if (inp->inp_flags & INP_DROPPED) { > + INP_WUNLOCK(inp); > + inp = NULL; > + goto findpcb; > + } > if ((inp->inp_flowtype == M_HASHTYPE_NONE) && > (M_HASHTYPE_GET(m) != M_HASHTYPE_NONE) && > ((inp->inp_socket == NULL) || > @@ -981,6 +991,10 @@ relocked: > if (in_pcbrele_wlocked(inp)) { > inp = NULL; > goto findpcb; > + } else if (inp->inp_flags & INP_DROPPED) { > + INP_WUNLOCK(inp); > + inp = NULL; > + goto findpcb; > } > } else > ti_locked = TI_RLOCKED; > @@ -1040,6 +1054,10 @@ relocked: > if (in_pcbrele_wlocked(inp)) { > inp = NULL; > goto findpcb; > + } else if (inp->inp_flags & INP_DROPPED) { > + INP_WUNLOCK(inp); > + inp = NULL; > + goto findpcb; > } > goto relocked; > } else > > -- > Julien > From owner-freebsd-stable@freebsd.org Mon Oct 10 13:32:24 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F092C0BFD6 for ; Mon, 10 Oct 2016 13:32:24 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F1131C08; Mon, 10 Oct 2016 13:32:23 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1btagm-000AdC-Dh; Mon, 10 Oct 2016 16:32:20 +0300 Date: Mon, 10 Oct 2016 16:32:20 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161010133220.GU54003@zxy.spb.ru> References: <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 13:32:24 -0000 On Mon, Oct 10, 2016 at 01:26:12PM +0200, Julien Charbon wrote: > > Hi, > > On 10/6/16 1:10 PM, Slawa Olhovchenkov wrote: > > On Thu, Oct 06, 2016 at 09:28:06AM +0200, Julien Charbon wrote: > > > >> 2. thread1: In tcp_close() the inp is marked with INP_DROPPED flag, the > >> process continues and calls INP_WUNLOCK() here: > >> > >> https://github.com/freebsd/freebsd/blob/releng/11.0/sys/netinet/tcp_subr.c#L1568 > > > > Look also to sys/netinet/tcp_timewait.c:488 > > > > And check other locks from r160549 > > You are right, and here the a fix proposal for this issue: > > Fix a double-free when an inp transitions to INP_TIMEWAIT state after > having been dropped > https://reviews.freebsd.org/D8211 > > It basically enforces in_pcbdrop() logic in tcp_input(): A INP_DROPPED > inpcb should never be proceed further. > > Slawa, as you are the only one to reproduce this issue currently, could > test this patch? (And remove the temporary patch I did provided to you > before). > > I will wait for your tests results before pushing further. > > Thanks! > > diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c > index c72f01f..37f27e0 100644 > --- a/sys/netinet/tcp_input.c > +++ b/sys/netinet/tcp_input.c > @@ -921,6 +921,16 @@ findpcb: > goto dropwithreset; > } > INP_WLOCK_ASSERT(inp); > + /* > + * While waiting for inp lock during the lookup, another thread > + * can have droppedt the inpcb, in which case we need to loop back > + * and try to find a new inpcb to deliver to. > + */ > + if (inp->inp_flags & INP_DROPPED) { > + INP_WUNLOCK(inp); > + inp = NULL; > + goto findpcb; Are you sure about this goto? Can this cause infinite loop by found same inpcb? May be drop packet is more correct? From owner-freebsd-stable@freebsd.org Mon Oct 10 13:43:28 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62BB4C074CD for ; Mon, 10 Oct 2016 13:43:28 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1CCC0696 for ; Mon, 10 Oct 2016 13:43:27 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id u9ADhLAS084503 for ; Mon, 10 Oct 2016 09:43:26 -0400 (EDT) (envelope-from george+freebsd@m5p.com) Subject: Re: Failing to upgrade from 10.1-RELEASE to 10.3-STABLE To: freebsd-stable@freebsd.org References: <5942b107-349b-4a97-7f26-e24ea09079bb@m5p.com> <20161009195746.GC4560@home.opsec.eu> From: George Mitchell Message-ID: Date: Mon, 10 Oct 2016 09:43:21 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [10.100.0.247]); Mon, 10 Oct 2016 09:43:26 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 13:43:28 -0000 On 10/10/16 03:04, Matthew Seaman wrote: > [...] > Most of the time you should be able to upgrade from any patch level of > release to the latest on any supported release branch using > freebsd-update(8). However there have been a number of occasions where > changes to freebsd-update itself cause that not to work. This is one of > those occasions. > > As you've discovered, the answer is to update to the latest patch level > of the branch you're already on, which will pull in the necessary fixes > to freebsd-update(8), and then you can upgrade to a more recent branch. > > As other people have noted, you can't use freebsd-update(8) to get to > 10.3-STABLE -- for that, you need to build the OS from source. > > Cheers, > > Matthew > > Thanks for the further details. -- George From owner-freebsd-stable@freebsd.org Mon Oct 10 14:03:54 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD81FC07A63 for ; Mon, 10 Oct 2016 14:03:54 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f65.google.com (mail-lf0-f65.google.com [209.85.215.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FC541CD for ; Mon, 10 Oct 2016 14:03:53 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f65.google.com with SMTP id x79so6781253lff.2 for ; Mon, 10 Oct 2016 07:03:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=i2mYNuOQL8ywpSv9012bqUlqefmYluqHwG6qk56l4nU=; b=h9aZlyVPbzQ6NHhu/IOXzQAKKAhBoTSxQod7zOJ7uNvK1wa9tO3KfhuLpOnGaPXTx0 qSOyMIvgv6le9xhUKP+0/2qSSvF98bImo8DLqrnBpcUVTy6BJg9ZgRE4+vZgg0ZP3uEM 9CJasfeG9eIuHYpj4qjy/vOvAVOd6gQcdL9vgXtqgmM6B1ZJaJaCraS7rxZYTM//cX8M +RXRKDISQN0pOg4x5l6Q2F2282x69OKhcMVZ+iPi/T2VM4Kvc3enAsIV56tD80hEDqXn rO/J6NCzFtaqKw47EKekRFLSf21cG4QQxRAFEkyGWs1EKKuv21JtqsJvVTrQqzTxvm+T eX1A== X-Gm-Message-State: AA6/9RmnRlwb/EFsJyGVk8L+CZRMHxuq/cM7UsV8/RX7UqIsPjMO3ExjUnfjyV4LOTyNTQ== X-Received: by 10.25.35.6 with SMTP id j6mr9469918lfj.147.1476108225573; Mon, 10 Oct 2016 07:03:45 -0700 (PDT) Received: from [10.100.64.17] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id s63sm6515864lja.49.2016.10.10.07.03.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Oct 2016 07:03:44 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> Date: Mon, 10 Oct 2016 16:03:39 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161010133220.GU54003@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Uxu93BPJNKACrFgUljh7rp1RMd0auWIuX" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 14:03:54 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Uxu93BPJNKACrFgUljh7rp1RMd0auWIuX Content-Type: multipart/mixed; boundary="rAjpVOoPH2mHjtbjA6bhunC7dS7rgcUnF"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20160923200143.GG2840@zxy.spb.ru> <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> In-Reply-To: <20161010133220.GU54003@zxy.spb.ru> --rAjpVOoPH2mHjtbjA6bhunC7dS7rgcUnF Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/10/16 3:32 PM, Slawa Olhovchenkov wrote: > On Mon, Oct 10, 2016 at 01:26:12PM +0200, Julien Charbon wrote: >> On 10/6/16 1:10 PM, Slawa Olhovchenkov wrote: >>> On Thu, Oct 06, 2016 at 09:28:06AM +0200, Julien Charbon wrote: >>> >>>> 2. thread1: In tcp_close() the inp is marked with INP_DROPPED flag,= the >>>> process continues and calls INP_WUNLOCK() here: >>>> >>>> https://github.com/freebsd/freebsd/blob/releng/11.0/sys/netinet/tcp_= subr.c#L1568 >>> >>> Look also to sys/netinet/tcp_timewait.c:488 >>> >>> And check other locks from r160549 >> >> You are right, and here the a fix proposal for this issue: >> >> Fix a double-free when an inp transitions to INP_TIMEWAIT state after >> having been dropped >> https://reviews.freebsd.org/D8211 >> >> It basically enforces in_pcbdrop() logic in tcp_input(): A INP_DROPP= ED >> inpcb should never be proceed further. >> >> Slawa, as you are the only one to reproduce this issue currently, cou= ld >> test this patch? (And remove the temporary patch I did provided to yo= u >> before). >> >> I will wait for your tests results before pushing further. >> >> Thanks! >> >> diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c >> index c72f01f..37f27e0 100644 >> --- a/sys/netinet/tcp_input.c >> +++ b/sys/netinet/tcp_input.c >> @@ -921,6 +921,16 @@ findpcb: >> goto dropwithreset; >> } >> INP_WLOCK_ASSERT(inp); >> + /* >> + * While waiting for inp lock during the lookup, another threa= d >> + * can have droppedt the inpcb, in which case we need to loop= back >> + * and try to find a new inpcb to deliver to. >> + */ >> + if (inp->inp_flags & INP_DROPPED) { >> + INP_WUNLOCK(inp); >> + inp =3D NULL; >> + goto findpcb; >=20 > Are you sure about this goto? > Can this cause infinite loop by found same inpcb? > May be drop packet is more correct? Good question: Infinite loop is not possible here, as the next TCP hash lookup will return NULL or a fresh new and not dropped inp. You can check the current other usages of goto findpcb in tcp_input(). The rational here being: - Behavior before the patch: If the inp we found was deleted then goto findpcb. - Behavior after the patch: If the inp we found was deleted or dropped then goto findpcb. I just prefer having the same behavior applied everywhere: If tcp_input() loses the inp lock race and the inp was deleted or dropped then retry to find a new inpcb to deliver to. But you are right dropping the packet here will also fix the issue. Then the review process becomes quite helpful because people can argue: Dropping here is better because "blah", or goto findpcb is better because "bluh", etc. And at the review end you have a nice final patch. https://reviews.freebsd.org/D8211 -- Julien --rAjpVOoPH2mHjtbjA6bhunC7dS7rgcUnF-- --Uxu93BPJNKACrFgUljh7rp1RMd0auWIuX Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX+5+/AAoJEKVlQ5Je6dhx6lQH/Awtgic2tUHJdoFJkzB+DWng pMiInCMiaSkF978ngUgRXjltqLVfb1YBR0Odn7UvbY3W6scOyEEUqO0aIyVXS1mY FSoiQsBlJaHRmKth4RaUPXrBrktHgY2IzVSTNITlfZKSDg0pKjRJalNiQWjyAUr0 LmkmV58/x0rNAXKi/4ZLmmAjgjnMk5n4qVwIoXuA2H12KbE+ZbFu1WIB3FsOnr+i xlN07KtRxuN84obr0UhuanEsnFw2kITr8QiRe5j9yRN+qRMr80awv6Px1cpDsokP h4VsbW4ESmf5w1C3OqqETeiXpPlnF5JPnanw0iX1x/2jInD+fOmYRfFsHeoCmuU= =qFSj -----END PGP SIGNATURE----- --Uxu93BPJNKACrFgUljh7rp1RMd0auWIuX-- From owner-freebsd-stable@freebsd.org Mon Oct 10 14:27:50 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9705C0B17A for ; Mon, 10 Oct 2016 14:27:50 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-lf0-x233.google.com (mail-lf0-x233.google.com [IPv6:2a00:1450:4010:c07::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 67997FEA for ; Mon, 10 Oct 2016 14:27:50 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: by mail-lf0-x233.google.com with SMTP id l131so124893043lfl.2 for ; Mon, 10 Oct 2016 07:27:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=lNeQH9eh81xrjxNp903mxt3OWQUMKzMEuR7qtp2JMNU=; b=X19ioYFPnET+FMvu5T0XTd4vKu6jGPXOT7cB0tMQJbNR58GD1vqwz7UURTKKM4KTZQ QIKIeKe+nwlC5Dyq1IQsgSSFxbOh949hY4FptvAW+3tVFWkCL4OG29QNWwIFH/pVIdvo KEeKUKkX4OSVbii+zvJXusvLm1QmHNeUM2Qpg687H4J3KHftOdbEbZWCEDGhUnTxTPKq 8fzR8hnMYOZBS4ZqZbGJ1s0LWL59tinGT6R8o889RDDgKlc1ldDiuakqNUfzknOwWB83 yqpB5kIDz7ygvbcfXQZVE0BzRqe5kxNCsgtLvsi4xKpw3lXJMzWaabH7IGoi9LJ8W7kQ fclw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=lNeQH9eh81xrjxNp903mxt3OWQUMKzMEuR7qtp2JMNU=; b=MkvRE/FSfj3vEmlPRIb+OQW+Hja8XYOJurUiZhFQIfv22e9HlZmh2mJq0wz4ujbuIe XeygJb+EtABLv6m0k36hCHdoJacI/vLNnmIF/Hr3/IOjqPt8Cs39eviwGwv/+AHIvlGv YJ2R/qpPWaafAirTmPZ4C/n21EKhlhEnOX34s7SZz3kTo1QDI7PBZDxw+AGRio05JDO0 /DS6zMvxzKQF8U7yhxd5sylk3lUPDa9s0ZDW7y2hquZxDQ4TXz8AtT1VCFPmiQoqOH76 3py7JrMzf3+aMxNgVP1hGSD9s/5LpJw3clLExGDW80oaUW6lovDDXRmpTUGES9prONM+ GE/A== X-Gm-Message-State: AA6/9Rm/ZNCqadX312e1tISOUg59nXGxZlbwjGSpYmD6/As5OjkmpBZtYCgCqeh24VQ9Dw== X-Received: by 10.46.33.220 with SMTP id h89mr13850329lji.28.1476109666860; Mon, 10 Oct 2016 07:27:46 -0700 (PDT) Received: from Johans-MacBook-Air-2.local (92-111-79-242.static.chello.nl. [92.111.79.242]) by smtp.googlemail.com with ESMTPSA id h67sm6494255lji.29.2016.10.10.07.27.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Oct 2016 07:27:45 -0700 (PDT) Subject: Re: 11-current becoming 11-stable To: tech-lists References: <24b1b61b-5b4d-ae98-11e0-7939c11d5a61@zyxst.net> Cc: freebsd-stable@freebsd.org From: Johan Hendriks Message-ID: Date: Mon, 10 Oct 2016 16:27:43 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <24b1b61b-5b4d-ae98-11e0-7939c11d5a61@zyxst.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 14:27:51 -0000 Op 10/10/2016 om 12:48 schreef tech-lists: > Hi, > > I have a desktop that has run 11-current for at least the last year. I > periodically rebuild kernel and world to keep on top of things. As > it's a desktop, there are lots of ports installed. These are installed > via the older cd /port && make install clean way, rather than poudiere. > > Basically I'm asking that, when 11-stable becomes available (as far as > I can see, it's still at PRERELEASE), is it worthwhile/advisable to > recompile all my ports after doing the make world & make delete-old > dance? > > thanks, If you stay at FreeBSD 11 then there is no need to rebuild your ports. Only if you decide to go to FreeBSD 12 aka CURRENT, then you need to reinstall the ports! regards Johan From owner-freebsd-stable@freebsd.org Mon Oct 10 14:29:44 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56009C0B253 for ; Mon, 10 Oct 2016 14:29:44 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1392B1BB; Mon, 10 Oct 2016 14:29:44 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1btbaH-000C3n-Ro; Mon, 10 Oct 2016 17:29:41 +0300 Date: Mon, 10 Oct 2016 17:29:41 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161010142941.GV54003@zxy.spb.ru> References: <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 14:29:44 -0000 On Mon, Oct 10, 2016 at 04:03:39PM +0200, Julien Charbon wrote: > > Hi Slawa, > > On 10/10/16 3:32 PM, Slawa Olhovchenkov wrote: > > On Mon, Oct 10, 2016 at 01:26:12PM +0200, Julien Charbon wrote: > >> On 10/6/16 1:10 PM, Slawa Olhovchenkov wrote: > >>> On Thu, Oct 06, 2016 at 09:28:06AM +0200, Julien Charbon wrote: > >>> > >>>> 2. thread1: In tcp_close() the inp is marked with INP_DROPPED flag, the > >>>> process continues and calls INP_WUNLOCK() here: > >>>> > >>>> https://github.com/freebsd/freebsd/blob/releng/11.0/sys/netinet/tcp_subr.c#L1568 > >>> > >>> Look also to sys/netinet/tcp_timewait.c:488 > >>> > >>> And check other locks from r160549 > >> > >> You are right, and here the a fix proposal for this issue: > >> > >> Fix a double-free when an inp transitions to INP_TIMEWAIT state after > >> having been dropped > >> https://reviews.freebsd.org/D8211 > >> > >> It basically enforces in_pcbdrop() logic in tcp_input(): A INP_DROPPED > >> inpcb should never be proceed further. > >> > >> Slawa, as you are the only one to reproduce this issue currently, could > >> test this patch? (And remove the temporary patch I did provided to you > >> before). > >> > >> I will wait for your tests results before pushing further. > >> > >> Thanks! > >> > >> diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c > >> index c72f01f..37f27e0 100644 > >> --- a/sys/netinet/tcp_input.c > >> +++ b/sys/netinet/tcp_input.c > >> @@ -921,6 +921,16 @@ findpcb: > >> goto dropwithreset; > >> } > >> INP_WLOCK_ASSERT(inp); > >> + /* > >> + * While waiting for inp lock during the lookup, another thread > >> + * can have droppedt the inpcb, in which case we need to loop back > >> + * and try to find a new inpcb to deliver to. > >> + */ > >> + if (inp->inp_flags & INP_DROPPED) { > >> + INP_WUNLOCK(inp); > >> + inp = NULL; > >> + goto findpcb; > > > > Are you sure about this goto? > > Can this cause infinite loop by found same inpcb? > > May be drop packet is more correct? > > Good question: Infinite loop is not possible here, as the next TCP > hash lookup will return NULL or a fresh new and not dropped inp. You I am not expert in this api and don't see cause of this: I am assume hash lookup don't remove from hash returned args and I am don't see any removing of this inp. Why hash lookup don't return same inp? (assume this input patch interrupt callout code on the same CPU core). > can check the current other usages of goto findpcb in tcp_input(). The > rational here being: > > - Behavior before the patch: If the inp we found was deleted then goto > findpcb. > - Behavior after the patch: If the inp we found was deleted or dropped > then goto findpcb. > > I just prefer having the same behavior applied everywhere: If > tcp_input() loses the inp lock race and the inp was deleted or dropped > then retry to find a new inpcb to deliver to. > > But you are right dropping the packet here will also fix the issue. > > Then the review process becomes quite helpful because people can argue: > Dropping here is better because "blah", or goto findpcb is better > because "bluh", etc. And at the review end you have a nice final patch. > > https://reviews.freebsd.org/D8211 I am not sure, I am see to sys/netinet/in_pcb.h:#define INP_DROPPED 0x04000000 /* protocol drop flag */ and think this is a flag 'all packets must be droped' From owner-freebsd-stable@freebsd.org Mon Oct 10 14:49:09 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3D3FC0BA5D for ; Mon, 10 Oct 2016 14:49:09 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-ua0-x229.google.com (mail-ua0-x229.google.com [IPv6:2607:f8b0:400c:c08::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 50FE0CA9 for ; Mon, 10 Oct 2016 14:49:09 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-ua0-x229.google.com with SMTP id u68so86273695uau.2 for ; Mon, 10 Oct 2016 07:49:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=W73L/HNmJkNl07iJ6VgRSmT+H95OYrcGLvmEJR7Z4h8=; b=wNSGQxohAodx2gUSmuEp5yNbsv/Yx5emqIG0/bSPIHjvpc3kRx+wI28bY1nWgyzEnV aju1PGbfSGtt4IR3/QhQ9KBmpTZv6L6h8+/IeuBBi20SOhreH64R0O1s9gYEmkAjgdpm TJAvu7zo6Taruh+8Kx6eB9wyFAnGXJSKGEx93r/XjX1Q6OY4gPYbPCTbTg8YMsD7QN3L SR55zyTp2yReqiSm3Bb/dtdA06+8MPmdAmkvC5uadftsifCJhpn5gKgUyM4UaSuidDb0 RTJWUeSRj8MZ6wl1B2bxD6aD9utruSgttI8c1ayHb2/APDh1iqaoGPAqPdkoHggkS3UX 7qiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=W73L/HNmJkNl07iJ6VgRSmT+H95OYrcGLvmEJR7Z4h8=; b=hyfsOl9DkIqAcKWElaM9hxmqw/aA2F8Kqb2crhzCgXd9T0iA6tPmPXOCq+v8wQw184 +ai1gmp/1Z3lfR5rFqzYeS6O7aCmnpi8ItcVtVdVQSTq3xL9OWSQQrfJpO2M9nMY4fB5 VDAn71O/C2sVK7dB9FxSnsNzl4wF2DNF2wgjd6eMsyaiBKPqIs1Z2eQgbd3A5H8Fk8RB uNfUXvpGfSqxEo922geWE6sV9QrxyhK0cVc5qhGUAMfwjBNjYRKCID18GtL3YZpK/u9i 0alaNOCQUHYt/KCWtwZTG/OmtC4RtPRxR4o8EIqbn/B/l1/BrABsL2ypmvJtic1vUP1t lVrw== X-Gm-Message-State: AA6/9Rkmw0eNE0rzH6qJz5F/pUlzN3kQsFqST6fhVsf9ygjxZhFrijoxgwWIKINrwoVthhy+0tIGMpHH3y7nhg== X-Received: by 10.159.39.70 with SMTP id a64mr22073832uaa.145.1476110948139; Mon, 10 Oct 2016 07:49:08 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.118.78 with HTTP; Mon, 10 Oct 2016 07:49:07 -0700 (PDT) In-Reply-To: <24b1b61b-5b4d-ae98-11e0-7939c11d5a61@zyxst.net> References: <24b1b61b-5b4d-ae98-11e0-7939c11d5a61@zyxst.net> From: Kevin Oberman Date: Mon, 10 Oct 2016 07:49:07 -0700 X-Google-Sender-Auth: C8PjyD99FhNtZcZCI34Q1S6zCtU Message-ID: Subject: Re: 11-current becoming 11-stable To: tech-lists Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 14:49:09 -0000 On Mon, Oct 10, 2016 at 3:48 AM, tech-lists wrote: > Hi, > > I have a desktop that has run 11-current for at least the last year. I > periodically rebuild kernel and world to keep on top of things. As it's a > desktop, there are lots of ports installed. These are installed via the > older cd /port && make install clean way, rather than poudiere. > > Basically I'm asking that, when 11-stable becomes available (as far as I > can see, it's still at PRERELEASE), is it worthwhile/advisable to recompile > all my ports after doing the make world & make delete-old dance? > > thanks, > -- > J. > 11.0-RELEASE should be showing up any day. It is on Release Candidate 3. First, there is nothing outdated about building ports from source. The only time using poudiere makes real sense is when you need to build a "standard" set of ports for redistribution to other systems. If you are building for a single system, it really is just an added complication. The real choice is between building from ports or installing packages. If you need to build some ports with non-standard options, then using packages may not work for you. Mixing ports and packages can greatly complicate things and may result in unexpected issues, though it can work if you are careful. For my server I use all packages EXCEPT postfix. Since nothing depends on it, I am comfortable in building it from source whenever it is updated. That saves me a LOT of time. As to the need to update, most ports don't require rebuilding after the move to 11. FreeBSD libraries use symbol versioning so that ABI should never be broken when new versions change the API. The exception would be contributed code that is does not have symbol versioning. The most significant of these is probably OpenSSL. It is likely that ports that use the base system library will fail to run on 11. If they use the port version of OpenSSL or LibreSSL, this is not an issue. I strongly recommend that you install security/openssl and never link ports with the base system libraries. If you have done this, it is likely that any ports will need a rebuild. There are several other contributed libraries that are not commonly used by ports. These may cause some issues, but in general you can move from 10 to 11 and not re-build ports. You certainly won't need to re-build many. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Mon Oct 10 15:58:48 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3DEAC0C59A for ; Mon, 10 Oct 2016 15:58:48 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f65.google.com (mail-lf0-f65.google.com [209.85.215.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63CFA7B7 for ; Mon, 10 Oct 2016 15:58:48 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f65.google.com with SMTP id x79so7422228lff.2 for ; Mon, 10 Oct 2016 08:58:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=S4dlF0IFThTa2mSq15Q2F4KeQhJUQ3rz+pXgctGlAgc=; b=bZAYWFu2hcxfsWtAd5+cwnb3JiPCDAlB+bctvkg8m2ui3e+SI/En3BpiVN0PCtVi3K oGt5PbNHCb2kggSbybuAuZV+A0b5eY4xJpYvTF3gnpGWagWcbE0WfHweRJ2E86FI98XH C3K270ahq6roqCOJmODklCOvTbZgHqsbxCkRt+xnQvRskF7fI9J5oM/Li1WQcVLH/4jy sfLtuuBy71TX0ddoMtojvYtNtzbmnewVcIIHYiuvJRHCdZVaVe5d9TSEmiEz1XnpaD7X 6oE7CoAHd0+9vXGguGqIU8K/cSmS5DsVKldJgxbxViCYit+i6V8ycftnb+aH/+pYP7BN Gnkw== X-Gm-Message-State: AA6/9RliYBXrlO5T8Aowl5JfNDnt5Jrx27YXrU+RDkJO2PyG47Z1tJRBssAyWLtALYeePQ== X-Received: by 10.25.32.69 with SMTP id g66mr12973309lfg.15.1476114267376; Mon, 10 Oct 2016 08:44:27 -0700 (PDT) Received: from [10.100.64.17] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id 94sm6646687lja.10.2016.10.10.08.44.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Oct 2016 08:44:26 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> Date: Mon, 10 Oct 2016 17:44:21 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161010142941.GV54003@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="jgP2QijCNPFwwniMmTVgoP1jMrJMrPvsB" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 15:58:48 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --jgP2QijCNPFwwniMmTVgoP1jMrJMrPvsB Content-Type: multipart/mixed; boundary="Sp9pwUnhtrmlVgl2LKFvD2WtVtJmggQHQ"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20160925124626.GI2840@zxy.spb.ru> <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> In-Reply-To: <20161010142941.GV54003@zxy.spb.ru> --Sp9pwUnhtrmlVgl2LKFvD2WtVtJmggQHQ Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, On 10/10/16 4:29 PM, Slawa Olhovchenkov wrote: > On Mon, Oct 10, 2016 at 04:03:39PM +0200, Julien Charbon wrote: >> On 10/10/16 3:32 PM, Slawa Olhovchenkov wrote: >>> On Mon, Oct 10, 2016 at 01:26:12PM +0200, Julien Charbon wrote: >>>> On 10/6/16 1:10 PM, Slawa Olhovchenkov wrote: >>>>> On Thu, Oct 06, 2016 at 09:28:06AM +0200, Julien Charbon wrote: >>>>> >>>>>> 2. thread1: In tcp_close() the inp is marked with INP_DROPPED fla= g, the >>>>>> process continues and calls INP_WUNLOCK() here: >>>>>> >>>>>> https://github.com/freebsd/freebsd/blob/releng/11.0/sys/netinet/tc= p_subr.c#L1568 >>>>> >>>>> Look also to sys/netinet/tcp_timewait.c:488 >>>>> >>>>> And check other locks from r160549 >>>> >>>> You are right, and here the a fix proposal for this issue: >>>> >>>> Fix a double-free when an inp transitions to INP_TIMEWAIT state afte= r >>>> having been dropped >>>> https://reviews.freebsd.org/D8211 >>>> >>>> It basically enforces in_pcbdrop() logic in tcp_input(): A INP_DRO= PPED >>>> inpcb should never be proceed further. >>>> >>>> Slawa, as you are the only one to reproduce this issue currently, c= ould >>>> test this patch? (And remove the temporary patch I did provided to = you >>>> before). >>>> >>>> I will wait for your tests results before pushing further. >>>> >>>> Thanks! >>>> >>>> diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c >>>> index c72f01f..37f27e0 100644 >>>> --- a/sys/netinet/tcp_input.c >>>> +++ b/sys/netinet/tcp_input.c >>>> @@ -921,6 +921,16 @@ findpcb: >>>> goto dropwithreset; >>>> } >>>> INP_WLOCK_ASSERT(inp); >>>> + /* >>>> + * While waiting for inp lock during the lookup, another thr= ead >>>> + * can have droppedt the inpcb, in which case we need to lo= op back >>>> + * and try to find a new inpcb to deliver to. >>>> + */ >>>> + if (inp->inp_flags & INP_DROPPED) { >>>> + INP_WUNLOCK(inp); >>>> + inp =3D NULL; >>>> + goto findpcb; >>> >>> Are you sure about this goto? >>> Can this cause infinite loop by found same inpcb? >>> May be drop packet is more correct? >> >> Good question: Infinite loop is not possible here, as the next TCP >> hash lookup will return NULL or a fresh new and not dropped inp. You >=20 > I am not expert in this api and don't see cause of this: I am assume > hash lookup don't remove from hash returned args and I am don't see > any removing of this inp. Why hash lookup don't return same inp? >=20 > (assume this input patch interrupt callout code on the same CPU core). >=20 >> can check the current other usages of goto findpcb in tcp_input(). Th= e >> rational here being: >> >> - Behavior before the patch: If the inp we found was deleted then go= to >> findpcb. >> - Behavior after the patch: If the inp we found was deleted or dropp= ed >> then goto findpcb. >> >> I just prefer having the same behavior applied everywhere: If >> tcp_input() loses the inp lock race and the inp was deleted or dropped= >> then retry to find a new inpcb to deliver to. >> >> But you are right dropping the packet here will also fix the issue. >> >> Then the review process becomes quite helpful because people can argu= e: >> Dropping here is better because "blah", or goto findpcb is better >> because "bluh", etc. And at the review end you have a nice final patc= h. >> >> https://reviews.freebsd.org/D8211 >=20 > I am not sure, I am see to >=20 > sys/netinet/in_pcb.h:#define INP_DROPPED 0x04000000 /* p= rotocol drop flag */ >=20 > and think this is a flag 'all packets must be droped' On 10/10/16 4:29 PM, Slawa Olhovchenkov wrote: > On Mon, Oct 10, 2016 at 04:03:39PM +0200, Julien Charbon wrote: >> On 10/10/16 3:32 PM, Slawa Olhovchenkov wrote: >>> On Mon, Oct 10, 2016 at 01:26:12PM +0200, Julien Charbon wrote: >>>> On 10/6/16 1:10 PM, Slawa Olhovchenkov wrote: >>>>> On Thu, Oct 06, 2016 at 09:28:06AM +0200, Julien Charbon wrote: >>>>> >>>>>> 2. thread1: In tcp_close() the inp is marked with INP_DROPPED flag, the >>>>>> process continues and calls INP_WUNLOCK() here: >>>>>> >>>>>> https://github.com/freebsd/freebsd/blob/releng/11.0/sys/netinet/tcp_subr.= c#L1568 >>>>> >>>>> Look also to sys/netinet/tcp_timewait.c:488 >>>>> >>>>> And check other locks from r160549 >>>> >>>> You are right, and here the a fix proposal for this issue: >>>> >>>> Fix a double-free when an inp transitions to INP_TIMEWAIT state afte= r >>>> having been dropped >>>> https://reviews.freebsd.org/D8211 >>>> >>>> It basically enforces in_pcbdrop() logic in tcp_input(): A INP_DROPPED >>>> inpcb should never be proceed further. >>>> >>>> Slawa, as you are the only one to reproduce this issue currently, could >>>> test this patch? (And remove the temporary patch I did provided to = you >>>> before). >>>> >>>> I will wait for your tests results before pushing further. >>>> >>>> Thanks! >>>> >>>> diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c >>>> index c72f01f..37f27e0 100644 >>>> --- a/sys/netinet/tcp_input.c >>>> +++ b/sys/netinet/tcp_input.c >>>> @@ -921,6 +921,16 @@ findpcb: >>>> goto dropwithreset; >>>> } >>>> INP_WLOCK_ASSERT(inp); >>>> + /* >>>> + * While waiting for inp lock during the lookup, another thr= ead >>>> + * can have droppedt the inpcb, in which case we need to loop back >>>> + * and try to find a new inpcb to deliver to. >>>> + */ >>>> + if (inp->inp_flags & INP_DROPPED) { >>>> + INP_WUNLOCK(inp); >>>> + inp =3D NULL; >>>> + goto findpcb; >>> >>> Are you sure about this goto? >>> Can this cause infinite loop by found same inpcb? >>> May be drop packet is more correct? >> >> Good question: Infinite loop is not possible here, as the next TCP >> hash lookup will return NULL or a fresh new and not dropped inp. You > > I am not expert in this api and don't see cause of this: I am assume > hash lookup don't remove from hash returned args and I am don't see > any removing of this inp. Why hash lookup don't return same inp? > > (assume this input patch interrupt callout code on the same CPU core). > >> can check the current other usages of goto findpcb in tcp_input(). Th= e >> rational here being: >> >> - Behavior before the patch: If the inp we found was deleted then go= to >> findpcb. >> - Behavior after the patch: If the inp we found was deleted or dropp= ed >> then goto findpcb. >> >> I just prefer having the same behavior applied everywhere: If >> tcp_input() loses the inp lock race and the inp was deleted or dropped= >> then retry to find a new inpcb to deliver to. >> >> But you are right dropping the packet here will also fix the issue. >> >> Then the review process becomes quite helpful because people can argu= e: >> Dropping here is better because "blah", or goto findpcb is better >> because "bluh", etc. And at the review end you have a nice final patc= h. >> >> https://reviews.freebsd.org/D8211 > > I am not sure, I am see to > > sys/netinet/in_pcb.h:#define INP_DROPPED 0x04000000 /* protocol drop flag */ > > and think this is a flag 'all packets must be droped' Hm, I believe this flag means "this inp has been dropped by the TCP stack, so don't use it anymore". Actually this flag is better described in the function that sets it: "(INP_DROPPED) is used by TCP to mark an inpcb as unused and avoid future packet delivery or event notification when a socket remains open but TCP has closed." https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_pcb= =2Ec#L1320 /* * in_pcbdrop() removes an inpcb from hashed lists, releasing its address and * port reservation, and preventing it from being returned by inpcb looku= ps. * * It is used by TCP to mark an inpcb as unused and avoid future packet * delivery or event notification when a socket remains open but TCP has * closed. This might occur as a result of a shutdown()-initiated TCP cl= ose * or a RST on the wire, and allows the port binding to be reused while still * maintaining the invariant that so_pcb always points to a valid inpcb until * in_pcbdetach(). * */ void in_pcbdrop(struct inpcb *inp) { inp->inp_flags |=3D INP_DROPPED; ... The classical example where "goto findpcb" is useful: You receive a new connection request with a TCP SYN packet and this packet is unlucky and reached a inp being dropped: - with "goto findpcb" approach, the next lookup will most likely find the LISTEN inp and start the TCP hand-shake as usual - with "drop the packet" approach, the TCP client will need to re-transmit a TCP SYN packet It is not because a packet was unlucky once that it deserves to be dropped. :) -- Julien --Sp9pwUnhtrmlVgl2LKFvD2WtVtJmggQHQ-- --jgP2QijCNPFwwniMmTVgoP1jMrJMrPvsB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX+7dZAAoJEKVlQ5Je6dhxLFgH/RZKNAZlyImT1Pcw5YGevSTZ LHAtq7x84dKDQrUWZcE5K8GYvXrpOm3uEjnWMfbc6BfPz7T7emBHC3Y4GgIJ4X29 d6khxTPsgvBFTetRwDkiet5Gk8OrI7t5W3NcXvLpFcAJkBVBQ9lXP5RKqhfWxhJE 3KejwpOAyDLVLMTaN08omHmS4J72pckewe+Ud8/rRm+G/H1xuIDuRbiQGrBMVf8R HW8e7mwotOx3sJ9JIBBDFYsQ5CDUVPUgfLcN3/U4vWtcIaxuUY8AbY1s/aIh7ltW RZfJCbcWrXmZcbrp+Yw2uq7010IpPkpHJi/LaudwPJzg2izXUU9tThDhqJU+F5g= =paCp -----END PGP SIGNATURE----- --jgP2QijCNPFwwniMmTVgoP1jMrJMrPvsB-- From owner-freebsd-stable@freebsd.org Mon Oct 10 17:35:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6504CC0C7A9 for ; Mon, 10 Oct 2016 17:35:35 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2200E225; Mon, 10 Oct 2016 17:35:35 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bteU7-000GwC-Re; Mon, 10 Oct 2016 20:35:31 +0300 Date: Mon, 10 Oct 2016 20:35:31 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161010173531.GI6177@zxy.spb.ru> References: <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2016 17:35:35 -0000 On Mon, Oct 10, 2016 at 05:44:21PM +0200, Julien Charbon wrote: > >> can check the current other usages of goto findpcb in tcp_input(). The > >> rational here being: > >> > >> - Behavior before the patch: If the inp we found was deleted then goto > >> findpcb. > >> - Behavior after the patch: If the inp we found was deleted or dropped > >> then goto findpcb. > >> > >> I just prefer having the same behavior applied everywhere: If > >> tcp_input() loses the inp lock race and the inp was deleted or dropped > >> then retry to find a new inpcb to deliver to. > >> > >> But you are right dropping the packet here will also fix the issue. > >> > >> Then the review process becomes quite helpful because people can argue: > >> Dropping here is better because "blah", or goto findpcb is better > >> because "bluh", etc. And at the review end you have a nice final patch. > >> > >> https://reviews.freebsd.org/D8211 > > > > I am not sure, I am see to > > > > sys/netinet/in_pcb.h:#define INP_DROPPED 0x04000000 /* > protocol drop flag */ > > > > and think this is a flag 'all packets must be droped' > > Hm, I believe this flag means "this inp has been dropped by the TCP > stack, so don't use it anymore". Actually this flag is better described > in the function that sets it: > > "(INP_DROPPED) is used by TCP to mark an inpcb as unused and avoid > future packet delivery or event notification when a socket remains open > but TCP has closed." > > https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_pcb.c#L1320 > > /* > * in_pcbdrop() removes an inpcb from hashed lists, releasing its > address and > * port reservation, and preventing it from being returned by inpcb lookups. > * > * It is used by TCP to mark an inpcb as unused and avoid future packet > * delivery or event notification when a socket remains open but TCP has > * closed. This might occur as a result of a shutdown()-initiated TCP close > * or a RST on the wire, and allows the port binding to be reused while > still > * maintaining the invariant that so_pcb always points to a valid inpcb > until > * in_pcbdetach(). > * > */ > void > in_pcbdrop(struct inpcb *inp) > { > inp->inp_flags |= INP_DROPPED; > ... > > The classical example where "goto findpcb" is useful: You receive a > new connection request with a TCP SYN packet and this packet is unlucky > and reached a inp being dropped: > > - with "goto findpcb" approach, the next lookup will most likely find > the LISTEN inp and start the TCP hand-shake as usual > - with "drop the packet" approach, the TCP client will need to > re-transmit a TCP SYN packet > > It is not because a packet was unlucky once that it deserves to be > dropped. :) Thanks for explaining, very helpfull. In this situation (TCP SYN with same 4-tuple as existing socket) allocate new PCB is best. But for this we must destroy current PCB. I am think INP_WUNLOCK(inp) don't destroy it and in_pcblookup_mbuf find it again (I am think in_pcblookup_mbuf find this PCB on first turn). I am assume for classical example in_pcbrele_wlocked(inp) free and destroy current PCB for possibility in_pcblookup_mbuf allocate new one. From owner-freebsd-stable@freebsd.org Tue Oct 11 07:20:30 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2251C0D841 for ; Tue, 11 Oct 2016 07:20:30 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com [209.85.215.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 66A87CD7 for ; Tue, 11 Oct 2016 07:20:29 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f51.google.com with SMTP id l131so26096676lfl.2 for ; Tue, 11 Oct 2016 00:20:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=LsjeWxOmrNaZOmpZND6GDYNQ/mJkAtvf7yii/iqaPyc=; b=Q36srr1P8RjyJMeAxtK1JbMvOWzXnacNtbj3o2rHpUi8ITrhOQUCsGmecqAfZimSuy QBWqBlUBDt9gpur6SsiWwFmnlGBrUfsH5jut4ZvVoovcu5ORcrccMNaSIdsKlUSzq5jd 67MJ810CVh866iyL9In1MUs0h2MLV5m2wmuYp69h5WqztsZqiKvX67s2fD9A5ZDwsb93 cPTyYlW+A2fUNWV8X0sK9gffIRBvPvwS9oQ2mkYayrH8jpM+NTlTLWf+uL/hgrCTZX4p 4cyForkiDQ6xUXINlDGPzZELAaHCGNA5VbUmRk8V94b8j7tXequZB88nx9qs+588pshU yOmA== X-Gm-Message-State: AA6/9Rk4cWTxqEYoTFQ+3+krDd8Z1zFElYsoWVa/3B8o/oUUaOi/NhylU0UcoVhnEr0JIA== X-Received: by 10.25.163.17 with SMTP id m17mr1625073lfe.161.1476170425560; Tue, 11 Oct 2016 00:20:25 -0700 (PDT) Received: from [10.100.64.17] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id h68sm353068lji.20.2016.10.11.00.20.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Oct 2016 00:20:24 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> Date: Tue, 11 Oct 2016 09:20:17 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161010173531.GI6177@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="oq9rDkJ6isRwA1ckJwhFJjfq55qKQCU3W" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Oct 2016 07:20:31 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --oq9rDkJ6isRwA1ckJwhFJjfq55qKQCU3W Content-Type: multipart/mixed; boundary="6bRp3pkLFdj50Nvhlaf2QLjbLWh7J9Aq4"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20160926172159.GA54003@zxy.spb.ru> <62453d9c-b1e4-1129-70ff-654dacea37f9@gmail.com> <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> In-Reply-To: <20161010173531.GI6177@zxy.spb.ru> --6bRp3pkLFdj50Nvhlaf2QLjbLWh7J9Aq4 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/10/16 7:35 PM, Slawa Olhovchenkov wrote: > On Mon, Oct 10, 2016 at 05:44:21PM +0200, Julien Charbon wrote: >>>> can check the current other usages of goto findpcb in tcp_input(). = The >>>> rational here being: >>>> >>>> - Behavior before the patch: If the inp we found was deleted then = goto >>>> findpcb. >>>> - Behavior after the patch: If the inp we found was deleted or dro= pped >>>> then goto findpcb. >>>> >>>> I just prefer having the same behavior applied everywhere: If >>>> tcp_input() loses the inp lock race and the inp was deleted or dropp= ed >>>> then retry to find a new inpcb to deliver to. >>>> >>>> But you are right dropping the packet here will also fix the issue.= >>>> >>>> Then the review process becomes quite helpful because people can ar= gue: >>>> Dropping here is better because "blah", or goto findpcb is better >>>> because "bluh", etc. And at the review end you have a nice final pa= tch. >>>> >>>> https://reviews.freebsd.org/D8211 >>> >>> I am not sure, I am see to >>> >>> sys/netinet/in_pcb.h:#define INP_DROPPED 0x04000000 /*= >> protocol drop flag */ >>> >>> and think this is a flag 'all packets must be droped' >> >> Hm, I believe this flag means "this inp has been dropped by the TCP >> stack, so don't use it anymore". Actually this flag is better describ= ed >> in the function that sets it: >> >> "(INP_DROPPED) is used by TCP to mark an inpcb as unused and avoid >> future packet delivery or event notification when a socket remains ope= n >> but TCP has closed." >> >> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_= pcb.c#L1320 >> >> /* >> * in_pcbdrop() removes an inpcb from hashed lists, releasing its >> address and >> * port reservation, and preventing it from being returned by inpcb lo= okups. >> * >> * It is used by TCP to mark an inpcb as unused and avoid future packe= t >> * delivery or event notification when a socket remains open but TCP h= as >> * closed. This might occur as a result of a shutdown()-initiated TCP= close >> * or a RST on the wire, and allows the port binding to be reused whil= e >> still >> * maintaining the invariant that so_pcb always points to a valid inpc= b >> until >> * in_pcbdetach(). >> * >> */ >> void >> in_pcbdrop(struct inpcb *inp) >> { >> inp->inp_flags |=3D INP_DROPPED; >> ... >> >> The classical example where "goto findpcb" is useful: You receive a >> new connection request with a TCP SYN packet and this packet is unluck= y >> and reached a inp being dropped: >> >> - with "goto findpcb" approach, the next lookup will most likely find= >> the LISTEN inp and start the TCP hand-shake as usual >> - with "drop the packet" approach, the TCP client will need to >> re-transmit a TCP SYN packet >> >> It is not because a packet was unlucky once that it deserves to be >> dropped. :) >=20 > Thanks for explaining, very helpfull. > In this situation (TCP SYN with same 4-tuple as existing socket) > allocate new PCB is best. But for this we must destroy current PCB. I > am think INP_WUNLOCK(inp) don't destroy it and in_pcblookup_mbuf find > it again (I am think in_pcblookup_mbuf find this PCB on first turn). > I am assume for classical example in_pcbrele_wlocked(inp) free and > destroy current PCB for possibility in_pcblookup_mbuf allocate new > one. Astute question: Here, the same inp cannot be find again by in_pcblookup_mbuf(), the explanation is a bit long though: in_pcbdrop() does two things under INP_WLOCK lock: https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_pcb= =2Ec#L1334 1. Add INP_DROPPED flag 2. Remove the inp from the TCP hash table And once removed from the TCP hash table, in_pcblookup_mbuf() will return NULL when doing the same TCP hash table lookup again. It means that under a INP_WLOCK lock, these two changes are atomic: - Either an inp does not have INP_DROPPED flag and can be in TCP hash ta= ble - Either an inp has INP_DROPPED flag and is not in TCP hash table But when you don't have the INP_WLOCK lock, then you can witness the intermediate state where a inp is still in TCP hash table while a thread is adding the INP_DROPPED flag. Nothing unusual here. Then threads are competing for the INP_WLOCK lock. For the example, let's say the thread A wants to run tcp_input()/in_pcblookup_mbuf() and racing for this INP_WLOCK: https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_pcb= =2Ec#L1964 And thread B wants to run tcp_timer_2msl()/tcp_close()/in_pcbdrop() and racing for this INP_WLOCK: https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/tcp_ti= mer.c#L323 That leads to two cases: o Thread A wins the race: Thread A will continue tcp_input() as usal and INP_DROPPED flags is not set and inp is still in TCP hash table. Thread B is waiting on thread A to release INP_WLOCK after finishing tcp_input() processing, and thread B will continue tcp_timer_2msl()/tcp_close()/in_pcbdrop() processing. o Thread B wins the race: Thread B runs tcp_timer_2msl()/tcp_close()/in_pcbdrop() and inp INP_DROPPED is set and inp being removed from TCP hash table. In parallel, thread A has found the inp in TCP hash before is was removed, and waiting on the found inp INP_WLOCK lock. Once thread B has released the INP_WLOCK lock, thread A gets this lock and sees the INP_DROPPED flag and do "goto findpcb" but here because the inp is not more in TCP hash table and it will not be find again by in_pcblookup_mbuf(). Hopefully I am clear enough here. -- Julien --6bRp3pkLFdj50Nvhlaf2QLjbLWh7J9Aq4-- --oq9rDkJ6isRwA1ckJwhFJjfq55qKQCU3W Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/JK2AAoJEKVlQ5Je6dhxNxkH/RhKPtBgDzP+NpnwwumbRhEr PrbYsnFOKL60VpPZfbvY4vz7TNclgmaaS2VIRcuJphuJr2k6MfkETyFd7QSBqSQV vg4tAvNVXbNqG+Ubvrc2KvY/QeK+FbJc1aYlzZ9xS5trHLbf0uRjoIg+KsC+gyBA VPLizfIGdN+8tHFs0FKAuirfPgiqNBBgUDaTCqh0l8DOsRp3/QLPs05sok6UJAtF ooaba6b4WyFe1DCQawMvJM4JDZIrGMy3qjzZYQppM8FxYbbl3yCAqcTDV0wgksXF TViCJYuRb+xYbQNdvnOSPZUBGbjsZYa8UXq4fmqva9urD7UllFpiDzKJ8uPJBIw= =5V+B -----END PGP SIGNATURE----- --oq9rDkJ6isRwA1ckJwhFJjfq55qKQCU3W-- From owner-freebsd-stable@freebsd.org Tue Oct 11 09:11:20 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FA72C0CED1 for ; Tue, 11 Oct 2016 09:11:20 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (unknown [IPv6:2001:6a0:10f:1::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "plan-b.pwste.edu.pl", Issuer "plan-b.pwste.edu.pl" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 92638D52 for ; Tue, 11 Oct 2016 09:11:18 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (zarychtam@localhost [127.0.0.1]) by plan-b.pwste.edu.pl (8.15.2/8.15.2) with ESMTPS id u9B9BDrp041676 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 11 Oct 2016 11:11:13 +0200 (CEST) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: (from zarychtam@localhost) by plan-b.pwste.edu.pl (8.15.2/8.15.2/Submit) id u9B9BDr3041675; Tue, 11 Oct 2016 11:11:13 +0200 (CEST) (envelope-from zarychtam) Date: Tue, 11 Oct 2016 11:11:13 +0200 From: Marek Zarychta To: Nathan Lay Cc: freebsd-stable@FreeBSD.org Subject: Re: possible regression on i386 Message-ID: <20161011091113.GA41419@plan-b.pwste.edu.pl> References: <20161009205758.GB8329@plan-b.pwste.edu.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="6c2NcOVqGQ03X4Wi" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Oct 2016 09:11:20 -0000 --6c2NcOVqGQ03X4Wi Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Oct 09, 2016 at 08:11:55PM -0400, Nathan Lay wrote: > If your problem is anything like mine was, buildworld is trying to link > with /usr/lib/librt.so rather than the new one built during the buildworld > build. As per a recent commit, the new librt will have the additional > mq_getfd_np() symbol while the original /usr/lib/librt.so will not. That > causes those unresolved reference errors for new code trying to use the > mq_getfd_np() function. >=20 > Try building and installing librt manually: > cd /usr/src/lib/librt > make && make install >=20 > Then try buildworld again. >=20 Thank you for reply and advice. It was, of course my fault,=20 /etc/src.conf file was left unchanged since upgrade from 9.3 and=20 setting "CC=3Dclang" was causing compilation errors. The first impression is very important: FreeBSD 11 runs pretty smoothly=20 on this old machine being also more lightweight than 9.3. --=20 Marek Zarychta --6c2NcOVqGQ03X4Wi Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAABCAAGBQJX/KyvAAoJEHWf7P/9Uo0sg18IAJ620DO3pmFzWLBhdepvDB58 aSk94MUwZ8K8wVfYMd0WRqdqRd4/p2+orPhNfr8/j8viyomqq1rpzt8RIxFLassG TOf/LVG8VcJU5LwFt8rq3hrpwjPNUleOJvA6JK1CRa3GMuYFBHgqetUDPRwreJLS TS/94XD2MQxwUEA9ihKxFVGbJiR6P3yyIpKsWtSwSPNq5cVjLiQCJkJjIMGvRkXm qgyIk5eTKTCNkBtXI28r/KOhLO+WnjusepwiQsH62kiczaYSRyYwJnhUAXi+GNra uy1feQ4K6wqaHPsZcCUVS8r06k0rgO8Z3b9ImzOWCnNelC1+fVthXYOw+2XXt3k= =M3PA -----END PGP SIGNATURE----- --6c2NcOVqGQ03X4Wi-- From owner-freebsd-stable@freebsd.org Tue Oct 11 11:10:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D780DC0DE61 for ; Tue, 11 Oct 2016 11:10:47 +0000 (UTC) (envelope-from oliver.pinter@balabit.com) Received: from mail-qk0-x22e.google.com (mail-qk0-x22e.google.com [IPv6:2607:f8b0:400d:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FA11EAE for ; Tue, 11 Oct 2016 11:10:47 +0000 (UTC) (envelope-from oliver.pinter@balabit.com) Received: by mail-qk0-x22e.google.com with SMTP id f128so26604540qkb.1 for ; Tue, 11 Oct 2016 04:10:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=balabit-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ZgP7+ZucgGo0z2mDCYbYIFZMTABuBdjYzkb5qlEsTRQ=; b=qqndJYdmaP17vPi0zA0jb/UUGgmWVEh90edbEvzlReEZpF05bwXNdMfhi5+8fMSO/6 LDrj/leCEIMD8RDRWoq/R/2Ma2TzyxDcf2CqeupPauB1iD7VlNiB7dp4p9gsaAN5WKsR +RCoa10x+0BT9wNgbye0M2xG30SiMHXeNzrGBbcf9By/vixTfUp4aNqIMJKefeKopOnr 4wEgY5LMx446nrfLhpp8LxoHqwBVdlAK5NhPMu2uzhPB/JlbLubkTpVoQOeRIuc4LWPi l6xsaKTJuTlOk6mNIAd5UdLHUySlGNm+G1B0SaXyyRe24jbGoabIVikkNgq7MBFt1vp5 RCKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ZgP7+ZucgGo0z2mDCYbYIFZMTABuBdjYzkb5qlEsTRQ=; b=H0ZXUHYR/+9QDuB4eeCD3sC8bZrif10KZKL8Sw69sx79AOue06Wlo8XRRY68iaqnS2 yEe7HQae+Yo5pW4HjLiwkAKwJTe/s7eTEAENv4BB5vWlzNcPpK5zIy0rd1r9CDt0yXmp fr/Z5Bvk6kJmuVo/ee4/oAELiLCjgOajx9g92VKZQgovSEGbEJwQGFz15J6fVu3ae2q/ cZuii3BqXhXJgoM0Brfp/WQUComR2xSnlTB6eoy+EtdcHEEZ//IVfJMvDXQf43/kmgcU FtOlN9FQ/Q2L5cG2YyIDz3FJOjOep9rrcHv6M8R7bPctJnor1y8IJ8Z2KxoLFMH2m75f jPbw== X-Gm-Message-State: AA6/9Rl3Ru3CN5oJQaQGc2u4tffhoU/SheTvLkc+mcrkCVz0GH87bmmagsRzxHTlvWll+UHIgPHwKwwnNC6AKYlF X-Received: by 10.194.220.232 with SMTP id pz8mr4215320wjc.154.1476184246566; Tue, 11 Oct 2016 04:10:46 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.162.103 with HTTP; Tue, 11 Oct 2016 04:10:45 -0700 (PDT) In-Reply-To: <0f543bb5-468e-e559-1bd8-8e2cf3f8bbc3@freebsd.org> References: <7b732876-8cc3-a638-7ff1-e664060d4907@freebsd.org> <0f543bb5-468e-e559-1bd8-8e2cf3f8bbc3@freebsd.org> From: =?UTF-8?B?UGludMOpciwgT2xpdsOpcg==?= Date: Tue, 11 Oct 2016 13:10:45 +0200 Message-ID: Subject: Re: fix for use-after-free problem in 10.x To: Julian Elischer Cc: Oliver Pinter , freebsd , FreeBSD Stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Oct 2016 11:10:47 -0000 On Mon, Oct 10, 2016 at 7:07 AM, Julian Elischer wrote: > On 8/10/2016 5:36 AM, Oliver Pinter wrote: > >> On 10/5/16, Julian Elischer wrote: >> >>> In 11 and 12 the taskqueue code has been rewritten in this area but >>> under 10 this bug still occurs. >>> >>> On our appliances this bug stops the system from mounting the ZFS >>> root, so it is quite severe. >>> Basically while the thread is sleeping during the ZFS mount of root >>> (in the while loop), another thread can free the 'task' item it is >>> checking in that while loop and it can be reused or filled with >>> 'deadcode' etc., with the waiting code unaware of the change.. The fix >>> is to refetch the item at the end of the queue each time around the loop. >>> I don't really want to do the bigger change of MFCing the change in >>> 11, as it is more extensive, though if someone else does, that's ok by >>> me. (If it's ABI compatible) >>> >>> Any comments or suggestions? >>> >> Yes, please commit them. This patch fixes the ZFS + GELI + INVARIANTS >> problem for us. >> There is the FreeBSD PR about the issue: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209580 >> > > I committed a slightly better version to stable/10 > should I ask for a merge to releng/10.3? Yes, it would be really nice! Thanks your effort! > > > > > >> here's the fix in diff form: >>> >>> >>> [robot@porridge /usr/src]$ p4 diff -du ... >>> --- //depot/pbranches/jelischer/FreeBSD-PZ/10.3/sys/kern/subr_ta >>> skqueue.c >>> 2016-09-27 09:14:59.000000000 -0700 >>> +++ /usr/src/sys/kern/subr_taskqueue.c 2016-09-27 09:14:59.000000000 >>> -0700 >>> @@ -441,9 +441,10 @@ >>> >>> TQ_LOCK(queue); >>> task = STAILQ_LAST(&queue->tq_queue, task, ta_link); >>> - if (task != NULL) >>> - while (task->ta_pending != 0) >>> - TQ_SLEEP(queue, task, &queue->tq_mutex, PWAIT, >>> "-", >>> 0); >>> + while (task != NULL && task->ta_pending != 0) { >>> + TQ_SLEEP(queue, task, &queue->tq_mutex, PWAIT, "-", 0); >>> + task = STAILQ_LAST(&queue->tq_queue, task, ta_link); >>> + } >>> taskqueue_drain_running(queue); >>> KASSERT(STAILQ_EMPTY(&queue->tq_queue), >>> ("taskqueue queue is not empty after draining")); >>> >>> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@f >>> reebsd.org" >>> >>> > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Tue Oct 11 11:30:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57ABEC0D43F for ; Tue, 11 Oct 2016 11:30:18 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A28D8BD9 for ; Tue, 11 Oct 2016 11:30:17 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA25691; Tue, 11 Oct 2016 14:30:10 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1btvG5-000GYe-UL; Tue, 11 Oct 2016 14:30:09 +0300 Subject: Re: aibs(4) / atk0110 support for newer systems To: Torfinn Ingolfsen , freebsd-stable@FreeBSD.org References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> <20160930145704.4dbc9d90011154b38493964e@getmail.no> <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> <20160930205928.77d7e74f7bd1a35fcf1aa50a@getmail.no> <7a868c22-e0bd-f677-e4ad-2bdf6f3605d0@FreeBSD.org> <20161003201511.7258687453f12c44a46a361a@getmail.no> <20161005143723.4273657959160b67637a5adf@getmail.no> <4c431c8a-7c8c-4336-b948-6e9ae882c0fc@FreeBSD.org> <20161005222831.e674c3a134266c2e1edc523e@getmail.no> <19cb4aa4-f38e-1331-2939-281a08d1f78d@FreeBSD.org> From: Andriy Gapon Message-ID: Date: Tue, 11 Oct 2016 14:29:34 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <19cb4aa4-f38e-1331-2939-281a08d1f78d@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Oct 2016 11:30:18 -0000 On 06/10/2016 00:37, Andriy Gapon wrote: > On 05/10/2016 23:28, Torfinn Ingolfsen wrote: >> #6 0xffffffff80cd0081 in calltrap () >> at /usr/src/sys/amd64/amd64/exception.S:238 >> #7 0xffffffff81bcb078 in aibs_add_sensor () from /boot/kernel/aibs.ko >> #8 0xffffffff81bcb4b4 in aibs_attach_sif () from /boot/kernel/aibs.ko > > Argh, I've just spotted a very silly typo. > Could you please replace '0' with 'o' in > err = aibs_add_sensor(sc, 0, &as[i], &descr); > ? Ping. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Tue Oct 11 12:11:55 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 994F1C0B0F2 for ; Tue, 11 Oct 2016 12:11:55 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 573F6381; Tue, 11 Oct 2016 12:11:55 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1btvuL-000MyY-KS; Tue, 11 Oct 2016 15:11:45 +0300 Date: Tue, 11 Oct 2016 15:11:45 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161011121145.GJ6177@zxy.spb.ru> References: <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Oct 2016 12:11:55 -0000 On Tue, Oct 11, 2016 at 09:20:17AM +0200, Julien Charbon wrote: > Then threads are competing for the INP_WLOCK lock. For the example, > let's say the thread A wants to run tcp_input()/in_pcblookup_mbuf() and > racing for this INP_WLOCK: > > https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_pcb.c#L1964 > > And thread B wants to run tcp_timer_2msl()/tcp_close()/in_pcbdrop() and > racing for this INP_WLOCK: > > https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/tcp_timer.c#L323 > > That leads to two cases: > > o Thread A wins the race: > > Thread A will continue tcp_input() as usal and INP_DROPPED flags is > not set and inp is still in TCP hash table. > Thread B is waiting on thread A to release INP_WLOCK after finishing > tcp_input() processing, and thread B will continue > tcp_timer_2msl()/tcp_close()/in_pcbdrop() processing. > > o Thread B wins the race: > > Thread B runs tcp_timer_2msl()/tcp_close()/in_pcbdrop() and inp > INP_DROPPED is set and inp being removed from TCP hash table. > In parallel, thread A has found the inp in TCP hash before is was > removed, and waiting on the found inp INP_WLOCK lock. > Once thread B has released the INP_WLOCK lock, thread A gets this lock > and sees the INP_DROPPED flag and do "goto findpcb" but here because the > inp is not more in TCP hash table and it will not be find again by > in_pcblookup_mbuf(). > > Hopefully I am clear enough here. Thanks, very clear. Small qeustion: when both thread run on same CPU core, INP_WLOCK will be re-schedule? As I remeber race created by call tcp_twstart() at time of end tcp_close(), at path sofree()-tcp_usr_detach() and unexpected INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in tcp_twstart() After check source code I am found invocation of tcp_twstart() in sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c. Invocation from sys/netinet/tcp_stacks/fastpath.c and sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and now will be OK. Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see independed INP_WLOCK. Is this OK? Can be thread A wants do_peer_close() directed from chelsio IRQ handler, bypass tcp_input()? From owner-freebsd-stable@freebsd.org Wed Oct 12 08:18:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66591C0D3F1 for ; Wed, 12 Oct 2016 08:18:34 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com [74.125.82.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE35FD34 for ; Wed, 12 Oct 2016 08:18:33 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f52.google.com with SMTP id f193so16144297wmg.0 for ; Wed, 12 Oct 2016 01:18:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=X1bGVMaN2YuDZiyrnVxIRhKVSUV5+xvXsp4Mz3C5TKk=; b=TT02wam364AZOGoUk55WIpp3zzxXrbwhe9sxhog53WfuyOiuptvXbMMLx7iJ3m3KyL DIusOkpPD4R6TEc0mFcUN0sSEr8HTyTt2ZrfXYhzi/xH+LKLV/KRVdPHFLkZ23D4cjQY Olyk5c08iN4Wr8rvJsLOwPQ2+KBs3G6YyUeQOMgQ+L4enM3ceMBeTySXCVlUTH1DY7XW LETdjzCfDcTeHKQthg3XPiXOAmHKAWnOL/OQoDFmxIBt6P1Q6rpeHFpZEsNo/Ng6+eRK 4UOha+qruVi8ymrqKYSYcQqp8nNXwubA/IDiRGiWZFAQNwm5SCDjYrO7W32T8oRSTobk 5/qA== X-Gm-Message-State: AA6/9RmIxuBw9aYwC45NtfrmgNBk5+CZiVihib7PUlJ3Z9dhovjSv77Vq1G8i1Pb1PBKsw== X-Received: by 10.28.88.134 with SMTP id m128mr469141wmb.39.1476260305912; Wed, 12 Oct 2016 01:18:25 -0700 (PDT) Received: from [10.100.64.21] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id p78sm1609112wma.0.2016.10.12.01.18.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 01:18:25 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Wed, 12 Oct 2016 10:18:18 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161011121145.GJ6177@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="lo6WsNmCMpKpkEu6WS6lHh4jDTQdQVNts" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 08:18:34 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --lo6WsNmCMpKpkEu6WS6lHh4jDTQdQVNts Content-Type: multipart/mixed; boundary="LWk23kBf77cRBBkv9C82XIU038BlJnf1A"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20160928115909.GC54003@zxy.spb.ru> <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> In-Reply-To: <20161011121145.GJ6177@zxy.spb.ru> --LWk23kBf77cRBBkv9C82XIU038BlJnf1A Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/11/16 2:11 PM, Slawa Olhovchenkov wrote: > On Tue, Oct 11, 2016 at 09:20:17AM +0200, Julien Charbon wrote: >> Then threads are competing for the INP_WLOCK lock. For the example, >> let's say the thread A wants to run tcp_input()/in_pcblookup_mbuf() an= d >> racing for this INP_WLOCK: >> >> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_= pcb.c#L1964 >> >> And thread B wants to run tcp_timer_2msl()/tcp_close()/in_pcbdrop() a= nd >> racing for this INP_WLOCK: >> >> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/tcp= _timer.c#L323 >> >> That leads to two cases: >> >> o Thread A wins the race: >> >> Thread A will continue tcp_input() as usal and INP_DROPPED flags is >> not set and inp is still in TCP hash table. >> Thread B is waiting on thread A to release INP_WLOCK after finishing= >> tcp_input() processing, and thread B will continue >> tcp_timer_2msl()/tcp_close()/in_pcbdrop() processing. >> >> o Thread B wins the race: >> >> Thread B runs tcp_timer_2msl()/tcp_close()/in_pcbdrop() and inp >> INP_DROPPED is set and inp being removed from TCP hash table. >> In parallel, thread A has found the inp in TCP hash before is was >> removed, and waiting on the found inp INP_WLOCK lock. >> Once thread B has released the INP_WLOCK lock, thread A gets this lo= ck >> and sees the INP_DROPPED flag and do "goto findpcb" but here because t= he >> inp is not more in TCP hash table and it will not be find again by >> in_pcblookup_mbuf(). >> >> Hopefully I am clear enough here. >=20 > Thanks, very clear. > Small qeustion: when both thread run on same CPU core, INP_WLOCK will > be re-schedule? Hmm, a thread can re-scheduled but not a lock. Thus no sure I understand your question here. :) > As I remeber race created by call tcp_twstart() at time of end > tcp_close(), at path sofree()-tcp_usr_detach() and unexpected > INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in tcp_tws= tart() Exactly, thus the current fix is: If you already have the INP_DROPPED flag set you are not allowed to call tcp_twstart(), actually it is a good candidate for a new INVARIANT. Let me add that. > After check source code I am found invocation of tcp_twstart() in > sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, > sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c. >=20 > Invocation from sys/netinet/tcp_stacks/fastpath.c and > sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and now > will be OK. >=20 > Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and > sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see independed > INP_WLOCK. Is this OK? >=20 > Can be thread A wants do_peer_close() directed from chelsio IRQ > handler, bypass tcp_input()? If you look carefully INP_WLOCK is used in cxgb_cpl_io.c and t4_cpl_io.c before calling tcp_twstart(). -- Julien --LWk23kBf77cRBBkv9C82XIU038BlJnf1A-- --lo6WsNmCMpKpkEu6WS6lHh4jDTQdQVNts Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/fHPAAoJEKVlQ5Je6dhxftQIAOFpZ+iF1ve1hfyXLFdI7kSx 2WU04gzNQ40+NXmMJvdZ3hQovwNbXCAu3aM7rqHbOV016Nf8x23gBFTqH+5wgE4z VRrfWifZZk8k1UFXsx2c6u2iVAcfx4WqDnJg98EtYP0Nh6dRuKuqcvPlblchHQJy OO4Zxc8TmIASZrOy1ce8m/fBspLkshoh7c1UdLtXlpyMyboh9dA/B+hC3lN2tVf8 7ghELfaPletstCfTQh6emgxE+IBEBbjAkgChsTCTmCMDoCK4jQ5YtbdjGJ8ZfSLi vBciJ7jWZhRHlANW8ZHbydHXvkOBVCPakmvt+xa8NotQhUSzB4UKRFgZG9Cw9tM= =OisL -----END PGP SIGNATURE----- --lo6WsNmCMpKpkEu6WS6lHh4jDTQdQVNts-- From owner-freebsd-stable@freebsd.org Wed Oct 12 08:40:49 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A4C9C0DC5F for ; Wed, 12 Oct 2016 08:40:49 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ED452FDB; Wed, 12 Oct 2016 08:40:48 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buF5h-000Ge1-PU; Wed, 12 Oct 2016 11:40:45 +0300 Date: Wed, 12 Oct 2016 11:40:45 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161012084045.GA57714@zxy.spb.ru> References: <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 08:40:49 -0000 On Wed, Oct 12, 2016 at 10:18:18AM +0200, Julien Charbon wrote: > > Hi Slawa, > > On 10/11/16 2:11 PM, Slawa Olhovchenkov wrote: > > On Tue, Oct 11, 2016 at 09:20:17AM +0200, Julien Charbon wrote: > >> Then threads are competing for the INP_WLOCK lock. For the example, > >> let's say the thread A wants to run tcp_input()/in_pcblookup_mbuf() and > >> racing for this INP_WLOCK: > >> > >> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/in_pcb.c#L1964 > >> > >> And thread B wants to run tcp_timer_2msl()/tcp_close()/in_pcbdrop() and > >> racing for this INP_WLOCK: > >> > >> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/tcp_timer.c#L323 > >> > >> That leads to two cases: > >> > >> o Thread A wins the race: > >> > >> Thread A will continue tcp_input() as usal and INP_DROPPED flags is > >> not set and inp is still in TCP hash table. > >> Thread B is waiting on thread A to release INP_WLOCK after finishing > >> tcp_input() processing, and thread B will continue > >> tcp_timer_2msl()/tcp_close()/in_pcbdrop() processing. > >> > >> o Thread B wins the race: > >> > >> Thread B runs tcp_timer_2msl()/tcp_close()/in_pcbdrop() and inp > >> INP_DROPPED is set and inp being removed from TCP hash table. > >> In parallel, thread A has found the inp in TCP hash before is was > >> removed, and waiting on the found inp INP_WLOCK lock. > >> Once thread B has released the INP_WLOCK lock, thread A gets this lock > >> and sees the INP_DROPPED flag and do "goto findpcb" but here because the > >> inp is not more in TCP hash table and it will not be find again by > >> in_pcblookup_mbuf(). > >> > >> Hopefully I am clear enough here. > > > > Thanks, very clear. > > Small qeustion: when both thread run on same CPU core, INP_WLOCK will > > be re-schedule? > > Hmm, a thread can re-scheduled but not a lock. Thus no sure I > understand your question here. :) I am don't know how work INP_WLOCK in this case (all on same cpu): thread1: INP_WLOCK -interrupt-- thread2: INP_WLOCK if INP_WLOCK is like spinlock -- this is dead lock. if INP_WLOCK is like mutex -- thread1 resheduled. > > As I remeber race created by call tcp_twstart() at time of end > > tcp_close(), at path sofree()-tcp_usr_detach() and unexpected > > INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in tcp_twstart() > > Exactly, thus the current fix is: If you already have the INP_DROPPED > flag set you are not allowed to call tcp_twstart(), actually it is a > good candidate for a new INVARIANT. Let me add that. > > > After check source code I am found invocation of tcp_twstart() in > > sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, > > sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c. > > > > Invocation from sys/netinet/tcp_stacks/fastpath.c and > > sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and now > > will be OK. > > > > Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and > > sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see independed > > INP_WLOCK. Is this OK? > > > > Can be thread A wants do_peer_close() directed from chelsio IRQ > > handler, bypass tcp_input()? > > If you look carefully INP_WLOCK is used in cxgb_cpl_io.c and > t4_cpl_io.c before calling tcp_twstart(). Yes, and you remeber: sys/netinet/tcp_subr.c 1535 struct tcpcb * 1536 tcp_close(struct tcpcb *tp) 1537 { ... 1569 INP_WUNLOCK(inp); 1570 ACCEPT_LOCK(); 1571 SOCK_LOCK(so); 1572 so->so_state &= ~SS_PROTOREF; 1573 sofree(so); 1574 return (NULL); sofree() call tcp_usr_detach() and in tcp_usr_detach() we have unexpected INP_TIMEWAIT. From owner-freebsd-stable@freebsd.org Wed Oct 12 09:29:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4DCBC0EE98 for ; Wed, 12 Oct 2016 09:29:47 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8317BB0A; Wed, 12 Oct 2016 09:29:47 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buFr7-000I2l-Dr; Wed, 12 Oct 2016 12:29:45 +0300 Date: Wed, 12 Oct 2016 12:29:45 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161012092945.GB57714@zxy.spb.ru> References: <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 09:29:47 -0000 On Wed, Oct 12, 2016 at 11:19:48AM +0200, Julien Charbon wrote: > > if INP_WLOCK is like spinlock -- this is dead lock. > > if INP_WLOCK is like mutex -- thread1 resheduled. > > Thanks, I understand you question now. No an interrupt cannot bypass a > lock: Here INP_WLOCK is like mutex -- thread1 resheduled. Thanks, nice. > >>> As I remeber race created by call tcp_twstart() at time of end > >>> tcp_close(), at path sofree()-tcp_usr_detach() and unexpected > >>> INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in tcp_twstart() > >> > >> Exactly, thus the current fix is: If you already have the INP_DROPPED > >> flag set you are not allowed to call tcp_twstart(), actually it is a > >> good candidate for a new INVARIANT. Let me add that. > >> > >>> After check source code I am found invocation of tcp_twstart() in > >>> sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, > >>> sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c. > >>> > >>> Invocation from sys/netinet/tcp_stacks/fastpath.c and > >>> sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and now > >>> will be OK. > >>> > >>> Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and > >>> sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see independed > >>> INP_WLOCK. Is this OK? > >>> > >>> Can be thread A wants do_peer_close() directed from chelsio IRQ > >>> handler, bypass tcp_input()? > >> > >> If you look carefully INP_WLOCK is used in cxgb_cpl_io.c and > >> t4_cpl_io.c before calling tcp_twstart(). > > > > Yes, and you remeber: sys/netinet/tcp_subr.c > > > > 1535 struct tcpcb * > > 1536 tcp_close(struct tcpcb *tp) > > 1537 { > > ... > > 1569 INP_WUNLOCK(inp); > > 1570 ACCEPT_LOCK(); > > 1571 SOCK_LOCK(so); > > 1572 so->so_state &= ~SS_PROTOREF; > > 1573 sofree(so); > > 1574 return (NULL); > > > > sofree() call tcp_usr_detach() and in tcp_usr_detach() we have > > unexpected INP_TIMEWAIT. > > I see, thus just for the context: The TCP stack in sys/dev/cxgb* is a > TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a > separate/side TCP stack that is used only with TCP_OFFLOAD option. > > This TOE TCP stack actually has its own set of detach()/input() > functions and seems to check INP_DROPPED flag properly. I guess @np > check fixes in socket TCP stack and decides which one can also impact > the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, some > are only in TOE TCP stack. I am fear about other direction -- setting INP_TIMEWAIT in Chelsio TOE TCP stack and impact this to tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. From owner-freebsd-stable@freebsd.org Wed Oct 12 09:42:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41D6BC0C32A for ; Wed, 12 Oct 2016 09:42:47 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B763C371 for ; Wed, 12 Oct 2016 09:42:46 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f44.google.com with SMTP id o81so20408958wma.1 for ; Wed, 12 Oct 2016 02:42:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=SnqEA7S/x6xjmwHd0nT05P+nhQ1Qft7XAVGTipkB/g8=; b=GH2K3DSnWuwRRkPGmRM0wYqNQY7Me+oU0Nc1PRqdFTq+U42fZhzDNUtC+QhjXSkVtT gshHTxOV91+BJzEY67O3WYTt83bs07m7kfk6Ycqrlk4tENvctWx8XT+RVyJKGOMy3H0v ZEesm8T/U1US8QVuxMXIia19gnKOwrrKedxaq22+cnVY34t4f52a4Zn/kBQ0vrAxtgNM 2Nz1LtakTu2i/bwn8l6lbduBkVDgf66dXHkgU31sGEH+Tz66w19nm+cAJLWOKgh5cG7B 182zRb74HUqUeCYbqA9VETCIi4x91AA9nYF9lOghV3pa+hZ3FSUSm1txDqOL0XWSoWd9 fGmQ== X-Gm-Message-State: AA6/9RkbiFOLOhTIR9t59JDfargEKA5E/O1DI1S1jTIBLmHbMJaI54Kc/xADtKG9+quAew== X-Received: by 10.194.222.225 with SMTP id qp1mr154337wjc.161.1476265364687; Wed, 12 Oct 2016 02:42:44 -0700 (PDT) Received: from [10.100.64.21] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id af4sm1982414wjc.17.2016.10.12.02.42.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 02:42:43 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> Date: Wed, 12 Oct 2016 11:42:38 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161012092945.GB57714@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="N9Ce2544eUHm39KOxGT1kCICLAd7kvFl0" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 09:42:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --N9Ce2544eUHm39KOxGT1kCICLAd7kvFl0 Content-Type: multipart/mixed; boundary="fJwW0kuse1wwXC9OHGT2rDC3o0fcKpDKb"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> In-Reply-To: <20161012092945.GB57714@zxy.spb.ru> --fJwW0kuse1wwXC9OHGT2rDC3o0fcKpDKb Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 10/12/16 11:29 AM, Slawa Olhovchenkov wrote: > On Wed, Oct 12, 2016 at 11:19:48AM +0200, Julien Charbon wrote: >=20 >>> if INP_WLOCK is like spinlock -- this is dead lock. >>> if INP_WLOCK is like mutex -- thread1 resheduled. >> >> Thanks, I understand you question now. No an interrupt cannot bypass= a >> lock: Here INP_WLOCK is like mutex -- thread1 resheduled. >=20 > Thanks, nice. >=20 >>>>> As I remeber race created by call tcp_twstart() at time of end >>>>> tcp_close(), at path sofree()-tcp_usr_detach() and unexpected >>>>> INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in tcp= _twstart() >>>> >>>> Exactly, thus the current fix is: If you already have the INP_DROP= PED >>>> flag set you are not allowed to call tcp_twstart(), actually it is a= >>>> good candidate for a new INVARIANT. Let me add that. >>>> >>>>> After check source code I am found invocation of tcp_twstart() in >>>>> sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, >>>>> sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c. >>>>> >>>>> Invocation from sys/netinet/tcp_stacks/fastpath.c and >>>>> sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and no= w >>>>> will be OK. >>>>> >>>>> Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and >>>>> sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see independ= ed >>>>> INP_WLOCK. Is this OK? >>>>> >>>>> Can be thread A wants do_peer_close() directed from chelsio IRQ >>>>> handler, bypass tcp_input()? >>>> >>>> If you look carefully INP_WLOCK is used in cxgb_cpl_io.c and >>>> t4_cpl_io.c before calling tcp_twstart(). >>> >>> Yes, and you remeber: sys/netinet/tcp_subr.c >>> >>> 1535 struct tcpcb * >>> 1536 tcp_close(struct tcpcb *tp) >>> 1537 { >>> ... >>> 1569 INP_WUNLOCK(inp); >>> 1570 ACCEPT_LOCK(); >>> 1571 SOCK_LOCK(so); >>> 1572 so->so_state &=3D ~SS_PROTOREF; >>> 1573 sofree(so); >>> 1574 return (NULL); >>> >>> sofree() call tcp_usr_detach() and in tcp_usr_detach() we have >>> unexpected INP_TIMEWAIT. >> >> I see, thus just for the context: The TCP stack in sys/dev/cxgb* is = a >> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a >> separate/side TCP stack that is used only with TCP_OFFLOAD option. >> >> This TOE TCP stack actually has its own set of detach()/input() >> functions and seems to check INP_DROPPED flag properly. I guess @np >> check fixes in socket TCP stack and decides which one can also impact >> the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, so= me >> are only in TOE TCP stack. >=20 > I am fear about other direction -- setting INP_TIMEWAIT in Chelsio TOE > TCP stack and impact this to > tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. I see, I expect no problem on this side as tcp_timer_2msl() checks the INP_TIMEWAIT flag and do not call tcp_close() if set. -- Julien --fJwW0kuse1wwXC9OHGT2rDC3o0fcKpDKb-- --N9Ce2544eUHm39KOxGT1kCICLAd7kvFl0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/gWRAAoJEKVlQ5Je6dhxF4YH/3ttxM1WJJuEt5FgXKizUk2/ uKT4mPGegoTIslwUC6CJzs/OjCkWKzKwNCWN5fK630gdV1kgQygPaXINR4QDI90y /VqSVzPdhWltR/+xuziuOdaPeGxrEV51i7poHQYpM9v4vXN/yEj2QiP40lZlUsUi j8LatszYKcEq6u2VHdbzWgxzL7xWqz3PGZZfQaNTxh/yqAK18VUXlmiCqcWzfjz+ rL7kRRAwjpoMtkF0gLLEteGwMnRRNCrZ8aBX3kfcKJr+d32nVHZm9s7TC9uwV7FY iZPJIBiOLURF1QGepbhxQ/ldvJVPNRZtH4rfEXuoOhlX2El+pI9gMEt6foVo9ww= =PK4z -----END PGP SIGNATURE----- --N9Ce2544eUHm39KOxGT1kCICLAd7kvFl0-- From owner-freebsd-stable@freebsd.org Wed Oct 12 09:49:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7FF1C0C4C2 for ; Wed, 12 Oct 2016 09:49:19 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-wm0-f47.google.com (mail-wm0-f47.google.com [74.125.82.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7351C9D0 for ; Wed, 12 Oct 2016 09:49:19 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-wm0-f47.google.com with SMTP id c78so21350082wme.1 for ; Wed, 12 Oct 2016 02:49:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=8vqosM+kJbZfWpZsVxOl0e+t0WjWBvJ4ThUnE5Vn25c=; b=iklj1Y6Cv7KeISy7NR0Mblk/AOo6MAZPPA2H9gX/Bd7VAvOz6DCekTwrZ63WJY2fJC QILpwDnyDBrXRjHuv6AWkYi2k5q+zpT26GZtSlc64Zv8UWttq37L5BsNgwszjFob/jVC Z3Zrycn7jM9BY2nDCTrzVIL1W4u9rJVL9u7tZq2njVUSAagiHw9tKvuY8JMitYfYZlHW DhwRw1Zi4j3SnLwBH8AmhuslZvmNCDWEEBVqf1DNqTmFC1aNym8SooRnUEBTQegzHQIb Pvi4JDEyogS0EHZT31EwVSpx/GHUQTyB6FXoBu1A7ZJSPX6ompxeD5bjmI0lEso9WbWB JMQg== X-Gm-Message-State: AA6/9RlxoQk5jCljnh5dFmf5tYKaXWeoVDM3KeKAall3kupoaZYtHCZIsmWImwEqK+V/jA== X-Received: by 10.28.145.149 with SMTP id t143mr650616wmd.130.1476263995478; Wed, 12 Oct 2016 02:19:55 -0700 (PDT) Received: from [10.100.64.21] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id f2sm11421072wjr.2.2016.10.12.02.19.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 02:19:54 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Wed, 12 Oct 2016 11:19:48 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161012084045.GA57714@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="DGX6hbHiQg02lxOwk9lmjcNErfTg5diS7" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 09:49:19 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --DGX6hbHiQg02lxOwk9lmjcNErfTg5diS7 Content-Type: multipart/mixed; boundary="8aoSHgk0kipRFXJlwulQdE80ElhUCGXii"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> In-Reply-To: <20161012084045.GA57714@zxy.spb.ru> --8aoSHgk0kipRFXJlwulQdE80ElhUCGXii Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/12/16 10:40 AM, Slawa Olhovchenkov wrote: > On Wed, Oct 12, 2016 at 10:18:18AM +0200, Julien Charbon wrote: >> On 10/11/16 2:11 PM, Slawa Olhovchenkov wrote: >>> On Tue, Oct 11, 2016 at 09:20:17AM +0200, Julien Charbon wrote: >>>> Then threads are competing for the INP_WLOCK lock. For the example= , >>>> let's say the thread A wants to run tcp_input()/in_pcblookup_mbuf() = and >>>> racing for this INP_WLOCK: >>>> >>>> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/i= n_pcb.c#L1964 >>>> >>>> And thread B wants to run tcp_timer_2msl()/tcp_close()/in_pcbdrop()= and >>>> racing for this INP_WLOCK: >>>> >>>> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/t= cp_timer.c#L323 >>>> >>>> That leads to two cases: >>>> >>>> o Thread A wins the race: >>>> >>>> Thread A will continue tcp_input() as usal and INP_DROPPED flags i= s >>>> not set and inp is still in TCP hash table. >>>> Thread B is waiting on thread A to release INP_WLOCK after finishi= ng >>>> tcp_input() processing, and thread B will continue >>>> tcp_timer_2msl()/tcp_close()/in_pcbdrop() processing. >>>> >>>> o Thread B wins the race: >>>> >>>> Thread B runs tcp_timer_2msl()/tcp_close()/in_pcbdrop() and inp >>>> INP_DROPPED is set and inp being removed from TCP hash table. >>>> In parallel, thread A has found the inp in TCP hash before is was >>>> removed, and waiting on the found inp INP_WLOCK lock. >>>> Once thread B has released the INP_WLOCK lock, thread A gets this = lock >>>> and sees the INP_DROPPED flag and do "goto findpcb" but here because= the >>>> inp is not more in TCP hash table and it will not be find again by >>>> in_pcblookup_mbuf(). >>>> >>>> Hopefully I am clear enough here. >>> >>> Thanks, very clear. >>> Small qeustion: when both thread run on same CPU core, INP_WLOCK will= >>> be re-schedule? >> >> Hmm, a thread can re-scheduled but not a lock. Thus no sure I >> understand your question here. :) >=20 > I am don't know how work INP_WLOCK in this case (all on same cpu): >=20 > thread1: INP_WLOCK > -interrupt-- > thread2: INP_WLOCK >=20 > if INP_WLOCK is like spinlock -- this is dead lock. > if INP_WLOCK is like mutex -- thread1 resheduled. Thanks, I understand you question now. No an interrupt cannot bypass a lock: Here INP_WLOCK is like mutex -- thread1 resheduled. >>> As I remeber race created by call tcp_twstart() at time of end >>> tcp_close(), at path sofree()-tcp_usr_detach() and unexpected >>> INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in tcp_t= wstart() >> >> Exactly, thus the current fix is: If you already have the INP_DROPPE= D >> flag set you are not allowed to call tcp_twstart(), actually it is a >> good candidate for a new INVARIANT. Let me add that. >> >>> After check source code I am found invocation of tcp_twstart() in >>> sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, >>> sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c. >>> >>> Invocation from sys/netinet/tcp_stacks/fastpath.c and >>> sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and now >>> will be OK. >>> >>> Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and >>> sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see independed= >>> INP_WLOCK. Is this OK? >>> >>> Can be thread A wants do_peer_close() directed from chelsio IRQ >>> handler, bypass tcp_input()? >> >> If you look carefully INP_WLOCK is used in cxgb_cpl_io.c and >> t4_cpl_io.c before calling tcp_twstart(). >=20 > Yes, and you remeber: sys/netinet/tcp_subr.c >=20 > 1535 struct tcpcb * > 1536 tcp_close(struct tcpcb *tp) > 1537 { > ... > 1569 INP_WUNLOCK(inp); > 1570 ACCEPT_LOCK(); > 1571 SOCK_LOCK(so); > 1572 so->so_state &=3D ~SS_PROTOREF; > 1573 sofree(so); > 1574 return (NULL); >=20 > sofree() call tcp_usr_detach() and in tcp_usr_detach() we have > unexpected INP_TIMEWAIT. I see, thus just for the context: The TCP stack in sys/dev/cxgb* is a TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a separate/side TCP stack that is used only with TCP_OFFLOAD option. This TOE TCP stack actually has its own set of detach()/input() functions and seems to check INP_DROPPED flag properly. I guess @np check fixes in socket TCP stack and decides which one can also impact the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, some are only in TOE TCP stack. -- Julien --8aoSHgk0kipRFXJlwulQdE80ElhUCGXii-- --DGX6hbHiQg02lxOwk9lmjcNErfTg5diS7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/gA4AAoJEKVlQ5Je6dhxOMUH/iJ2YxJkegfdn6YiAfRffrNx RT+E1JvhSQmITZ1wCSSZpAG+y4VjYaph5Ey8V429wenXcq47MDAcbqrqmmDhj82v nNiFschA8A4lCRvZyPIrRV1UD+ojaE56ykzvrXqvf5JsdZ54oVzweTkEdP/ehnyk XADKx2XrwAydr7L5/6a+3yu5EgiMU/6KMvxEGT0PVGw1Tyur4kQjnstfLPfo5u8m miKVM8VFu5wewMn7FApb2amhBUGo0cSJZTDGSd+IMowZMuY8eB52IHLXUtJLEn6n MNjEuETcy37T8EU9u8LbTMJgLi6sGWusRfSc1FZ2LWM3xiavbKsPwxVgSt6lERQ= =7uvB -----END PGP SIGNATURE----- --DGX6hbHiQg02lxOwk9lmjcNErfTg5diS7-- From owner-freebsd-stable@freebsd.org Wed Oct 12 09:52:36 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2FF1C0C6ED for ; Wed, 12 Oct 2016 09:52:36 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92B36E47; Wed, 12 Oct 2016 09:52:36 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buGDC-000Ig8-0U; Wed, 12 Oct 2016 12:52:34 +0300 Date: Wed, 12 Oct 2016 12:52:33 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161012095233.GC57714@zxy.spb.ru> References: <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 09:52:36 -0000 On Wed, Oct 12, 2016 at 11:42:38AM +0200, Julien Charbon wrote: > On 10/12/16 11:29 AM, Slawa Olhovchenkov wrote: > > On Wed, Oct 12, 2016 at 11:19:48AM +0200, Julien Charbon wrote: > > > >>> if INP_WLOCK is like spinlock -- this is dead lock. > >>> if INP_WLOCK is like mutex -- thread1 resheduled. > >> > >> Thanks, I understand you question now. No an interrupt cannot bypass a > >> lock: Here INP_WLOCK is like mutex -- thread1 resheduled. > > > > Thanks, nice. > > > >>>>> As I remeber race created by call tcp_twstart() at time of end > >>>>> tcp_close(), at path sofree()-tcp_usr_detach() and unexpected > >>>>> INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in tcp_twstart() > >>>> > >>>> Exactly, thus the current fix is: If you already have the INP_DROPPED > >>>> flag set you are not allowed to call tcp_twstart(), actually it is a > >>>> good candidate for a new INVARIANT. Let me add that. > >>>> > >>>>> After check source code I am found invocation of tcp_twstart() in > >>>>> sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, > >>>>> sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c. > >>>>> > >>>>> Invocation from sys/netinet/tcp_stacks/fastpath.c and > >>>>> sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and now > >>>>> will be OK. > >>>>> > >>>>> Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and > >>>>> sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see independed > >>>>> INP_WLOCK. Is this OK? > >>>>> > >>>>> Can be thread A wants do_peer_close() directed from chelsio IRQ > >>>>> handler, bypass tcp_input()? > >>>> > >>>> If you look carefully INP_WLOCK is used in cxgb_cpl_io.c and > >>>> t4_cpl_io.c before calling tcp_twstart(). > >>> > >>> Yes, and you remeber: sys/netinet/tcp_subr.c > >>> > >>> 1535 struct tcpcb * > >>> 1536 tcp_close(struct tcpcb *tp) > >>> 1537 { > >>> ... > >>> 1569 INP_WUNLOCK(inp); > >>> 1570 ACCEPT_LOCK(); > >>> 1571 SOCK_LOCK(so); > >>> 1572 so->so_state &= ~SS_PROTOREF; > >>> 1573 sofree(so); > >>> 1574 return (NULL); > >>> > >>> sofree() call tcp_usr_detach() and in tcp_usr_detach() we have > >>> unexpected INP_TIMEWAIT. > >> > >> I see, thus just for the context: The TCP stack in sys/dev/cxgb* is a > >> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a > >> separate/side TCP stack that is used only with TCP_OFFLOAD option. > >> > >> This TOE TCP stack actually has its own set of detach()/input() > >> functions and seems to check INP_DROPPED flag properly. I guess @np > >> check fixes in socket TCP stack and decides which one can also impact > >> the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, some > >> are only in TOE TCP stack. > > > > I am fear about other direction -- setting INP_TIMEWAIT in Chelsio TOE > > TCP stack and impact this to > > tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. > > I see, I expect no problem on this side as tcp_timer_2msl() checks the > INP_TIMEWAIT flag and do not call tcp_close() if set. I am about case when at time of first INP_WUNLOCK() tcp_timer_2msl() don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNLOCK() and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpected INP_TIMEWAIT in tcp_usr_detach(). From owner-freebsd-stable@freebsd.org Wed Oct 12 12:07:23 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB9C9C0EAFD for ; Wed, 12 Oct 2016 12:07:23 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f49.google.com (mail-lf0-f49.google.com [209.85.215.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C07939C for ; Wed, 12 Oct 2016 12:07:23 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f49.google.com with SMTP id x79so74525093lff.0 for ; Wed, 12 Oct 2016 05:07:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=tnbnF+N97MKQZJlClhdG556ibwmuS29jsy8bIiIHUsw=; b=knIDDxXbf5OsPQbR2aq1B4IeX4e/dc1VWRtLSH668zVPVuk4ThzYd+WiFq/zdTB1y+ xO1xfwjnVjMEloCPhjmzQsiwXdrzzNkrjK81ygSE0UWQLF2NXebQlWiivn7n3bRgjs27 19755Xyp4RBDxxFAyXZdcRdpQZ2FIWkhDY+Ed26dChv3gJzEzZA33AHQIivtBCUqIaSP ExG+HrZ2pTvAzs76GxXf0Slmg0rVRgszgDeYwz8HngS69NhXugmTfHvfbsPTGhkYXw6y NOzJjDjr3IDGxnueI6G8VkNEvU+RDoHc3yLtMayydGTkVGt9kxvv5I3OrPqm2OhpjlDo RKJg== X-Gm-Message-State: AA6/9RlK9iS8OD4fB4Xx6Z2nqL4oLYccMxEPHVoxZSUeIQZ3CGcVdawL8WpqypxbBcq/Mg== X-Received: by 10.25.76.7 with SMTP id z7mr804771lfa.126.1476274035012; Wed, 12 Oct 2016 05:07:15 -0700 (PDT) Received: from [10.100.64.21] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id p3sm2069434lfe.1.2016.10.12.05.07.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 05:07:14 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Wed, 12 Oct 2016 14:06:59 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161012095233.GC57714@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="QUlFkUkrLXFqeCg5Mmgu2mfi4vc2xCgp0" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 12:07:24 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --QUlFkUkrLXFqeCg5Mmgu2mfi4vc2xCgp0 Content-Type: multipart/mixed; boundary="S2SvsBPGCh1kCtCwqdd8V9n6muFopH9rD"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> In-Reply-To: <20161012095233.GC57714@zxy.spb.ru> --S2SvsBPGCh1kCtCwqdd8V9n6muFopH9rD Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/12/16 11:52 AM, Slawa Olhovchenkov wrote: > On Wed, Oct 12, 2016 at 11:42:38AM +0200, Julien Charbon wrote: >> On 10/12/16 11:29 AM, Slawa Olhovchenkov wrote: >>> On Wed, Oct 12, 2016 at 11:19:48AM +0200, Julien Charbon wrote: >>> >>>>> if INP_WLOCK is like spinlock -- this is dead lock. >>>>> if INP_WLOCK is like mutex -- thread1 resheduled. >>>> >>>> Thanks, I understand you question now. No an interrupt cannot bypa= ss a >>>> lock: Here INP_WLOCK is like mutex -- thread1 resheduled. >>> >>> Thanks, nice. >>> >>>>>>> As I remeber race created by call tcp_twstart() at time of end >>>>>>> tcp_close(), at path sofree()-tcp_usr_detach() and unexpected >>>>>>> INP_TIMEWAIT state in the tcp_usr_detach(). INP_TIMEWAIT set in t= cp_twstart() >>>>>> >>>>>> Exactly, thus the current fix is: If you already have the INP_DR= OPPED >>>>>> flag set you are not allowed to call tcp_twstart(), actually it is= a >>>>>> good candidate for a new INVARIANT. Let me add that. >>>>>> >>>>>>> After check source code I am found invocation of tcp_twstart() in= >>>>>>> sys/netinet/tcp_stacks/fastpath.c, sys/netinet/tcp_input.c, >>>>>>> sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c, sys/dev/cxgbe/tom/t4_cpl_io.c= =2E >>>>>>> >>>>>>> Invocation from sys/netinet/tcp_stacks/fastpath.c and >>>>>>> sys/netinet/tcp_input.c guarded by INP_WLOCK in tcp_input(), and = now >>>>>>> will be OK. >>>>>>> >>>>>>> Invocation from sys/dev/cxgb/ulp/tom/cxgb_cpl_io.c and >>>>>>> sys/dev/cxgbe/tom/t4_cpl_io.c is not clear to me, I am see indepe= nded >>>>>>> INP_WLOCK. Is this OK? >>>>>>> >>>>>>> Can be thread A wants do_peer_close() directed from chelsio IRQ >>>>>>> handler, bypass tcp_input()? >>>>>> >>>>>> If you look carefully INP_WLOCK is used in cxgb_cpl_io.c and >>>>>> t4_cpl_io.c before calling tcp_twstart(). >>>>> >>>>> Yes, and you remeber: sys/netinet/tcp_subr.c >>>>> >>>>> 1535 struct tcpcb * >>>>> 1536 tcp_close(struct tcpcb *tp) >>>>> 1537 { >>>>> ... >>>>> 1569 INP_WUNLOCK(inp); >>>>> 1570 ACCEPT_LOCK(); >>>>> 1571 SOCK_LOCK(so); >>>>> 1572 so->so_state &=3D ~SS_PROTOREF; >>>>> 1573 sofree(so); >>>>> 1574 return (NULL); >>>>> >>>>> sofree() call tcp_usr_detach() and in tcp_usr_detach() we have >>>>> unexpected INP_TIMEWAIT. >>>> >>>> I see, thus just for the context: The TCP stack in sys/dev/cxgb* i= s a >>>> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a >>>> separate/side TCP stack that is used only with TCP_OFFLOAD option. >>>> >>>> This TOE TCP stack actually has its own set of detach()/input() >>>> functions and seems to check INP_DROPPED flag properly. I guess @np= >>>> check fixes in socket TCP stack and decides which one can also impac= t >>>> the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, = some >>>> are only in TOE TCP stack. >>> >>> I am fear about other direction -- setting INP_TIMEWAIT in Chelsio TO= E >>> TCP stack and impact this to >>> tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. >> >> I see, I expect no problem on this side as tcp_timer_2msl() checks th= e >> INP_TIMEWAIT flag and do not call tcp_close() if set. >=20 > I am about case when at time of first INP_WUNLOCK() tcp_timer_2msl() > don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNLOCK() > and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set > INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpected > INP_TIMEWAIT in tcp_usr_detach(). Sure, basically the same bug that in classic TCP stack. If you think it can happen, send an email describing that to np@ and he will check and fix that. He is a TOE TCP stack expert and I am not. In all cases, if this issue is possible in TOE TCP stack context, the patch will be straightforward: If the INP_DROPPED flag is set do not call tcp_twstart(= ). The current patch focuses only on the classic TCP stack. -- Julien --S2SvsBPGCh1kCtCwqdd8V9n6muFopH9rD-- --QUlFkUkrLXFqeCg5Mmgu2mfi4vc2xCgp0 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/idwAAoJEKVlQ5Je6dhxHcoIALKEpseITVVpDlpMkteujLge nKwhtpaTyxAh1S7CDbvuGoHSDcvmy8ajHW928lD+Y7zIT325Zk2LCBjxfNqcMoRz rTXrFcph7nhEhHawEXCeKHhxP3VLmy76axfm13SG9stT0vWYOFwbdiEF1BiGrIZb 7Qu3JPumAGsW8CGPlhFu1kTXx7msVCxnCBC4N84UmhU/oVL6oKKyDlvNZ7barKXM kOknO5B2JDujT/WispMGtcKjqwLbtrAlgustfPGBzxSKpMw4DyUT+ps8yZRY/Zyn T8RduPXnvzwvTY5CufUoc3nYySvb6U8Nbxk9fmrlUsTNFIvkHlBs5WW3ezwgJPY= =uWUU -----END PGP SIGNATURE----- --QUlFkUkrLXFqeCg5Mmgu2mfi4vc2xCgp0-- From owner-freebsd-stable@freebsd.org Wed Oct 12 12:13:32 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49930C0EF35 for ; Wed, 12 Oct 2016 12:13:32 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 04671BCB; Wed, 12 Oct 2016 12:13:32 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buIPS-000Mec-Q8; Wed, 12 Oct 2016 15:13:22 +0300 Date: Wed, 12 Oct 2016 15:13:22 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161012121322.GB57876@zxy.spb.ru> References: <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 12:13:32 -0000 On Wed, Oct 12, 2016 at 02:06:59PM +0200, Julien Charbon wrote: > >>>>> sofree() call tcp_usr_detach() and in tcp_usr_detach() we have > >>>>> unexpected INP_TIMEWAIT. > >>>> > >>>> I see, thus just for the context: The TCP stack in sys/dev/cxgb* is a > >>>> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a > >>>> separate/side TCP stack that is used only with TCP_OFFLOAD option. > >>>> > >>>> This TOE TCP stack actually has its own set of detach()/input() > >>>> functions and seems to check INP_DROPPED flag properly. I guess @np > >>>> check fixes in socket TCP stack and decides which one can also impact > >>>> the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, some > >>>> are only in TOE TCP stack. > >>> > >>> I am fear about other direction -- setting INP_TIMEWAIT in Chelsio TOE > >>> TCP stack and impact this to > >>> tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. > >> > >> I see, I expect no problem on this side as tcp_timer_2msl() checks the > >> INP_TIMEWAIT flag and do not call tcp_close() if set. > > > > I am about case when at time of first INP_WUNLOCK() tcp_timer_2msl() > > don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNLOCK() > > and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set > > INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpected > > INP_TIMEWAIT in tcp_usr_detach(). > > Sure, basically the same bug that in classic TCP stack. If you think > it can happen, send an email describing that to np@ and he will check > and fix that. He is a TOE TCP stack expert and I am not. In all cases, > if this issue is possible in TOE TCP stack context, the patch will be > straightforward: If the INP_DROPPED flag is set do not call tcp_twstart(). > > The current patch focuses only on the classic TCP stack. May be current workaround (with logging) in tcp_usr_detach() is good solutuion for preventing system lockout by similar bugs? From owner-freebsd-stable@freebsd.org Wed Oct 12 12:35:30 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3665AC0DA9F for ; Wed, 12 Oct 2016 12:35:30 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f43.google.com (mail-lf0-f43.google.com [209.85.215.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF1FDB5A for ; Wed, 12 Oct 2016 12:35:29 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f43.google.com with SMTP id b75so73267797lfg.3 for ; Wed, 12 Oct 2016 05:35:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=kFFQtg0LJUxGHYydcW00BzgchaUcTdpDh/o52HJFOwE=; b=ItjcNWDTwigwVq9PXCwsCE5Imv/n5xbfnLmXuJR5fMpygTSGEZPmEgvO3rB4/Klzz6 U6TYql/cQ9jYJSn/Lboqcd8V1Tv9m0v7s565zvW3I8QahwjlC4AmtJXSr+2lszaWxNLn WdZNpfL+e54cpIItYeU9FvmkWZTrP2Sy6z7ILkohnx8JFwvCeiB7+IF09dvZoMpxuymO 6Q3QGDru+kl/mb9Bo4DgHk+gZoZmwLnpHtl6KVeDuOSLaIeXh+f6dn2tFqf8kyjokZoO go5p0C+wPpYSj6JRmL+qGps3u2K6c1xGi8aVNzg/HNNuFzHX5jVSfZCR9y23irDuLtD8 fWzw== X-Gm-Message-State: AA6/9RkYIr3TZAtk1HLC7sa7e2gyb+lGpyWXS/A2mZJzp+HI6DyY9wSn/pcDFh1EkmS0mA== X-Received: by 10.25.32.69 with SMTP id g66mr977794lfg.15.1476275721677; Wed, 12 Oct 2016 05:35:21 -0700 (PDT) Received: from [10.100.64.21] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id 94sm2128958lja.10.2016.10.12.05.35.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 05:35:20 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> Date: Wed, 12 Oct 2016 14:35:11 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161012121322.GB57876@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="p0Bsep5oESfbx0NfeiJT4H3hKPE0HcIq2" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 12:35:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --p0Bsep5oESfbx0NfeiJT4H3hKPE0HcIq2 Content-Type: multipart/mixed; boundary="EGjrQK3qasa9e1amPuPHTuUJUJQEj6Bc8"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> In-Reply-To: <20161012121322.GB57876@zxy.spb.ru> --EGjrQK3qasa9e1amPuPHTuUJUJQEj6Bc8 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/12/16 2:13 PM, Slawa Olhovchenkov wrote: > On Wed, Oct 12, 2016 at 02:06:59PM +0200, Julien Charbon wrote: >>>>>>> sofree() call tcp_usr_detach() and in tcp_usr_detach() we have >>>>>>> unexpected INP_TIMEWAIT. >>>>>> >>>>>> I see, thus just for the context: The TCP stack in sys/dev/cxgb*= is a >>>>>> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a >>>>>> separate/side TCP stack that is used only with TCP_OFFLOAD option.= >>>>>> >>>>>> This TOE TCP stack actually has its own set of detach()/input() >>>>>> functions and seems to check INP_DROPPED flag properly. I guess @= np >>>>>> check fixes in socket TCP stack and decides which one can also imp= act >>>>>> the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack= , some >>>>>> are only in TOE TCP stack. >>>>> >>>>> I am fear about other direction -- setting INP_TIMEWAIT in Chelsio = TOE >>>>> TCP stack and impact this to >>>>> tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. >>>> >>>> I see, I expect no problem on this side as tcp_timer_2msl() checks = the >>>> INP_TIMEWAIT flag and do not call tcp_close() if set. >>> >>> I am about case when at time of first INP_WUNLOCK() tcp_timer_2msl() >>> don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNLOCK(= ) >>> and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set >>> INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpected >>> INP_TIMEWAIT in tcp_usr_detach(). >> >> Sure, basically the same bug that in classic TCP stack. If you think= >> it can happen, send an email describing that to np@ and he will check >> and fix that. He is a TOE TCP stack expert and I am not. In all case= s, >> if this issue is possible in TOE TCP stack context, the patch will be >> straightforward: If the INP_DROPPED flag is set do not call tcp_twsta= rt(). >> >> The current patch focuses only on the classic TCP stack. >=20 > May be current workaround (with logging) in tcp_usr_detach() is good > solutuion for preventing system lockout by similar bugs? Good question, the quick workaround in tcp_usr_detach() does not handle all the cases. If it reduces the number of crashes you can still find scenarios where it can have unexpected side effect. Long term solution is to enforce: If the inp has the INP_DROPPED flag just stop processing it and return. If you grep the INP_DROPPED flag in kernel sources, you can see that this test is already done in almost all tcp_*() processing functions but tcp_input(). I would say that even without this issue tcp_input() should check INP_DROPPED flags after INP_WLOCK anyway. Same for the TOE TCP stack, you are simply not supposed to process a inp with INP_DROPPED flag. -- Julien --EGjrQK3qasa9e1amPuPHTuUJUJQEj6Bc8-- --p0Bsep5oESfbx0NfeiJT4H3hKPE0HcIq2 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/i4HAAoJEKVlQ5Je6dhxB3MH/R4zM90cMHFNgzG2fkXaRFUz C0YVYDO6SxffHON9HVNQrTE+hV5D/n/FrtPHSqeCK+JnKi3SO2Td5fwvIb1FnhQs SF0iK640pJO5fpzrgYRX6FwmEM192XWylAiBTAzuMyPRpl4QHWEIxv7F4bt01Xtj QCV/PJt7JUipvv6Q7KCuCvLAOKOTOSjWRXwhatCeVXS9kKVVOmZrjARMxLRFnQJG a2Ww2fStHSodnmaJCIWWomU5vll+NNo0QHA6q6JCOMHFO9UD8evGHsP4vDuGQiyr 244KogxvJBU7K/F2rFhUekTR0tFcce/dGcyxP+RJO6wUL+ikSpBDpXSnB/jCQxU= =MQ14 -----END PGP SIGNATURE----- --p0Bsep5oESfbx0NfeiJT4H3hKPE0HcIq2-- From owner-freebsd-stable@freebsd.org Wed Oct 12 13:01:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE006C0E158 for ; Wed, 12 Oct 2016 13:01:05 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9CFD075; Wed, 12 Oct 2016 13:01:05 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buJ9b-000Nuh-A8; Wed, 12 Oct 2016 16:01:03 +0300 Date: Wed, 12 Oct 2016 16:01:03 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161012130103.GD57714@zxy.spb.ru> References: <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 13:01:06 -0000 On Wed, Oct 12, 2016 at 02:35:11PM +0200, Julien Charbon wrote: > > Hi Slawa, > > On 10/12/16 2:13 PM, Slawa Olhovchenkov wrote: > > On Wed, Oct 12, 2016 at 02:06:59PM +0200, Julien Charbon wrote: > >>>>>>> sofree() call tcp_usr_detach() and in tcp_usr_detach() we have > >>>>>>> unexpected INP_TIMEWAIT. > >>>>>> > >>>>>> I see, thus just for the context: The TCP stack in sys/dev/cxgb* is a > >>>>>> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a > >>>>>> separate/side TCP stack that is used only with TCP_OFFLOAD option. > >>>>>> > >>>>>> This TOE TCP stack actually has its own set of detach()/input() > >>>>>> functions and seems to check INP_DROPPED flag properly. I guess @np > >>>>>> check fixes in socket TCP stack and decides which one can also impact > >>>>>> the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, some > >>>>>> are only in TOE TCP stack. > >>>>> > >>>>> I am fear about other direction -- setting INP_TIMEWAIT in Chelsio TOE > >>>>> TCP stack and impact this to > >>>>> tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. > >>>> > >>>> I see, I expect no problem on this side as tcp_timer_2msl() checks the > >>>> INP_TIMEWAIT flag and do not call tcp_close() if set. > >>> > >>> I am about case when at time of first INP_WUNLOCK() tcp_timer_2msl() > >>> don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNLOCK() > >>> and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set > >>> INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpected > >>> INP_TIMEWAIT in tcp_usr_detach(). > >> > >> Sure, basically the same bug that in classic TCP stack. If you think > >> it can happen, send an email describing that to np@ and he will check > >> and fix that. He is a TOE TCP stack expert and I am not. In all cases, > >> if this issue is possible in TOE TCP stack context, the patch will be > >> straightforward: If the INP_DROPPED flag is set do not call tcp_twstart(). > >> > >> The current patch focuses only on the classic TCP stack. > > > > May be current workaround (with logging) in tcp_usr_detach() is good > > solutuion for preventing system lockout by similar bugs? > > Good question, the quick workaround in tcp_usr_detach() does not handle > all the cases. If it reduces the number of crashes you can still find > scenarios where it can have unexpected side effect. This is best then guaranted lockout. > Long term solution is to enforce: If the inp has the INP_DROPPED flag > just stop processing it and return. If you grep the INP_DROPPED flag in > kernel sources, you can see that this test is already done in almost all > tcp_*() processing functions but tcp_input(). > > I would say that even without this issue tcp_input() should check > INP_DROPPED flags after INP_WLOCK anyway. Same for the TOE TCP stack, > you are simply not supposed to process a inp with INP_DROPPED flag. Absolutly acceptant! May point is: more check and good handling of check result is best for stability. I.e. AND check INP_DROPPED in tcp_input AND workaroud INP_TIMEWAIT in tcp_usr_detach (with logging) and check of some posible cases in XXX TOE. Current TCP stack too complex and have many corner cases. This is need additional guards where posible (not caused kernel panic). From owner-freebsd-stable@freebsd.org Wed Oct 12 14:29:55 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A42BC0ED51 for ; Wed, 12 Oct 2016 14:29:55 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB68AF5 for ; Wed, 12 Oct 2016 14:29:54 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.ijs.si (Postfix) with ESMTPS id 3svGTq0F8Rz1K4 for ; Wed, 12 Oct 2016 16:29:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:organization:subject:subject:from:from :date:date:content-transfer-encoding:content-type:content-type :mime-version:received:received:received:received; s=jakla4; t= 1476282587; x=1478874588; bh=0L1HILwL5rzVrMl9JGZ29eTSC3F3yaJ98ZU iTHY5t/o=; b=I1+sqCnQqTX2q4buPbP87J+PBdNzKLl/48iGPQRQE7dTaKxtZiI 2fMNgbYDG5Xn/yBlLfE9GFB+yGGTN8SDKAYTgcWU3+mQQS+4tb0EjUNoKxGQlf+q ZpJj2ctE5dILJiIpOUhH1OYCQAxHUX8TEHOcdLqkWR850vxLVYq1/fSs= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id nA2JIgCsdwDZ for ; Wed, 12 Oct 2016 16:29:47 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 3svGTl0MQxz1K1 for ; Wed, 12 Oct 2016 16:29:46 +0200 (CEST) Received: from nabiralnik.ijs.si (nabiralnik.ijs.si [IPv6:2001:1470:ff80::80:16]) by mildred.ijs.si (Postfix) with ESMTP id 3svGTk5yydz1XG for ; Wed, 12 Oct 2016 16:29:46 +0200 (CEST) Received: from neli.ijs.si (2001:1470:ff80:88:21c:c0ff:feb1:8c91) by webmail.ijs.si with HTTP (HTTP/1.1 POST); Wed, 12 Oct 2016 16:29:46 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 12 Oct 2016 16:29:46 +0200 From: Mark Martinec To: freebsd-stable@FreeBSD.org Subject: update.FreeBSD.org unresponsive? Organization: Jozef Stefan Institute Message-ID: X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.2.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 14:29:55 -0000 Trying to upgrade a couple of hosts (11.0-RC2, 11.0-RC3, 10.3-RELEASE-p10) to 11.0 (using: freebsd-update upgrade -r 11.0-RELEASE), and it seems the fetch(1) always fails with a timeout. Even a simple (freebsd-update fetch) in an attempt to bump a 10.3-RELEASE-p9 to 10.3-RELEASE-p10 now fails with a timeout, while previously it worked reliably and fast. The interesting thing is that both the ping and ping6 to update.FreeBSD.org work flawlessly with no packet loss. I tried it several times yesterday, and again today. Our network connectivity is otherwise good and fast. To rule out a possibility of a firewall or routing issue I even tried it from a different network (different ISP), and also over Hurricane-Electric tunnel. Same thing over IPv4 or IPv6. Going through a web proxy doesn't help either. tcpdump / wireshark shows that the three-way SYN handshake succeeds, then the client sends a GET, re-sends the packet several times, but nothing comes back any more. Or sometimes even the SYN handshake fails to complete. Tried to use curl to fetch the same file, it fails too: $ curl -6 http://update.FreeBSD.org/11.0-RC3/amd64/t/78e79429ffc2730cbb467270372d754165c6a0812805d9a0522d412b3e9b7d7e curl: (7) Failed to connect to update.FreeBSD.org port 80: Operation timed out $ curl -4 http://update.FreeBSD.org/11.0-RC3/amd64/t/78e79429ffc2730cbb467270372d754165c6a0812805d9a0522d412b3e9b7d7e curl: (56) Recv failure: Operation timed out So, do we just need to be patient, or is the update.FreeBSD.org hosed? Mark From owner-freebsd-stable@freebsd.org Wed Oct 12 15:17:51 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E9DBC0F7A6 for ; Wed, 12 Oct 2016 15:17:51 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f47.google.com (mail-lf0-f47.google.com [209.85.215.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE152CC1 for ; Wed, 12 Oct 2016 15:17:50 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f47.google.com with SMTP id l131so46686537lfl.2 for ; Wed, 12 Oct 2016 08:17:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=NCthbeygx3Gm7GeKSL/Dqm0XGzA2Cry3iKdPYiJVy7s=; b=lBXTBSp7WoDCUlOzQuIdu5sqmK3ts08mtPfNu8Ki0BQuxFaGjKOrrGFHr3ZmJ5vEf9 0mnHegAHyYTzGaoGq5o2mFIla1zVeUvHtSg5v2KE+JlvEIh9codPKaQUAOceG8MiymLs O5XZKHnnBM8aa4nL+D0EXz3oiPlem/tpRjJRpd07k93Y3x6vIVuZk3y7jUSM3bqjEgl3 iO4Sj1UdfGr8NvSW5wK+IymIvYWlw1CfHuC1lAsXr0MpLP2VtDiO1pjheQvjS64QoW4/ 6nrODUQzsx24ci8k9avKsz6nBVhODketAZ2VQpVz6cADn2r99D+VgXE3oQGl9uG03jYD hbYw== X-Gm-Message-State: AA6/9RmCCkg9A8AHvz6kd3S1OaW06sp1LxDO2939Z2mfzbZPDq2VJKDgaWNc+V6ML2lQmw== X-Received: by 10.25.29.1 with SMTP id d1mr2013538lfd.121.1476285462905; Wed, 12 Oct 2016 08:17:42 -0700 (PDT) Received: from [10.100.64.21] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id 87sm2393679lfs.0.2016.10.12.08.17.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 08:17:42 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Wed, 12 Oct 2016 17:17:35 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161012130103.GD57714@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="9RACJdhco9AOMMH2HacDKhuumGKC9UBNd" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 15:17:51 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9RACJdhco9AOMMH2HacDKhuumGKC9UBNd Content-Type: multipart/mixed; boundary="9bJTnMPbIV13a4h14mNLwvCpKSwoufKs7"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> In-Reply-To: <20161012130103.GD57714@zxy.spb.ru> --9bJTnMPbIV13a4h14mNLwvCpKSwoufKs7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/12/16 3:01 PM, Slawa Olhovchenkov wrote: > On Wed, Oct 12, 2016 at 02:35:11PM +0200, Julien Charbon wrote: >> On 10/12/16 2:13 PM, Slawa Olhovchenkov wrote: >>> On Wed, Oct 12, 2016 at 02:06:59PM +0200, Julien Charbon wrote: >>>>>>>>> sofree() call tcp_usr_detach() and in tcp_usr_detach() we have >>>>>>>>> unexpected INP_TIMEWAIT. >>>>>>>> >>>>>>>> I see, thus just for the context: The TCP stack in sys/dev/cxg= b* is a >>>>>>>> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a >>>>>>>> separate/side TCP stack that is used only with TCP_OFFLOAD optio= n. >>>>>>>> >>>>>>>> This TOE TCP stack actually has its own set of detach()/input()= >>>>>>>> functions and seems to check INP_DROPPED flag properly. I guess= @np >>>>>>>> check fixes in socket TCP stack and decides which one can also i= mpact >>>>>>>> the Chelsio TOE TCP stack. Some bugs are only in socket TCP sta= ck, some >>>>>>>> are only in TOE TCP stack. >>>>>>> >>>>>>> I am fear about other direction -- setting INP_TIMEWAIT in Chelsi= o TOE >>>>>>> TCP stack and impact this to >>>>>>> tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. >>>>>> >>>>>> I see, I expect no problem on this side as tcp_timer_2msl() check= s the >>>>>> INP_TIMEWAIT flag and do not call tcp_close() if set. >>>>> >>>>> I am about case when at time of first INP_WUNLOCK() tcp_timer_2msl(= ) >>>>> don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNLOC= K() >>>>> and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set >>>>> INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpected >>>>> INP_TIMEWAIT in tcp_usr_detach(). >>>> >>>> Sure, basically the same bug that in classic TCP stack. If you thi= nk >>>> it can happen, send an email describing that to np@ and he will chec= k >>>> and fix that. He is a TOE TCP stack expert and I am not. In all ca= ses, >>>> if this issue is possible in TOE TCP stack context, the patch will b= e >>>> straightforward: If the INP_DROPPED flag is set do not call tcp_tws= tart(). >>>> >>>> The current patch focuses only on the classic TCP stack. >>> >>> May be current workaround (with logging) in tcp_usr_detach() is good >>> solutuion for preventing system lockout by similar bugs? >> >> Good question, the quick workaround in tcp_usr_detach() does not hand= le >> all the cases. If it reduces the number of crashes you can still find= >> scenarios where it can have unexpected side effect. >=20 > This is best then guaranted lockout. >=20 >> Long term solution is to enforce: If the inp has the INP_DROPPED fla= g >> just stop processing it and return. If you grep the INP_DROPPED flag = in >> kernel sources, you can see that this test is already done in almost a= ll >> tcp_*() processing functions but tcp_input(). >> >> I would say that even without this issue tcp_input() should check >> INP_DROPPED flags after INP_WLOCK anyway. Same for the TOE TCP stack,= >> you are simply not supposed to process a inp with INP_DROPPED flag. >=20 > Absolutly acceptant! > May point is: more check and good handling of check result is best for > stability. >=20 > I.e. AND check INP_DROPPED in tcp_input AND workaroud INP_TIMEWAIT in > tcp_usr_detach (with logging) and check of some posible cases in XXX TO= E. >=20 > Current TCP stack too complex and have many corner cases. This is need > additional guards where posible (not caused kernel panic). I see your point: Even if this issue is caught by this assert: KASSERT(tp =3D=3D NULL, ("tcp_detach: INP_TIMEWAIT && " "INP_DROPPED && tp !=3D NULL")); https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/tcp_us= rreq.c#L213 you might not have INVARIANT option, then you will get a lockout quite difficult to debug. Thus what we can do is: - If INVARIANT is set: kernel panic to get all the details in the core.= - If INVARIANT is not set: Log this error with an explicit kernel log(LOG_ERR) describing the issue, and then use the workaround to avoid the double-free to let the system to good enough state. Something like: tcp_detach() { ... if (inp->inp_flags & INP_TIMEWAIT) { ... if (inp->inp_flags & INP_DROPPED) { in_pcbdetach(inp); if (__predict_true(tp =3D=3D NULL)) { in_pcbfree(inp); } else { #ifdef INVARIANTS panic("tcp_detach: tp !=3D NULL, That's not good because 'blah'\n= "); #else log(LOG_ERR, "tcp_detach: tp !=3D NULL, That's not good because 'blah'\n"); #endif INP_WUNLOCK(inp); } } } =2E.. } -- Julien --9bJTnMPbIV13a4h14mNLwvCpKSwoufKs7-- --9RACJdhco9AOMMH2HacDKhuumGKC9UBNd Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/lQVAAoJEKVlQ5Je6dhxjs4H/R2s88vWMX7pZf18nWtnvHhV bfSxX4ZTlwczbqsmzEhx8VdvwrbU1aZJsrBkFFqIV7ccxKxVdfQYxZajDqFLkShU a7VuqzYN5p+hNGkEgvt315KVRVl5ABTiFikKm2heMtvFnlrn3FO1HbuAyrVSdWlD QUw7+ecIU5RFpMJlc1VkRJPdSAKS+lCnZcfzvOdc5VHvwNSIW2atKXa3Wvw7nDcO XAACGSgXpeZRyi0+3iIhlc6+uwRIOFj9QdPso5vxx4Y9YTyI7scfdl1wxXi8AlOG fnhyBE6VhVf0DyIg9n6sddYFtwhR+eh4y501hNhKe20F8vSJbTEFVwTdznfupcs= =hbxA -----END PGP SIGNATURE----- --9RACJdhco9AOMMH2HacDKhuumGKC9UBNd-- From owner-freebsd-stable@freebsd.org Wed Oct 12 15:23:15 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39768C0FA20 for ; Wed, 12 Oct 2016 15:23:15 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA820129 for ; Wed, 12 Oct 2016 15:23:14 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.ijs.si (Postfix) with ESMTPS id 3svHgP3GXXzNX for ; Wed, 12 Oct 2016 17:23:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1476285789; x=1478877790; bh=nR3 vk/ZaM2pMRqKDccVunjCZtbPMImlo8KZj8R8qwJk=; b=VzkGongWETwUFn9D9I/ HV7Xa6llv3Vvtx+G1dHmgtV3AaVZKP3FfK/U55pyitS/CBuZmmBTLIhVK3aAubK5 NenTGtsoqLnLbAqOXbt4zzrifPiWbw1QBSDHSewJNcCBvAKaavzqlE+92S5hnn2C 8DeYVBmgGDjYvjKmVH7qfemQ= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id 4AT5GaReLRwR for ; Wed, 12 Oct 2016 17:23:09 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 3svHgK4wNtzNW for ; Wed, 12 Oct 2016 17:23:09 +0200 (CEST) Received: from nabiralnik.ijs.si (nabiralnik.ijs.si [IPv6:2001:1470:ff80::80:16]) by mildred.ijs.si (Postfix) with ESMTP id 3svHgK45vlzNN for ; Wed, 12 Oct 2016 17:23:09 +0200 (CEST) Received: from neli.ijs.si (2001:1470:ff80:88:21c:c0ff:feb1:8c91) by webmail.ijs.si with HTTP (HTTP/1.1 POST); Wed, 12 Oct 2016 17:23:09 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 12 Oct 2016 17:23:09 +0200 From: Mark Martinec To: freebsd-stable@freebsd.org Subject: Re: update.FreeBSD.org unresponsive? Organization: Jozef Stefan Institute In-Reply-To: References: Message-ID: X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.2.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 15:23:15 -0000 Whatever you did, it started to work now normally. Thank you! (no changes at our side) Mark 2016-10-12 16:29, Mark Martinec wrote: > Trying to upgrade a couple of hosts (11.0-RC2, 11.0-RC3, > 10.3-RELEASE-p10) > to 11.0 (using: freebsd-update upgrade -r 11.0-RELEASE), and it seems > the fetch(1) always fails with a timeout. Even a simple (freebsd-update > fetch) > in an attempt to bump a 10.3-RELEASE-p9 to 10.3-RELEASE-p10 now fails > with a timeout, while previously it worked reliably and fast. > > The interesting thing is that both the ping and ping6 to > update.FreeBSD.org > work flawlessly with no packet loss. > > I tried it several times yesterday, and again today. Our network > connectivity is otherwise good and fast. To rule out a possibility > of a firewall or routing issue I even tried it from a different > network (different ISP), and also over Hurricane-Electric tunnel. > Same thing over IPv4 or IPv6. Going through a web proxy doesn't > help either. > > tcpdump / wireshark shows that the three-way SYN handshake succeeds, > then the client sends a GET, re-sends the packet several times, > but nothing comes back any more. Or sometimes even the SYN handshake > fails to complete. > > Tried to use curl to fetch the same file, it fails too: > > $ curl -6 > http://update.FreeBSD.org/11.0-RC3/amd64/t/78e79429ffc2730cbb467270372d754165c6a0812805d9a0522d412b3e9b7d7e > curl: (7) Failed to connect to update.FreeBSD.org port 80: Operation > timed out > > $ curl -4 > http://update.FreeBSD.org/11.0-RC3/amd64/t/78e79429ffc2730cbb467270372d754165c6a0812805d9a0522d412b3e9b7d7e > curl: (56) Recv failure: Operation timed out > > > So, do we just need to be patient, or is the update.FreeBSD.org hosed? > > Mark From owner-freebsd-stable@freebsd.org Wed Oct 12 15:42:32 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40F93C0E07E for ; Wed, 12 Oct 2016 15:42:32 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2F35182; Wed, 12 Oct 2016 15:42:31 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buLfp-00023w-LU; Wed, 12 Oct 2016 18:42:29 +0300 Date: Wed, 12 Oct 2016 18:42:29 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161012154229.GC57876@zxy.spb.ru> References: <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 15:42:32 -0000 On Wed, Oct 12, 2016 at 05:17:35PM +0200, Julien Charbon wrote: > >>>>>>>> I see, thus just for the context: The TCP stack in sys/dev/cxgb* is a > >>>>>>>> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a > >>>>>>>> separate/side TCP stack that is used only with TCP_OFFLOAD option. > >>>>>>>> > >>>>>>>> This TOE TCP stack actually has its own set of detach()/input() > >>>>>>>> functions and seems to check INP_DROPPED flag properly. I guess @np > >>>>>>>> check fixes in socket TCP stack and decides which one can also impact > >>>>>>>> the Chelsio TOE TCP stack. Some bugs are only in socket TCP stack, some > >>>>>>>> are only in TOE TCP stack. > >>>>>>> > >>>>>>> I am fear about other direction -- setting INP_TIMEWAIT in Chelsio TOE > >>>>>>> TCP stack and impact this to > >>>>>>> tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. > >>>>>> > >>>>>> I see, I expect no problem on this side as tcp_timer_2msl() checks the > >>>>>> INP_TIMEWAIT flag and do not call tcp_close() if set. > >>>>> > >>>>> I am about case when at time of first INP_WUNLOCK() tcp_timer_2msl() > >>>>> don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNLOCK() > >>>>> and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set > >>>>> INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpected > >>>>> INP_TIMEWAIT in tcp_usr_detach(). > >>>> > >>>> Sure, basically the same bug that in classic TCP stack. If you think > >>>> it can happen, send an email describing that to np@ and he will check > >>>> and fix that. He is a TOE TCP stack expert and I am not. In all cases, > >>>> if this issue is possible in TOE TCP stack context, the patch will be > >>>> straightforward: If the INP_DROPPED flag is set do not call tcp_twstart(). I am email to np@ > >>>> The current patch focuses only on the classic TCP stack. > >>> > >>> May be current workaround (with logging) in tcp_usr_detach() is good > >>> solutuion for preventing system lockout by similar bugs? > >> > >> Good question, the quick workaround in tcp_usr_detach() does not handle > >> all the cases. If it reduces the number of crashes you can still find > >> scenarios where it can have unexpected side effect. > > > > This is best then guaranted lockout. > > > >> Long term solution is to enforce: If the inp has the INP_DROPPED flag > >> just stop processing it and return. If you grep the INP_DROPPED flag in > >> kernel sources, you can see that this test is already done in almost all > >> tcp_*() processing functions but tcp_input(). > >> > >> I would say that even without this issue tcp_input() should check > >> INP_DROPPED flags after INP_WLOCK anyway. Same for the TOE TCP stack, > >> you are simply not supposed to process a inp with INP_DROPPED flag. > > > > Absolutly acceptant! > > May point is: more check and good handling of check result is best for > > stability. > > > > I.e. AND check INP_DROPPED in tcp_input AND workaroud INP_TIMEWAIT in > > tcp_usr_detach (with logging) and check of some posible cases in XXX TOE. > > > > Current TCP stack too complex and have many corner cases. This is need > > additional guards where posible (not caused kernel panic). > > I see your point: Even if this issue is caught by this assert: > > KASSERT(tp == NULL, ("tcp_detach: INP_TIMEWAIT && " > "INP_DROPPED && tp != NULL")); > https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/tcp_usrreq.c#L213 > > you might not have INVARIANT option, then you will get a lockout quite > difficult to debug. Thus what we can do is: > > - If INVARIANT is set: kernel panic to get all the details in the core. > - If INVARIANT is not set: Log this error with an explicit kernel > log(LOG_ERR) describing the issue, and then use the workaround to avoid > the double-free to let the system to good enough state. > > Something like: Yes, thanks! > tcp_detach() { > > ... > if (inp->inp_flags & INP_TIMEWAIT) { > > ... > if (inp->inp_flags & INP_DROPPED) { > > in_pcbdetach(inp); > if (__predict_true(tp == NULL)) { > in_pcbfree(inp); > } else { > #ifdef INVARIANTS > panic("tcp_detach: tp != NULL, That's not good because 'blah'\n"); > #else > log(LOG_ERR, "tcp_detach: tp != NULL, That's not good because > 'blah'\n"); May be some more info in log can help to detect root cause of issuse? I am don't know what info, may be flags or number of references? > #endif > INP_WUNLOCK(inp); > } > } > } > > ... > > } > > -- > Julien > From owner-freebsd-stable@freebsd.org Wed Oct 12 17:32:20 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79A4DC0F315 for ; Wed, 12 Oct 2016 17:32:20 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-lf0-x22a.google.com (mail-lf0-x22a.google.com [IPv6:2a00:1450:4010:c07::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E511CE9; Wed, 12 Oct 2016 17:32:20 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-lf0-x22a.google.com with SMTP id b75so85969278lfg.3; Wed, 12 Oct 2016 10:32:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=1WbpoDfVjWL1x9WXB+GxeJ1TCvH4ckqy+Im3/UsmoiI=; b=cGo6X2LGrzmLcOLA4tFokWpUx0GojtUPSOCsai78MTxkEHdhRea+dlqPhOXS7yzelF 1Q18NdZNOoyItFfIU0BQK4jwKkaWisNltjExP5FvkVBF1LdPQm3TIk6mFFlFEFniIOUe twG6iE8eUXhgSCjZjRwyG1y5/xSXifOTomLqhlCT2BAaSF4OmFMHjR9OwXSBH21znWW6 FM1v4dskgV4X3yge/CpUHqiU6vJVgP5QdJmMC0uK+b+2ga34HTwiFXksbQUIea4stN3/ XnUW7PNBMfyAVOCsVsyR+YH1STIQZLnk5gaqJDCmLLTaunrLeX8PGFkIUZi+GfxcgvgG cGBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=1WbpoDfVjWL1x9WXB+GxeJ1TCvH4ckqy+Im3/UsmoiI=; b=a79PhgevpPSHnAeOwwmDq9T2n5qcFYG/jSy/NW6NnCLAdWeyVK0M75beUG7GX8Tokl isKDplWTLjGOKmAVoxP/oW/NfdgCTADiPSraqssFlW8fqOBb9sp1MdaL5s+CFjKtPUeo tFk+E583Zx0pqCjKP4stDcMtgAUKJOV+IJFwQ3rvhICdjGTHxJxCCWnoQ8VgcTtaFAGo krUvcuvKu1mjFhvdnk7xUN3d+lIWOqDNqUe8KOpobS+W6Udg8093fipbX87oNjfnNjBT DhldIzw1BdmdIdh4Bdx2Mv2LrdIq11K/9xSvWJfr93+vvFF2buSbmv1ZmtjJ5CRe/iq9 Rl5g== X-Gm-Message-State: AA6/9Rm/2Hp/PC7WvEeg4tB9WQOefo1zHNyocsjr11ZknwuAzFzUrD2Y6md5o1SgW6Kr/g== X-Received: by 10.25.32.69 with SMTP id g66mr2940237lfg.15.1476293538091; Wed, 12 Oct 2016 10:32:18 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [12.32.117.8]) by smtp.googlemail.com with ESMTPSA id f198sm2569430lfe.10.2016.10.12.10.32.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 10:32:17 -0700 (PDT) Sender: Navdeep Parhar Subject: Re: 11.0 stuck on high network load To: Julien Charbon , Slawa Olhovchenkov References: <20161006111043.GH54003@zxy.spb.ru> <1431484c-c00e-24c5-bd76-714be8ae5ed5@freebsd.org> <20161010133220.GU54003@zxy.spb.ru> <23f1200e-383e-befb-b76d-c88b3e1287b0@freebsd.org> <20161010142941.GV54003@zxy.spb.ru> <52d634aa-639c-bef7-1f10-c46dbadc4d85@freebsd.org> <20161010173531.GI6177@zxy.spb.ru> <8143cd8f-c007-2378-b004-b2b037402d03@freebsd.org> <20161011121145.GJ6177@zxy.spb.ru> <20161012084045.GA57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Navdeep Parhar Message-ID: <121fe05f-37fb-9a65-1241-9206aed1e77a@FreeBSD.org> Date: Wed, 12 Oct 2016 10:32:14 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 17:32:20 -0000 > I see, thus just for the context: The TCP stack in sys/dev/cxgb* is a > TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a > separate/side TCP stack that is used only with TCP_OFFLOAD option. > > This TOE TCP stack actually has its own set of detach()/input() > functions and seems to check INP_DROPPED flag properly. I guess @np > check fixes in socket TCP stack and decides which one can also impact Yes, I do keep an eye on the changes in the stack and keep the TOE drivers up to date. The good part is that those drivers are trying to do the exact same thing with the locks and various bits of state as the software stack so they don't have any special requirements. If any patch comes out of this discussion I'll update the TOE drivers (if needed). btw, if you're looking at TOE drivers then read the sys/dev/cxgbe/t4_tom code (instead of the old T3 chip's cxgb/ulp/tom) as that's the most actively maintained and tested. Regards, Navdeep From owner-freebsd-stable@freebsd.org Wed Oct 12 18:13:36 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0F2BC0FB16 for ; Wed, 12 Oct 2016 18:13:36 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "StartCom Class 1 DV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AB8394 for ; Wed, 12 Oct 2016 18:13:36 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.14.9) with ESMTPS id u9CID6QN006680 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 12 Oct 2016 20:13:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.14.9/Submit) with UUCP id u9CID6Wj006679 for freebsd-stable@FreeBSD.ORG; Wed, 12 Oct 2016 20:13:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.14.9/8.14.9) with ESMTP id u9CHv29M008622 for ; Wed, 12 Oct 2016 19:57:02 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.14.9/8.14.9) with ESMTP id u9CHs4op008155 for ; Wed, 12 Oct 2016 19:54:04 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.14.9/8.14.9/Submit) id u9CHs4In008154 for freebsd-stable@FreeBSD.ORG; Wed, 12 Oct 2016 19:54:04 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: ZFS l2arc broken in 10.3 Date: Wed, 12 Oct 2016 19:46:02 +0200 Organization: even some more stinky socks Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Date: Wed, 12 Oct 2016 17:46:02 +0000 (UTC) Injection-Info: oper.dinoex.de; logging-data="6707"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 X-Mozilla-News-Host: news://localhost:119 Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (uucp.dinoex.sub.de [194.45.71.2]); Wed, 12 Oct 2016 20:13:07 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 18:13:37 -0000 details to follow From owner-freebsd-stable@freebsd.org Wed Oct 12 21:13:15 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32932C0EA06 for ; Wed, 12 Oct 2016 21:13:15 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "StartCom Class 1 DV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A5644C08 for ; Wed, 12 Oct 2016 21:13:14 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.14.9) with ESMTPS id u9CLD6G0042198 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 12 Oct 2016 23:13:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.14.9/Submit) with UUCP id u9CLD6aU042197 for freebsd-stable@FreeBSD.ORG; Wed, 12 Oct 2016 23:13:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.14.9/8.14.9) with ESMTP id u9CK5nWd022319 for ; Wed, 12 Oct 2016 22:05:49 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.14.9/8.14.9) with ESMTP id u9CK428d022120 for ; Wed, 12 Oct 2016 22:04:03 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.14.9/8.14.9/Submit) id u9CK42DC022116 for freebsd-stable@FreeBSD.ORG; Wed, 12 Oct 2016 22:04:02 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: [fixed] ZFS l2arc broken in 10.3 Date: Wed, 12 Oct 2016 21:54:23 +0200 Organization: even some more stinky socks Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------050104010405000902000902" Injection-Date: Wed, 12 Oct 2016 19:54:23 +0000 (UTC) Injection-Info: oper.dinoex.de; logging-data="20686"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 In-Reply-To: Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (uucp.dinoex.sub.de [194.45.71.2]); Wed, 12 Oct 2016 23:13:07 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 21:13:15 -0000 This is a multi-part message in MIME format. --------------050104010405000902000902 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit sendbug seems not to work anymore, I end up on websites with marketing- babble and finally get asked to provide some login and passwd. :( But the former mail looks like having come back to me, so it seems I'm still allowed to post here... --------------050104010405000902000902 Content-Type: text/plain; charset=UTF-8; name="zfs-l2arc" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="zfs-l2arc" KioqIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvYXJj LmMub3JpZwlXZWQgT2N0IDEyIDIxOjA3OjI1IDIwMTYKLS0tIHN5cy9jZGRsL2NvbnRyaWIv b3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMvYXJjLmMJV2VkIE9jdCAxMiAyMTo0Njox NiAyMDE2CioqKioqKioqKioqKioqKgoqKiogNjUwOCw2NTE0ICoqKioKICAJCQkgKi8KICAJ CQlidWZfc3ogPSBoZHItPmJfc2l6ZTsKICAJCQlhbGlnbiA9IChzaXplX3QpMSA8PCBkZXYt PmwyYWRfdmRldi0+dmRldl9hc2hpZnQ7CiEgCQkJYnVmX2Ffc3ogPSBQMlJPVU5EVVAoYnVm X3N6LCBhbGlnbik7CiAgCiAgCQkJaWYgKCh3cml0ZV9hc2l6ZSArIGJ1Zl9hX3N6KSA+IHRh cmdldF9zeikgewogIAkJCQlmdWxsID0gQl9UUlVFOwotLS0gNjUwOCw2NTE0IC0tLS0KICAJ CQkgKi8KICAJCQlidWZfc3ogPSBoZHItPmJfc2l6ZTsKICAJCQlhbGlnbiA9IChzaXplX3Qp MSA8PCBkZXYtPmwyYWRfdmRldi0+dmRldl9hc2hpZnQ7CiEgCQkJYnVmX2Ffc3ogPSBQMlJP VU5EVVBfVFlQRUQoYnVmX3N6LCBhbGlnbiwgdWludDY0X3QpOwogIAogIAkJCWlmICgod3Jp dGVfYXNpemUgKyBidWZfYV9zeikgPiB0YXJnZXRfc3opIHsKICAJCQkJZnVsbCA9IEJfVFJV RTsK --------------050104010405000902000902-- From owner-freebsd-stable@freebsd.org Wed Oct 12 21:13:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1277EC0EA08 for ; Wed, 12 Oct 2016 21:13:16 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "StartCom Class 1 DV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A998AC09 for ; Wed, 12 Oct 2016 21:13:15 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.14.9) with ESMTPS id u9CLD7tr042205 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 12 Oct 2016 23:13:07 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.14.9/Submit) with UUCP id u9CLD7B4042204 for freebsd-stable@FreeBSD.ORG; Wed, 12 Oct 2016 23:13:07 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.14.9/8.14.9) with ESMTP id u9CKQfZ8005558 for ; Wed, 12 Oct 2016 22:26:41 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.14.9/8.14.9) with ESMTP id u9CKO8De005148 for ; Wed, 12 Oct 2016 22:24:09 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.14.9/8.14.9/Submit) id u9CKO89b005147 for freebsd-stable@FreeBSD.ORG; Wed, 12 Oct 2016 22:24:08 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: Re: ZFS l2arc broken in 10.3 Date: Wed, 12 Oct 2016 22:18:44 +0200 Organization: even some more stinky socks Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Date: Wed, 12 Oct 2016 20:18:44 +0000 (UTC) Injection-Info: oper.dinoex.de; logging-data="4323"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 In-Reply-To: Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (uucp.dinoex.sub.de [194.45.71.2]); Wed, 12 Oct 2016 23:13:08 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 21:13:16 -0000 Details: After upgrading 2 machines from 9.3 to 10.3-STABLE, on one of them the l2arc stays empty (capacity alloc = 0), although it is online and gets accessed. It did work well on 9.3. I did the following tests: * Create a zpool on a stick, with two volumes: one filesystem and one cache. The cache stays with alloc=0. Export it and move it into the other machine. The cache immediately fills. Move it back, the cache stays with alloc=0. -> this rules out all zpool/zfs get/set options, as they should walk with the pool. * Boot the GENERIC kernel. l2arc stays with alloc=0. -> this rules out all my nonstandard kernel options. * Boot in single user mode. l2arc stays with alloc=0. -> this rules out all /etc/* config files. * Delete the zpool.cache and reimport pools. l2arc stays with alloc=0. * Copy the /boot/loader.conf settings to the other machine. The l2arc still works there. I could not think of any remaining place where this could come from, except the kernel code itself. From there, I found these counters nicely incrementing each second: kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 50758 kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 27121 kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 40589375488 But also this counter incrementing: kstat.zfs.misc.arcstats.l2_write_full: 14604 Then with some printf in the code I saw these values provided: buf_sz = hdr->b_size; align = (size_t)1 << dev->l2ad_vdev->vdev_ashift; buf_a_sz = P2ROUNDUP(buf_sz, align); if ((write_asize + buf_a_sz) > target_sz) { full = B_TRUE; mutex_exit(hash_lock); ARCSTAT_BUMP(arcstat_l2_write_full); break; } buf_sz = 1536 align = 512 buf_a_sz = 18446744069414585856 write_asize = 0 target_sz = 16777216 where buf_a_sz is obviousely off by (2^64 - 2^32). Maybe this is an effect of crosscompiling i386 on amd64. But anyway, as long as i386 is still supported, it should not happen. Now, my real concern is: if this really obvious ... made it undetected until 10.3, how many other missing typecasts are still in the code?? From owner-freebsd-stable@freebsd.org Wed Oct 12 23:16:32 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6291BC0F6B6 for ; Wed, 12 Oct 2016 23:16:32 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [193.2.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 16CAAD90 for ; Wed, 12 Oct 2016 23:16:31 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.ijs.si (Postfix) with ESMTPS id 3svV4J1xghzCj for ; Thu, 13 Oct 2016 01:12:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1476313912; x=1478905913; bh=FFD XW+LOKdXmRVeYjMAbRqguF4qJfVGT4N+pOasPdsA=; b=fJAC/CDNO9SfzyGuI31 uxMCKmxK2NfipCWiJ353IVkZXE7c67bA/qd1OomE09Vll9fHfA8qD7pXzJOO9hcr GOM1maPVz0TCAOMU7rUc1OU94z7Wrtyxmw0QJzqOcl7pcMhQuKcEiq5wxU6rkq8F JG4lFiU1/ucBpQf6hZxZKc7U= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id nUWf0-gRcbF7 for ; Thu, 13 Oct 2016 01:11:52 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 3svV4851fCzCh for ; Thu, 13 Oct 2016 01:11:51 +0200 (CEST) Received: from nabiralnik.ijs.si (nabiralnik.ijs.si [IPv6:2001:1470:ff80::80:16]) by mildred.ijs.si (Postfix) with ESMTP id 3svV473lnDzGm for ; Thu, 13 Oct 2016 01:11:51 +0200 (CEST) Received: from sleepy.ijs.si (2001:1470:ff80:e001::1:1) by webmail.ijs.si with HTTP (HTTP/1.1 POST); Thu, 13 Oct 2016 01:11:51 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 13 Oct 2016 01:11:51 +0200 From: Mark Martinec To: freebsd-stable@freebsd.org Subject: Re: update.FreeBSD.org unresponsive? Organization: Jozef Stefan Institute In-Reply-To: References: Message-ID: <6bd02094b3cc341424b36791e4b9327b@mailbox.ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.2.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2016 23:16:32 -0000 Perhaps it's time to replace Apache httpd/2.2.16 (released 6+ years ago) running on update.FreeBSD.org with something lighter and more agile like nginx (or at least with a fresher version of Apache httpd). The accf_http(9) (with: accept_filter=httpready) may help too. Mark 2016-10-12 17:23, Mark Martinec wrote: > Whatever you did, it started to work now normally. Thank you! > (no changes at our side) > Mark > > > 2016-10-12 16:29, Mark Martinec wrote: >> Trying to upgrade a couple of hosts (11.0-RC2, 11.0-RC3, >> 10.3-RELEASE-p10) >> to 11.0 (using: freebsd-update upgrade -r 11.0-RELEASE), and it seems >> the fetch(1) always fails with a timeout. Even a simple >> (freebsd-update fetch) >> in an attempt to bump a 10.3-RELEASE-p9 to 10.3-RELEASE-p10 now fails >> with a timeout, while previously it worked reliably and fast. >> >> The interesting thing is that both the ping and ping6 to >> update.FreeBSD.org >> work flawlessly with no packet loss. >> I tried it several times yesterday, and again today. [...] From owner-freebsd-stable@freebsd.org Thu Oct 13 04:04:13 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB7EDC0F224 for ; Thu, 13 Oct 2016 04:04:13 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A843330C for ; Thu, 13 Oct 2016 04:04:13 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-io0-x22b.google.com with SMTP id i202so72013344ioi.2 for ; Wed, 12 Oct 2016 21:04:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=LQGVy4gbhsSBx5dGEZb+zf3N+duA/IlN9lchieGch5U=; b=UX6Mf5gGMIxE2E54o4Dat6SIMjUt7Xqmczqk540D8FgEqsXuNvlNlh6KsPvrahWfIU 0O6QopDT+ZMPCKLBfju0Z8GI51ghyI7R0md7Zk2y78dcgKxLJf3h5T9AH8dpuD4HMT3J ClB/UhEzoHDz64KnKnqhka5JwpmjBDleZh2MYYWkY5+PFFLdRtU/Y/DUjHcVS9utPg9s cD9gEjtyurIiO/j0lPRfvvBaFnvrQarnp+HRFqhZk0cYe6XgF43wiNza70K+DFy2FU87 KJAu9AXJR2f4z6BcU1BRke3zmytN96RJ07A36DVNQguHvO/fIU0sCzlbtjVfPhptrUrE VuJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=LQGVy4gbhsSBx5dGEZb+zf3N+duA/IlN9lchieGch5U=; b=dHPnOXp/yc7/akiErP6u1WllLY1giRWihmudND4b1T9Jt54g/cAQbl9cSJZStHERhN eB+U24sep6a2FnH3KsCP5WkiB8w75VMI3ZjyigV684UlM8SU7noi0S6+734S8xcmVTEL q97oALVz7DjPUGabkgx5fepwXgc0CX0RVC7G4aAvzzPatFSgzLCUS94lQDspuzOOowbj 9WPdxvnh/lv6PA2zbJmaDnxLFJAV6oO0ljmXjP3iATAMq6BmDqQUFsp3n8CWCksAe9VB R9nR82PaVTCglyOiVwCOgOsozkH7HBlyzRPUteGZMNUE/6U9KQdGylazZ0XwvHeUVpCn /A1A== X-Gm-Message-State: AA6/9RkVcicMDlPO5HhpAgKh/wsIibuRGr0FsD5Ct0XOl77nKexIe0qpxfKHv0r6598Y5buqMOzwK2FK1Lx7GA== X-Received: by 10.107.37.212 with SMTP id l203mr4896169iol.101.1476331452957; Wed, 12 Oct 2016 21:04:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.87.140 with HTTP; Wed, 12 Oct 2016 21:04:12 -0700 (PDT) From: Aryeh Friedman Date: Thu, 13 Oct 2016 00:04:12 -0400 Message-ID: Subject: 11-RELEASE PL1 doesn't load kernel modules with ZFS-on-Root To: FreeBSD Stable List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 04:04:13 -0000 FreeBSD thin.lan.fnwe.net 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 root@thin:~ # df -k Filesystem 1024-blocks Used Avail Capacity Mounted on zroot/ROOT/default 927326328 8035732 919290596 1% / devfs 1 1 0 100% /dev zroot/tmp 919319840 29244 919290596 0% /tmp zroot/usr/home 919290736 140 919290596 0% /usr/home zroot/usr/ports 925242208 5951612 919290596 1% /usr/ports zroot/usr/src 920521220 1230624 919290596 0% /usr/src zroot/var/audit 919290692 96 919290596 0% /var/audit zroot/var/crash 919290692 96 919290596 0% /var/crash zroot/var/log 919290816 220 919290596 0% /var/log zroot/var/mail 919290724 128 919290596 0% /var/mail zroot/var/tmp 919290692 96 919290596 0% /var/tmp zroot 919290692 96 919290596 0% /zroot root@thin:~ # kldstat Id Refs Address Size Name 1 10 0xffffffff80200000 1fa7c38 kernel 2 1 0xffffffff821a9000 30aec0 zfs.ko 3 2 0xffffffff824b4000 adc0 opensolaris.ko root@thin:~ # kldload linux kldload: an error occurred while loading the module. Please check dmesg(8) for more details. -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-stable@freebsd.org Thu Oct 13 07:27:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72F2CC0F793 for ; Thu, 13 Oct 2016 07:27:05 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B03E9B39 for ; Thu, 13 Oct 2016 07:27:04 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA03693; Thu, 13 Oct 2016 10:27:02 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1buaPu-000J85-Et; Thu, 13 Oct 2016 10:27:02 +0300 Subject: Re: 11-RELEASE PL1 doesn't load kernel modules with ZFS-on-Root To: Aryeh Friedman , FreeBSD Stable List References: From: Andriy Gapon Message-ID: <1a717aad-66a5-4c78-4d3e-c600e4793dee@FreeBSD.org> Date: Thu, 13 Oct 2016 10:26:26 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 07:27:05 -0000 On 13/10/2016 07:04, Aryeh Friedman wrote: > Please check dmesg(8) > for more details. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Thu Oct 13 07:32:58 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF395C0FB59 for ; Thu, 13 Oct 2016 07:32:58 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 77EB82F9; Thu, 13 Oct 2016 07:32:58 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-io0-x22e.google.com with SMTP id j37so76187556ioo.3; Thu, 13 Oct 2016 00:32:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=NGEsjLNU86umrD+A31CrqX2Nu4qNC4LkxrhjSYvYa3w=; b=Xkv47NpWqIGftrdphdz9Wm7rJwF0z7Oh2/6reqc266oOhz3pNiTvLmtKGMs8Uj4N6p rFUAlv3z7Qn3NYOTvB/qy8wCXIY/KYOYqRXRErg9+McUI4kSae2g/eMds4wW8iYHakt0 zueFMaRYhOj2GrOzWNdimV5wcQg29RxHwFzquT4jrlUx78zk6+rJWCumgOFvWvLvXMyL htTjqdrwCScBK4wXJQy0D6DMAg1cbCSLi1hsUEhDwJzjOTv9/AKl50WZpQdrcyh/U0Qb En/SyLEXrC8TL+WOK5ggRpzbmIm3+vcBHeG4UJ7hcTLjShkno6yhJFT+L2PN7qIIXf6u aq4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=NGEsjLNU86umrD+A31CrqX2Nu4qNC4LkxrhjSYvYa3w=; b=Gf4Nw+rnMhB0HQ80K9NkCDyedcb9s1ODxx5j6HPqPvjFkM5hMGZqMns+vSOXs1nZxg 5L6hUqINP/CUfP0gRXnFJk9HgnW1JrHV6BrA9OSfoDQDQlmHjJJ0ffazk6lPaiR/ZJcI jK1TV9LLakYcRJkIyvr/0+o8FjtmQ/eQM3h9eySRBCo/Us4y5aQjjjqK2/SDaDwi/p+e lH1AYOG0cd5D6HecjhNj6GnGi4ocwb37sEjRsyS8EuwJYnBkNjfVicD7sHds8RJr/bbG dsofuP3GqyCOrsGtaZoeF95a9DJfAzOz3t/DMnaYIh1V2tONHSnmaMr+inX43vb0xYno w75Q== X-Gm-Message-State: AA6/9RnuA2CwQTVn4EDC+0tjFEmVvt4B5J36ZwDoqhUYhQfFHx+yaSVAzyPr/o0ETLXjIoLCx6VC8woYvYo0Xw== X-Received: by 10.107.17.229 with SMTP id 98mr5175168ior.96.1476343977865; Thu, 13 Oct 2016 00:32:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.87.140 with HTTP; Thu, 13 Oct 2016 00:32:57 -0700 (PDT) In-Reply-To: References: <20161013053918.GE51420@home.opsec.eu> From: Aryeh Friedman Date: Thu, 13 Oct 2016 03:32:57 -0400 Message-ID: Subject: Fwd: 11-RELEASE PL1 doesn't load kernel modules with ZFS-on-Root To: FreeBSD Stable List , Andriy Gapon Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 07:32:58 -0000 ---------- Forwarded message ---------- From: Aryeh Friedman Date: Thu, Oct 13, 2016 at 1:59 AM Subject: Re: 11-RELEASE PL1 doesn't load kernel modules with ZFS-on-Root To: Kurt Jaeger On Thu, Oct 13, 2016 at 1:39 AM, Kurt Jaeger wrote: > Hi! > > > root@thin:~ # kldload linux > > kldload: an error occurred while loading the module. Please check > dmesg(8) > > for more details. > > What does the dmesg log show ? > KLD linux.ko: depends on kernel - not available or version mismatch linker_load_file: Unsupported file type Fresh install from FreeBSD-11.0-RELEASE-amd64-bootonly.iso. Source tree generated with a svn checkout aryehl@thin:~ % ls -lt /boot/kernel/linux.ko -r-xr-xr-x 1 root wheel 582728 Oct 11 11:26 /boot/kernel/linux.ko aryehl@thin:~ % uname -a FreeBSD thin.lan.fnwe.net 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 On Thu, Oct 13, 2016 at 2:31 AM, Kurt Jaeger wrote: Hi! > KLD linux.ko: depends on kernel - not available or version mismatch > linker_load_file: Unsupported file type What does uname -a say ? What does file /boot/kernel/linux.ko say ? What does ls -lt /boot/kernel/ show ? Maybe the file was not replaced during upgrade and is still on some previous version ? -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-stable@freebsd.org Thu Oct 13 09:59:53 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBF39C0EADC for ; Thu, 13 Oct 2016 09:59:53 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (ingresso-1-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:411::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AAB0D227 for ; Thu, 13 Oct 2016 09:59:53 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6]) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bucnn-000EIS-MZ for freebsd-stable@freebsd.org; Thu, 13 Oct 2016 09:59:51 +0000 Subject: Re: ZFS l2arc broken in 10.3 To: freebsd-stable@freebsd.org References: From: Pete French Message-ID: <530d4222-535e-b29d-aba7-9520644dc6e5@ingresso.co.uk> Date: Thu, 13 Oct 2016 10:59:51 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 09:59:53 -0000 Ok, thats a bit worry if true - but I can confirm that l2arc works fine under 10.3 on amd64, so what you say about cross-compling might be true. Am taking an inetrest in this as I have just dpeloyed a lot of machines which are going to be relying on l2arc working to get reasobale performance. -pete. On 10/12/16 21:18, Peter wrote: > Details: > After upgrading 2 machines from 9.3 to 10.3-STABLE, on one of them the > l2arc stays empty (capacity alloc = 0), although it is online and gets > accessed. It did work well on 9.3. > > I did the following tests: > * Create a zpool on a stick, with two volumes: one filesystem and one > cache. The cache stays with alloc=0. > Export it and move it into the other machine. The cache immediately > fills. > Move it back, the cache stays with alloc=0. > -> this rules out all zpool/zfs get/set options, as they should > walk with the pool. > * Boot the GENERIC kernel. l2arc stays with alloc=0. > -> this rules out all my nonstandard kernel options. > * Boot in single user mode. l2arc stays with alloc=0. > -> this rules out all /etc/* config files. > * Delete the zpool.cache and reimport pools. l2arc stays with alloc=0. > * Copy the /boot/loader.conf settings to the other machine. The l2arc > still works there. > > I could not think of any remaining place where this could come from, > except the kernel code itself. > From there, I found these counters nicely incrementing each second: > kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 50758 > kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 27121 > kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 40589375488 > But also this counter incrementing: > kstat.zfs.misc.arcstats.l2_write_full: 14604 > > Then with some printf in the code I saw these values provided: > buf_sz = hdr->b_size; > align = (size_t)1 << dev->l2ad_vdev->vdev_ashift; > buf_a_sz = P2ROUNDUP(buf_sz, align); > if ((write_asize + buf_a_sz) > target_sz) { > full = B_TRUE; > mutex_exit(hash_lock); > ARCSTAT_BUMP(arcstat_l2_write_full); > break; > } > > buf_sz = 1536 > align = 512 > buf_a_sz = 18446744069414585856 > write_asize = 0 > target_sz = 16777216 > > where buf_a_sz is obviousely off by (2^64 - 2^32). > > Maybe this is an effect of crosscompiling i386 on amd64. But anyway, as > long as i386 is still supported, it should not happen. > > > Now, my real concern is: if this really obvious ... made it undetected > until 10.3, how many other missing typecasts are still in the code?? > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu Oct 13 11:24:25 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF247C0E748 for ; Thu, 13 Oct 2016 11:24:25 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 357DFB19 for ; Thu, 13 Oct 2016 11:24:24 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA04922; Thu, 13 Oct 2016 14:24:22 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bue7a-000JOZ-F0; Thu, 13 Oct 2016 14:24:22 +0300 Subject: Re: ZFS l2arc broken in 10.3 To: Peter , freebsd-stable@FreeBSD.org References: From: Andriy Gapon Message-ID: <32ee460e-64ba-2270-d587-d27007bbb3ab@FreeBSD.org> Date: Thu, 13 Oct 2016 14:23:25 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 11:24:26 -0000 On 12/10/2016 23:18, Peter wrote: > Details: > After upgrading 2 machines from 9.3 to 10.3-STABLE, on one of them the > l2arc stays empty (capacity alloc = 0), although it is online and gets > accessed. It did work well on 9.3. > > I did the following tests: > * Create a zpool on a stick, with two volumes: one filesystem and one > cache. The cache stays with alloc=0. > Export it and move it into the other machine. The cache immediately > fills. > Move it back, the cache stays with alloc=0. > -> this rules out all zpool/zfs get/set options, as they should > walk with the pool. > * Boot the GENERIC kernel. l2arc stays with alloc=0. > -> this rules out all my nonstandard kernel options. > * Boot in single user mode. l2arc stays with alloc=0. > -> this rules out all /etc/* config files. > * Delete the zpool.cache and reimport pools. l2arc stays with alloc=0. > * Copy the /boot/loader.conf settings to the other machine. The l2arc > still works there. > > I could not think of any remaining place where this could come from, > except the kernel code itself. > From there, I found these counters nicely incrementing each second: > kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 50758 > kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 27121 > kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 40589375488 > But also this counter incrementing: > kstat.zfs.misc.arcstats.l2_write_full: 14604 > > Then with some printf in the code I saw these values provided: > buf_sz = hdr->b_size; > align = (size_t)1 << dev->l2ad_vdev->vdev_ashift; > buf_a_sz = P2ROUNDUP(buf_sz, align); > if ((write_asize + buf_a_sz) > target_sz) { > full = B_TRUE; > mutex_exit(hash_lock); > ARCSTAT_BUMP(arcstat_l2_write_full); > break; > } > > buf_sz = 1536 > align = 512 > buf_a_sz = 18446744069414585856 > write_asize = 0 > target_sz = 16777216 > > where buf_a_sz is obviousely off by (2^64 - 2^32). > > Maybe this is an effect of crosscompiling i386 on amd64. Yes, the problem is specific to 32-bit platforms where size_t is 32-bit. > But anyway, as long as > i386 is still supported, it should not happen. Certainly. > Now, my real concern is: if this really obvious ... made it undetected until > 10.3, how many other missing typecasts are still in the code?? No need to be dramatic here. That particular piece code is very new. I committed it to head in April (r297848), MFC-ed even later. Apparently no one else who uses 32-bit systems and has L2ARC configured had a chance to run into the bug. Thank you very much for discovering and analyzing the bug and providing a fix for it! -- Andriy Gapon From owner-freebsd-stable@freebsd.org Thu Oct 13 12:01:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F28DC0FB8E for ; Thu, 13 Oct 2016 12:01:59 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 072BC666 for ; Thu, 13 Oct 2016 12:01:58 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-qk0-f177.google.com with SMTP id n189so85029374qke.0 for ; Thu, 13 Oct 2016 05:01:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=mZ4RJDpkoQln62QShN+/m/XifmxBTrvMFO33zBvLKa8=; b=ddfgUGheDi45kpXLXTQwzoMxhmLOsIDXMsyMu1GHLyFlPLPYfztNa8Da+nceaTgGv1 IV4Oxyi3erXDDn3iALrXdlnbr53LYyABoMiwnkyaReSp3pclEzcGBENDM728D2QX84br G863qPTz/gnYiDnTYk0VOJhFPY/Nh0HMYk6gFZagDvGX9tIuxcZpeI6l+PmssXdO+2xf 46L7+ZrNWn49NYyzjln8PMcrzq9DWFYYqdmLzvO503eDTsYVSdMRIPoAXiSRdfFG+KfU ZKvxUHnJTnIauffjU/Ul/ip7y5UnJXo5tcotTpIm8OH2AXOy02xVuSE3ZUOWk23zmewb 2Ejg== X-Gm-Message-State: AA6/9RnoiKF2Q76TRVaDqgYm1iUfiEbneJhSOck8NRxTHs7tgkX8flfgx8OLPvPpEXjvlA== X-Received: by 10.55.140.131 with SMTP id o125mr5632449qkd.17.1476359785368; Thu, 13 Oct 2016 04:56:25 -0700 (PDT) Received: from [10.100.64.26] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id a94sm4574417qkh.11.2016.10.13.04.56.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Oct 2016 04:56:24 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Thu, 13 Oct 2016 13:56:21 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161012154229.GC57876@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="IaBdSCtk01RvXNmT49NC0DLBOsIESQDPV" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 12:01:59 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --IaBdSCtk01RvXNmT49NC0DLBOsIESQDPV Content-Type: multipart/mixed; boundary="3JqlvwLibke83UhlriI3GVwN2NOMKIGGQ"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20161012084045.GA57714@zxy.spb.ru> <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> In-Reply-To: <20161012154229.GC57876@zxy.spb.ru> --3JqlvwLibke83UhlriI3GVwN2NOMKIGGQ Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/12/16 5:42 PM, Slawa Olhovchenkov wrote: > On Wed, Oct 12, 2016 at 05:17:35PM +0200, Julien Charbon wrote: >=20 >>>>>>>>>> I see, thus just for the context: The TCP stack in sys/dev/c= xgb* is a >>>>>>>>>> TOE (TCP Offload Engine?) TCP stack for Chelsio NICs, it is a >>>>>>>>>> separate/side TCP stack that is used only with TCP_OFFLOAD opt= ion. >>>>>>>>>> >>>>>>>>>> This TOE TCP stack actually has its own set of detach()/input= () >>>>>>>>>> functions and seems to check INP_DROPPED flag properly. I gue= ss @np >>>>>>>>>> check fixes in socket TCP stack and decides which one can also= impact >>>>>>>>>> the Chelsio TOE TCP stack. Some bugs are only in socket TCP s= tack, some >>>>>>>>>> are only in TOE TCP stack. >>>>>>>>> >>>>>>>>> I am fear about other direction -- setting INP_TIMEWAIT in Chel= sio TOE >>>>>>>>> TCP stack and impact this to >>>>>>>>> tcp_timer_2msl()/tcp_close()/sofree()/tcp_usr_detach() path. >>>>>>>> >>>>>>>> I see, I expect no problem on this side as tcp_timer_2msl() che= cks the >>>>>>>> INP_TIMEWAIT flag and do not call tcp_close() if set. >>>>>>> >>>>>>> I am about case when at time of first INP_WUNLOCK() tcp_timer_2ms= l() >>>>>>> don't see INP_TIMEWAIT, call tcp_close(), tcp_close() do INP_WUNL= OCK() >>>>>>> and now Chelsio TOE take INP_WLOCK, do tcp_twstart() and set >>>>>>> INP_TIMEWAIT. After this tcp_timer_2msl resume and have unexpecte= d >>>>>>> INP_TIMEWAIT in tcp_usr_detach(). >>>>>> >>>>>> Sure, basically the same bug that in classic TCP stack. If you t= hink >>>>>> it can happen, send an email describing that to np@ and he will ch= eck >>>>>> and fix that. He is a TOE TCP stack expert and I am not. In all = cases, >>>>>> if this issue is possible in TOE TCP stack context, the patch will= be >>>>>> straightforward: If the INP_DROPPED flag is set do not call tcp_t= wstart(). >=20 > I am email to np@ >=20 >>>>>> The current patch focuses only on the classic TCP stack. >>>>> >>>>> May be current workaround (with logging) in tcp_usr_detach() is goo= d >>>>> solutuion for preventing system lockout by similar bugs? >>>> >>>> Good question, the quick workaround in tcp_usr_detach() does not ha= ndle >>>> all the cases. If it reduces the number of crashes you can still fi= nd >>>> scenarios where it can have unexpected side effect. >>> >>> This is best then guaranted lockout. >>> >>>> Long term solution is to enforce: If the inp has the INP_DROPPED f= lag >>>> just stop processing it and return. If you grep the INP_DROPPED fla= g in >>>> kernel sources, you can see that this test is already done in almost= all >>>> tcp_*() processing functions but tcp_input(). >>>> >>>> I would say that even without this issue tcp_input() should check >>>> INP_DROPPED flags after INP_WLOCK anyway. Same for the TOE TCP stac= k, >>>> you are simply not supposed to process a inp with INP_DROPPED flag. >>> >>> Absolutly acceptant! >>> May point is: more check and good handling of check result is best fo= r >>> stability. >>> >>> I.e. AND check INP_DROPPED in tcp_input AND workaroud INP_TIMEWAIT in= >>> tcp_usr_detach (with logging) and check of some posible cases in XXX = TOE. >>> >>> Current TCP stack too complex and have many corner cases. This is nee= d >>> additional guards where posible (not caused kernel panic). >> >> I see your point: Even if this issue is caught by this assert: >> >> KASSERT(tp =3D=3D NULL, ("tcp_detach: INP_TIMEWAIT && " >> "INP_DROPPED && tp !=3D NULL")); >> https://github.com/freebsd/freebsd/blob/release/11.0.0/sys/netinet/tcp= _usrreq.c#L213 >> >> you might not have INVARIANT option, then you will get a lockout quit= e >> difficult to debug. Thus what we can do is: >> >> - If INVARIANT is set: kernel panic to get all the details in the co= re. >> - If INVARIANT is not set: Log this error with an explicit kernel >> log(LOG_ERR) describing the issue, and then use the workaround to avoi= d >> the double-free to let the system to good enough state. >> >> Something like: >=20 > Yes, thanks! Proposed changes added in the review: https://reviews.freebsd.org/D8211 tell me when you have three days without issue with this change. >> tcp_detach() { >> >> ... >> if (inp->inp_flags & INP_TIMEWAIT) { >> >> ... >> if (inp->inp_flags & INP_DROPPED) { >> >> in_pcbdetach(inp); >> if (__predict_true(tp =3D=3D NULL)) { >> in_pcbfree(inp); >> } else { >> #ifdef INVARIANTS >> panic("tcp_detach: tp !=3D NULL, That's not good because 'blah= '\n"); >> #else >> log(LOG_ERR, "tcp_detach: tp !=3D NULL, That's not good becaus= e >> 'blah'\n"); >=20 > May be some more info in log can help to detect root cause of issuse? > I am don't know what info, may be flags or number of references? For this kind of issue, the useful part is the stacktrace. INVARIANT will give you that trace in the core, and without INVARIANT then it is better to use dtrace: $ cat tcp-twstart-dropped.d fbt::tcp_twstart:entry /args[0]->t_inpcb->inp_flags & 0x04000000/ { stack(); printf("INP_DROPPED in tcp_twstart: %x", args[0]->t_inpcb->inp_flags); } -- Julien --3JqlvwLibke83UhlriI3GVwN2NOMKIGGQ-- --IaBdSCtk01RvXNmT49NC0DLBOsIESQDPV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/3ZpAAoJEKVlQ5Je6dhxffgIAOs9QX6B0NiH5oaiTPhODLEX MRP9AcYRipoFEw4xIv07mI/Bqy8uAV062ZsYxjC3OppAH+bkNqIHX8exdpz5Hrct MvFzBtAWU5/t7zk6B4ZokLerUo5r9F6rkAjqgCFLK/WoU2EesH77GX1JFlo18qSn RqArn8F4AvvKFonKut21KChOoUB3VJ9fYKLOrcwheC8LRApTyPxQy6KyITxq8+p3 xxi1oc44TO0WK78OzWj1w9/tzw61oxaEmEmWDm2LNAkOEuF2fkq5aLUjPGrkaBVj xg1pXtbEipUK4gtHxqbTmz4dMsdEzZcZrH0w5tpVhodi988d7njzOpku/tJgsUE= =XhjT -----END PGP SIGNATURE----- --IaBdSCtk01RvXNmT49NC0DLBOsIESQDPV-- From owner-freebsd-stable@freebsd.org Thu Oct 13 13:13:24 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDC79C10506 for ; Thu, 13 Oct 2016 13:13:24 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "StartCom Class 1 DV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D961197 for ; Thu, 13 Oct 2016 13:13:24 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.14.9) with ESMTPS id u9DDDAHo044764 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 13 Oct 2016 15:13:11 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.14.9/Submit) with UUCP id u9DDDAsL044763 for freebsd-stable@FreeBSD.ORG; Thu, 13 Oct 2016 15:13:10 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.14.9/8.14.9) with ESMTP id u9DCOrH7034986 for ; Thu, 13 Oct 2016 14:24:53 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.14.9/8.14.9) with ESMTP id u9DCO22S034839 for ; Thu, 13 Oct 2016 14:24:03 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.14.9/8.14.9/Submit) id u9DCO26F034838 for freebsd-stable@FreeBSD.ORG; Thu, 13 Oct 2016 14:24:02 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: Re: ZFS l2arc broken in 10.3 Date: Thu, 13 Oct 2016 14:16:36 +0200 Organization: even some more stinky socks Message-ID: References: <530d4222-535e-b29d-aba7-9520644dc6e5@ingresso.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Date: Thu, 13 Oct 2016 12:17:10 +0000 (UTC) Injection-Info: oper.dinoex.de; logging-data="33908"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 In-Reply-To: <530d4222-535e-b29d-aba7-9520644dc6e5@ingresso.co.uk> Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (uucp.dinoex.sub.de [194.45.71.2]); Thu, 13 Oct 2016 15:13:11 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 13:13:25 -0000 Pete French wrote: > > Ok, thats a bit worry if true - but I can confirm that l2arc works fine > under 10.3 on amd64, so what you say about cross-compling might be true. > Am taking an inetrest in this as I have just dpeloyed a lot of machines > which are going to be relying on l2arc working to get reasobale performance. Sure on my amd64 it also works fine. AFAIK such things are tolerated when compiling in 64bit. But I was pointed to another point interim: my source is from STABLE branch; in the 10.3 RELEASE the code is different. Obviousely there were recent changes, and that explains why the problem was not yet detected. From owner-freebsd-stable@freebsd.org Thu Oct 13 14:38:36 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A56ECC10D19 for ; Thu, 13 Oct 2016 14:38:36 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 63D608B3; Thu, 13 Oct 2016 14:38:36 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buh9N-000EyH-JC; Thu, 13 Oct 2016 17:38:25 +0300 Date: Thu, 13 Oct 2016 17:38:25 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161013143825.GK57714@zxy.spb.ru> References: <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 14:38:36 -0000 On Thu, Oct 13, 2016 at 01:56:21PM +0200, Julien Charbon wrote: > >> Something like: > > > > Yes, thanks! > > Proposed changes added in the review: > > https://reviews.freebsd.org/D8211 > > tell me when you have three days without issue with this change. > > >> tcp_detach() { > >> > >> ... > >> if (inp->inp_flags & INP_TIMEWAIT) { > >> > >> ... > >> if (inp->inp_flags & INP_DROPPED) { > >> > >> in_pcbdetach(inp); > >> if (__predict_true(tp == NULL)) { > >> in_pcbfree(inp); > >> } else { > >> #ifdef INVARIANTS > >> panic("tcp_detach: tp != NULL, That's not good because 'blah'\n"); > >> #else > >> log(LOG_ERR, "tcp_detach: tp != NULL, That's not good because > >> 'blah'\n"); > > > > May be some more info in log can help to detect root cause of issuse? > > I am don't know what info, may be flags or number of references? > > For this kind of issue, the useful part is the stacktrace. INVARIANT Like this? #ifdef KDB kdb_backtrace(); #endif as found in sys/netgraph/ng_base.c > will give you that trace in the core, and without INVARIANT then it is > better to use dtrace: > > $ cat tcp-twstart-dropped.d > fbt::tcp_twstart:entry > /args[0]->t_inpcb->inp_flags & 0x04000000/ > { > stack(); > printf("INP_DROPPED in tcp_twstart: %x", args[0]->t_inpcb->inp_flags); > } Same code may be insert there too, IMHO. > -- > Julien > > From owner-freebsd-stable@freebsd.org Thu Oct 13 15:06:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90133C0E985 for ; Thu, 13 Oct 2016 15:06:18 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-qk0-f178.google.com (mail-qk0-f178.google.com [209.85.220.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47675F27 for ; Thu, 13 Oct 2016 15:06:17 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-qk0-f178.google.com with SMTP id n189so95296169qke.0 for ; Thu, 13 Oct 2016 08:06:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=rh1+rbd2zgGGduc6WjISZr8tOL4qyXq3XWCYXd7DlAI=; b=j4QZ41insy6LLmsOmI1Ry5f5D+avtcmwOCRUMOIWAAfLRCJQBFYaMBKcj4vs+2qoar EiOLNyVH+wbJ+fBlZ1GHNJUt4LRDIcpm1s82WcTNBJHPuwGNGkkDZwYM+Wo/0WQWyi2L JE6+5vPxfYSp2B7LrJO76ldgw1dBXcE0a/f8V5rOqZJltLD7uzvfO6w0GoGhKaZ2DDij gxDmu4PkiFJQy4V0RO9GzrCJcRxjUT67rAYfVOBYMkrzOAw+lsJXdaaRFshLNXiIIkm+ oFnlYNzEX7MqtmDVIksFP64wfOCdVQ/AsrexXbnQqOUz22YOPNZ9iz0PXeT2/COOtTQb L3Ow== X-Gm-Message-State: AA6/9RmeSNIW6bqMr4L9gPqQaRJd0OZNZEyx//lYJAQNt91xh3Aav9HBidnhb1IGFgA33A== X-Received: by 10.194.141.233 with SMTP id rr9mr7087673wjb.18.1476371163975; Thu, 13 Oct 2016 08:06:03 -0700 (PDT) Received: from [10.100.64.26] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id y2sm23172921wjx.20.2016.10.13.08.06.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Oct 2016 08:06:02 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> Date: Thu, 13 Oct 2016 17:06:00 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161013143825.GK57714@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="a85w2Ti06SRN3wnPEEmo3Nli51ICXgk05" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 15:06:18 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --a85w2Ti06SRN3wnPEEmo3Nli51ICXgk05 Content-Type: multipart/mixed; boundary="eS5gOKKhWQfOwIqLPd6aU4a7ATAckBMFo"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20161012092945.GB57714@zxy.spb.ru> <4b0d4b58-6d13-3cd5-6991-27163f27acca@freebsd.org> <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> In-Reply-To: <20161013143825.GK57714@zxy.spb.ru> --eS5gOKKhWQfOwIqLPd6aU4a7ATAckBMFo Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi Slawa, On 10/13/16 4:38 PM, Slawa Olhovchenkov wrote: > On Thu, Oct 13, 2016 at 01:56:21PM +0200, Julien Charbon wrote: >>>> Something like: >>> >>> Yes, thanks! >> >> Proposed changes added in the review: >> >> https://reviews.freebsd.org/D8211 >> >> tell me when you have three days without issue with this change. >> >>>> tcp_detach() { >>>> >>>> ... >>>> if (inp->inp_flags & INP_TIMEWAIT) { >>>> >>>> ... >>>> if (inp->inp_flags & INP_DROPPED) { >>>> >>>> in_pcbdetach(inp); >>>> if (__predict_true(tp =3D=3D NULL)) { >>>> in_pcbfree(inp); >>>> } else { >>>> #ifdef INVARIANTS >>>> panic("tcp_detach: tp !=3D NULL, That's not good because 'bl= ah'\n"); >>>> #else >>>> log(LOG_ERR, "tcp_detach: tp !=3D NULL, That's not good beca= use >>>> 'blah'\n"); >>> >>> May be some more info in log can help to detect root cause of issuse?= >>> I am don't know what info, may be flags or number of references? >> >> For this kind of issue, the useful part is the stacktrace. INVARIANT= >=20 > Like this? >=20 > #ifdef KDB > kdb_backtrace(); > #endif >=20 > as found in sys/netgraph/ng_base.c It is overkill dtrace can do that. >> will give you that trace in the core, and without INVARIANT then it is= >> better to use dtrace: >> >> $ cat tcp-twstart-dropped.d >> fbt::tcp_twstart:entry >> /args[0]->t_inpcb->inp_flags & 0x04000000/ >> { >> stack(); >> printf("INP_DROPPED in tcp_twstart: %x", args[0]->t_inpcb->inp_flags= ); >> } >=20 > Same code may be insert there too, IMHO. Hmm, I don't think so: - If you have INVARIANT, the kernel will panic in tcp_twstart() or tcp_detach() and you will have everything you need to debug. - If you don't, dtrace is the right tool to use in all cases anyway. -- Julien --eS5gOKKhWQfOwIqLPd6aU4a7ATAckBMFo-- --a85w2Ti06SRN3wnPEEmo3Nli51ICXgk05 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/6LcAAoJEKVlQ5Je6dhxlUAH/jHhMvvTBIoxoK8eodIPzE8m +5quGdpDvht+3mRfY6fGpYGCTT60rT3xMsN2L0xNJ8i6qXiY9oB48nTO8P9359Nk 5oolU1L7WCFeKAeTkmOCWim9YxTGT9f+4MQxVliKsMa3uDlR09RMj1JLucVLAXZ1 tbc6u96+AlhTZu21EzGiejv4otp+KbLEbDCmFS1jbufLd8tnN2A3S/olMzv4kadi plhwDu+wAXk5cu0hK8ETDwjlmVV+MXY7r2yAZp3jk5QXvh3wSbDK+2Re8WQpnaze Ee2N7zdibXVpOTCdACK02srQZlAASC8DwRy41cCLVV8mvmEoG8ijJxKYLXL0KBw= =oU3g -----END PGP SIGNATURE----- --a85w2Ti06SRN3wnPEEmo3Nli51ICXgk05-- From owner-freebsd-stable@freebsd.org Thu Oct 13 15:17:17 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92858C0C3DA for ; Thu, 13 Oct 2016 15:17:17 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 507B580F; Thu, 13 Oct 2016 15:17:17 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buhkx-000G4V-23; Thu, 13 Oct 2016 18:17:15 +0300 Date: Thu, 13 Oct 2016 18:17:15 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161013151715.GL57714@zxy.spb.ru> References: <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 15:17:17 -0000 On Thu, Oct 13, 2016 at 05:06:00PM +0200, Julien Charbon wrote: > >> will give you that trace in the core, and without INVARIANT then it is > >> better to use dtrace: > >> > >> $ cat tcp-twstart-dropped.d > >> fbt::tcp_twstart:entry > >> /args[0]->t_inpcb->inp_flags & 0x04000000/ > >> { > >> stack(); > >> printf("INP_DROPPED in tcp_twstart: %x", args[0]->t_inpcb->inp_flags); > >> } > > > > Same code may be insert there too, IMHO. > > Hmm, I don't think so: > > - If you have INVARIANT, the kernel will panic in tcp_twstart() or > tcp_detach() and you will have everything you need to debug. > - If you don't, dtrace is the right tool to use in all cases anyway. dtrace don't executed in may case w/ diagnostic "dtrace: processing aborted: Abort due to systemic unresponsiveness". This is for tcp_close. May be tcp_twstart will be more successuful, may be not. Also, using dtrace too complex in production (need complex startup under screen and capture output) and for many peoples. kdb_backtrace() have too less administrative overhead. From owner-freebsd-stable@freebsd.org Thu Oct 13 16:21:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3122C101B7 for ; Thu, 13 Oct 2016 16:21:47 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f54.google.com (mail-lf0-f54.google.com [209.85.215.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AEBE864 for ; Thu, 13 Oct 2016 16:21:46 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f54.google.com with SMTP id l131so104751084lfl.2 for ; Thu, 13 Oct 2016 09:21:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=all5m+BTK7kD5RX1fkFFwz3PP+wUgvlobKnibh+sRyA=; b=kWk7nwFYQv0faoMp8elkS1iuGLaiC4lPS8eOtoMs5YvsJ0BxnE2uACDGbPEXDkEk2A Vd6XDFCBwheOGN3A1jypiXFkLs2luHXFiLTKxmG0BWn4BbTR6trGulQdeB2N6aFWKYqG 7ayW61XzcowCeGYrjE1+dBn7ZPWKlhj5Trs8CbtfQr4SJ0td6+MEV9Ug9WbWYcr/uYUp xt5J2ddpphV0c0KUMejgRnIsFe4wSDBoWYIY4Q20sEaIoBns+hyDBbq6uOG9HvZ2ay4B fR1lfskHWv/po0utCIRnRTIQXw0eZKqOGX61NbMNjx8QLUPCPYc8yDiNPuMfPMravhBn dcCQ== X-Gm-Message-State: AA6/9Rm5a1jPL/dg+HZKaGTEYs724VGFwPO97XHx/r52t7zENMX76i+xlqV+JPKjzcFmHA== X-Received: by 10.25.20.228 with SMTP id 97mr1420974lfu.93.1476375277688; Thu, 13 Oct 2016 09:14:37 -0700 (PDT) Received: from [10.100.64.26] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id u68sm4018876lfg.31.2016.10.13.09.14.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Oct 2016 09:14:36 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> <20161013151715.GL57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: Date: Thu, 13 Oct 2016 18:14:29 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161013151715.GL57714@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="77t2wdg15S3ivEXxVVQvo5c8JV1DO52u6" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 16:21:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --77t2wdg15S3ivEXxVVQvo5c8JV1DO52u6 Content-Type: multipart/mixed; boundary="o2IrimuSsOFSk8pIWPSFxEP7aw5M0QFgs"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: Subject: Re: 11.0 stuck on high network load References: <20161012095233.GC57714@zxy.spb.ru> <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> <20161013151715.GL57714@zxy.spb.ru> In-Reply-To: <20161013151715.GL57714@zxy.spb.ru> --o2IrimuSsOFSk8pIWPSFxEP7aw5M0QFgs Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 10/13/16 5:17 PM, Slawa Olhovchenkov wrote: > On Thu, Oct 13, 2016 at 05:06:00PM +0200, Julien Charbon wrote: >=20 >>>> will give you that trace in the core, and without INVARIANT then it = is >>>> better to use dtrace: >>>> >>>> $ cat tcp-twstart-dropped.d >>>> fbt::tcp_twstart:entry >>>> /args[0]->t_inpcb->inp_flags & 0x04000000/ >>>> { >>>> stack(); >>>> printf("INP_DROPPED in tcp_twstart: %x", args[0]->t_inpcb->inp_fla= gs); >>>> } >>> >>> Same code may be insert there too, IMHO. >> >> Hmm, I don't think so: >> >> - If you have INVARIANT, the kernel will panic in tcp_twstart() or >> tcp_detach() and you will have everything you need to debug. >> - If you don't, dtrace is the right tool to use in all cases anyway. >=20 > dtrace don't executed in may case w/ diagnostic "dtrace: processing > aborted: Abort due to systemic unresponsiveness". This is for > tcp_close. May be tcp_twstart will be more successuful, may be not. It does and will. > Also, using dtrace too complex in production (need complex startup > under screen and capture output) and for many peoples. > kdb_backtrace() have too less administrative overhead. I still think it is overkill. The main goal of this change is to fix a quite tricky and old TCP stack locking issue. Let's try to do that first, it is complex enough by itself. Once the fix is validated and pushed, feel free to propose your own patch/review to add kdb_backtrace(), log(), etc.. to get other devs point of view. I don't remember who said: "Never ever optimize error cases"... -- Julien --o2IrimuSsOFSk8pIWPSFxEP7aw5M0QFgs-- --77t2wdg15S3ivEXxVVQvo5c8JV1DO52u6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJX/7LtAAoJEKVlQ5Je6dhxl0gH/1exUJnXktv5wXl3Pt9u41Sq stw7JS2C3l1TSk5m1cIw9U2vfD0Ok7iY/ksmXEnNtUlOB3MLF5Z954AVdghUtBt2 D7gGpcTYbxyiU8MxKsT6I4UO0aEQoQBkJqPDLQUMpS3QLc73tYzypkj8PO7ype6m HsY4ONga5GlP/WoeKovHmAxocXL8mgz+xYAbuovqtgy6yEhc1w6bRI9/u/5lONQb tjiOcgwZAB+erJqNXoZ/8WllgbwJ/ujBl3XzLXqMEvy82infmZ+luvaCkAACbbJV OWTeqiA8PW6+6MxGnhkaGu8iSf2X0tGLsfDck0ny9TZkO4xwEfxvIaszqMxj6Jw= =g/dO -----END PGP SIGNATURE----- --77t2wdg15S3ivEXxVVQvo5c8JV1DO52u6-- From owner-freebsd-stable@freebsd.org Thu Oct 13 18:32:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9144EC109F7; Thu, 13 Oct 2016 18:32:05 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx2.enfer-du-nord.net (mx2.enfer-du-nord.net [IPv6:2001:41d0:d:3049:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6037DD76; Thu, 13 Oct 2016 18:32:05 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e04:6101:b4e6:a589:6c42:4ad2] (p2003008C2E046101B4E6A5896C424AD2.dip0.t-ipconnect.de [IPv6:2003:8c:2e04:6101:b4e6:a589:6c42:4ad2]) by mx2.enfer-du-nord.net (Postfix) with ESMTPSA id 3svzpq55nlzQ9V; Thu, 13 Oct 2016 20:32:03 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: FreeBSD 11 : running blacklistd needed for 520.pfdenied? From: Michael Grimm In-Reply-To: Date: Thu, 13 Oct 2016 20:32:02 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <3EF5E845-A3D9-4802-B9DD-A788CB09197C@ellael.org> References: To: freebsd-questions@freebsd.org, freebsd-stable@freebsd.org X-Virus-Scanned: clamav-milter 0.99.2 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 18:32:05 -0000 Hi - On 15.08.2016, at 19:01, Michael Grimm wrote: > I recently upgraded from 10.3-STABLE to 11.0-PRERELEASE. Now, I am = missing those parts in my daily security report regarding pf, e.g.: >=20 > example.private pf denied packets: > +block drop in on ix0 all [ Evaluations: 12757684 Packets: = 133590 Bytes: 7477681 States: 0 ] > +block drop in log quick on ix0 from to any [ = Evaluations: 12754165 Packets: 3753 Bytes: 269612 States: 0 ] > +block drop quick on ix0 from any to [ Evaluations: = 790740 Packets: 873 Bytes: 295032 States: 0 ] >=20 > I do believe that those lines should be generated by = /etc/periodic/security/520.pfdenied (stripped to the relevant part): >=20 > TMP=3D`mktemp -t security`=20 > touch ${TMP}=20 > for _a in "" blacklistd=20 > do=20 > pfctl -a ${_a} -sr -v -z 2>/dev/null | \=20 > nawk '{if (/^block/) {buf=3D$0; getline; gsub(" +"," = ",$0); if ($5 > 0) print buf$0;} }' >> ${TMP}=20 > done=20 Well, one needs to add the "old" functionality of 10.3-STABLE's = /etc/periodic/security/520.pfdenied to get those lines reappear again. = The new script in 11-STABLE (and presumably 11-RELEASE) assumes a = running blacklistd which isn't necessarily the case in every = installation running pf firewalls. Patch: ++++++++++++++++++++++++++++++++++++++++++++++++++++++ SNIP = ++++++++++++++++++++++++++++++++++++++++++++++++++++++ --- 520.pfdenied 2016-08-15 18:59:11.532831000 +0200 +++ 520.pfdenied.new 2016-10-13 20:03:28.891362000 +0200 @@ -50,6 +50,8 @@ pfctl -a ${_a} -sr -v -z 2>/dev/null | \ nawk '{if (/^block/) {buf=3D$0; getline; gsub(" +"," = ",$0); if ($5 > 0) print buf$0;} }' >> ${TMP} done + pfctl -sr -v 2>/dev/null | \ + nawk '{if (/^block/) {buf=3D$0; getline; gsub(" +"," ",$0); if = ($5 > 0) print buf$0;} }' >> ${TMP} if [ -s ${TMP} ]; then check_diff new_only pf ${TMP} "${host} pf denied = packets:" fi ++++++++++++++++++++++++++++++++++++++++++++++++++++++ SNAP = ++++++++++++++++++++++++++++++++++++++++++++++++++++++ Regards, Michael From owner-freebsd-stable@freebsd.org Fri Oct 14 09:00:40 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED26DC112F5; Fri, 14 Oct 2016 09:00:40 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9ACFEF82; Fri, 14 Oct 2016 09:00:40 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id u9E90aqQ076254; Fri, 14 Oct 2016 11:00:37 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id A3008E9B; Fri, 14 Oct 2016 11:00:36 +0200 (CEST) Message-ID: <58009EB4.30708@omnilan.de> Date: Fri, 14 Oct 2016 11:00:36 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: FreeBSD Net , FreeBSD Stable Subject: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Fri, 14 Oct 2016 11:00:37 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 09:00:41 -0000 Dear all, I found great papers about netmap(4)s desigen and implementation details, and I'm sure it's one other masterpeace of rizzo-quality :-) Thanks to all participants for that great code! To be honest, I haven't read all of that, because I'm short in time and my first mission is to see if FreeBSD 11 will replace some of my ESXi machines. One key element seems netmap(4). It's quiet hard to find userland documentation. So far, I've discovered that there are three essential tools waiting in _usr/src/tools/tools/netmap_ to be compiled (resulting in *./vale-ctl*, *./bridge*, *./pkt-gen*) While the latter is often referenced in netmap(4) documentation, it's not of interest for me, because I'll be doing real-world performance tests and I'm convinced that all the impressive numbers presented in the netmap documentation are valid :-) So *vale-ctl(-8)* seems to be of interest (I'm using (-8) becaus currently there is no man8 part (I guess that's the reason for these tools not beeing integrated into base binaries)) Accidentally I found out that 'vale-ctl -n testif0' creates a artificial interface, which is reported by ifconfig(8): testif0: flags=8801 metric 0 mtu 1500 options=80000 ether 00:be:eb:8d:f8:00 nd6 options=21 But I can't assign a IP address: 'ifconfig testif0 203.0.113.1/24' ifconfig: ioctl (SIOCAIFADDR): Invalid argument I guess couldn't geti the picture of the netmap(4) world yet. Probably, testif0 is available only in netmap(4) world, not in "host world". I'm assuming, because I found vale-ctl(-8)s "-h" switch. So another very little peace I'm aware of the netmap(4) world, is how to attach physical interfaces to virtual switches: '/usr/src/tools/tools/netmap/vale-ctl -a vale0:em1' Now vale-ctl(-8) shows: bdg_ctl [149] bridge:0 port:0 vale0:em1 /* To share my experience: One cannot use any other than vale[[:digit:]] for defining the on-demand to be created virtual switch instance, so e.g. "vale-ctl -a vale-test:em1" doesn't work, although found in netmap(4) man page in FreeBSD-11: »valeXXX:YYY (arbitrary XXX and YYY) the file descriptor is bound to port YYY of a VALE switch called XXX, both dynamically created if necessary. The string cannot exceed IFNAMSIZ characters, and YYY cannot be the name of any existing OS network interface« I was about to give up on netmap(4) investigations because I thought it isn't production ready yet (in FreeBSD), since even andding the first physical interface fails: '/usr/src/tools/tools/netmap/vale-ctl -a vale-test:em1' vale-test:em1: Invalid argument Probably accidentally I used vale[[:digit:]] instead and wondered whay it suddenly works… To get back to vale-ctl(-8)s "-h" switch: */ If I add a physical interface with -h instead of -a, the host's IP stack doesn't get disconnected from the interface, so it's still usable by host applications and vale-ctl(-8) lists one line more: bdg_ctl [149] bridge:0 port:0 vale0:em1 bdg_ctl [149] bridge:0 port:1 vale0:em1^ So my assumption that netmap(4) lives decapsuled from the well known FreeBSD IP world. Now my question: How can I plug a jail's or vmm's artificial interface to a VALE virtual switch, bridging frames to real-world via physical interfaces? (the latter part should work with vale-ctl -h vale0:em1, but what interface to use for jail(8) vnet.interface and how to create/attach?) Thanks, -harry From owner-freebsd-stable@freebsd.org Fri Oct 14 09:35:51 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44E81C11E76 for ; Fri, 14 Oct 2016 09:35:51 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 048F4B93; Fri, 14 Oct 2016 09:35:51 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buyu3-000MMy-1k; Fri, 14 Oct 2016 12:35:47 +0300 Date: Fri, 14 Oct 2016 12:35:47 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161014093546.GN57714@zxy.spb.ru> References: <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> <20161013151715.GL57714@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 09:35:51 -0000 On Thu, Oct 13, 2016 at 06:14:29PM +0200, Julien Charbon wrote: > On 10/13/16 5:17 PM, Slawa Olhovchenkov wrote: > > On Thu, Oct 13, 2016 at 05:06:00PM +0200, Julien Charbon wrote: > > > >>>> will give you that trace in the core, and without INVARIANT then it is > >>>> better to use dtrace: > >>>> > >>>> $ cat tcp-twstart-dropped.d > >>>> fbt::tcp_twstart:entry > >>>> /args[0]->t_inpcb->inp_flags & 0x04000000/ > >>>> { > >>>> stack(); > >>>> printf("INP_DROPPED in tcp_twstart: %x", args[0]->t_inpcb->inp_flags); > >>>> } > >>> > >>> Same code may be insert there too, IMHO. > >> > >> Hmm, I don't think so: > >> > >> - If you have INVARIANT, the kernel will panic in tcp_twstart() or > >> tcp_detach() and you will have everything you need to debug. > >> - If you don't, dtrace is the right tool to use in all cases anyway. > > > > dtrace don't executed in may case w/ diagnostic "dtrace: processing > > aborted: Abort due to systemic unresponsiveness". This is for > > tcp_close. May be tcp_twstart will be more successuful, may be not. > > It does and will. > > > Also, using dtrace too complex in production (need complex startup > > under screen and capture output) and for many peoples. > > kdb_backtrace() have too less administrative overhead. > > I still think it is overkill. The main goal of this change is to fix a > quite tricky and old TCP stack locking issue. Let's try to do that > first, it is complex enough by itself. > > Once the fix is validated and pushed, feel free to propose your own > patch/review to add kdb_backtrace(), log(), etc.. to get other devs > point of view. > > I don't remember who said: "Never ever optimize error cases"... This is not optimeze error cases, this is error recovery and diagnostic of error cases in other subsystems. Currently FreeBSD internals too complex for just always trust on correct of other subsystem or do panic on any incosystency. INVARIANTS too expensive now (20Gbit drops to 8Gbits). PS: I am applay patch. Wait till monday. Thanks very match for this hard work! From owner-freebsd-stable@freebsd.org Fri Oct 14 09:57:08 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54254C0F682 for ; Fri, 14 Oct 2016 09:57:08 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com [209.85.215.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EBFE3883 for ; Fri, 14 Oct 2016 09:57:07 +0000 (UTC) (envelope-from julien.charbon@gmail.com) Received: by mail-lf0-f45.google.com with SMTP id l131so146113247lfl.2 for ; Fri, 14 Oct 2016 02:57:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=2GS/c1N4adglh8O8c8hOjwuNV2f+Gd6fb7HJt1nPi2I=; b=QvGrapeOX9QCcxsnsq/9GU9RcYoEsbAAC8V0frJv7l1i6cTTKg/YDxrRYnyNFlMuxz R9kbVhYYWfzyBe9vzBYtmC/x5pcPlyITbbE0cpz18himeQ+xyGXBq2negwZsB98UTPIP foQqaMZZdz3nKOgQIG70rNjb52LT7y5Fid5VW96HojHmelolscvoibNSEtrxowpL7L3i m3sgh9U9aVR+UGBW+15k9s5DdvKAAb3BttmeX/qICAnj1hN+gcRQMZoUX9jRisPuzsOO tf2+lrvNVM84ams/YtPGgXpt3CqYln8qa9sTZX2TDGYbYGHOFLC3hZTtq9Wu+0VIZez/ LQug== X-Gm-Message-State: AA6/9RkRWyfnVMGuFRDnhIUbdSs5SV9sBZplAvYfO7j7zPbYD1wwgfUtn3Zm5QuCwB0uzw== X-Received: by 10.25.139.86 with SMTP id n83mr2570752lfd.72.1476438525106; Fri, 14 Oct 2016 02:48:45 -0700 (PDT) Received: from [10.100.64.8] ([217.30.88.7]) by smtp.gmail.com with ESMTPSA id n14sm4860609lfn.7.2016.10.14.02.48.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Oct 2016 02:48:44 -0700 (PDT) Subject: Re: 11.0 stuck on high network load To: Slawa Olhovchenkov References: <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> <20161013151715.GL57714@zxy.spb.ru> <20161014093546.GN57714@zxy.spb.ru> Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara From: Julien Charbon Message-ID: <9f2eb420-897f-e231-834e-c8b256c81130@freebsd.org> Date: Fri, 14 Oct 2016 11:48:38 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161014093546.GN57714@zxy.spb.ru> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="oQtGu5WxAv55Vq1jCjCv1IBIOsQGk4tTP" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 09:57:08 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --oQtGu5WxAv55Vq1jCjCv1IBIOsQGk4tTP Content-Type: multipart/mixed; boundary="OqVWeOCDacKtisbe40sROktEM7EB10mp7"; protected-headers="v1" From: Julien Charbon To: Slawa Olhovchenkov Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Message-ID: <9f2eb420-897f-e231-834e-c8b256c81130@freebsd.org> Subject: Re: 11.0 stuck on high network load References: <20161012121322.GB57876@zxy.spb.ru> <62d8861c-673e-6d86-e96e-751399e505e5@freebsd.org> <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> <20161013151715.GL57714@zxy.spb.ru> <20161014093546.GN57714@zxy.spb.ru> In-Reply-To: <20161014093546.GN57714@zxy.spb.ru> --OqVWeOCDacKtisbe40sROktEM7EB10mp7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hi, On 10/14/16 11:35 AM, Slawa Olhovchenkov wrote: > On Thu, Oct 13, 2016 at 06:14:29PM +0200, Julien Charbon wrote: >> On 10/13/16 5:17 PM, Slawa Olhovchenkov wrote: >>> On Thu, Oct 13, 2016 at 05:06:00PM +0200, Julien Charbon wrote: >>> >>>>>> will give you that trace in the core, and without INVARIANT then i= t is >>>>>> better to use dtrace: >>>>>> >>>>>> $ cat tcp-twstart-dropped.d >>>>>> fbt::tcp_twstart:entry >>>>>> /args[0]->t_inpcb->inp_flags & 0x04000000/ >>>>>> { >>>>>> stack(); >>>>>> printf("INP_DROPPED in tcp_twstart: %x", args[0]->t_inpcb->inp_f= lags); >>>>>> } >>>>> >>>>> Same code may be insert there too, IMHO. >>>> >>>> Hmm, I don't think so: >>>> >>>> - If you have INVARIANT, the kernel will panic in tcp_twstart() or >>>> tcp_detach() and you will have everything you need to debug. >>>> - If you don't, dtrace is the right tool to use in all cases anyway= =2E >>> >>> dtrace don't executed in may case w/ diagnostic "dtrace: processing >>> aborted: Abort due to systemic unresponsiveness". This is for >>> tcp_close. May be tcp_twstart will be more successuful, may be not. >> >> It does and will. >> >>> Also, using dtrace too complex in production (need complex startup >>> under screen and capture output) and for many peoples. >>> kdb_backtrace() have too less administrative overhead. >> >> I still think it is overkill. The main goal of this change is to fix= a >> quite tricky and old TCP stack locking issue. Let's try to do that >> first, it is complex enough by itself. >> >> Once the fix is validated and pushed, feel free to propose your own >> patch/review to add kdb_backtrace(), log(), etc.. to get other devs >> point of view. >> >> I don't remember who said: "Never ever optimize error cases"... >=20 > This is not optimeze error cases, this is error recovery and > diagnostic of error cases in other subsystems. Sure, I guess this quote is more geared toward: "Always spend 50x more time on improving the main path than the error path". > Currently FreeBSD internals too complex for just always trust on > correct of other subsystem or do panic on any incosystency. >=20 > INVARIANTS too expensive now (20Gbit drops to 8Gbits). I do agree. I am not expert enough to see all the side effects of calling kdb_backtrace() from the TCP stack, might be way too slow, tricky in interruption context, etc. You can see that kdb_backtrace() is rarely called in the kernel source. That's why it is better if you propose a review on adding this line to get comments from other devs on just this question. > PS: I am applay patch. Wait till monday. >=20 > Thanks very match for this hard work! No problem, thanks for your time. But it is not over yet: We have to wait for final test. -- Julien --OqVWeOCDacKtisbe40sROktEM7EB10mp7-- --oQtGu5WxAv55Vq1jCjCv1IBIOsQGk4tTP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJYAKn6AAoJEKVlQ5Je6dhxz3YH/3ucJr741f6lrt7Pk367/lX+ PwduNL80rE+WfLsp47UCbk3yUinpWjaTk5VzjA/IUKcQ8SsNS7JLbaEfhCL3WYfI RhJDdiwU3rljVMhAFA58NJgOwXX+RVlA/kw5ba38FGxUO9xGMwx3eZd8H/d3tz0r rFVf1ng+RnTVuTj8Lhm/REiEvx9KhIT/sVW2iQquHriK582Js7XQ9ZBi5UfbkalQ ueYAsM+5PIjSVKShSL4ZaE5HSZ0sqTLVxIXcpPUmweH3wzjOvW5uwT+XWxtwFCur hLTlTqi4TGjmT8NhnWd/hAglsjQ0VuPLoA9i6uC+uto5sqqqdAB/VvtF9P2mzpU= =BTqZ -----END PGP SIGNATURE----- --oQtGu5WxAv55Vq1jCjCv1IBIOsQGk4tTP-- From owner-freebsd-stable@freebsd.org Fri Oct 14 10:02:25 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C61EC0F9DD for ; Fri, 14 Oct 2016 10:02:25 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B23ED33; Fri, 14 Oct 2016 10:02:25 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1buzJm-000N6x-Na; Fri, 14 Oct 2016 13:02:22 +0300 Date: Fri, 14 Oct 2016 13:02:22 +0300 From: Slawa Olhovchenkov To: Julien Charbon Cc: Konstantin Belousov , freebsd-stable@FreeBSD.org, hiren panchasara Subject: Re: 11.0 stuck on high network load Message-ID: <20161014100222.GO57714@zxy.spb.ru> References: <20161012130103.GD57714@zxy.spb.ru> <20161012154229.GC57876@zxy.spb.ru> <20161013143825.GK57714@zxy.spb.ru> <33ab0bfc-7009-95a7-7752-c2c439092e85@freebsd.org> <20161013151715.GL57714@zxy.spb.ru> <20161014093546.GN57714@zxy.spb.ru> <9f2eb420-897f-e231-834e-c8b256c81130@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9f2eb420-897f-e231-834e-c8b256c81130@freebsd.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 10:02:25 -0000 On Fri, Oct 14, 2016 at 11:48:38AM +0200, Julien Charbon wrote: > >>> Also, using dtrace too complex in production (need complex startup > >>> under screen and capture output) and for many peoples. > >>> kdb_backtrace() have too less administrative overhead. > >> > >> I still think it is overkill. The main goal of this change is to fix a > >> quite tricky and old TCP stack locking issue. Let's try to do that > >> first, it is complex enough by itself. > >> > >> Once the fix is validated and pushed, feel free to propose your own > >> patch/review to add kdb_backtrace(), log(), etc.. to get other devs > >> point of view. > >> > >> I don't remember who said: "Never ever optimize error cases"... > > > > This is not optimeze error cases, this is error recovery and > > diagnostic of error cases in other subsystems. > > Sure, I guess this quote is more geared toward: "Always spend 50x more > time on improving the main path than the error path". > > > Currently FreeBSD internals too complex for just always trust on > > correct of other subsystem or do panic on any incosystency. > > > > INVARIANTS too expensive now (20Gbit drops to 8Gbits). > > I do agree. I am not expert enough to see all the side effects of > calling kdb_backtrace() from the TCP stack, might be way too slow, > tricky in interruption context, etc. You can see that kdb_backtrace() I think about this. This is example take from netgraph and this similar case (about interruption context and etc). Occurrence to rare (one per day, may be one per two hour) for any overhead. OK, I am see you point: you expirence don't allow to put this code and need separete review and commit. Right, np. > is rarely called in the kernel source. That's why it is better if you > propose a review on adding this line to get comments from other devs on > just this question. > > > PS: I am applay patch. Wait till monday. > > > > Thanks very match for this hard work! > > No problem, thanks for your time. But it is not over yet: We have to > wait for final test. Currently system don't use Chelsio TOE, after monday I am update system with Chelsio TOE. With chelsio I am see this occurrence very rare, one in few month. From owner-freebsd-stable@freebsd.org Fri Oct 14 13:08:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E50C5C11728; Fri, 14 Oct 2016 13:08:56 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x241.google.com (mail-oi0-x241.google.com [IPv6:2607:f8b0:4003:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8BE3391; Fri, 14 Oct 2016 13:08:56 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x241.google.com with SMTP id d132so7502769oib.2; Fri, 14 Oct 2016 06:08:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DzbAULD0FK8r0nJDWyxtYJQ0zBLG9Erw+7jm9OkOCPI=; b=pNb3r3tP4i14GiQbQkKCAkg1k//VU0L0+ub47TseVSAQ9K3Cct8Y+3CQYP2a7gzKH/ a6F029BJIqyAZYsSvJuUljsuznJT/TF5c4BdwL/OkvyeoNhEqPIkg/Rls8r+oYk9+5Q3 Hl+5kHqzvbAJDObtL0cyK1QYTvhNCHRAnHK60pL7Eta2ytbYi6G3xwQHo7zqVbeuz9Pu VygcnnACANnJvFKQRFHZn/Y/OO+swr9Wto94aDgOydeYZN7DDnaq79M9Pqhz/FTo+MPz CvKtW6sIoTVg7ji+Bm9nkoOZfa6OnBaL2NTuxi5PpwIFiB30iHxaTilu7C6eJsxBjsW6 PP4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DzbAULD0FK8r0nJDWyxtYJQ0zBLG9Erw+7jm9OkOCPI=; b=lDuhyaIKhys0KT6AigvXF5CSewSLCi+0wv5S7WuE1J2FMLUmfds+ZuR7sSEsy9bwge 3ZLuFpudzzJAaRuF2hOmFenSUab6Vs0kg5+4R0KAr7GG3C6mk4tgZ9JSWv8tJ72qT8qC 8QtBsCQCh+NHnja3zMKxhtVrgSU65yssXDI23BLPR+StdR2BVhiGOp9Dv7Zqnd626w0O /suOzo9yy6+6uKVqHBDUGsqidsXGyNypfVIvdv04CEjobtEoRIbqiJBTZ3lmQdpwH17N kUqnhpQYSitaCBT7aocIS+dQIuWjLdCklfkLb/IPHM9Rs/pO25dgZ0ga+NHHSEJ5qDRs DxSg== X-Gm-Message-State: AA6/9RkIhejN+qx6sydDXtsmC6LgTl0WRJsvOxc7F3jV0PyUNaTv4D/VYq4fET1Sm1bHDi3bmF/LTu3o0n/wpQ== X-Received: by 10.157.42.167 with SMTP id e36mr5549928otb.232.1476450536015; Fri, 14 Oct 2016 06:08:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.61.52 with HTTP; Fri, 14 Oct 2016 06:08:55 -0700 (PDT) In-Reply-To: <58009EB4.30708@omnilan.de> References: <58009EB4.30708@omnilan.de> From: Vincenzo Maffione Date: Fri, 14 Oct 2016 15:08:55 +0200 Message-ID: Subject: Re: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] To: Harry Schmalzbauer Cc: FreeBSD Net , FreeBSD Stable , Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 13:08:57 -0000 Hi, Thanks for your feedback. 2016-10-14 11:00 GMT+02:00 Harry Schmalzbauer : > Dear all, > > I found great papers about netmap(4)s desigen and implementation > details, and I'm sure it's one other masterpeace of rizzo-quality :-) > Thanks to all participants for that great code! > > To be honest, I haven't read all of that, because I'm short in time and > my first mission is to see if FreeBSD 11 will replace some of my ESXi > machines. > > One key element seems netmap(4). > It's quiet hard to find userland documentation. > > So far, I've discovered that there are three essential tools waiting in > _usr/src/tools/tools/netmap_ to be compiled > (resulting in *./vale-ctl*, *./bridge*, *./pkt-gen*) > > While the latter is often referenced in netmap(4) documentation, it's > not of interest for me, because I'll be doing real-world performance > tests and I'm convinced that all the impressive numbers presented in the > netmap documentation are valid :-) > > So *vale-ctl(-8)* seems to be of interest (I'm using (-8) becaus > currently there is no man8 part (I guess that's the reason for these > tools not beeing integrated into base binaries)) > > Accidentally I found out that 'vale-ctl -n testif0' creates a artificial > interface, which is reported by ifconfig(8): > testif0: flags=3D8801 metric 0 mtu 1500 > options=3D80000 > ether 00:be:eb:8d:f8:00 > nd6 options=3D21 > > But I can't assign a IP address: 'ifconfig testif0 203.0.113.1/24' > ifconfig: ioctl (SIOCAIFADDR): Invalid argument > > I guess couldn't geti the picture of the netmap(4) world yet. > Probably, testif0 is available only in netmap(4) world, not in "host > world". > I'm assuming, because I found vale-ctl(-8)s "-h" switch. > Yes, those are the "persistent" VALE ports. They are a recent feature, and probably you don't need to use them if you are going to play with Virtual Machines and jails (see below). > > So another very little peace I'm aware of the netmap(4) world, is how to > attach physical interfaces to virtual switches: > '/usr/src/tools/tools/netmap/vale-ctl -a vale0:em1' > Now vale-ctl(-8) shows: > bdg_ctl [149] bridge:0 port:0 vale0:em1 > > /* > To share my experience: One cannot use any other than vale[[:digit:]] > for defining the on-demand to be created virtual switch instance, so > e.g. "vale-ctl -a vale-test:em1" doesn't work, although found in > netmap(4) man page in FreeBSD-11: > =C2=BBvaleXXX:YYY (arbitrary XXX and YYY) > the file descriptor is bound to port YYY of a VALE switch called > XXX, both dynamically created if necessary. The string cannot > exceed IFNAMSIZ characters, and YYY cannot be the name of any > existing OS network interface=C2=AB > > I was about to give up on netmap(4) investigations because I thought it > isn't production ready yet (in FreeBSD), since even andding the first > physical interface fails: '/usr/src/tools/tools/netmap/vale-ctl -a > vale-test:em1' > vale-test:em1: Invalid argument > > Probably accidentally I used vale[[:digit:]] instead and wondered whay > it suddenly works=E2=80=A6 > Correct, this seems to be an inconsistency between the manual and the implementation, we will fix the manual. > > To get back to vale-ctl(-8)s "-h" switch: > */ > > If I add a physical interface with -h instead of -a, the host's IP stack > doesn't get disconnected from the interface, so it's still usable by > host applications and vale-ctl(-8) lists one line more: > bdg_ctl [149] bridge:0 port:0 vale0:em1 > bdg_ctl [149] bridge:0 port:1 vale0:em1^ > So my assumption that netmap(4) lives decapsuled from the well known > FreeBSD IP world. > > > Now my question: > > How can I plug a jail's or vmm's artificial interface to a VALE virtual > switch, bridging frames to real-world via physical interfaces? > (the latter part should work with vale-ctl -h vale0:em1, but what > interface to use for jail(8) vnet.interface and how to create/attach?) > If you use bhyve/vmm, you can attach the VM TAP interface to the VALE switch, as you would do for "em1". Regarding jails, I don't know exactly how networking works there, but I guess epair(4) interface (or similar) are used. If this is the case, then you would have one end of the epair only visible in the jail, and the other end only visible in the "host"; then you could attach the host end to a VALE switch again with "vale-ctl -a". Unfortunately, the performance you would get in any case is not great, because TAP and epair interface do not have netmap "native support". Moreover, when using bhyve, you have to pay the cost of the emulation of the vtnet device, since each packet passes through this device (other than passing across netmap). However, consider the following: a consistent netmap update is going to happen in FreeBSD-CURRENT, in short. This is going to align the netmap code which is now in FreeBSD to the code on the official github repository ( https://github.com/luigirizzo/netmap). Among the new features, there is a new solution for bhyve networking, which will let you attach your bhyve VMs directly to a VALE switch, without paying additional overheads related to TAPs, epairs, and vtnet emulation. You can find additional information, code and performance numbers here: https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel. Cheers, Vincenzo > > Thanks, > > -harry > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" --=20 Vincenzo Maffione From owner-freebsd-stable@freebsd.org Fri Oct 14 13:10:22 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 225A6C11A8E for ; Fri, 14 Oct 2016 13:10:22 +0000 (UTC) (envelope-from dmitryluhtionov@gmail.com) Received: from mail-lf0-x233.google.com (mail-lf0-x233.google.com [IPv6:2a00:1450:4010:c07::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BA90AB4 for ; Fri, 14 Oct 2016 13:10:21 +0000 (UTC) (envelope-from dmitryluhtionov@gmail.com) Received: by mail-lf0-x233.google.com with SMTP id b81so192639383lfe.1 for ; Fri, 14 Oct 2016 06:10:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=O2joEv5QOsIFz0AiBJ57/+Gm+jGVpJtmBr6Ar/U1vsc=; b=xDDdiKwzJuVrhV5vJilrAIE9opCEHbV/ueSk65dJnVf50CKbN5/Ow2U21akXe1iUe9 +GRO/4Br83rlxtinOM3+cnu1WDPJwJ2K6xjPq/bZ/TnkcpQ6usO8VbBVB4/b1KjDHfwa HMNowKE+zomCa5wNaZ3/X2uWOsQ9OH2Al01G8AYGi6kfP354vdbtGM3mAcDHU2UzUdUw nD13LVCsPHdQ/N/k3qOsOoL6J439KP1cH/sEkbOQNoCvr+GlutwsCfTScKtCB+KbijTv vNF34P0E7CpNOE7xOJaq3BPF+kE1kDYhW3pfohh1leXfUn1SbFDc7jFj0yPbB4qTW8Bm wKcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=O2joEv5QOsIFz0AiBJ57/+Gm+jGVpJtmBr6Ar/U1vsc=; b=PneX9wxm42QwePC1EOInQR4532piK6pxIKN2m0hHkCFiiXTIXnQ0sO1WoFcPqILu52 WqG/PJgz3XfnDieQ001/LfRqRia5avjfHUBwl4G+qf48DMw+3h5e0/FHxBWxKyH5ZJr2 0Kkd7u7WXkEynxZimGGB+kNxbID52OzS2oeHkNN6IcbMPj7Am1yE1KbdpL3s9mNdE6fP vv8D2KYTSl2zlwTylrDpeRNzC5Rkh3gmG7vBpnnMSvwXYwZMHy6VohTRhe9ZfhPhB9ig sCCT2/9oOMZ0xsaMlaDzRhuIEPBu0QqAtSgNWDGnT+9QmXVrkcEML8FFjPvTmk6iGVqp sqIA== X-Gm-Message-State: AA6/9Rmd6x6Hc+XqKpHmDlfTQd/U7MyxnVfcE5h8Y02c1rDDGs5wEHhPuG1LdNlR6v3pZe5Tp5w3Cl9KSFTfqw== X-Received: by 10.25.190.71 with SMTP id o68mr3757333lff.23.1476450618819; Fri, 14 Oct 2016 06:10:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.16.202 with HTTP; Fri, 14 Oct 2016 06:10:18 -0700 (PDT) From: Dmitry Luhtionov Date: Fri, 14 Oct 2016 16:10:18 +0300 Message-ID: Subject: make buildwotrld can not find zlib,h To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 13:10:22 -0000 c++ -O2 -pipe -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/tools/clang/include -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support -I. -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" -DDEFAULT_SYSROOT=\"/usr/obj/usr/src/tmp\" -MD -MF.depend.Compression.o -MTCompression.o -Qunused-arguments -I/usr/obj/usr/src/tmp/legacy/usr/include -std=c++11 -fno-exceptions -fno-rtti -stdlib=libc++ -Wno-c++11-extensions -c /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/Compression.cpp -o Compression.o /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/Compression.cpp:21:10: fatal error: 'zlib.h' file not found #include ^ 1 error generated. From owner-freebsd-stable@freebsd.org Fri Oct 14 13:38:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CD98C108FA; Fri, 14 Oct 2016 13:38:05 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A3BF24A; Fri, 14 Oct 2016 13:38:04 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id u9EDc2oO078823; Fri, 14 Oct 2016 15:38:02 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 3322AF08; Fri, 14 Oct 2016 15:38:02 +0200 (CEST) Message-ID: <5800DFB9.5030102@omnilan.de> Date: Fri, 14 Oct 2016 15:38:01 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Vincenzo Maffione CC: FreeBSD Net , FreeBSD Stable , Luigi Rizzo Subject: Re: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] References: <58009EB4.30708@omnilan.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Fri, 14 Oct 2016 15:38:03 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 13:38:05 -0000 Bezüglich Vincenzo Maffione's Nachricht vom 14.10.2016 15:08 (localtime): > Hi, > > Thanks for your feedback. > … >> Accidentally I found out that 'vale-ctl -n testif0' creates a artificial >> interface, which is reported by ifconfig(8): >> testif0: flags=8801 metric 0 mtu 1500 >> options=80000 >> ether 00:be:eb:8d:f8:00 >> nd6 options=21 >> >> But I can't assign a IP address: 'ifconfig testif0 203.0.113.1/24' >> ifconfig: ioctl (SIOCAIFADDR): Invalid argument >> >> I guess couldn't geti the picture of the netmap(4) world yet. >> Probably, testif0 is available only in netmap(4) world, not in "host >> world". >> I'm assuming, because I found vale-ctl(-8)s "-h" switch. >> > > Yes, those are the "persistent" VALE ports. They are a recent feature, and > probably you don't need to use them if you are going to play with Virtual > Machines and jails (see below). Hello Vincenzo, thank you very much for your help!!! … >> Now my question: >> >> How can I plug a jail's or vmm's artificial interface to a VALE virtual >> switch, bridging frames to real-world via physical interfaces? >> (the latter part should work with vale-ctl -h vale0:em1, but what >> interface to use for jail(8) vnet.interface and how to create/attach?) >> > > If you use bhyve/vmm, you can attach the VM TAP interface to the VALE > switch, as you would do for "em1". Regarding jails, I don't know exactly > how networking works there, but I guess epair(4) interface (or similar) are > used. If this is the case, then you would have one end of the epair only > visible in the jail, and the other end only visible in the "host"; then you I'm familar with epair(4), but not with tap(4). I don't understand the man page for tap, perhaps I should read pty(4)… But I guess I don't have to know the details of tap(4), since you confirmed that it can be connected to VALE. So one could summarize: VALE (as part of netmap(4)) can act as a if_bridge(4) replacement in FreeBSD-10/11, keeping everything else involved untouched. Please correct me if I'm wrong. > could attach the host end to a VALE switch again with "vale-ctl -a". > Unfortunately, the performance you would get in any case is not great, > because TAP and epair interface do not have netmap "native support". > Moreover, when using bhyve, you have to pay the cost of the emulation of > the vtnet device, since each packet passes through this device (other than > passing across netmap). I understand, thanks. In fact, I expected that at first hand, but have had some oddities with if_bridge(4) some years ago, so I thought I'd better try something new ;-) Can I expect any resource savings over if_bridge(4)? I guess if so, the ammount isn't relevant considering the whole bhyve scenarium. > However, consider the following: a consistent netmap update is going to > happen in FreeBSD-CURRENT, in short. This is going to align the netmap code > which is now in FreeBSD to the code on the official github repository ( > https://github.com/luigirizzo/netmap). Among the new features, there is a > new solution for bhyve networking, which will let you attach your bhyve VMs > directly to a VALE switch, without paying additional overheads related to > TAPs, epairs, and vtnet emulation. You can find additional information, > code and performance numbers here: > https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel. Thanks for that hint! I guess it's about ptnetmap(4)? I read papers but haven't considered it could be production-ready for FreeBSD in the near future. It's extremely interesting and I'd love to be eraly adopter, but my (ESXi) setups are currently doing well and I don't have spare time or any business project to try out… :-( Is it likely that there will a MFC happen? Or will it be a exclusive 12.0 feature? If ptnetmap will be MFCd I'll definitely give it a try next summer and stay with 11.0 for my replacement machines for now. Otherwise I'm unsure… best, -Harry From owner-freebsd-stable@freebsd.org Fri Oct 14 16:44:53 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8737C11866 for ; Fri, 14 Oct 2016 16:44:53 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (unknown [IPv6:2001:6a0:10f:1::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "plan-b.pwste.edu.pl", Issuer "plan-b.pwste.edu.pl" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4800BBF3 for ; Fri, 14 Oct 2016 16:44:50 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (zarychtam@localhost [127.0.0.1]) by plan-b.pwste.edu.pl (8.15.2/8.15.2) with ESMTPS id u9EGijpd006889 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 14 Oct 2016 18:44:45 +0200 (CEST) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: (from zarychtam@localhost) by plan-b.pwste.edu.pl (8.15.2/8.15.2/Submit) id u9EGijTH006888; Fri, 14 Oct 2016 18:44:45 +0200 (CEST) (envelope-from zarychtam) Date: Fri, 14 Oct 2016 18:44:45 +0200 From: Marek Zarychta To: Dmitry Luhtionov Cc: freebsd-stable@freebsd.org Subject: Re: make buildwotrld can not find zlib,h Message-ID: <20161014164445.GA6808@plan-b.pwste.edu.pl> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="2oS5YaxWCcQjTEyO" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 16:44:53 -0000 --2oS5YaxWCcQjTEyO Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 14, 2016 at 04:10:18PM +0300, Dmitry Luhtionov wrote: > c++ -O2 -pipe > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/tools/clang/inc= lude > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support -I. > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/clang= /include > -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS > -D__STDC_CONSTANT_MACROS -DNDEBUG -fno-strict-aliasing > -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd11.0\" > -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd11.0\" > -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/src/tmp\" -MD -MF.depend.Compression.o > -MTCompression.o -Qunused-arguments > -I/usr/obj/usr/src/tmp/legacy/usr/include -std=3Dc++11 -fno-exceptions > -fno-rtti -stdlib=3Dlibc++ -Wno-c++11-extensions -c > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/Compr= ession.cpp > -o Compression.o > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/Compr= ession.cpp:21:10: > fatal error: 'zlib.h' file not found > #include > ^ > 1 error generated. Maybe some incorrect or stale statements in src.conf ? --=20 Marek Zarychta --2oS5YaxWCcQjTEyO Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAABCAAGBQJYAQt5AAoJEHWf7P/9Uo0s5woIAIbBuwtNGuNTZV2vT4GQQyax UkF0eE1xNLMK35IxFf8WRk6+1zRqYj8PrYp/Ad0PEORDqxE3fidE7mscup9uf4i5 +ydFdu63OYHRINmC1TUpgAGIB/+uIWsDORu4eQ8Q3FFqyB8tVTnyewD1Hf30jVsJ sWEjbfkfPhgMvsceHVqRDmXrqEZYFf2q3j/EmQCznakmB3MbaaNL/rjx4BDJQxEG ox19A9CDgXX5cS6peYaHbfpdEm9ekYRjuKBAwaOZpIL9zyGu703ecXwhPoLTg1Lw 7pjLlzUOItg/nNtrC+SNirUCRje8ZSFNs2zcaIrekD0vpENZncnNhMJceIDzCEQ= =67kt -----END PGP SIGNATURE----- --2oS5YaxWCcQjTEyO-- From owner-freebsd-stable@freebsd.org Fri Oct 14 18:06:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9DACC12DB4 for ; Fri, 14 Oct 2016 18:06:56 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x22a.google.com (mail-vk0-x22a.google.com [IPv6:2607:f8b0:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76924228 for ; Fri, 14 Oct 2016 18:06:56 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x22a.google.com with SMTP id 2so123106117vkb.3 for ; Fri, 14 Oct 2016 11:06:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Ud46Yx6BAVrqtDlwUSRCNIJ03SF5v9XnDYpk7Pdfou8=; b=c5bRL3HTBPGCxfjkezGj9Cugcy899aOhn7Osi8aBYRx9UDX5sVhlWeQ6gT6xWICved KzLprK27lsBCi+rQdKW+IOjqZbG3jZiHUuBbLbUKfO3U0CU0+YXIHnpKIFvDmtU/O6q1 o9Kf+9Aa976pke1eTXkPH2ImpDlwbJeH+M/wRwPbBZcEBfCKcBmXltLJGVLlkbIKZnCr ltgesFrIuaVADl6lSLvqdSXiljepA9eCrDd7SZZx0lmI2EqvUQAjgPSvoNbVQ8grTC4x QFMfc7V3cPe1g1tU+aH82yiBrXKATvCgLktUyzoa+nDq/mGkxlD4yvPKzPoIt8qksFT0 uIhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Ud46Yx6BAVrqtDlwUSRCNIJ03SF5v9XnDYpk7Pdfou8=; b=blgapIGiDAtt9v0gW5nukl09hEKuahHZTnmshLfJV9vXeHRcikAgnMh7ehUBvP+kQp SnOK9tjJ0yQc5MIjiF8eRK8jADrPCI1E7b46LFNxL4IknoQGi84TgZWMZs5lKI2LoxYD 4lBsQHkrPUL02KZyhkwhRnB+XhPkUnmZpbTk2yEd+XvlZ+jR79QlbIZrpMZ9XQpSLI8/ 7OoaFtwCojlmNkCh7Hz8ASmcUYHjHGq1Jf3FQGod0BcXUYw1z9pba4fcxqkQT86lLnG+ wuM9mnxL8dMnMzkeT1EgBP2XFngrGo0aRsQGPflI3U82J+3YrojnAX5S05qHomtLyeF5 rAgQ== X-Gm-Message-State: AA6/9RkXIhGVYUjqd3dWOrbCyUC0I8zc8xIh9SmhiVbSl+3Wo8+h0qmzdUwl1KmY6do4AViCCG+xpXhniaoRJQ== X-Received: by 10.31.237.67 with SMTP id l64mr10275745vkh.56.1476468415545; Fri, 14 Oct 2016 11:06:55 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.118.78 with HTTP; Fri, 14 Oct 2016 11:06:55 -0700 (PDT) In-Reply-To: References: From: Kevin Oberman Date: Fri, 14 Oct 2016 11:06:55 -0700 X-Google-Sender-Auth: A2y0aGhlCfxcgWipJpY6wd_scb0 Message-ID: Subject: Re: make buildwotrld can not find zlib,h To: Dmitry Luhtionov Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 18:06:56 -0000 On Fri, Oct 14, 2016 at 6:10 AM, Dmitry Luhtionov wrote: > c++ -O2 -pipe > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/tools/clang/ > include > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support -I. > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/clang/ > include > -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS > -D__STDC_CONSTANT_MACROS -DNDEBUG -fno-strict-aliasing > -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" > -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" > -DDEFAULT_SYSROOT=\"/usr/obj/usr/src/tmp\" -MD -MF.depend.Compression.o > -MTCompression.o -Qunused-arguments > -I/usr/obj/usr/src/tmp/legacy/usr/include -std=c++11 -fno-exceptions > -fno-rtti -stdlib=libc++ -Wno-c++11-extensions -c > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > Compression.cpp > -o Compression.o > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > Compression.cpp:21:10: > fatal error: 'zlib.h' file not found > #include > ^ > 1 error generated. > Very odd. /usr/include/zlib.h is and long has been a standard component of FreeBSD and should be present on your system. Can you confirm its absence? Anything that could be in /etc/src.conf that might trigger this? (I can't see anything obvious, but src.conf(5) is very long.) I'm also not sure whether, at this point in the build, you should be using the system's include files or those in /usr/obj/usr/src/tmp/usr/include/ or /usr/src/lib/libz/zlib.h, which is what should be copied to /usr/obj/usr/src/tmp/usr/include. Normally the system's files are not used during the build. Have you tried completely removing /usr/obj (rm -r /usr/obj/*) before starting the build with -DNO_CLEAN? -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Fri Oct 14 18:49:38 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A78C5C1275A for ; Fri, 14 Oct 2016 18:49:38 +0000 (UTC) (envelope-from tkawakit@kumamoto-iri.jp) Received: from amakusa.kumamoto-iri.jp (ns-i2.kmt-iri.go.jp [IPv6:2001:240:174:1001::9]) by mx1.freebsd.org (Postfix) with ESMTP id 79D31F71 for ; Fri, 14 Oct 2016 18:49:38 +0000 (UTC) (envelope-from tkawakit@kumamoto-iri.jp) Received: from amakusa.kumamoto-iri.jp (localhost [127.0.0.1]) by amakusa.kumamoto-iri.jp (Postfix) with ESMTP id E99BB17002FA; Sat, 15 Oct 2016 03:49:34 +0900 (JST) Received: from shobu.kumamoto-iri.jp (shobu.kumamoto-iri.jp [202.24.93.170]) by amakusa.kumamoto-iri.jp (Postfix) with ESMTP id E8CD717000F0; Sat, 15 Oct 2016 03:49:34 +0900 (JST) Received: from [192.168.1.5] (FL1-125-195-110-104.kmm.mesh.ad.jp [125.195.110.104]) by shobu.kumamoto-iri.jp (Postfix) with ESMTPSA id B4D4C60089D; Sat, 15 Oct 2016 03:49:34 +0900 (JST) User-Agent: K-9 Mail for Android MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Subject: Re: make buildwotrld can not find zlib,h From: =?UTF-8?B?5rKz5YyX6ZqG55Sf?= Date: Sat, 15 Oct 2016 03:49:33 +0900 To: Dmitry Luhtionov CC: FreeBSD-STABLE Mailing List Message-ID: <2EA1F46A-57CE-4152-82C8-E8C0AEECC4CB@kumamoto-iri.jp> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 18:49:38 -0000 On Fri, Oct 14, 2016 at 6:10 AM, Dmitry Luhtionov wrote: > c++ -O2 -pipe > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/tools/clang/ > include > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support -I. > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/clang/ > include > -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS > -D__STDC_CONSTANT_MACROS -DNDEBUG -fno-strict-aliasing > -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" > -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" > -DDEFAULT_SYSROOT=\"/usr/obj/usr/src/tmp\" -MD -MF.depend.Compression.o > -MTCompression.o -Qunused-arguments > -I/usr/obj/usr/src/tmp/legacy/usr/include -std=c++11 -fno-exceptions > -fno-rtti -stdlib=libc++ -Wno-c++11-extensions -c > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > Compression.cpp > -o Compression.o > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > Compression.cpp:21:10: > fatal error: 'zlib.h' file not found > #include > ^ > 1 error generated. > Very odd. /usr/include/zlib.h is and long has been a standard component of FreeBSD and should be present on your system. Can you confirm its absence? Anything that could be in /etc/src.conf that might trigger this? (I can't see anything obvious, but src.conf(5) is very long.) I'm also not sure whether, at this point in the build, you should be using the system's include files or those in /usr/obj/usr/src/tmp/usr/include/ or /usr/src/lib/libz/zlib.h, which is what should be copied to /usr/obj/usr/src/tmp/usr/include. Normally the system's files are not used during the build. Have you tried completely removing /usr/obj (rm -r /usr/obj/*) before starting the build with -DNO_CLEAN? -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" --- 河北隆生＠熊本県産業技術センター From owner-freebsd-stable@freebsd.org Fri Oct 14 18:54:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51F5DC1290F for ; Fri, 14 Oct 2016 18:54:34 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from lamora.getmail.no (lamora.getmail.no [84.210.184.7]) by mx1.freebsd.org (Postfix) with ESMTP id 0BA453EB for ; Fri, 14 Oct 2016 18:54:33 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id B48958E7DD for ; Fri, 14 Oct 2016 20:49:02 +0200 (CEST) Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id pmUleB2BwQsv for ; Fri, 14 Oct 2016 20:49:02 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by lamora.getmail.no (Postfix) with ESMTP id 6199D8E7DE for ; Fri, 14 Oct 2016 20:49:02 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.9.2 lamora.getmail.no 6199D8E7DE DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=getmail.no; s=8A9C8B4C-D727-11E2-8095-B6466E6B3FA2; t=1476470942; bh=gfqHapqk19ieSDCL/fL96L9sgWg74OSRcRVTu9byN7Y=; h=Date:From:To:Subject:Message-Id:Mime-Version:Content-Type: Content-Transfer-Encoding; b=NjjibFy4U7622IdLccAK28u/g3/Kpp+WDeyM+MQL6Oq+A1n5RWO7RxDB0a4qfmAY8 uH/op1UvBtWeJabsdjGdI70dqDbZsHuK0lqttJ+/gxD9/RYvI1XQKbmL+I1shvlPut um8S/1fYIbJiFjtWs4n7tM1ZGBjeEJWrAj83rlU4= X-Virus-Scanned: amavisd-new at lamora.get.c.bitbit.net Received: from lamora.getmail.no ([127.0.0.1]) by localhost (lamora.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id w_cbQ-D1zJZm for ; Fri, 14 Oct 2016 20:49:02 +0200 (CEST) Received: from kg-core1.kg4.no (cm-84.209.39.108.getinternet.no [84.209.39.108]) by lamora.getmail.no (Postfix) with ESMTPSA id 3DFA78E7DD for ; Fri, 14 Oct 2016 20:49:02 +0200 (CEST) Date: Fri, 14 Oct 2016 20:49:01 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Subject: Re: aibs(4) / atk0110 support for newer systems Message-Id: <20161014204901.a61269485c5e5aa4fc33deaa@getmail.no> In-Reply-To: References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> <20160930145704.4dbc9d90011154b38493964e@getmail.no> <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> <20160930205928.77d7e74f7bd1a35fcf1aa50a@getmail.no> <7a868c22-e0bd-f677-e4ad-2bdf6f3605d0@FreeBSD.org> <20161003201511.7258687453f12c44a46a361a@getmail.no> <20161005143723.4273657959160b67637a5adf@getmail.no> <4c431c8a-7c8c-4336-b948-6e9ae882c0fc@FreeBSD.org> <20161005222831.e674c3a134266c2e1edc523e@getmail.no> <19cb4aa4-f38e-1331-2939-281a08d1f78d@FreeBSD.org> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 18:54:34 -0000 On Tue, 11 Oct 2016 14:29:34 +0300 Andriy Gapon wrote: > On 06/10/2016 00:37, Andriy Gapon wrote: > > On 05/10/2016 23:28, Torfinn Ingolfsen wrote: > >> #6 0xffffffff80cd0081 in calltrap () > >> at /usr/src/sys/amd64/amd64/exception.S:238 > >> #7 0xffffffff81bcb078 in aibs_add_sensor () from /boot/kernel/aibs.ko > >> #8 0xffffffff81bcb4b4 in aibs_attach_sif () from /boot/kernel/aibs.ko > > > > Argh, I've just spotted a very silly typo. > > Could you please replace '0' with 'o' in > > err = aibs_add_sensor(sc, 0, &as[i], &descr); > > ? > > Ping. Done - see other messages in this thread. Sorry about the delay. -- Torfinn Ingolfsen From owner-freebsd-stable@freebsd.org Fri Oct 14 18:54:37 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 959E8C1291A for ; Fri, 14 Oct 2016 18:54:37 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from bouvier.getmail.no (bouvier.getmail.no [84.210.184.8]) by mx1.freebsd.org (Postfix) with ESMTP id 26CEB3EC for ; Fri, 14 Oct 2016 18:54:36 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id BABA248029 for ; Fri, 14 Oct 2016 20:48:17 +0200 (CEST) Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id HeoJBlWLccgO for ; Fri, 14 Oct 2016 20:48:17 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id 4294F4802F for ; Fri, 14 Oct 2016 20:48:17 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.9.2 bouvier.getmail.no 4294F4802F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=getmail.no; s=8A9C8B4C-D727-11E2-8095-B6466E6B3FA2; t=1476470897; bh=6hO87Qd0vRTvaJot6/HespySKKISOtnnF4MiDGbu+iE=; h=Date:From:To:Subject:Message-Id:Mime-Version:Content-Type: Content-Transfer-Encoding; b=XaaRGdJgzCpokeuB8WpM2o7H1MpFLjdaLJ4eG79sWaI3RiSVhVu1Smv4c8CkVoOgs zmfNBLjhP4ezd1ziBNpUSpFqz5JeXnWJRNI3e0If3qjNgLiDV7zbZDjo3IfqXczjZI KAckvWYl6HFThAlm+bOtGdt9WJ5O2LLp0mXPaV4A= X-Virus-Scanned: amavisd-new at bouvier.get.c.bitbit.net Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id tLuzVqUASOj9 for ; Fri, 14 Oct 2016 20:48:17 +0200 (CEST) Received: from kg-core1.kg4.no (cm-84.209.39.108.getinternet.no [84.209.39.108]) by bouvier.getmail.no (Postfix) with ESMTPSA id 16CD648029 for ; Fri, 14 Oct 2016 20:48:17 +0200 (CEST) Date: Fri, 14 Oct 2016 20:48:16 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Subject: Re: aibs(4) / atk0110 support for newer systems Message-Id: <20161014204816.140c1c51543f7458a044b3cd@getmail.no> In-Reply-To: <19cb4aa4-f38e-1331-2939-281a08d1f78d@FreeBSD.org> References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> <20160930145704.4dbc9d90011154b38493964e@getmail.no> <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> <20160930205928.77d7e74f7bd1a35fcf1aa50a@getmail.no> <7a868c22-e0bd-f677-e4ad-2bdf6f3605d0@FreeBSD.org> <20161003201511.7258687453f12c44a46a361a@getmail.no> <20161005143723.4273657959160b67637a5adf@getmail.no> <4c431c8a-7c8c-4336-b948-6e9ae882c0fc@FreeBSD.org> <20161005222831.e674c3a134266c2e1edc523e@getmail.no> <19cb4aa4-f38e-1331-2939-281a08d1f78d@FreeBSD.org> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 18:54:37 -0000 On Thu, 6 Oct 2016 00:37:16 +0300 Andriy Gapon wrote: > On 05/10/2016 23:28, Torfinn Ingolfsen wrote: > > #6 0xffffffff80cd0081 in calltrap () > > at /usr/src/sys/amd64/amd64/exception.S:238 > > #7 0xffffffff81bcb078 in aibs_add_sensor () from /boot/kernel/aibs.ko > > #8 0xffffffff81bcb4b4 in aibs_attach_sif () from /boot/kernel/aibs.ko > > Argh, I've just spotted a very silly typo. > Could you please replace '0' with 'o' in > err = aibs_add_sensor(sc, 0, &as[i], &descr); > ? Yes, that fixes it - aibs.ko no longer panics my machine when I load it. Output from /var/log/messages: Oct 14 20:32:18 kg-core1 kernel: aibs0: on acpi0 Oct 14 20:32:18 kg-core1 kernel: aibs0: v0: 0x06020000 Vcore Voltage 850 / 1600 Oct 14 20:32:18 kg-core1 kernel: aibs0: v1: 0x06020001 +3.3 Voltage 2970 / 3630 Oct 14 20:32:18 kg-core1 kernel: aibs0: v2: 0x06020002 +5 Voltage 4500 / 5500 Oct 14 20:32:18 kg-core1 kernel: aibs0: v3: 0x06020003 +12 Voltage 10200 / 13800 Oct 14 20:32:18 kg-core1 kernel: aibs0: t0: 0x06030000 CPU Temperature 600 / 950 Oct 14 20:32:18 kg-core1 kernel: aibs0: t1: 0x06030001 MB Temperature 450 / 750 Oct 14 20:32:18 kg-core1 kernel: aibs0: f0: 0x06040000 CPU FAN Speed 600 / 7200 Oct 14 20:32:18 kg-core1 kernel: aibs0: f1: 0x06040001 CHASSIS FAN Speed 600 / 7200 it looks almost the same as it did previously: tingo@kg-core1$ grep aibs /tmp/c1-dmesg-9.3-stable-20160826.txt aibs0: on acpi0 aibs0: V0: 0x06020000 Vcore Voltage 850 / 1600 0x1 aibs0: V1: 0x06020001 +3.3 Voltage 2970 / 3630 0x1 aibs0: V2: 0x06020002 +5 Voltage 4500 / 5500 0x1 aibs0: V3: 0x06020003 +12 Voltage 10200 / 13800 0x1 aibs0: T0: 0x06030000 CPU Temperature 600 / 950 0x10001 aibs0: T1: 0x06030001 MB Temperature 450 / 750 0x10001 aibs0: F0: 0x06040000 CPU FAN Speed 600 / 7200 0x10001 aibs0: F1: 0x06040001 CHASSIS FAN Speed 600 / 7200 0x10001 and from sysctl dev.aibs root@kg-core1# sysctl dev.aibs dev.aibs.0.volt.0: 1404 850 1600 dev.aibs.0.volt.1: 3265 2970 3630 dev.aibs.0.volt.2: 5070 4500 5500 dev.aibs.0.volt.3: 12028 10200 13800 dev.aibs.0.temp.0: 51.9C 59.9C 94.9C dev.aibs.0.temp.1: 30.9C 44.9C 74.9C dev.aibs.0.fan.0: 2678 600 7200 dev.aibs.0.fan.1: 2678 600 7200 dev.aibs.0.%parent: acpi0 dev.aibs.0.%pnpinfo: _HID=ATK0110 _UID=16843024 dev.aibs.0.%location: handle=\_SB_.PCI0.SBRG.ASOC dev.aibs.0.%driver: aibs dev.aibs.0.%desc: ASUSTeK AI Booster (ACPI ASOC ATK0110) -- Torfinn Ingolfsen From owner-freebsd-stable@freebsd.org Sat Oct 15 07:32:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D43B1C125F2; Sat, 15 Oct 2016 07:32:05 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x22b.google.com (mail-oi0-x22b.google.com [IPv6:2607:f8b0:4003:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 986F8928; Sat, 15 Oct 2016 07:32:05 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x22b.google.com with SMTP id y2so161956221oie.0; Sat, 15 Oct 2016 00:32:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=75ToIbowHPkVtbCDF6Fe5mSLlbDBvtVurs9POYokY34=; b=yI257TKxuK8hKhI71LR3jZc5MzOITSkrKOWV43GVJinsrLdRkzNGwvmNuQhm+GxyTw a3qtMtCdA4eIzmTxfcQqTcp0JiufmRCI1S7o24HwlfzVGVOu0l+JbPc3F9/nub0JJEE+ AmpIjqrN0dR/wvkZ1wXGOG1EhyMVy1bp4nkoWonsdQkx/2anK6ARUzWk07EFo/MB9qdc AzfNK9RFr0VIC1g3XH5vy6xjULe7NYfOjW5gzNlRX5RE+ZEg+6eW46qvWf1EfsBTkJw/ 4nem0QOrmPgSnXXUJCAZ0bQ5ilRXssQ8zCeXiZMharu1s/2qSbI3MPYZ3x76RMQ4wEPD hmQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=75ToIbowHPkVtbCDF6Fe5mSLlbDBvtVurs9POYokY34=; b=MarY9jWA6eC02WUDZNssrQzKfxenUTD8UeTtbG/Q76uaPdZypX4eudXXjqQlLT1Z+d wwxmxK72TzcajDizhGtaC6FJ+jnSv+S13KCpx2GVxTW4Paun/MF8Vl5IYT8b72RNmWSK IPDTco8tvrJws+KyuKn+Bsuduc7C1UizSeDci6+X8cMh7y4gp9kJPjX6lOK9AleK4ZeA jrWGDQWjHI5t+NWD+fjJvjINW+zXX5RiLr2H++Hfx9dstZZFF1HYaTxTCsZGmhDGhgON hqu33l+dUfMWhiHXSht3cWWlxuPXjkX0d88p8o3APro0M+UClq2OydgLdajIG8MypJKd vhCA== X-Gm-Message-State: AA6/9RkEo8zjtff9osfDR88Z3tWashqJSdxjDFs919f8opQ0AjvjM6I5Cg8aU1LcNZo2Tdry+NEMoqQMv5zJfg== X-Received: by 10.157.47.232 with SMTP id b37mr8503831otd.67.1476516724794; Sat, 15 Oct 2016 00:32:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.61.52 with HTTP; Sat, 15 Oct 2016 00:32:04 -0700 (PDT) In-Reply-To: <5800DFB9.5030102@omnilan.de> References: <58009EB4.30708@omnilan.de> <5800DFB9.5030102@omnilan.de> From: Vincenzo Maffione Date: Sat, 15 Oct 2016 09:32:04 +0200 Message-ID: Subject: Re: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] To: Harry Schmalzbauer Cc: FreeBSD Net , FreeBSD Stable , Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Oct 2016 07:32:05 -0000 2016-10-14 15:38 GMT+02:00 Harry Schmalzbauer : > Bez=C3=BCglich Vincenzo Maffione's Nachricht vom 14.10.2016 15:08 (localt= ime): > > Hi, > > > > Thanks for your feedback. > > > =E2=80=A6 > >> Accidentally I found out that 'vale-ctl -n testif0' creates a artifici= al > >> interface, which is reported by ifconfig(8): > >> testif0: flags=3D8801 metric 0 mtu 1500 > >> options=3D80000 > >> ether 00:be:eb:8d:f8:00 > >> nd6 options=3D21 > >> > >> But I can't assign a IP address: 'ifconfig testif0 203.0.113.1/24' > >> ifconfig: ioctl (SIOCAIFADDR): Invalid argument > >> > >> I guess couldn't geti the picture of the netmap(4) world yet. > >> Probably, testif0 is available only in netmap(4) world, not in "host > >> world". > >> I'm assuming, because I found vale-ctl(-8)s "-h" switch. > >> > > > > Yes, those are the "persistent" VALE ports. They are a recent feature, > and > > probably you don't need to use them if you are going to play with Virtu= al > > Machines and jails (see below). > > Hello Vincenzo, > > thank you very much for your help!!! > > > =E2=80=A6 > >> Now my question: > >> > >> How can I plug a jail's or vmm's artificial interface to a VALE virtua= l > >> switch, bridging frames to real-world via physical interfaces? > >> (the latter part should work with vale-ctl -h vale0:em1, but what > >> interface to use for jail(8) vnet.interface and how to create/attach?) > >> > > > > If you use bhyve/vmm, you can attach the VM TAP interface to the VALE > > switch, as you would do for "em1". Regarding jails, I don't know exactl= y > > how networking works there, but I guess epair(4) interface (or similar) > are > > used. If this is the case, then you would have one end of the epair onl= y > > visible in the jail, and the other end only visible in the "host"; then > you > > I'm familar with epair(4), but not with tap(4). > I don't understand the man page for tap, perhaps I should read pty(4)=E2= =80=A6 > But I guess I don't have to know the details of tap(4), since you > confirmed that it can be connected to VALE. > It's not necessary to understand the details. However, a TAP device is conceptually similar to the two ends of an epair, with the difference that in the TAP a network interface (e.g. tap0) is conecptually "connected" back-to-back to a file descriptor. The file descriptor is written/read by the hypervisor (to inject/intercept packets to/from the network stack), while the tap0 interface can be attached to if_bridge. > > So one could summarize: > VALE (as part of netmap(4)) can act as a if_bridge(4) replacement in > FreeBSD-10/11, keeping everything else involved untouched. > Please correct me if I'm wrong. > For simple cases yes. if_bridge may have features that are not supported by netmap (i.e. configure ports as VLAN access ports). Moreover, if_bridge has a interface (br0), whereas VALE bridges doesn't. > > > > could attach the host end to a VALE switch again with "vale-ctl -a". > > Unfortunately, the performance you would get in any case is not great, > > because TAP and epair interface do not have netmap "native support". > > Moreover, when using bhyve, you have to pay the cost of the emulation o= f > > the vtnet device, since each packet passes through this device (other > than > > passing across netmap). > > I understand, thanks. > In fact, I expected that at first hand, but have had some oddities with > if_bridge(4) some years ago, so I thought I'd better try something new ;-= ) > Can I expect any resource savings over if_bridge(4)? I guess if so, the > ammount isn't relevant considering the whole bhyve scenarium. > I don't think you can expect resource savings in terms of memory, since netmap is trading off memory for performance, using pre-allocation rather than dynamic allocation (of each packet). I don't know what happens in terms of CPU saving, but using TAPs, epair and vtnet makes the system as a whole not really efficient (as if_bridge), so you may end up with nothing. > > > > However, consider the following: a consistent netmap update is going to > > happen in FreeBSD-CURRENT, in short. This is going to align the netmap > code > > which is now in FreeBSD to the code on the official github repository ( > > https://github.com/luigirizzo/netmap). Among the new features, there is > a > > new solution for bhyve networking, which will let you attach your bhyve > VMs > > directly to a VALE switch, without paying additional overheads related = to > > TAPs, epairs, and vtnet emulation. You can find additional information, > > code and performance numbers here: > > https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel. > > Thanks for that hint! > I guess it's about ptnetmap(4)? I read papers but haven't considered it > could be production-ready for FreeBSD in the near future. > It's extremely interesting and I'd love to be eraly adopter, but my > (ESXi) setups are currently doing well and I don't have spare time or > any business project to try out=E2=80=A6 :-( > Yes, it's ptnetmap. However, bhyve is going to have support for VALE ports anyway (even without ptnetmap), as QEMU already does, so at least you will be able to replace TAPs with VALE ports (while still using vtnet devices for the VM). > > Is it likely that there will a MFC happen? Or will it be a exclusive > 12.0 feature? If ptnetmap will be MFCd I'll definitely give it a try > next summer and stay with 11.0 for my replacement machines for now. > Otherwise I'm unsure=E2=80=A6 > No idea, I guess this is up to the committers. First step, however, is to merge in CURRENT :) Cheers, Vincenzo > > best, > > -Harry > --=20 Vincenzo Maffione From owner-freebsd-stable@freebsd.org Sat Oct 15 09:12:50 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F55BC12663 for ; Sat, 15 Oct 2016 09:12:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CEE67AA1 for ; Sat, 15 Oct 2016 09:12:49 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id MAA12749; Sat, 15 Oct 2016 12:12:47 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bvL1L-000Mmc-Ap; Sat, 15 Oct 2016 12:12:47 +0300 Subject: Re: aibs(4) / atk0110 support for newer systems To: Torfinn Ingolfsen , freebsd-stable@FreeBSD.org References: <86cf8380-ac6f-55f0-f0f8-16000d7f04b2@FreeBSD.org> <20160930145704.4dbc9d90011154b38493964e@getmail.no> <7d498084-ec05-d4c9-5f49-6aef32495caf@FreeBSD.org> <20160930205928.77d7e74f7bd1a35fcf1aa50a@getmail.no> <7a868c22-e0bd-f677-e4ad-2bdf6f3605d0@FreeBSD.org> <20161003201511.7258687453f12c44a46a361a@getmail.no> <20161005143723.4273657959160b67637a5adf@getmail.no> <4c431c8a-7c8c-4336-b948-6e9ae882c0fc@FreeBSD.org> <20161005222831.e674c3a134266c2e1edc523e@getmail.no> <19cb4aa4-f38e-1331-2939-281a08d1f78d@FreeBSD.org> <20161014204901.a61269485c5e5aa4fc33deaa@getmail.no> From: Andriy Gapon Message-ID: <1212d05e-8210-5ff1-1ee4-dbf063c41f48@FreeBSD.org> Date: Sat, 15 Oct 2016 12:11:50 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161014204901.a61269485c5e5aa4fc33deaa@getmail.no> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Oct 2016 09:12:50 -0000 On 14/10/2016 21:49, Torfinn Ingolfsen wrote: > On Tue, 11 Oct 2016 14:29:34 +0300 > Andriy Gapon wrote: > >> On 06/10/2016 00:37, Andriy Gapon wrote: >>> On 05/10/2016 23:28, Torfinn Ingolfsen wrote: >>>> #6 0xffffffff80cd0081 in calltrap () >>>> at /usr/src/sys/amd64/amd64/exception.S:238 >>>> #7 0xffffffff81bcb078 in aibs_add_sensor () from /boot/kernel/aibs.ko >>>> #8 0xffffffff81bcb4b4 in aibs_attach_sif () from /boot/kernel/aibs.ko >>> >>> Argh, I've just spotted a very silly typo. >>> Could you please replace '0' with 'o' in >>> err = aibs_add_sensor(sc, 0, &as[i], &descr); >>> ? >> >> Ping. > > Done - see other messages in this thread. Sorry about the delay. > No worries. Thank you very much for your testing! I've just committed the change to head. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Sat Oct 15 09:42:10 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D212CC12D61; Sat, 15 Oct 2016 09:42:10 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82FFF78F; Sat, 15 Oct 2016 09:42:09 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id u9F9g7jn092749; Sat, 15 Oct 2016 11:42:07 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 94F7C197; Sat, 15 Oct 2016 11:42:06 +0200 (CEST) Message-ID: <5801F9EE.1000407@omnilan.de> Date: Sat, 15 Oct 2016 11:42:06 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Vincenzo Maffione CC: FreeBSD Net , FreeBSD Stable , Luigi Rizzo Subject: Re: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] References: <58009EB4.30708@omnilan.de> <5800DFB9.5030102@omnilan.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 15 Oct 2016 11:42:07 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Oct 2016 09:42:10 -0000 Bezüglich Vincenzo Maffione's Nachricht vom 15.10.2016 09:32 (localtime): > 2016-10-14 15:38 GMT+02:00 Harry Schmalzbauer : … >> I'm familar with epair(4), but not with tap(4). >> I don't understand the man page for tap, perhaps I should read pty(4)… >> But I guess I don't have to know the details of tap(4), since you >> confirmed that it can be connected to VALE. >> > > It's not necessary to understand the details. However, a TAP device is > conceptually similar to the two ends of an epair, with the difference that > in the TAP a network interface (e.g. tap0) is conecptually "connected" > back-to-back to a file descriptor. The file descriptor is written/read by > the hypervisor (to inject/intercept packets to/from the network stack), > while the tap0 interface can be attached to if_bridge. Hi Vincenzo, thanks for your explanation! >> >> So one could summarize: >> VALE (as part of netmap(4)) can act as a if_bridge(4) replacement in >> FreeBSD-10/11, keeping everything else involved untouched. >> Please correct me if I'm wrong. >> > > For simple cases yes. if_bridge may have features that are not supported by > netmap (i.e. configure ports as VLAN access ports). Moreover, if_bridge has > a interface (br0), whereas VALE bridges doesn't. Again, thank you for your time! (R)STP comes to my mind (which I don't need any more). And I'm not sure if VALE really lacks that, but I guess it wouldn't match VALEs philosophy/design at all… … >>> https://github.com/luigirizzo/netmap). Among the new features, there is >> a >>> new solution for bhyve networking, which will let you attach your bhyve >> VMs >>> directly to a VALE switch, without paying additional overheads related to >>> TAPs, epairs, and vtnet emulation. You can find additional information, >>> code and performance numbers here: >>> https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel. >> >> Thanks for that hint! >> I guess it's about ptnetmap(4)? I read papers but haven't considered it >> could be production-ready for FreeBSD in the near future. >> It's extremely interesting and I'd love to be eraly adopter, but my >> (ESXi) setups are currently doing well and I don't have spare time or >> any business project to try out… :-( >> > > Yes, it's ptnetmap. However, bhyve is going to have support for VALE ports > anyway (even without ptnetmap), as QEMU already does, so at least you will > be able to replace TAPs with VALE ports (while still using vtnet devices > for the VM). Oic, I wasn't aware that there will be a VALE-vtnet direct path! That is really great news :-) And a big achievment for guests preferring "standard" drivers, ptnetmap could limit the guest OS choice I guess. For now, I'm happy having been in touch with netmap(4) – at least with a very little fraction of natmap – but I'll stay the legacy way utilizing if_bridge(4) and see if there are still oddities and try to find some time to track them down (involving LACP, VLANs, Jumbo-Frames and IPv6 – that was the problematic constellation) Since I have extra PHYs, I can do PCIe-passthrough like before (with ESXi) for some special guests. I'm looking forward to find out how this works with bhyve! Best, -Harry From owner-freebsd-stable@freebsd.org Sat Oct 15 16:21:37 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76309C13693 for ; Sat, 15 Oct 2016 16:21:37 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 35164E0B for ; Sat, 15 Oct 2016 16:21:36 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id F363D302E66 for ; Sat, 15 Oct 2016 11:12:31 -0500 (CDT) To: freebsd-stable@freebsd.org From: Karl Denninger Subject: Errata notice for 11.0 -- question Message-ID: <2b381f8e-e47e-9806-91d1-872cf6372c78@denninger.net> Date: Sat, 15 Oct 2016 11:12:31 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms050401080207000005090104" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Oct 2016 16:21:37 -0000 This is a cryptographically signed message in MIME format. --------------ms050401080207000005090104 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I noted this /after /svn updating my 11.x box to the most-current -STABLE= : A bug was diagnosed in interaction of the |pmap_activate()| function and TLB shootdownIPI handler on amd64 systems which have PCID features but do not implement theINVPCID instruction. On such machines, such as SandyBridge=E2=84=A2 and IvyBridge=E2=84=A2 microarchitectures, set the loader tunable |vm.pmap.pcid_enabled=3D0| during boot: set vm.pmap.pcid_enabled=3D0 boot Add this line to |/boot/loader.conf| for the change to persist across reboots: To check if the system is affected, check dmesg(8) for PCID listed in the "Features2", and absence of INVPCID in the "Structured Extended Features". If the PCID feature is not present, or INVPCID is present, system is not affected. Well, I'm allegedly subject to this: Copyright (c) 1992-2016 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved.= FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-STABLE #13 r307318M: Fri Oct 14 09:23:46 CDT 2016 karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0) VT(vga): text 80x25 CPU: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz (2400.13-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0x206c2 Family=3D0x6 Model=3D0x2c Step= ping=3D2 =20 Features=3D0xbfebfbff =20 Features2=3D0x29ee3ff AMD Features=3D0x2c100800 AMD Features2=3D0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics And I do _*not*_ have it turned off at this moment. vm.pmap.invpcid_works: 0 vm.pmap.pcid_enabled: 1 But.... I've also yet to see any sort of misbehavior. So the questions are: 1. What's the misbehavior I should expect to see if this is not shut off (and it isn't)? 2. Should I take this machine down immediately to reboot it with vm.pmap.pcid_enabled=3D0 in /boot/loader.conf? A svn log perusal didn't see anything post the errata date that appears to be related.... which makes me wonder. Thanks in advance! --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms050401080207000005090104 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTUxNjEyMzFaME8GCSqGSIb3DQEJBDFCBEBX WccYvSZORgAOZBsSlX6YIxEcQV0Wicv2z1oAzZLfe/TE72AeEtUFuYalNXD01/MAKkvaOIdf Q41uzSIsAWB0MGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAHpQEtFSZ JTUkJaBN7cRMsaYxphGtCqn6USOpfsOxvZZpQVT2QUTiMO0ORyxkBPM7bXd0gcHdLObK/PqY RXL98A0znXH6qSZ47IfgZx9p6bo8oPB+Sz8/Nw+YqU5Ox4XwYn/7i6gqDc0YVbZs7ncoX+0J nD4OVABhZnwNOnogk7bsg4HWMIofeB6Z6cSAGInaM5zzxgKWkE65ivl/zMn8ZW1Mubbry1EK pg0wTJWsdUPMDGzWdnqWCsBJDYuAZ8JY3em2nHUhmaXJS3UCDwQHhAefyEoEhzYNFTnmwjiV QNH2KgCEBqPBcprh6Fra+3HZBUXnRJl+C29dZPyVl49kRp25gEPy6Bfr5qaOb0122JkW8m6C di+/gMoeswOz/RLW5sDUQNjzIWj0AF6H+MyQcyyytpTtgBmmJJEdvQ2czVBzLkrfB86rwiTK NvKTJgx5Aqep5mh1l50RzJ0PV3E4HEVEffCIxKbj1cW/SV224u8rQSjBJNz6G0DcFpTeLPoq KZUQuIfqx3C0GJzIlkwODyc6vpxpjw37+k8vksRUqF2g2EFJx5ft4+59voUDdjdH4vCXwztH vzfAtlkVTTuyyP6bb5TtMtIDOvRYHmoltKwWGOC8mr4XGxCffln9EbvrsFqGA+hlMio5yiCU gD8v9mbvgVCxZMpL4X583eEqKMYAAAAAAAA= --------------ms050401080207000005090104--