From owner-freebsd-net@freebsd.org Tue Dec 26 11:36:02 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2119E87230 for ; Tue, 26 Dec 2017 11:36:02 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward104j.mail.yandex.net (forward104j.mail.yandex.net [IPv6:2a02:6b8:0:801:2::107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 730A87130A for ; Tue, 26 Dec 2017 11:36:02 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mxback20j.mail.yandex.net (mxback20j.mail.yandex.net [IPv6:2a02:6b8:0:1619::114]) by forward104j.mail.yandex.net (Yandex) with ESMTP id 74E5E4111B; Tue, 26 Dec 2017 14:35:59 +0300 (MSK) Received: from smtp4j.mail.yandex.net (smtp4j.mail.yandex.net [2a02:6b8:0:1619::15:6]) by mxback20j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id Q5lWXZPsKI-ZxYiYY2Z; Tue, 26 Dec 2017 14:35:59 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1514288159; bh=gJ/Gy2SCeU90i24913+aQNKi0om+cjUZcZKClhTQtDo=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=ERDwMJ0Iw+KdlWWMz5E4jchi899LuXlWWSqAAUSDuhhnYh2u8D/mdcucqSgjeS6Fx dS1LtSHVUaj72ZsB4bRahpQGycgKWKII2s17jiMNElB6iD0G9uFUdAGiRwFjkUysLL iXfLS6xv4azgZqC3/l1ee/RakAhYuYa+qyYZ0M+4= Received: by smtp4j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id LT5bzeAmOe-Zwx4R7oc; Tue, 26 Dec 2017 14:35:58 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1514288158; bh=gJ/Gy2SCeU90i24913+aQNKi0om+cjUZcZKClhTQtDo=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=oyZzsC4BvtDANyfR9CmtQ1w4Y/Iz9ThJ5J0buKogWJu3jtITEQsOvpNc9jEm02+dT zkyRvxAxXZmEoxvOiHrTmDkwxeF35PmJCW4kq5EX90MQNDlomSJd2nuhSTEIuBz4uC 9TKR5VpSaKEWMG17vZBK8Nb1DQxHURXFm5I18B/Q= Authentication-Results: smtp4j.mail.yandex.net; dkim=pass header.i=@yandex.ru Subject: Re: [freebsd-current]Who should reset M_PKTHDR flag in m_buf when IP packets are fragmented. m_unshare panic throw when IPSec is enabled To: Harsh Jain , freebsd-net@freebsd.org References: <73302ead-b2e9-c25b-4d11-475f38dec1a1@chelsio.com> <993c58bb-3bf2-d6a3-9a05-13e1631aec87@yandex.ru> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Message-ID: Date: Tue, 26 Dec 2017 14:33:04 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="WUp1Gkiol6r8GwihO60R2BQFxQ43xhLLL" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Dec 2017 11:36:02 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WUp1Gkiol6r8GwihO60R2BQFxQ43xhLLL Content-Type: multipart/mixed; boundary="LXau3uXE3xeG2wKXx8OV7wRRdsfm6Kn9U"; protected-headers="v1" From: "Andrey V. Elsukov" To: Harsh Jain , freebsd-net@freebsd.org Message-ID: Subject: Re: [freebsd-current]Who should reset M_PKTHDR flag in m_buf when IP packets are fragmented. m_unshare panic throw when IPSec is enabled References: <73302ead-b2e9-c25b-4d11-475f38dec1a1@chelsio.com> <993c58bb-3bf2-d6a3-9a05-13e1631aec87@yandex.ru> In-Reply-To: --LXau3uXE3xeG2wKXx8OV7wRRdsfm6Kn9U Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 26.12.2017 13:22, Harsh Jain wrote: >>> panic: m_unshare: m0 0xfffff80020f82600, m 0xfffff8005d054100 has M_P= KTHDR >>> cpuid =3D 15 >>> time =3D 1495578455 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 0xfffffe0= 44e9bb890 >>> kdb_backtrace() at kdb_backtrace+0x53/frame 0xfffffe044e9bb960 >>> vpanic() at vpanic+0x269/frame 0xfffffe044e9bba30 >>> kassert_panic() at kassert_panic+0xc7/frame 0xfffffe044e9bbac0 >>> m_unshare() at m_unshare+0x578/frame 0xfffffe044e9bbbc0 >>> esp_output() at esp_output+0x44c/frame 0xfffffe044e9bbe40 >>> ipsec4_perform_request() at ipsec4_perform_request+0x5df/frame 0xffff= fe044e9bbff0 >> Hi, >> >> it seems unusual that IP reassembly happens on outbound path. > It can be re-produced with single Ping packet on chelsio(cxgbe) NIC. I = tried with Intel NIC. It seems they re-produce M_WRITEABLE() buffer(follo= ws different path in m_unshare) which is not true for cxgbe. In my view, IP fragmentation should occur in ip_output after IPsec encryption. Something like: 1. rip_output() has mbuf chain where only first mbuf has M_PKTHDR flag 2. ip_output() -> IPSEC_OUTPUT() -> esp_output() -> m_unshare(). We should still have only one mbuf with M_PKTHDR flag here. 3. esp_output_cb() -> ipsec_process_done() -> ip_output() 4. Now IP fragmentation should occur: ip_fragment() creates chain of mbufs to send, where M_PKTHDR flag will be set for each fragment. >> Do you have some packet normalization using firewall? > Default FREEBSD current installation. No explicit firewall. > What you think above patch makes sense. It is not clear to me why it helps. The panic happens on outbound path, where mbuf should be allocated by network stack and should be writeable. ip_reass() usually used on inbound path. I think the patch just hides the problem in another place. Do you mean that cxgbe can produce !WRITEABLE mbuf for received packet and then pass it to the network stack? --=20 WBR, Andrey V. Elsukov --LXau3uXE3xeG2wKXx8OV7wRRdsfm6Kn9U-- --WUp1Gkiol6r8GwihO60R2BQFxQ43xhLLL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAlpCM3EACgkQAcXqBBDI oXoKSQf+LOnwK7M78ZKfmidNEe+2Fzk4fW6KmlNRWNq2nc9n9qGz0wY3sPTx40u/ nj/oOfstoIPjJdQ31iFSLwLFW18beb85ysd9UJlDORlfVNS+DZHV5wCFOmG5u9jj SHEGF5OY2TrA8ek3XaBwRs6pl+eG7h1poupdSgR8FNO1567onqeKXegydstIsNwT n2Q/LpJ2vd49eHigHB+C2qzt+GQGbBNiY8Iw+hdpQsSDR86HthJTlkPY06UPNHyu PrPRa838TI+BSpw1oe76SQ6mhFLPjz0zN9uRSahYtSoUhHnGrSpbKG332CwOHrsX btptPf1wiidsLBpZeXjqEsELw91QaQ== =Inge -----END PGP SIGNATURE----- --WUp1Gkiol6r8GwihO60R2BQFxQ43xhLLL--