From owner-freebsd-current@freebsd.org Fri Jan 8 18:09:27 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C64C2A68A63 for ; Fri, 8 Jan 2016 18:09:27 +0000 (UTC) (envelope-from jtl@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8132A10B9 for ; Fri, 8 Jan 2016 18:09:27 +0000 (UTC) (envelope-from jtl@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 7EDA3A68A62; Fri, 8 Jan 2016 18:09:27 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 646B6A68A61 for ; Fri, 8 Jan 2016 18:09:27 +0000 (UTC) (envelope-from jtl@freebsd.org) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0128.outbound.protection.outlook.com [207.46.100.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F347A10B7; Fri, 8 Jan 2016 18:09:26 +0000 (UTC) (envelope-from jtl@freebsd.org) Received: from SN1PR05CA0031.namprd05.prod.outlook.com (10.163.68.169) by DM2PR0501MB1392.namprd05.prod.outlook.com (10.161.224.139) with Microsoft SMTP Server (TLS) id 15.1.361.13; Fri, 8 Jan 2016 14:37:26 +0000 Received: from BL2FFO11FD048.protection.gbl (2a01:111:f400:7c09::175) by SN1PR05CA0031.outlook.office365.com (2a01:111:e400:5197::41) with Microsoft SMTP Server (TLS) id 15.1.365.18 via Frontend Transport; Fri, 8 Jan 2016 14:37:25 +0000 Authentication-Results: spf=softfail (sender IP is 66.129.239.19) smtp.mailfrom=freebsd.org; catwhisker.org; dkim=none (message not signed) header.d=none; catwhisker.org; dmarc=none action=none header.from=freebsd.org; Received-SPF: SoftFail (protection.outlook.com: domain of transitioning freebsd.org discourages use of 66.129.239.19 as permitted sender) Received: from p-emfe01b-sac.jnpr.net (66.129.239.19) by BL2FFO11FD048.mail.protection.outlook.com (10.173.161.210) with Microsoft SMTP Server (TLS) id 15.1.355.15 via Frontend Transport; Fri, 8 Jan 2016 14:37:25 +0000 Received: from magenta.juniper.net (172.17.27.123) by p-emfe01b-sac.jnpr.net (172.24.192.21) with Microsoft SMTP Server (TLS) id 14.3.123.3; Fri, 8 Jan 2016 06:34:28 -0800 Received: from [172.29.34.89] (gkovich-sslvpn-nc.jnpr.net [172.29.34.89]) by magenta.juniper.net (8.11.3/8.11.3) with ESMTP id u08EYPD34964; Fri, 8 Jan 2016 06:34:25 -0800 (PST) (envelope-from jtl@freebsd.org) User-Agent: Microsoft-MacOutlook/14.5.9.151119 Date: Fri, 8 Jan 2016 09:34:23 -0500 Subject: Re: panic: sbappendstream 1 [head/amd64 @r293419] From: "Jonathan T. Looney" Sender: Jonathan Looney To: David Wolfskill CC: , Gleb Smirnoff Message-ID: Thread-Topic: panic: sbappendstream 1 [head/amd64 @r293419] References: <20160108140518.GM2781@albert.catwhisker.org> In-Reply-To: <20160108140518.GM2781@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD048; 1:wo1cT5GtBSVrtssZ5XeISx8Rt91jfKsQePv7rwwgO5mgYum26ox5ob83QJcP297cxVRijL9UblP6TW85iu4cgy61Nsr+z96Os6UvJpQjcg8hKJWGk4f7NNM4q6z1re0i7uyvcCxO3wzSGc7TAwitIkDzv4p52/W/euQYB7ucWKqJ9P5Pl2SqvMBfx1FKAge7gDe+/ea3hcr9vVBW9jEaTWXMkRICtPopF54tQ5s36/sHUP/uw0tacA1RThw4msTAYgpiL3FVEBo6+7d5wC78VgTrx3NVHFmhHb9kbIAExNkm7EmSx41kGxsuzP5y4Pw+/VKkZQbl6g+R+b2cgRxVmHy9LnX+L5CqgvfQ3lFOxFI= X-Forefront-Antispam-Report: CIP:66.129.239.19; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(2980300002)(377454003)(24454002)(479174004)(189002)(199003)(83506001)(2906002)(1220700001)(81156007)(4001350100001)(1096002)(6806005)(106466001)(36756003)(110136002)(16796002)(23726003)(47776003)(586003)(15975445007)(230700001)(5001960100002)(189998001)(4326007)(87936001)(575784001)(97736004)(86362001)(46406003)(19580405001)(92566002)(77096005)(11100500001)(76176999)(54356999)(19580395003)(50986999)(50466002)(69596002)(2950100001)(42262002); DIR:OUT; SFP:1102; SCL:1; SRVR:DM2PR0501MB1392; H:p-emfe01b-sac.jnpr.net; FPR:; SPF:SoftFail; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; DM2PR0501MB1392; 2:gNXc0L+ephdJYRL+T0G0VYOA6NTRmHlK8OsPVtyVSE5gqPdypiafcddZCkFgBHG3NwubYEH7hxOWl8MDjRFkm4NC5cv1zuNiKsEUlBuzGE5+kqPj5g+THnJW+sB2ZnKDQYntI4RdBC2BYG1ho7H1AA==; 3:5jmvsVfAvLYc4E8olrkHU6WJiPLGzpGoZki7nyIZ1O4+362n82MJhRvVyeXbDRip4sYh7fQMsFtrGhGMbC7/O+q699bLPK8kKqaQdst/6mf2acWoj3UaMb+wrgKXjLUIjfbXD8iKm6J/T09VQ5dip6r/pn9ReO6W4CHvmH45PvXsPNmMq9FDNlEBtvlpjS1JHPXSVA/axijHCyb7eE7Nha5rsG3tGdHNzPVuiYPmnbk=; 25:H0gqJKSFHXJUcdCJ/QH+lvAj4W2kLj4GV6++0JSxEvNcfzRDAK+whbl8vMP+xbVOqyWL7jNfV7a3VHokG79aPcxa7u3VKFk3yrC+QLnWOpB8tp762YDuMB0dyY9mq5ZbO+s9/WqMtLkMYQ1ikmycGe7SH1M9kk0lileFRyi0iknqSV9MjSJAxZ6oorVfrC/3vSndo5lBBNsDLSwyu/+rfmf+KG1+PlQ5ovK97XbhVBVyzz+eDKmpE4UsRNUDCxCL X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM2PR0501MB1392; X-MS-Office365-Filtering-Correlation-Id: cd9451f3-ba44-445e-3e77-08d318393a00 X-Microsoft-Exchange-Diagnostics: 1; DM2PR0501MB1392; 20:puJmW4MftNlxXHFSYtHoidvrSVX7dNkvKrKT1Uz9j82O2cAIunFnuo2JvGjTXfA4MzXPY460JO8mP7NRaI+AikLjC/L0Z/t6Ny0u8DEnAo9WzTJxzEr6qIiPp6+tR6TaNo9v96Wvx4jl2PG6NeNpRtki3yLHKe4KXVMJj7Sv4AJ7RF+oF5QANkL1MPGSikVauIKh3DIJc+w4avtZRn/y5omJqV+7KreZFd7RqJtGYpkvlo8GwbT8j2c2OzmURyAbZjZnBp42kiJsH6oIqtv/wI3KongA6AeHuCwzXqZyabbWco4eXPjyS06Ir2UvSh2Jz+uxtR9StAi/tIGSJw3yPxRKjzA/HEbQuyKDlmc6ePlGtfbhwuq+AdLfPC6zdUrGIOXFn8GN5GDtMC1+loiuv4o9LC6OJ+eLrhcn8oPsGQFOjfohfuhyxH4Vcm5OL/uyaBxh7l6EePNpzzuGQb0gqFO8tz2nN7o9fnb1XbzkNcL+MHKAdKED7fp4vfjxrpPI; 4:0Cm6kQ0vxoSQuc98Dkh04R+XVjZt7WJ6C5pXw6ZTfL/C0I2Tx0QGeOhp7jTsTK4pPIwi7FKnzZjdogvcqVU2dqOHJOKlv0FLaswPcTblMLol07f8BJZYhT+7k8JbqYy14B67+/V1zlZ0WNPtedAkk7Ni/TBKSA6Q1vh2JHjq56RrFPreK1RzSU4wvUtDuDH5Xsk+ENxfBqkchh+f6EmkTX/ZvltbtD9RWlh6BuP1Y3yrJxWx8TFjhkAVphr2n8x0Zecg87BH9uqB4h3nbIHrhGbW08VsGddi4Mle6fZaIswsghnogdqOdzKEvSWSrJko5jNGuWhnTxcgGVFwFomHH7ZjQXa7GkosehOSPaAWSSOXoKvRtZKqreMpfw2gGD6/Y95veGE0KcuPwnGzag4M0OXeZwaM0Vyn1gDOLrL10meOkijhDDlp4EZ8B0/okWDX X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(520078)(13015025)(13017025)(5005006)(8121501046)(13018025)(10201501046)(3002001); SRVR:DM2PR0501MB1392; BCL:0; PCL:0; RULEID:; SRVR:DM2PR0501MB1392; X-Forefront-PRVS: 0815F8251E X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM2PR0501MB1392; 23:DwFFHGdFJzTvVSoaf3A8GmPugfemTrL6dGDoVk+?= =?us-ascii?Q?MRHfJEdLmPhNLjyBR0c8rEOphacj3jQ+VyKLfXp0Rn1zvuWMniCICGiUFZyC?= =?us-ascii?Q?nQdPKjjwOPXOLiEHDkHjly4WENKcenXMqLsAH88vc4a1YtSO79t8Vs+uNRpg?= =?us-ascii?Q?/w5QkPGcD9PX2Uec3jytnkQo8mqUJnF7/2l3RYR4IXBOZoJSVGh0Z7EpFcYC?= =?us-ascii?Q?sWtJutQi6JOXgOHcIQz0Udu9yzZJ3+6eEiiRUxmMQoxYVamGAaXg5axVCftl?= =?us-ascii?Q?+CL23JefBCUr5eMkfrz2WZ5YrNWadR9oF5x6AzuxEncvDyPurwvIztwSXrE7?= =?us-ascii?Q?TWoHyCJHeOXzAjTSSphXsPHyPPJvNFPEygLosC7JsC/8MfWKLw3PCor9j5d/?= =?us-ascii?Q?tlbXd2e+qgybLSr8cQkIeCnOQ0M2KUxU6seZNGzuJKz4DoEr254OvUXN7hRy?= =?us-ascii?Q?11AyVele+Apgw+z9tgFVS+MJBGBHtikBmhKGUWfcwfe2E8BSsb/ay65BPBL+?= =?us-ascii?Q?sBwmb+OsCVNAm58sWbmBHEVFFGtzXtHaanJgIMdKIQZhwjJLIjsW8VMkG2C6?= =?us-ascii?Q?nRFsy5nQSEgaS7Q+5e3h5ZS9pCmR/b5VzeJJP6hVSZgpcg4LzbU2340h9VdE?= =?us-ascii?Q?34NumxP8/lh0m75ApPX7VLPOhkTnwKr6Jwy/R5HgkLSiE3ucY8Zu5poNA31s?= =?us-ascii?Q?ChvOr3QREDPD/uh+HYddjtje5urrvR8IQlLt9t3kMJ/LTnX+nq0In6wOccka?= =?us-ascii?Q?4h1c7GKY0poi7zCNOSDTwTnxqkdCzqsPsVwk+qeVEnEzxgL5d5athvQXTKL4?= =?us-ascii?Q?crVpQS4fQOqQ8kZvvGQlkCJM5Lmjxcls757GuFl4KtGT1VuiKzNmyXz0V2EP?= =?us-ascii?Q?cUxAys6PN4vK/IWvL5YkXNdK8mom3uRYA7byRbjCJ0uX04A0C5i29dpJ/nX1?= =?us-ascii?Q?2kzYwagZqe2Eaor8/a+CBIzcYu3iU2s+sQw2uXAShojeZR3NOo5OsL+uUGwg?= =?us-ascii?Q?7BLmL1VQ4IeQfM7xaD7GlwWQQA0qJhtK8tHWhKED0BpckfFPPyh81Z+f48lN?= =?us-ascii?Q?Ppeerd+34WRKZkSa3wXu7hw7zfBsJYDb6XgEwwu3f9rL4pu/tCjueyS3fga2?= =?us-ascii?Q?YuDvjRJ9EJdHD5tCAwB73MFn1A7ZN9Slz?= X-Microsoft-Exchange-Diagnostics: 1; DM2PR0501MB1392; 5:9+UF2ah6jbq59x93N71ONNXr5t+ZwbQ981Ng2CiGfZAkoDnKsm1eSm/HU1aFCHcT4ySYsnF87FbSOlwd5BCLjCohFCqSO8EP9RO0u7JFWL/IHutwfw5SpmxW+nv3SBDPsBilxLY+QRIlz9aA+LzoZQ==; 24:ofEciEdEDlDMyWFTLFXzYk0tQmHQdhbacA8XAOjRoVQ7EssJSsLMlm0WCQ/6dsA+PN+XPYYAbJ3uMBsxsW/EEEb2Zk96G6vPBmJNl4zEsLI= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jan 2016 14:37:25.1080 (UTC) X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4; Ip=[66.129.239.19]; Helo=[p-emfe01b-sac.jnpr.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0501MB1392 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2016 18:09:27 -0000 On 1/8/16, 9:05 AM, "David Wolfskill" wrote: >After the first panic, I rebuilt the kernel without -DNO_CLEAN; the >crash dump & other diagnostic info is from the clean build. > >January 8, 2016 at 05:57:27 AM PST > >FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1954 >r293419M/293420:1100093: Fri Jan 8 05:09:57 PST 2016 >root@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC amd64 > >panic: sbappendstream 1 > >... >Unread portion of the kernel message buffer: >panic: sbappendstream 1 >cpuid = 7 >KDB: stack backtrace: >db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >0xfffffe085e0595b0 >vpanic() at vpanic+0x182/frame 0xfffffe085e059630 >kassert_panic() at kassert_panic+0x126/frame 0xfffffe085e0596a0 >sbappendstream_locked() at sbappendstream_locked+0xa5/frame >0xfffffe085e0596d0 >uipc_send() at uipc_send+0x942/frame 0xfffffe085e059780 >sosend_generic() at sosend_generic+0x42f/frame 0xfffffe085e059840 >kern_sendit() at kern_sendit+0x21b/frame 0xfffffe085e0598f0 >sendit() at sendit+0x126/frame 0xfffffe085e059940 >sys_sendmsg() at sys_sendmsg+0x61/frame 0xfffffe085e0599a0 >amd64_syscall() at amd64_syscall+0x2db/frame 0xfffffe085e059ab0 >Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe085e059ab0 The likely suspect here looks like r293405, which changed uipc_send() to use sbappendstream_locked() instead of sbappend_locked(). However, I can't explain *why* that change is causing this problem without further investigation. Can you try reverting the change to see if that solves the problem you are seeing? Thanks! Jonathan >--- syscall (28, FreeBSD ELF64, sys_sendmsg), rip = 0x801270dfa, rsp = >0x7fffffffa098, rbp = 0x7fffffffa0d0 --- >KDB: enter: panic >... >Loaded symbols for /boot/kernel/autofs.ko >#0 doadump (textdump=0) at pcpu.h:221 >221 pcpu.h: No such file or directory. > in pcpu.h >(kgdb) #0 doadump (textdump=0) at pcpu.h:221 >#1 0xffffffff8038205b in db_dump (dummy=, >dummy2=false, > dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533 >#2 0xffffffff80381e4e in db_command (cmd_table=0x0) > at /usr/src/sys/ddb/db_command.c:440 >#3 0xffffffff80381be4 in db_command_loop () > at /usr/src/sys/ddb/db_command.c:493 >#4 0xffffffff8038467b in db_trap (type=, code=0) > at /usr/src/sys/ddb/db_main.c:251 >#5 0xffffffff80a5cfe3 in kdb_trap (type=3, code=0, tf=out>) > at /usr/src/sys/kern/subr_kdb.c:654 >#6 0xffffffff80e6a2a8 in trap (frame=0xfffffe085e0594e0) > at /usr/src/sys/amd64/amd64/trap.c:549 >#7 0xffffffff80e4a317 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:234 >#8 0xffffffff80a5c6cb in kdb_enter (why=0xffffffff8137af3c "panic", > msg=0x80
) at cpufunc.h:63 >#9 0xffffffff80a1fb8f in vpanic (fmt=, > ap=) at /usr/src/sys/kern/kern_shutdown.c:750 >#10 0xffffffff80a1f9e6 in kassert_panic (fmt=) > at /usr/src/sys/kern/kern_shutdown.c:647 >#11 0xffffffff80aa3375 in sbappendstream_locked (sb=0xfffff80044212378, > m=0xfffff800108c7200, flags=0) at /usr/src/sys/kern/uipc_sockbuf.c:642 >#12 0xffffffff80ab1a42 in uipc_send (so=0xfffff80044212000, flags=0, > m=, nam=0x0, control=, > td=0xfffff8001078e9a0) at /usr/src/sys/kern/uipc_usrreq.c:984 >#13 0xffffffff80aa5f5f in sosend_generic (so=0xfffff80044212000, >addr=0x0, > uio=0xfffffe085e059890, top=, > control=, flags=, > td=0xfffffe085e059880) at /usr/src/sys/kern/uipc_socket.c:1349 >#14 0xffffffff80aac36b in kern_sendit (td=0xfffff8001078e9a0, s=6, > mp=, flags=0, control=0x0, segflg=UIO_USERSPACE) > at /usr/src/sys/kern/uipc_syscalls.c:906 >#15 0xffffffff80aac666 in sendit (td=0xfffff8001078e9a0, > s=, mp=0xfffffe085e059958, flags=0) > at /usr/src/sys/kern/uipc_syscalls.c:833 >#16 0xffffffff80aac6f1 in sys_sendmsg (td=0xfffff8001078e9a0, > uap=0xfffffe085e059a40) at /usr/src/sys/kern/uipc_syscalls.c:1035 >#17 0xffffffff80e6b13b in amd64_syscall (td=0xfffff8001078e9a0, traced=0) > at subr_syscall.c:135 >#18 0xffffffff80e4a5fb in Xfast_syscall () > at /usr/src/sys/amd64/amd64/exception.S:394 >#19 0x0000000801270dfa in ?? () >Previous frame inner to this frame (corrupt stack?) >Current language: auto; currently minimal >(kgdb) >..... > >As indicated above, this is with a GENERIC kernel. My laptop (running >a kernel built with the same sources, but a slightly customized kernel >config) gets to the point of allowing me to login (via xdm), but when I >fire off a command that creates xterms & tries to run tmux(1) in them, >locks up (as far as I can tell), and a power-cycle is needed to recover. > >I can poke at the crash dump (given hints), make the dump and core.txt >file >available. > >Peace, >david >-- >David H. Wolfskill david@catwhisker.org >Those who would murder in the name of God or prophet are blasphemous >cowards. > >See http://www.catwhisker.org/~david/publickey.gpg for my public key.