From owner-freebsd-drivers@freebsd.org Thu Jan 2 17:47:25 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 40BCE1D5381; Thu, 2 Jan 2020 17:47:25 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47pb8X3GGFz3D4V; Thu, 2 Jan 2020 17:47:24 +0000 (UTC) (envelope-from rajfbsd@gmail.com) Received: by mail-wm1-x32b.google.com with SMTP id q9so6248205wmj.5; Thu, 02 Jan 2020 09:47:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=MOd87AULyvVBfd6Nv6bmpREQuj3bxLKRK3+dUD6v/xU=; b=eXEJWaIDl0kxtJM5GT6XHV6XYAH1KY1zLyHVMtqjnewZrHNswq+G3AQtfG5fJoaWsL yMEcdTjkPOlfpibvCk9vcUHiUoppcmKO0fPfIa+S2k4B4l5hrPnIbWEdpXQPEydjB0am SKKj+P2N8ecBuT/2wKk/K2CseJhSt42D7uIRPXBRU8PCPk/HR/EzEGYvTr8F9i+goq4a VxkOVDtHhc0tRLqfPz1FcpAE7yij7WVZj8bWGfkOl0pskZqZD3SveevxRTIUH0A3z304 m2+HcrTGsfWmbKUFc02DBGC1hBmEWAPeXQ7Fv+zuBnMgxEZSAhO+62cx5V12WnP216ZK 2G7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=MOd87AULyvVBfd6Nv6bmpREQuj3bxLKRK3+dUD6v/xU=; b=HlX7lQQKoL9mlzOlgDblxRkbqTugdcmhHZdbcPFWr472XpvaimxF/St2ctIpoi1v6+ TIzi58Qaq4W6jbT+bsJhvKxABQULqGZMcVBp6iFuY7yv0Uf9Ij6qLa6f0DJazuK/XEsa MazVjQtuvF9hHdohrR3VgicwYLD6LfY1b/VR6OZzLPxQ+ytig+zC//HFqwb6E4nBsdDB dUFyrr5g1w/d1rdCU+Piy0GvQrm2OuvQJ2A1z8UO5667gi6L07olGcDhEsoX0mMF8whj xePvFS1nnS0Ac/QYdZ8nsTSDXtU4aeDTG7u1UbyX+1m4ACtRBv8HtcrqIxU2h8JDscWY o3sg== X-Gm-Message-State: APjAAAWh23tRe7fsdbeZMcRb+BhgG1oOiu03xdO1hpR9sZGI/GYAEoUz Gixhx9aUJiz8pmcJ2HhL10ZytNrmaxfpS5GsqR0d+bzT X-Google-Smtp-Source: APXvYqxzpgkGKgbtCmJ3dBmj3wTgBJcphFzjxp8g54EQZzCN0RKWZaIYH83rN3gCP1LZLR6XBmhujV7BRfPTAbVbt8g= X-Received: by 2002:a1c:e007:: with SMTP id x7mr14582442wmg.3.1577987241947; Thu, 02 Jan 2020 09:47:21 -0800 (PST) MIME-Version: 1.0 From: Rajesh Kumar Date: Thu, 2 Jan 2020 23:17:10 +0530 Message-ID: Subject: panic: smp_targeted_tlb_shootdown: interrupts disabled - FreeBSD 12.0 To: freebsd-drivers@freebsd.org, freebsd-amd64@freebsd.org X-Rspamd-Queue-Id: 47pb8X3GGFz3D4V X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=eXEJWaID; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of rajfbsd@gmail.com designates 2a00:1450:4864:20::32b as permitted sender) smtp.mailfrom=rajfbsd@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(0.00)[ip: (-9.17), ipnet: 2a00:1450::/32(-2.63), asn: 15169(-1.87), country: US(-0.05)]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[b.2.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jan 2020 17:47:25 -0000 Hi, I am hitting the below panic when I am testing couple of my drivers. When i looked at the source, the reason is PSL_I bit (Interrupt enabled) bit is not set when read in "smp_targeted_tlb_shootdown" routine. But I am wondering what leads to this. *Driver test scenario:* 1. Driver has a kernel thread, which allocates memory and dma maps it and send the dma addr to Driver 2. Then wait (msleep) till it gets a notification from Driver 2. 2. Driver 2 has a kernel thread which will submit the received address from Driver1 to hardware and wait (msleep) for an interrupt 3. Driver 2 receives a interrupt and wakes up the thread (wakeup_one) in Driver2. Thread in Driver 2 then notifies Driver1 through a callback. 4. Callback in Driver 1 wakeup (wakeup_one) it's sleeping kernel thread. The kernel thread then dma unmap the buffer the frees the memory Kernel Panics while the memory is freed at last. I am sure this is a driver, but not sure what am I doing wrong here. Please help. panic: smp_targeted_tlb_shootdown: interrupts disabled cpuid = 44 time = 1577463701 KDB: stack backtrace: #0 0xffffffff80be7977 at kdb_backtrace+0x67 #1 0xffffffff80b9b563 at vpanic+0x1a3 #2 0xffffffff80b9b3b3 at panic+0x43 #3 0xffffffff811eea8f at smp_targeted_tlb_shootdown+0x40f #4 0xffffffff811eeb22 at smp_masked_invlpg_range+0x42 #5 0xffffffff8105e67c at pmap_invalidate_range+0x25c #6 0xffffffff81061cdd at pmap_remove_ptes+0x6d #7 0xffffffff8106079b at pmap_remove+0x35b #8 0xffffffff80ee0366 at _kmem_unback+0x36 #9 0xffffffff80ee04cd at kmem_free+0x2d #10 0xffffffff829fc5f8 at driver_thread_work+0x798 #11 0xffffffff80bf9cb4 at taskqueue_run_locked+0x154 #12 0xffffffff80bfae18 at taskqueue_thread_loop+0x98 #13 0xffffffff80b5bf33 at fork_exit+0x83 #14 0xffffffff810501be at fork_trampoline+0xe Please let me know if any details needed. Thanks, Rajesh.