From owner-freebsd-current@freebsd.org Sun Jul 14 20:12:27 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 120F3A4F9D for ; Sun, 14 Jul 2019 20:12:27 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd42.google.com (mail-io1-xd42.google.com [IPv6:2607:f8b0:4864:20::d42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D69928A7A3 for ; Sun, 14 Jul 2019 20:12:26 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd42.google.com with SMTP id q22so30926452iog.4 for ; Sun, 14 Jul 2019 13:12:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=aZMDomoKzBkxVuO+R5biaMUtwJAnssFtwaV+3BE0tfk=; b=J9/j5G+HpVP/smZPVQ4EIbeYlp2mpKFXE8vRBzUM7EmhUSESCcMnEiE8t7tYF4+0db yNQBGuLBtL6t0Mjw/Lk2NZIzpijbeTdcg8WZ07jdDyJRIpP/KzUrXyk/18ncBilugVjW X8hEKD5AK+v2F5s+AESlbJCMoFcuDIf/ulaqQf578j4DDjKvRE2wJrm+8x1cGcgk5fIx uf3EWeztipKRe16BgVBi1QsZcQhwdtpfqFGoP340axAI9zGwDs1+lAEShtY7iweuApTk 87moIqorQFmiXbWybxgSKSEWLqwMdyQkBrM27tzvvgX7bAbyuvPdnuYT59shkvTfWBQY 9EOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=aZMDomoKzBkxVuO+R5biaMUtwJAnssFtwaV+3BE0tfk=; b=iKhnLJuoUgduf6Y2qFM4VpDOBK95sNymmyCx1WdZDi1umzzpkPaIVun7e3mqZ/nR0J LbmNLe6c5+6ECxhMu9Mm4fOE88n91MSxhPZfbqCbHbUh2ZvCozJ1dQhig1C9yhS1Fpwl humtbPLcYtOoPDuwZjRtuTn32PQn4J/gT510M+jbi7xqW/jXpENwaBHeyQZJkI4CH1Dx gyrUxDNbMAafLAhiyzjzab6+eUYyWjqsC4T2kBWQhadDNHm4IG7DazHzu5N16q3jT/3i 9g1Exm4PIujNKOTJavZYc1IP9qKyF0XFL9gFgn4MVPIZtBeW6pei1f7lKS4pvuOeslpl 70jg== X-Gm-Message-State: APjAAAW/+RdG4eIPi8/03PToQfwdhhCV8pNyJSCxQGmqkXcjGN/6CyHe REX2LB0wtMxmZJ9ljP5KUkat3f/A X-Google-Smtp-Source: APXvYqw+fkvsJ7Z8GBCptzi38xJnrgkRGrr5TrHBrDR1e3Ri6jNZhGPa5mi3LtHBu08eKhfqu8whfA== X-Received: by 2002:a6b:8bd1:: with SMTP id n200mr22280075iod.134.1563135146032; Sun, 14 Jul 2019 13:12:26 -0700 (PDT) Received: from raichu (toroon0560w-lp140-05-70-29-85-38.dsl.bell.ca. [70.29.85.38]) by smtp.gmail.com with ESMTPSA id m4sm14681003iok.68.2019.07.14.13.12.25 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sun, 14 Jul 2019 13:12:25 -0700 (PDT) Sender: Mark Johnston Date: Sun, 14 Jul 2019 16:12:23 -0400 From: Mark Johnston To: Konstantin Belousov Cc: Larry Rosenman , Freebsd current Subject: Re: panic: vm_page_free_prep: freeing mapped page Message-ID: <20190714201223.GD82719@raichu> References: <20190713221457.GP47193@kib.kiev.ua> <20190714193420.GC82719@raichu> <20190714200354.GT47193@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190714200354.GT47193@kib.kiev.ua> User-Agent: Mutt/1.12.1 (2019-06-15) X-Rspamd-Queue-Id: D69928A7A3 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.989,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jul 2019 20:12:27 -0000 On Sun, Jul 14, 2019 at 11:03:54PM +0300, Konstantin Belousov wrote: > On Sun, Jul 14, 2019 at 03:34:20PM -0400, Mark Johnston wrote: > > On Sun, Jul 14, 2019 at 01:14:57AM +0300, Konstantin Belousov wrote: > > > On Sat, Jul 13, 2019 at 04:50:57PM -0500, Larry Rosenman wrote: > > > > I have cores. Ideas? > > > > svn rev: r349976 > > > > > > > > [I] ➜ more core.txt.12 > > > > borg.lerctr.org dumped core - see /var/crash/vmcore.12 > > > > > > > > Sat Jul 13 16:47:03 CDT 2019 > > > > > > > > FreeBSD borg.lerctr.org 13.0-CURRENT FreeBSD 13.0-CURRENT r349976 > > > > LER-MINIMAL amd64 > > > > > > > > panic: vm_page_free_prep: freeing mapped page 0xfffff82031044790 > > > > > > What was the process which caused the panic ? Was it threaded ? > > > > I looked at some of the kernel dumps. In all cases the crashing process > > is postgres. We have: > > > > (kgdb) p/x *m->md.pv_list.tqh_first > > $20 = { > > pv_va = 0x801c00000, > > pv_next = { > > tqe_next = 0xfffff80b3aacb568, > > tqe_prev = 0xfffff81faf7ecbe8 > > } > > } > > (kgdb) p/x *m->md.pv_list.tqh_first->pv_next.tqe_next > > $21 = { > > pv_va = 0x801c00000, > > pv_next = { > > tqe_next = 0x0, > > tqe_prev = 0xfffff80b3ab905d0 > > } > > } > > > > We can find the corresponding pmaps for these mappings by going > > through the corresponding pv_chunks. Then I looked up the other > > processes with mappings of the page. They are also postgres processes. > > We have: > > > > $33 = { > > prev = 0xfffff818baa2be00, > > next = 0xfffff80e7e9875b0, > > left = 0xfffff814df7a6310, > > right = 0xfffff80e7e9875b0, > > start = 0x801c00000, > > end = 0x80aa4a000, > > next_read = 0x801c00000, > > max_free = 0x3000, > > object = { > > vm_object = 0xfffff80bbeb94400, > > sub_map = 0xfffff80bbeb94400 > > }, > > offset = 0x600000, > > eflags = 0x0, > The eflags value is slightly strange. Yeah, I'd expect MAP_ENTRY_COW. > > protection = 0x3, > > max_protection = 0x7, > > inheritance = 0x0, > > read_ahead = 0xf, > > wired_count = 0x0, > > cred = 0x0, > > wiring_thread = 0x0 > > } > > > > and > > > > (kgdb) p $33->object.vm_object->ref_count > > $34 = 0 > > (kgdb) p $33->object.vm_object->shadow_count > > $35 = 5 > What is the object type ? Can you please print the object(s) ? OBJT_DEFAULT. $34 = { lock = { lock_object = { lo_name = 0xffffffff808571d4 "vm object", lo_flags = 627245056, lo_data = 0, lo_witness = 0x0 }, rw_lock = 18446735366207213568 }, object_list = { tqe_next = 0xfffff80bbeb94500, tqe_prev = 0xfffff80bbeb94320 }, shadow_head = { lh_first = 0xfffff8066f43b600 }, shadow_list = { le_next = 0xfffff815ca45de00, le_prev = 0xfffff80665064438 }, memq = { tqh_first = 0xfffff81faf7ecbb0, tqh_last = 0xfffff81fb1719ec0 }, rtree = { rt_root = 18446735385885095616 }, size = 37995, domain = { dr_policy = 0x0, dr_iter = 0 }, generation = 1, ref_count = 0, shadow_count = 5, memattr = 6 '\006', type = 0 '\000', flags = 4104, pg_color = 5632, paging_in_progress = 0, resident_page_count = 26097, backing_object = 0x0, backing_object_offset = 0, pager_object_list = { tqe_next = 0x0, tqe_prev = 0x0 }, rvq = { lh_first = 0x0 }, handle = 0x0, un_pager = { vnp = { vnp_size = 81, writemappings = 0 }, devp = { devp_pglist = { tqh_first = 0x51, tqh_last = 0x0 }, ops = 0x0, dev = 0x0 }, sgp = { sgp_pglist = { tqh_first = 0x51, tqh_last = 0x0 } }, swp = { swp_tmpfs = 0x51, swp_blks = { pt_root = 0 } } }, cred = 0xfffff8104e939600, charge = 149336064, umtx_data = 0x0 }