From owner-freebsd-xen@freebsd.org Sun Jul 29 22:30:32 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F751106588C for ; Sun, 29 Jul 2018 22:30:32 +0000 (UTC) (envelope-from akshay1994.leo@gmail.com) Received: from mail-lj1-f181.google.com (mail-lj1-f181.google.com [209.85.208.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 848367CE80; Sun, 29 Jul 2018 22:30:31 +0000 (UTC) (envelope-from akshay1994.leo@gmail.com) Received: by mail-lj1-f181.google.com with SMTP id j19-v6so8756993ljc.7; Sun, 29 Jul 2018 15:30:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vS9H3r7Y/87a7vyrILh9R9mS3kAk1s3iimbsMfOp3RU=; b=Y9uOsZQKl3phRo8wYnfH3HwkY+YrTOoBsyhOEy+rjT+vxTgBnonFqtz1hEJPpmCg2N X/x3TXA0N93JQbtut4+xYN93L1FfAcEaH1D5UUwE7AbjQTxESEPmZ+yBSIODQGMxuBHR gmmDwVgKTGl8MKuRJEf9xmIWN4SNwCrJSGy3p9Tne9R2yqNnCIH/Pn05Ndd6G3FeO74c YHfgG1WVp//JpPsbzfNM7S9dbLlrW3k/CN/U/AJ2BmSt/QTA/+ACtjsePDBIAbHrBDGA 6ROm7hoMPAl0QRabXsEpMYACGT/p/0qwJRAj0HlQ4kcYpK/2RDOs7DeZe88W73CrQKe3 wWUQ== X-Gm-Message-State: AOUpUlHiBSKgiiLg2Eh2lrVw7HGE7kdWTktemTYOGAs8OCNH4D+6IzLE 2k9XSUAfyBo9EM4ywtPoT/opbfk+gVY= X-Google-Smtp-Source: AAOMgpeho0sPaomiOXdMpXzte2n9GI7ZkaGOSh+uZaYik8bktcdF3PDzYuVAIw2UReVSmXwpNI6bmw== X-Received: by 2002:a2e:8950:: with SMTP id b16-v6mr5802384ljk.111.1532896919843; Sun, 29 Jul 2018 13:41:59 -0700 (PDT) Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com. [209.85.167.47]) by smtp.gmail.com with ESMTPSA id m9-v6sm1317879lfg.0.2018.07.29.13.41.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Jul 2018 13:41:59 -0700 (PDT) Received: by mail-lf1-f47.google.com with SMTP id u14-v6so6735791lfu.0; Sun, 29 Jul 2018 13:41:59 -0700 (PDT) X-Received: by 2002:a19:c403:: with SMTP id u3-v6mr8380254lff.87.1532896919118; Sun, 29 Jul 2018 13:41:59 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Akshay Jaggi Date: Sun, 29 Jul 2018 21:41:47 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Recursion in non-recursive mutex when using the grant table free callbacks To: Pratyush Yadav Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , Edward Napierala , FreeBSD-Xen Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 22:30:32 -0000 Roger would be a better person to answer this and I would wait for him to respond, but here are my 2 cents. First let's finalize and record the semantics of the callbacks. What guarantee do we want to provide callbacks about locks held (no locks held OR x,y,z locks held)? This is mostly a product decision. Usually callbacks are called with no locks held. Post this problem is simple. If the list_lock would be held during callbacks, we need to provide a lock-free function (which assumes lock would be held before function is called and use that) and if the lock would not be held we should fix the locking to ensure the case you mentioned should not happen. WDYT? Regards, Akshay On Sun, 29 Jul 2018 at 14:43, Pratyush Yadav wrote: > Hi, > > Currently, the grant table free callbacks can not work. This is > because of a recursion on a non-recursive mutex that causes a kernel > panic. The cause of the recursion is: check_free_callbacks() is always > called with the lock gnttab_list_lock held. So, the callback function > is called with the lock held. So, when the client uses any of the > grant reference allocation methods get_free_entries() is called, which > tries to acquire gnttab_list_lock(grant_table.c:77 [0]), causing a > recursion on the lock. > > I'm not sure what the correct fix would be though. One way I can think > of is that check_free_callback() should be called without the lock > held. But with this fix, it is possible for the callback to be called > even though the grant references it needs are not available. This > would happen when another thread takes those references while the > current thread has completed the check if(gnttab_free_count >= > callback->count) but has not yet called the callback > (grant_table,c:105 [1]). > > I think a better way to fix this would be to have a check in > get_free_entries() whether the current thread holds the lock, so it > does not try to acquire the lock if the current thread already holds > it. > > If you agree, I will send a patch. > > > > [0] > https://github.com/freebsd/freebsd/blob/master/sys/dev/xen/grant_table/grant_table.c#L77 > [1] > https://github.com/freebsd/freebsd/blob/master/sys/dev/xen/grant_table/grant_table.c#L105 > > -- > Regards, > Pratyush Yadav >