From owner-freebsd-fs@freebsd.org Sat Dec 12 01:09:00 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFC8F47953B for ; Sat, 12 Dec 2020 01:09:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ct8gJ370Mz3hJK for ; Sat, 12 Dec 2020 01:09:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 0BC18rUV060892 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 12 Dec 2020 03:08:56 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 0BC18rUV060892 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 0BC18r27060891; Sat, 12 Dec 2020 03:08:53 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 12 Dec 2020 03:08:53 +0200 From: Konstantin Belousov To: J David Cc: freebsd-fs@freebsd.org Subject: Re: Major issues with nfsv4 Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Ct8gJ370Mz3hJK X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2020 01:09:00 -0000 On Fri, Dec 11, 2020 at 03:30:29PM -0500, J David wrote: > On Thu, Dec 10, 2020 at 1:20 PM Konstantin Belousov wrote: > > E means exiting process. Is it multithreaded ? > > Show procstat -kk -p output for it. > > To answer this separately, procstat -kk of an exiting process > generating huge volumes of getattr requests produces nothing but the > headers: > > # ps Haxlww | fgrep DNE > 0 21281 18549 1 20 0 11196 2560 piperd S+ 1 > 0:00.00 fgrep DNE > 125428 9661 1 0 36 15 0 16 nfsreq DNE+J 3- > 3:22.54 job_exec > # proctstat -kk 9661 > PID TID COMM TDNAME KSTACK > > This happened while retesting on NFSv4.1. Although I don't know if > the process was originally multithreaded, it appears it wasn't even > single-threaded by the time it got into this state. Ok, do 'procstat -kk -a' instead. Exiting processes are not excluded from the kstack sysctl, might be you just raced with termination. Or, if you have serial console, enter ddb, then do 'bt '. Or if you have kernel built with symbols, # kgdb /boot/kernel/kernel /dev/mem (gdb) proc (gdb) bt but this has low chances of work for running process. procstat -kk -a output might be the most informative anyway.