Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 6 Jun 2018 09:17:55 -0400
From:      James E Keenan <jkeenan@pobox.com>
To:        freebsd-questions@freebsd.org
Subject:   Re: Unable to kill processes using either Ctrl-C or 'kill'
Message-ID:  <e385581a-ae27-451d-a0da-c3c51e5c5087@pobox.com>
In-Reply-To: <CADqw_gLwsSKT3w8iyY7d9%2Bisqyt7YH4CvifRL2WG54OQdvK7Xw@mail.gmail.com>
References:  <9a7f62c4-80aa-7eea-91ec-6712612a0451@pobox.com> <CAOZUxFu7LkafvT30H_ZZG6uJ-CkU537RD=dSHcEP=UVRgOdrZw@mail.gmail.com> <CADqw_gLwsSKT3w8iyY7d9%2Bisqyt7YH4CvifRL2WG54OQdvK7Xw@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On 06/03/2018 03:03 PM, Michael Schuster wrote:
> On Sun, Jun 3, 2018 at 8:47 PM, Duane Whitty <duane@nofroth.com> wrote:
> 
>> On Sun, Jun 3, 2018 at 1:20 PM, James E Keenan <jkeenan@pobox.com> wrote:
>>
>>> On a FreeBSD-11 host, I am unable to kill processes using either Ctrl-C
>> or
>>> 'kill'.
>>>
>>> 1. The problem first became manifest when I was attempting to use Vagrant
>>> to download a Vagrant box from vagrantup.com.   The box in question was
>> a
>>> VirtualBox called 'generic/freebsd11'.  I have successfully downloaded,
>>> installed and used this box several times already, so I anticipated no
>>> problems.
>>>
>>> #####
>>> $ vagrant init generic/freebsd11
>>> $ vagrant up --provision | tee vagrant-up-provision.log.20180603100900
>>> Bringing machine 'default' up with 'virtualbox' provider...
>>> ==> default: Checking if box 'generic/freebsd11' is up to date...
>>> ==> default: Clearing any previously set forwarded ports...
>>> ==> default: Fixed port collision for 22 => 2222. Now on port 2202.
>>> ==> default: Clearing any previously set network interfaces...
>>> ==> default: Preparing network interfaces based on configuration...
>>>      default: Adapter 1: nat
>>> ==> default: Forwarding ports...
>>>      default: 22 (guest) => 2202 (host) (adapter 1)
>>> ==> default: Running 'pre-boot' VM customizations...
>>> ==> default: Booting VM...
>>> ==> default: Waiting for machine to boot. This may take a few minutes...
>>>      default: SSH address: 127.0.0.1:2202
>>>      default: SSH username: vagrant
>>>      default: SSH auth method: private key
>>> #####
>>>
>>> Based on recent experience, I would have expected the "few minutes" to be
>>> 1 or 2 minutes at most and possibly be accompanied by "retrying" methods.
>>>
>>> However, at this point, the screen hung indefinitely.  I tried Ctrl-C;
>>> that command was printed in my terminal but otherwise nothing happened.
>>>
>>> 2. I ssh-ed to the host in a fresh terminal and called
>>>
>>> #####
>>> tail -f vagrant-up-provision.log.20180603100900
>>> #####
>>>
>>> That command displayed the output posted above and hung at the same
>>> point.  This process also could not be killed by Ctrl-C.
>>>
>>> 3. I then ssh-ed to the host in a third terminal, called 'ps aux', and
>>> then tried to kill suspect processes with 'kill -9 <pid>'.  Those
>> processes
>>> were not, in fact, killed; their status was changed to 'T' -- "Marks a
>>> stopped process" according to 'man ps'.  Some excerpts from 'ps auxwww':
>>>
>>> #####
>>> vmuser   7169   0.0  0.1   81356    4444  0  T+   14:09        0:01.99
>>> /usr/local/bin/ruby24 /usr/local/bin/vagrant up --provision
>>> ...
>>> jkeenan 67787   0.0  0.0    6296       0  1  WW+  -            0:00.00
>>> tail -f /home/vmuser/vagrant-images/generic-freebsd11-201806030939/
>>> vagrant-up-provision.log.20180603100900
>>> ...
>>> jkeenan 74119   0.0  0.0    7064       0  2  WW+  -            0:00.00
>>> /bin/sh /usr/bin/man ps
>>> #####
>>>
>>> 4. I have now opened quite a few connections to the host.  If I issue a
>>> command there such as 'man ps' or 'less' that entails paging, I can page
>>> through the output, but the process does not close by itself and does not
>>> respond to Ctrl-C.  If I then try to kill that process from another
>>> terminal, the best that happens is that its status gets changed to 'WW+'
>> --
>>> "Marks an idle interrupt thread" and "The process is swapped out".
>>>
>>> Internet searches turn up links like this one,
>>> https://forums.freebsd.org/threads/cant-kill-process-in-the-
>>> stop-state.56319/, that suggest that there are certain processes that do
>>> not respond to 'kill' signals.  That seems to be what's happening here.
>>>
>>> Can anyone suggest the cause of the problem?
>>>
>>> Short of requesting that the sysadmin shut down and reboot the system, is
>>> there anyway for a non-root user to solve this problem?
>>>
>>> Thank you very much.
>>> Jim Keenan
>>> _______________________________________________
>>> freebsd-questions@freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
>>> To unsubscribe, send any mail to "freebsd-questions-unsubscribe
>>> @freebsd.org"
>>>
>>
>>
>> Can you get added to sudoers?  I realize that still implies a level of root
>> access but I really don't know of any other way to kill processes which
>> don't belong to you.  I don't see why the sysadmin would need to reboot.
>>
> 
> most likely, being root or equivalent won't help in this case. If a
> processes owner cannot kill it (using -9, which cannot be caught) that
> implies that the process is hung in the kernel (signal delivery happens
> when a process leaves kernel context).
> 
> regards
> Michael
> 

Update:  While some sysadmin intervention got the hangs cleared up on 
Sunday night, the problem recurred Tuesday morning.  Once again, the 
hang occurred during 'vagrant up' at the "Waiting for machine to boot" 
point.  Once again, that session refused to accept terminal input, so I 
could not Ctrl-C out of it.  Once again, when I logged on from another 
terminal, any "vagrant xxx" command would hang -- even something as 
simple as "vagrant global-status".  And once again, any command that 
entailed paging or interactivity would hang, e.g., 'less' would hang 
when it reached EOF.

This output from 'top | cat' may support the "process is hung in the 
kernel" hypothesis:

#####
$ top | cat
last pid: 23582;  load averages:  0.21,  0.15,  0.10  up 39+18:08:01 
13:00:49
49 processes:  1 running, 25 sleeping, 3 stopped, 4 zombie, 16 waiting

Mem: 4192K Active, 852K Inact, 56K Laundry, 7858M Wired, 42M Free
ARC: 859M Total, 332M MFU, 161M MRU, 32K Anon, 12M Header, 353M Other
      234M Compressed, 588M Uncompressed, 2.51:1 Ratio
Swap: 4096M Total, 295M Used, 3801M Free, 7% Inuse


   PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU 
COMMAND
21147 vmuser       22  20    0  2483M  2336M select  3  32:36   0.00% 
VBoxHeadless
84461 jkeenan      13  20    0 61840K  6136K select  0  27:14   0.00% 
VBoxSVC
95384 jkeenan       3  20    0  2380M  2245M STOP    2  26:59   0.00% 
VBoxHeadless
45500 jkeenan      23  20    0   438M   301M select  0  19:31   0.00% 
VBoxHeadless
83727 jkeenan       1  20    0 20664K  4856K select  3   8:59   0.00% 
VBoxXPCOMIPCD
94095 jkeenan       3  20    0  1303M  1165M STOP    1   8:41   0.00% 
VBoxHeadless
...
#####

Note the high value for Wired memory.  "Memory in use by the Kernel. 
This memory cannot be swapped out." 
(https://unix.stackexchange.com/questions/134862/what-do-the-different-memory-counters-in-freebsd-mean#137254)

I have not yet heard back from the sysadmin concerning this second 
occurrence, so any suggestions you might have would be appreciated.

Thank you very much.
Jim Keenan



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e385581a-ae27-451d-a0da-c3c51e5c5087>