From owner-freebsd-virtualization@freebsd.org Tue Dec 13 06:32:19 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFF6CC74DA0 for ; Tue, 13 Dec 2016 06:32:19 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 6918415D0 for ; Tue, 13 Dec 2016 06:32:19 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id C423520C1C05 for ; Tue, 13 Dec 2016 16:31:59 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id B6474280502 for ; Tue, 13 Dec 2016 16:31:59 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 3_O_WMxwJiOg for ; Tue, 13 Dec 2016 16:31:59 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 6BDCC2809E1; Tue, 13 Dec 2016 16:31:57 +1000 (AEST) Subject: Re: Query about bhyve's blockif_cancel and the signalling mechanisms To: Ian Campbell , Tycho Nightingale References: Cc: freebsd-virtualization@freebsd.org, anil@recoil.org From: Peter Grehan Message-ID: <631f775d-8d61-55ba-1e7b-8ce4fcadcbf3@freebsd.org> Date: Mon, 12 Dec 2016 22:32:11 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.2 cv=YJDv8VOx c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17 a=N659UExz7-8A:10 a=n5n_aSjo0skA:10 a=Goinh_sa6ktK6CdGHDYA:9 a=pILNOxqGKmIA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 06:32:19 -0000 Hi Ian, > To recap my understanding of the mechanisms at work (glossing over the > queue handling and condvars involved etc), the bhyve block_if > infrastructure registers a callback for SIGCONT with the mevent > subsystem, which is a kevent/kqueue thing which delivers events to the > main thread (mevent_dispatch is the last thing in main()) it also sets > SIGCONT to SIG_IGN. That's correct. The intent was to have the signal delivered via the kevent callback rather than standard signal delivery. > When a disk controller device model wants to > cancel a block request (e.g. in ahci_port_stop) it calls > blockif_cancel which sends a SIGCONT to the blkio thread which has > claimed the request, notionally to kick it out of whatever blocking > system call it is in and cause it to return an error to the device > model. Yep, that's correct. > The main thing I do not follow is whether or not the blkio thread is > actually interrupted at all when the signal has been configured to be > delivered via the kevent/kqueue mechanisms to a 3rd unrelated thread. It is interrupted on FreeBSD. > I've dug around in the FreeBSD kevent and signal man pages but I > cannot find any part which describes anything of the semantics which > bhyve seems to be relying on (which seems to be that the system call > in the target thread will return EINTR at some point before the thread > which is "handling" the signal via kevent/kqueue sees that event). > > Have I missed something here or is bhyve relying on some subtle > underlying semantics? I didn't think it too FreeBSD-specific - if a thread is blocked in a system call, sending a signal should force it to exit on most Unices. > I have a secondary concern which is what happens if the IO thread is > on its way to making a blocking system call in blockif_proc but has > not actually done so when the signal is delivered. It seems like it > would simply carry on and make the blocking call with perhaps > unexpected consequences (i/o getting wedged, perhaps only until a > second reset attempt). I've not actually seen this happening though > and there's a chance I'm simply over thinking things after staring at > them for so long! I believe this case is handled - I discussed this at length with Tycho when the code was committed a while back. Tycho - any thoughts ? later, Peter.