From owner-freebsd-hackers@freebsd.org Wed Jan 2 12:53:09 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0191D141B84E for ; Wed, 2 Jan 2019 12:53:09 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [176.74.240.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C846F8AD88; Wed, 2 Jan 2019 12:53:07 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 7FDA0A2D9E; Wed, 2 Jan 2019 13:53:00 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qnj-7ZeoAanv; Wed, 2 Jan 2019 13:52:59 +0100 (CET) Received: from [192.168.11.152] (unknown [192.168.11.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 19B62A2D9A; Wed, 2 Jan 2019 13:52:59 +0100 (CET) Subject: Re: Using kqueue with aio_read/write To: Alan Somers Cc: FreeBSD Hackers References: <8753521a-4555-ec2a-5efc-dee2660b4d9b@digiware.nl> <969d9a38-d3dd-78d0-c974-ba14ec4747db@digiware.nl> From: Willem Jan Withagen Message-ID: <016fda76-8f02-ca62-af7d-062262bd4a09@digiware.nl> Date: Wed, 2 Jan 2019 13:53:00 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: nl X-Rspamd-Queue-Id: C846F8AD88 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of wjw@digiware.nl designates 176.74.240.9 as permitted sender) smtp.mailfrom=wjw@digiware.nl X-Spamd-Result: default: False [-4.18 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[digiware.nl]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; IP_SCORE(-0.69)[asn: 28878(-3.46), country: NL(0.03)]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[9.240.74.176.list.dnswl.org : 127.0.9.2]; MX_GOOD(-0.01)[cached: smtp.digiware.nl]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.98)[-0.980,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:28878, ipnet:176.74.224.0/19, country:NL]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Jan 2019 12:53:09 -0000 On 02/01/2019 03:23, Alan Somers wrote: > On Tue, Jan 1, 2019 at 6:56 PM Willem Jan Withagen wrote: >> On 28/12/2018 02:47, Alan Somers wrote: >>> On Thu, Dec 27, 2018 at 6:15 PM Willem Jan Withagen wrote: >>>> Hi, >>>> >>>> Im trying to understand why I cannot get so code to work. >>>> This is the smallest extract I can make to show my problem. >>>> >>>> I would expect the kevent() call to return every timeo tick. >>>> Even if I tell it NOT to time-out I get these spurts of errors >>>> >>>> Since there is nothing to trigger the AIO-event, I would expect kqueue >>>> to hold indefinitly. >>>> >>>> But it does not generate anything other than errors >>>> And instead it repeatedly complains that there is a permission error: >>>> get_events_kevent: EV_Error(1) kevent(): Operation not permitted >>>> >>>> But I'm not getting where that would the case... >>>> >>>> Surely a pilot error, but I do overlook it al the time. >>>> So suggestions are welcome. >>>> >>>> Thanx, >>>> --WjW >>>> >>>> #include >>>> #include >>>> #include >>>> #include >>>> #include >>>> #include >>>> #include >>>> #include >>>> #include >>>> >>>> #define BUFFER_SIZE 512 >>>> #define MAX_EVENTS 32 >>>> >>>> #define FILENAME "/tmp/aio_test" >>>> char filename[256]; >>>> int fd; >>>> int done = 0; >>>> >>>> void get_events_kevent(int fd, int kq) >>>> { >>>> printf("get_events function fd = %d, kq = %d\n", fd, kq); >>>> int i = 0, errcnt = 0, err, ret, reterr, rev; >>>> int search = 1; >>>> >>>> int timeout_ms = 10; >>>> struct timespec timeo = { >>>> timeout_ms / 1000, >>>> (timeout_ms % 1000) * 1000 * 1000 >>>> }; >>>> struct kevent filter[16]; >>>> struct kevent changed[16]; >>>> >>>> EV_SET(&filter[0], fd, EVFILT_AIO, >>>> EV_ADD, >>>> 0, 0, 0 ); >>> This is the first problem. There's no need to explicitly set >>> EVFILT_AIO on the kqueue. It gets set by the aio_read(2) or similar >>> syscall. And this invocation wouldn't be correct anyway, because for >>> AIO the ident field refers to the address of the struct aiocb, not the >>> file descriptor. If the only events you care about are AIO, then you >>> can pass NULL as the filter argument to kevent. I suspect this is the >>> cause of your problem. The kernel probably thinks you're trying to >>> register for an aiocb that's outside of your address space or >>> something like that. >>> >>> >>>> while (!done) { >>>> printf("+"); >>>> rev = kevent(kq, filter, 1, changed, 16, 0); //&timeo); >>>> if (rev < 0) { >>>> perror("kevent error"); >>>> } else if (rev == 0) { >>>> printf("T"); >>>> } else { >>>> printf("rev(%d)\n", rev); >>>> if (changed[0].flags == EV_ERROR) { >>>> errno = changed[0].data; >>>> printf( "%s: EV_Error(%d) kevent(): %s\n", __func__, errno, >>>> strerror(errno)); >>>> memset(&changed[0], 0, sizeof(struct kevent)); >>>> } else { >>>> err = aio_error((struct aiocb*)changed[0].udata); >>> No need to call aio_error(2) after kevent(2) returns. You can go >>> straight to aio_return. aio_error shouldn't hurt, but it isn't >>> necessary. >> According to kevent(2) calling kevent can return errors on the called >> aio_calls. >> It then returns with EV_ERROR in flags, and errno is stored in the >> event.data. >> >> But what would be going on when the event's flag contains EV_ERROR but >> event's data is still 0??? >> >> the udata field still seems to point to the aio data that was passed >> into the aio block when calling aio_read(). >> >> Should I ignore this as a non-error? >> >> --WjW > Are you sure you bzero()ed your aiocb before initializing it? Any > stack garbage that was present in its > aio_sigevent.sigev_notify_kevent_flags field will be dutifully copied > into the returned kevent. And in any case, the definitive way to get > the final status of a completed aio operation is with aio_return. That seems to help in getting things clear...    -13> 2019-01-02 13:32:31.834 dc15a80  1 bdev:327 paio get_next_completed processing event i = 0 aio_return(22) (22) Invalid argument Disadvantage is that it is not clear yet which of the many arguments that is? --WjW