Date: Tue, 19 Jan 2021 22:46:37 +0200 From: Andriy Gapon <avg@FreeBSD.org> To: meloun.michal@gmail.com, John Baldwin <jhb@FreeBSD.org>, Jessica Clarke <jrtc27@freebsd.org>, Oleksandr Tymoshenko <gonzo@FreeBSD.org> Cc: "src-committers@freebsd.org" <src-committers@FreeBSD.org>, "dev-commits-src-all@freebsd.org" <dev-commits-src-all@FreeBSD.org>, "dev-commits-src-main@freebsd.org" <dev-commits-src-main@FreeBSD.org> Subject: Re: git: 248f0cabca75 - main - make maximum interrupt number tunable on ARM, ARM64, MIPS, and RISC-V Message-ID: <a075c53f-66b8-ffda-31d6-038e65c10550@FreeBSD.org> In-Reply-To: <80a3722c-16d1-0e8e-b6e1-4bdcfb554aa4@gmail.com> References: <202101190036.10J0aj3O033256@gitrepo.freebsd.org> <8FE79D0F-602A-4C98-893A-806E72ED991B@freebsd.org> <f3f34011-90ba-f9d5-f0e2-0c58f382108d@FreeBSD.org> <80a3722c-16d1-0e8e-b6e1-4bdcfb554aa4@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-01-19 21:42, Michal Meloun wrote: > > > On 19.01.2021 19:41, John Baldwin wrote: >> On 1/19/21 8:43 AM, Jessica Clarke wrote: >>> On 19 Jan 2021, at 00:36, Oleksandr Tymoshenko <gonzo@FreeBSD.org> >>> wrote: >>>> @@ -142,21 +143,15 @@ static bool irq_assign_cpu = false; >>>> #endif >>>> #endif >>>> >>>> -/* >>>> - * - 2 counters for each I/O interrupt. >>>> - * - MAXCPU counters for each IPI counters for SMP. >>>> - */ >>>> -#ifdef SMP >>>> -#define INTRCNT_COUNT (NIRQ * 2 + INTR_IPI_COUNT * MAXCPU) >>>> -#else >>>> -#define INTRCNT_COUNT (NIRQ * 2) >>>> -#endif >>>> +int intr_nirq = NIRQ; >>>> +SYSCTL_UINT(_machdep, OID_AUTO, nirq, CTLFLAG_RDTUN, &intr_nirq, 0, >>>> + "Number of IRQs"); >>> >>> Unsigned integer, given the SYSCTL_UINT? >>> >>> What's stopping us from dynamically resizing rather than forcing the >>> user to use a tunable (which also means that, in practice, the limit >>> will remain unchanged, because you want GENERIC kernels to work out of >>> the box)? >> >> For x86 (and probably the same here), the biggest headache is that there >> is a dynamically allocated array indexed by IRQ value at interrupt >> invocation time. Currently it uses no locks since the value of the >> pointer is set during boot time and then never changes. Making it >> dynamic would require allowing that pointer to change and dealing with >> what that entails in a very hot path. Generally speaking, the majority >> of interrupts used in a given system are allocated during boot and not >> by hotplug devices added post-boot, so there is also not a lot of utility >> in having this run-time tunable vs a boot-time tunable. >> > To be exact, the main headaches are intrcnt and intrnames arrays and > their relationship to the sysctl interface. These fields are used > without locking, they do not have the ability to mark interrupt at given > index as unused. This is the real reason why we can't unload a driver > that acts as an interrupt controller (e.g. USB parallel port with the > possibility of interrupts on each pin). > Everything else in subr_intr.c should be prepared for dynamic > allocation/delete of interrupt sources. x86 does not use subr_intr / intrng though. I've been playing with an idea of using two-level interrupt "arrays" for things like interrupt counts and descriptiopns. That is, having fixed size chunks and a dynamically growing array of pointers to the chunks with a restriction that once a chunk is allocated it is never deallocated. So a pointer to an entry stored in such a chunk would remain stable. I also tried to use vmem(9) to keep track of allocated interrupts (interrupt sources) so that they could be dynamically created and removed. I think that it is a feasible idea, but I got lost in implementation details. -- Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a075c53f-66b8-ffda-31d6-038e65c10550>