Date: Fri, 12 Nov 2021 12:52:52 -0700 From: Warner Losh <imp@bsdimp.com> To: Chris Ross <cross+freebsd@distal.com> Cc: Ronald Klop <ronald-lists@klop.ws>, freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: swap_pager: cannot allocate bio Message-ID: <CANCZdfrY9YZ%2BrLpnhJgjxtkuYi5GnNcGU6SkZtJqhR9%2B_U44RA@mail.gmail.com> In-Reply-To: <4008C512-31F1-4BE3-B674-A270CF674757@distal.com> References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <op.1cpimpsmkndu52@joepie> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <42006135.15.1636709757975@mailrelay> <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> <CANCZdfpW3YJ7c_EO82BYwLCFhDXdCp2W_fxmxAXzYvr7HNmnZQ@mail.gmail.com> <4008C512-31F1-4BE3-B674-A270CF674757@distal.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000a67ddf05d09ccebc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Nov 12, 2021 at 12:50 PM Chris Ross <cross+freebsd@distal.com> wrote: > > > > On Nov 12, 2021, at 11:15, Warner Losh <imp@bsdimp.com> wrote: > > So the root cause of this problem is well known. You have a memory > shortage, so you want to page out dirty pages to reclaim memory. > > However, there's not enough memory to allocate the structures you need > to do I/O and so the swapout I/O fails half way down > > the stack not being able to allocate a bio. Some paths through the > swapper cope with this well, other parts that execute less > > often cope less well. > > > > There's some hacks in the tree today to help with the GELI case: we > prioritize swapping I/O. But there's no g_alloc_bio_swapping() interface > > for swapping I/O to get priority on allocating a bio to start with. > Places that use g_clone_bio() could have the clone's copy allocated > > from a special swap pool, but that starts to get messy and isn't done > today. And the upper layers like geom_cfs and ZFS are > > inconsistent in allocations, so there's work needed to make it robust i= n > ZFS, but I have only a vague notion of what's needed. At the very > > least, the swapping I/O that comes into the top of ZFS won't have > swapping I/O marked coming out the bottom because the > > BIO_SWAP flag is quite new. > > > > So until then, swapping on zvols is fraught with deadlocks like this an= d > in the past there's been a strong admonishment > > against it. > > Apologies, Warner, but I=E2=80=99m not sure I=E2=80=99m understanding thi= s last > statement. If you mean swapping _onto_ zvols, I=E2=80=99m not doing that= . If you > mean swapping in any way, while having zvols, then yes I am doing that. > > My swap is on a partition on the non-ZFS disk. A physical disk as far as > the kernel knows, hardware RAID1. > > # pstat -s > Device 1K-blocks Used Avail Capacity > /dev/da0p3 445682648 1018524 444664124 0% > OK. That's well supported and should work w/o some of the issues that I raised. I'd misunderstood and thought you were swapping to zvols... > Let me know if what you=E2=80=99re saying above is true to my case, and a= ny advice > as to how I can avoid it. I had a =E2=80=9Cnot enough swap space=E2=80= =9D a while back, > and accordingly increased the size of my swap partition. I have 128GB of > memory, though between the ARC and the big process I was running, that > fills it easily. > Yea, this is a 'memory is exhausted' problem, and more swap won't help that. It's unclear why we run out so fast, and why the separate zones for the bio isn't providing a good level of insulation from out of memory scenarios. Warner --000000000000a67ddf05d09ccebc--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrY9YZ%2BrLpnhJgjxtkuYi5GnNcGU6SkZtJqhR9%2B_U44RA>