Date: Sat, 20 Aug 2011 13:20:03 -0500 From: Alan Cox <alc@rice.edu> To: Kostik Belousov <kostikbel@gmail.com> Cc: alc@freebsd.org, freebsd-stable@freebsd.org, perryh@pluto.rain.com, "Alexander V. Chernikov" <melifaro@ipfw.ru>, daniel@digsys.bg Subject: Re: 32GB limit per swap device? Message-ID: <4E4FFAD3.4090706@rice.edu> In-Reply-To: <20110820174147.GW17489@deviant.kiev.zoral.com.ua> References: <4E4143A6.6030307@digsys.bg> <935F8EC2-88E0-45A3-BE8B-7210BE223BC5@mac.com> <4e42a0c0.e2t/9MF98O3HFjb1%perryh@pluto.rain.com> <4E4CCA6C.8020408@ipfw.ru> <CAJUyCcMc7m65c_XjHNFi0A4cHHySC1brLS7HdivstxeOi6uFQw@mail.gmail.com> <20110820174147.GW17489@deviant.kiev.zoral.com.ua>
index | next in thread | previous in thread | raw e-mail
On 08/20/2011 12:41, Kostik Belousov wrote: > On Sat, Aug 20, 2011 at 12:33:29PM -0500, Alan Cox wrote: >> On Thu, Aug 18, 2011 at 3:16 AM, Alexander V. Chernikov<melifaro@ipfw.ru>wrote: >> >>> On 10.08.2011 19:16, perryh@pluto.rain.com wrote: >>> >>>> Chuck Swiger<cswiger@mac.com> wrote: >>>> >>>> On Aug 9, 2011, at 7:26 AM, Daniel Kalchev wrote: >>>>>> I am trying to set up 64GB partitions for swap for a system that >>>>>> has 64GB of RAM (with the idea to dump kernel core etc). But, on >>>>>> 8-stable as of today I get: >>>>>> >>>>>> WARNING: reducing size to maximum of 67108864 blocks per swap unit >>>>>> >>>>>> Is there workaround for this limitation? >>>>>> >>> Another interesting question: >>> >>> swap pager operates in page blocks (PAGE_SIZE=4k on common arch). >>> >>> Block device size in passed to swaponsomething() in number of _disk_ blocks >>> (e.g. in DEV_BSIZE=512). After that, kernel b-lists (on top of which swap >>> pager is build) maximum objects check is enforced. >>> >>> The (possible) problem is that real object count we will operate on is not >>> the value passed to swaponsomething() since it is calculated in wrong units. >>> >>> we should check b-list limit on (X * DEV_BSIZE512 / PAGE_SIZE) value which >>> is rough (X / 8) so we should be able to address 32*8=256G. >>> >>> The code should look like this: >>> >>> Index: vm/swap_pager.c >>> ==============================**==============================**======= >>> --- vm/swap_pager.c (revision 223877) >>> +++ vm/swap_pager.c (working copy) >>> @@ -2129,6 +2129,15 @@ swaponsomething(struct vnode *vp, void *id, u_long >>> u_long mblocks; >>> >>> /* >>> + * nblks is in DEV_BSIZE'd chunks, convert to PAGE_SIZE'd chunks. >>> + * First chop nblks off to page-align it, then convert. >>> + * >>> + * sw->sw_nblks is in page-sized chunks now too. >>> + */ >>> + nblks&= ~(ctodb(1) - 1); >>> + nblks = dbtoc(nblks); >>> + >>> + /* >>> >>> * If we go beyond this, we get overflows in the radix >>> * tree bitmap code. >>> */ >>> @@ -2138,14 +2147,6 @@ swaponsomething(struct vnode *vp, void *id, u_long >>> mblocks); >>> nblks = mblocks; >>> } >>> - /* >>> - * nblks is in DEV_BSIZE'd chunks, convert to PAGE_SIZE'd chunks. >>> - * First chop nblks off to page-align it, then convert. >>> - * >>> - * sw->sw_nblks is in page-sized chunks now too. >>> - */ >>> - nblks&= ~(ctodb(1) - 1); >>> - nblks = dbtoc(nblks); >>> >>> sp = malloc(sizeof *sp, M_VMPGDATA, M_WAITOK | M_ZERO); >>> sp->sw_vp = vp; >>> >>> >>> (move pages recalculation before b-list check) >>> >>> >>> Can someone comment on this? >>> >>> >> I believe that you are correct. Have you tried testing this change on a >> large swap device? > I probably agree too, but I am in the process of re-reading the swap code, > and I do not quite believe in the limit. > I'm uncertain whether the current limit, "0x40000000 / BLIST_META_RADIX", is exact or not, but I doubt that it is too large. > When the initial code was committed, our daddr_t was 32bit, I checked > the RELENG_4 sources. Current code uses int64_t for daddr_t. My impression > right now is that we only utilize the low 32bits of daddr_t. > > Esp. interesting looks the following typedef: > typedef uint32_t u_daddr_t; /* unsigned disk address */ > which (correctly) means that typical mask (u_daddr_t)-1 is 0xffffffff. > > I wonder whether we could just use full 64bit and de-facto remove the > limitation on the swap partition size. I would rather argue first that the subr_list code should not be using daddr_t all. The code is abusing daddr_t and defining u_daddr_t to represent things that are not disk addresses. Instead, it should either define its own type or directly use (u)int*_t. Then, as for choosing between 32 and 64 bits, I'm skeptical of using this structure for managing more than 32 bits worth of blocks, given the amount of RAM it will use.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4E4FFAD3.4090706>