Date: Sun, 7 Aug 2022 11:12:20 -0700 From: David Christensen <dpchrist@holgerdanske.com> To: questions@freebsd.org Subject: Re: zfs and git upload-pack Message-ID: <348470bf-0f11-f7b3-e782-881c3f864ffb@holgerdanske.com> In-Reply-To: <20220807195750.0233e2f3@bureaucracy.de> References: <20220807102839.7c69f387@bureaucracy.de> <a94ec43d-8ae8-fd20-a5a9-0e1e67750107@holgerdanske.com> <20220807195750.0233e2f3@bureaucracy.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On 8/7/22 10:57, Philipp Takacs wrote: > On Sun, 7 Aug 2022 09:54:41 -0700 > David Christensen <dpchrist@holgerdanske.com> wrote: > >> On 8/7/22 01:28, Philipp wrote: >>> Hi all >>> >>> I host a quite uncommon git repository mostly out of binary files. I >>> have the problem every time this repo is cloned the host allocate >>> memory and going to swap. This leads to the host being unusable and >>> need to force rebooted. >>> >>> The repo is stored on a zfs and nullmounted in a jail to run the git >>> service over ssh. The host is a FreeBSD 13.1 with 4GB RAM and 4GB >>> swap. >>> >>> What I have noticed is that the biggest memory consumtion is from >>> mmap() a pack file. For the given repo this has the size of 6,7G. I >>> suspect this file is mapped in memory but not correctly >>> handled/unmaped (by the kernel) when not enough memory is available. >>> >>> I have tested some options to solve/workaround this issue: >>> >>> * limit the zfs ARC size in loader.conf >>> * zfs set primarycache none for the dataset >>> * limit datasize, memoryuse and vmemoryuse via login.conf >>> * limit git packedGitLimit >>> >>> None of them have solved the issue. >>> >>> On my last test I have noticed that the memory is not freed after >>> the git commands are stopped. To free the memory I had to unmount >>> the zfs dataset: >>> >>> satanist@confusion:~ $ freecolor -om >>> total used free shared buffers >>> cached Mem: 3923 3881 41 0 >>> 0 0 Swap: 4096 456 3639 >>> satanist@confusion:~ $ sudo zfs umount zroot/data/git/bigrepository >>> satanist@confusion:~ $ freecolor -om >>> total used free shared buffers >>> cached Mem: 3923 278 3645 0 >>> 0 0 Swap: 4096 456 3639 >>> >>> Any idea how to debug this or what else I could try solve this? >>> >>> Philipp >> >> >> Add memory; the more, the better. > > Can you borrow me your credit card? > > Jokes aside. this repo gets cloned a few times a month. Currently the > Host dies because one client try to clone this repo. This should be work > with 4GB memory. Please post console sessions that demonstrate cloning without failure and cloning with failure. David
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?348470bf-0f11-f7b3-e782-881c3f864ffb>