Date: Thu, 5 Apr 2012 10:43:04 +0300 From: Gleb Kurtsou <gleb.kurtsou@gmail.com> To: David Wolfskill <david@catwhisker.org>, current@freebsd.org Subject: Re: Using TMPFS for /tmp and /var/run? Message-ID: <20120405074304.GA1048@reks> In-Reply-To: <20120404133858.GB1420@albert.catwhisker.org> References: <20120404095035.GA1929@reks> <20120404133858.GB1420@albert.catwhisker.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--tThc/1wpZn/ma/RB Content-Type: text/plain; charset=utf-8 Content-Disposition: inline On (04/04/2012 06:38), David Wolfskill wrote: > On Wed, Apr 04, 2012 at 12:50:35PM +0300, Gleb Kurtsou wrote: > > ... > > tmpfs-32bit-size_max.patch.txt should fix the problem. I don't have i386 > > installations to test it myself. > > > > Do you run PAE kernel? Could you try filling up /tmp at least to 10g. > > ... > > After updating source to r233868, applying the patch, then updating, > here are the results of my testing so far (not using PAE). > > Summary: as before, I believe that the patch didn't hurt anything, but > it also doesn't restrict the usable size of /tmp to the specified size > (from /etc/fstab): > I've checked on i386 and patch worked as expected, but it required previous patch. I've combined both patches. Could you try it. Thanks, Gleb. > > Script started on Wed Apr 4 06:23:25 2012 > > g1-227(10.0-C)[1] _do > uid=0(root) gid=0(wheel) groups=0(wheel),5(operator) > /dev/ada0s2a on / (ufs, local, soft-updates) > devfs on /dev (devfs, local, multilabel) > tmpfs on /tmp (tmpfs, local) > /dev/ada0s2d on /usr (ufs, local, soft-updates) > /dev/ada0s4e on /var (ufs, local, soft-updates) > /dev/ada0s4g on /common (ufs, local, soft-updates) > fdescfs on /dev/fd (fdescfs) > FreeBSD g1-227.catwhisker.org 10.0-CURRENT FreeBSD 10.0-CURRENT #2 233868M: Wed Apr 4 06:02:25 PDT 2012 root@g1-227.catwhisker.org:/usr/obj/usr/src/sys/CANARY i386 > Wed Apr 4 06:23:27 PDT 2012 > >>> Removing old libraries > Please be sure no application still uses those libraries, else you > can not start such an application. Consult UPDATING for more > information regarding how to cope with the removal/revision bump > of a specific library. > >>> Old libraries removed > Wed Apr 4 06:23:28 PDT 2012 > g1-227(10.0-C)[2] exit > > Script done on Wed Apr 4 06:23:29 2012 > Script started on Wed Apr 4 06:23:35 2012 > > g1-227(10.0-C)[1] df -hi /tmp > Filesystem Size Used Avail Capacity iused ifree %iused Mounted on > tmpfs 23G 12k 23G 0% 17 2.1G 0% /tmp > g1-227(10.0-C)[2] grep tmpfs /etc/fstab > # tmpfs /tmp tmpfs rw,size=2147483648 0 0 > tmpfs /tmp tmpfs rw,size=8g 0 0 > > g1-227(10.0-C)[3] ls -lhT /bkp/tmp/test > -rw-r--r-- 1 david wheel 8.0G Mar 25 10:42:49 2012 /bkp/tmp/test > g1-227(10.0-C)[4] dd bs=1m if=!$ of=/tmp/test > dd bs=1m if=/bkp/tmp/test of=/tmp/test > 8192+0 records in > 8192+0 records out > 8589934592 bytes transferred in 186.178099 secs (46138266 bytes/sec) > g1-227(10.0-C)[5] df -hi /tmp > Filesystem Size Used Avail Capacity iused ifree %iused Mounted on > tmpfs 23G 8.0G 15G 35% 18 2.1G 0% /tmp > g1-227(10.0-C)[6] dd bs=1m if=/bkp/tmp/test of=/tmp/test1 > 8192+0 records in > 8192+0 records out > 8589934592 bytes transferred in 220.254916 secs (38999968 bytes/sec) > g1-227(10.0-C)[7] df -hi /tmp > Filesystem Size Used Avail Capacity iused ifree %iused Mounted on > tmpfs 23G 16G 7.0G 70% 19 2.1G 0% /tmp > g1-227(10.0-C)[8] exit > > Script done on Wed Apr 4 06:33:12 2012 > g1-227(10.0-C)[5] > > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Depriving a girl or boy of an opportunity for education is evil. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="tmpfs-memlimit2.patch.txt" diff --git a/sys/fs/tmpfs/tmpfs.h b/sys/fs/tmpfs/tmpfs.h index efa7c6d..3fc72ab 100644 --- a/sys/fs/tmpfs/tmpfs.h +++ b/sys/fs/tmpfs/tmpfs.h @@ -337,11 +337,10 @@ struct tmpfs_mount { * system, set during mount time. This variable must never be * used directly as it may be bigger than the current amount of * free memory; in the extreme case, it will hold the SIZE_MAX - * value. Instead, use the TMPFS_PAGES_MAX macro. */ + * value. */ size_t tm_pages_max; - /* Number of pages in use by the file system. Cannot be bigger - * than the value returned by TMPFS_PAGES_MAX in any case. */ + /* Number of pages in use by the file system. */ size_t tm_pages_used; /* Pointer to the node representing the root directory of this @@ -486,58 +485,32 @@ int tmpfs_truncate(struct vnode *, off_t); * Memory management stuff. */ -/* Amount of memory pages to reserve for the system (e.g., to not use by +/* + * Amount of memory pages to reserve for the system (e.g., to not use by * tmpfs). - * XXX: Should this be tunable through sysctl, for instance? */ -#define TMPFS_PAGES_RESERVED (4 * 1024 * 1024 / PAGE_SIZE) + */ +#define TMPFS_PAGES_MINRESERVED (4 * 1024 * 1024 / PAGE_SIZE) /* - * Returns information about the number of available memory pages, - * including physical and virtual ones. - * - * Remember to remove TMPFS_PAGES_RESERVED from the returned value to avoid - * excessive memory usage. - * + * Number of reserved swap pages should not be lower than + * swap_pager_almost_full high water mark. */ -static __inline size_t -tmpfs_mem_info(void) -{ +#define TMPFS_SWAP_MINRESERVED 1024 - return (swap_pager_avail + cnt.v_free_count + cnt.v_cache_count); -} +size_t tmpfs_mem_avail(void); -/* Returns the maximum size allowed for a tmpfs file system. This macro - * must be used instead of directly retrieving the value from tm_pages_max. - * The reason is that the size of a tmpfs file system is dynamic: it lets - * the user store files as long as there is enough free memory (including - * physical memory and swap space). Therefore, the amount of memory to be - * used is either the limit imposed by the user during mount time or the - * amount of available memory, whichever is lower. To avoid consuming all - * the memory for a given mount point, the system will always reserve a - * minimum of TMPFS_PAGES_RESERVED pages, which is also taken into account - * by this macro (see above). */ static __inline size_t -TMPFS_PAGES_MAX(struct tmpfs_mount *tmp) +tmpfs_pages_used(struct tmpfs_mount *tmp) { - size_t freepages; - - freepages = tmpfs_mem_info(); - freepages -= freepages < TMPFS_PAGES_RESERVED ? - freepages : TMPFS_PAGES_RESERVED; + const size_t node_size = sizeof(struct tmpfs_node) + + sizeof(struct tmpfs_dirent); + size_t meta_pages; - return MIN(tmp->tm_pages_max, freepages + tmp->tm_pages_used); + meta_pages = howmany((uintmax_t)tmp->tm_nodes_inuse * node_size, + PAGE_SIZE); + return (meta_pages + tmp->tm_pages_used); } -/* Returns the available space for the given file system. */ -#define TMPFS_META_PAGES(tmp) (howmany((tmp)->tm_nodes_inuse * (sizeof(struct tmpfs_node) \ - + sizeof(struct tmpfs_dirent)), PAGE_SIZE)) -#define TMPFS_FILE_PAGES(tmp) ((tmp)->tm_pages_used) - -#define TMPFS_PAGES_AVAIL(tmp) (TMPFS_PAGES_MAX(tmp) > \ - TMPFS_META_PAGES(tmp)+TMPFS_FILE_PAGES(tmp)? \ - TMPFS_PAGES_MAX(tmp) - TMPFS_META_PAGES(tmp) \ - - TMPFS_FILE_PAGES(tmp):0) - #endif /* --------------------------------------------------------------------- */ diff --git a/sys/fs/tmpfs/tmpfs_subr.c b/sys/fs/tmpfs/tmpfs_subr.c index fe596aa..5123fcc 100644 --- a/sys/fs/tmpfs/tmpfs_subr.c +++ b/sys/fs/tmpfs/tmpfs_subr.c @@ -59,6 +59,76 @@ __FBSDID("$FreeBSD$"); SYSCTL_NODE(_vfs, OID_AUTO, tmpfs, CTLFLAG_RW, 0, "tmpfs file system"); +static long tmpfs_swap_reserved = TMPFS_SWAP_MINRESERVED * 2; + +static long tmpfs_pages_reserved = TMPFS_PAGES_MINRESERVED; + +static int +sysctl_mem_reserved(SYSCTL_HANDLER_ARGS) +{ + int error; + long pages, bytes, reserved; + + pages = *(long *)arg1; + bytes = pages * PAGE_SIZE; + + error = sysctl_handle_long(oidp, &bytes, 0, req); + if (error || !req->newptr) + return (error); + + pages = bytes / PAGE_SIZE; + if (arg1 == &tmpfs_swap_reserved) + reserved = TMPFS_SWAP_MINRESERVED; + else + reserved = TMPFS_PAGES_MINRESERVED; + if (pages < reserved) + return (EINVAL); + + *(long *)arg1 = pages; + return (0); +} + +SYSCTL_PROC(_vfs_tmpfs, OID_AUTO, memory_reserved, CTLTYPE_LONG|CTLFLAG_RW, + &tmpfs_pages_reserved, 0, sysctl_mem_reserved, "L", "reserved memory"); +SYSCTL_PROC(_vfs_tmpfs, OID_AUTO, swap_reserved, CTLTYPE_LONG|CTLFLAG_RW, + &tmpfs_swap_reserved, 0, sysctl_mem_reserved, "L", "reserved swap memory"); + +size_t +tmpfs_mem_avail(void) +{ + vm_ooffset_t avail_swap, avail_mem; + + avail_swap = swap_pager_avail - tmpfs_swap_reserved; + if (__predict_false(avail_swap <= 0)) { + /* FIXME No swap or disabled swap check */ + if (swap_pager_avail == 0) + avail_swap = 0; + else + return (0); + } + avail_mem = cnt.v_free_count + cnt.v_cache_count - tmpfs_pages_reserved; + if (__predict_false(avail_mem < 0)) + avail_mem = 0; + return (avail_swap + avail_mem); +} + +static size_t +tmpfs_pages_check_avail(struct tmpfs_mount *tmp, size_t req_pages) +{ + size_t avail; + + avail = tmpfs_mem_avail(); + if (avail < req_pages) + return (0); + + if (tmp->tm_pages_max != SIZE_MAX) + avail = tmp->tm_pages_max - tmpfs_pages_used(tmp); + if (avail < req_pages) + return (0); + + return (1); +} + /* --------------------------------------------------------------------- */ /* @@ -99,6 +169,8 @@ tmpfs_alloc_node(struct tmpfs_mount *tmp, enum vtype type, if (tmp->tm_nodes_inuse >= tmp->tm_nodes_max) return (ENOSPC); + if (tmpfs_pages_check_avail(tmp, 1) == 0) + return (ENOSPC); nnode = (struct tmpfs_node *)uma_zalloc_arg( tmp->tm_node_pool, tmp, M_WAITOK); @@ -917,7 +989,7 @@ tmpfs_reg_resize(struct vnode *vp, off_t newsize, boolean_t ignerr) MPASS(oldpages == uobj->size); newpages = OFF_TO_IDX(newsize + PAGE_MASK); if (newpages > oldpages && - newpages - oldpages > TMPFS_PAGES_AVAIL(tmp)) + tmpfs_pages_check_avail(tmp, newpages - oldpages) == 0) return (ENOSPC); VM_OBJECT_LOCK(uobj); diff --git a/sys/fs/tmpfs/tmpfs_vfsops.c b/sys/fs/tmpfs/tmpfs_vfsops.c index 0647ba2..2ad7eb4 100644 --- a/sys/fs/tmpfs/tmpfs_vfsops.c +++ b/sys/fs/tmpfs/tmpfs_vfsops.c @@ -129,17 +129,62 @@ tmpfs_node_fini(void *mem, int size) mtx_destroy(&node->tn_interlock); } +/* + * XXX Rename to vfs_getopt_size() + */ +static int +tmpfs_getopt_size(struct vfsoptlist *opts, const char *name, u_quad_t *data) +{ + char *opt_value, *vtp; + quad_t iv; + int error, opt_len; + + error = vfs_getopt(opts, name, (void **)&opt_value, &opt_len); + if (error != 0) + return (error); + if (opt_len == 0 || opt_value == NULL) + return (EINVAL); + if (opt_value[0] == '\0' || opt_value[opt_len - 1] != '\0') + return (EINVAL); + + iv = strtoq(opt_value, &vtp, 0); + if (vtp == opt_value || (vtp[0] != '\0' && vtp[1] != '\0')) + return (EINVAL); + if (iv < 0) + return (EINVAL); + switch (vtp[0]) { + case 't': + case 'T': + iv *= 1024; + case 'g': + case 'G': + iv *= 1024; + case 'm': + case 'M': + iv *= 1024; + case 'k': + case 'K': + iv *= 1024; + case '\0': + break; + default: + return (EINVAL); + } + *data = iv; + + return (0); +} + static int tmpfs_mount(struct mount *mp) { + const size_t nodes_per_page = howmany(PAGE_SIZE, + sizeof(struct tmpfs_dirent) + sizeof(struct tmpfs_node)); struct tmpfs_mount *tmp; struct tmpfs_node *root; - size_t pages; - uint32_t nodes; int error; /* Size counters. */ - u_int nodes_max; - u_quad_t size_max, maxfilesize; + u_quad_t pages, nodes_max, size_max, maxfilesize; /* Root node attributes. */ uid_t root_uid; @@ -175,46 +220,47 @@ tmpfs_mount(struct mount *mp) if (mp->mnt_cred->cr_ruid != 0 || vfs_scanopt(mp->mnt_optnew, "mode", "%ho", &root_mode) != 1) root_mode = va.va_mode; - if (vfs_scanopt(mp->mnt_optnew, "inodes", "%u", &nodes_max) != 1) + if (tmpfs_getopt_size(mp->mnt_optnew, "inodes", &nodes_max) != 0) nodes_max = 0; - if (vfs_scanopt(mp->mnt_optnew, "size", "%qu", &size_max) != 1) + if (tmpfs_getopt_size(mp->mnt_optnew, "size", &size_max) != 0) size_max = 0; - if (vfs_scanopt(mp->mnt_optnew, "maxfilesize", "%qu", - &maxfilesize) != 1) + if (tmpfs_getopt_size(mp->mnt_optnew, "maxfilesize", &maxfilesize) != 0) maxfilesize = 0; /* Do not allow mounts if we do not have enough memory to preserve * the minimum reserved pages. */ - if (tmpfs_mem_info() < TMPFS_PAGES_RESERVED) + if (tmpfs_mem_avail() < TMPFS_PAGES_MINRESERVED) return ENOSPC; /* Get the maximum number of memory pages this file system is * allowed to use, based on the maximum size the user passed in * the mount structure. A value of zero is treated as if the * maximum available space was requested. */ - if (size_max < PAGE_SIZE || size_max > SIZE_MAX - PAGE_SIZE) + if (size_max < PAGE_SIZE || size_max > OFF_MAX - PAGE_SIZE || + (SIZE_MAX < OFF_MAX && size_max / PAGE_SIZE >= SIZE_MAX)) pages = SIZE_MAX; else pages = howmany(size_max, PAGE_SIZE); MPASS(pages > 0); if (nodes_max <= 3) { - if (pages > UINT32_MAX - 3) - nodes = UINT32_MAX; + if (pages < INT_MAX / nodes_per_page) + nodes_max = pages * nodes_per_page; else - nodes = pages + 3; - } else - nodes = nodes_max; - MPASS(nodes >= 3); + nodes_max = INT_MAX; + } + if (nodes_max > INT_MAX) + nodes_max = INT_MAX; + MPASS(nodes_max >= 3); /* Allocate the tmpfs mount structure and fill it. */ tmp = (struct tmpfs_mount *)malloc(sizeof(struct tmpfs_mount), M_TMPFSMNT, M_WAITOK | M_ZERO); mtx_init(&tmp->allnode_lock, "tmpfs allnode lock", NULL, MTX_DEF); - tmp->tm_nodes_max = nodes; + tmp->tm_nodes_max = nodes_max; tmp->tm_nodes_inuse = 0; - tmp->tm_maxfilesize = maxfilesize > 0 ? maxfilesize : UINT64_MAX; + tmp->tm_maxfilesize = maxfilesize > 0 ? maxfilesize : OFF_MAX; LIST_INIT(&tmp->tm_nodes_used); tmp->tm_pages_max = pages; @@ -384,22 +430,26 @@ tmpfs_fhtovp(struct mount *mp, struct fid *fhp, int flags, static int tmpfs_statfs(struct mount *mp, struct statfs *sbp) { - fsfilcnt_t freenodes; struct tmpfs_mount *tmp; + size_t used; tmp = VFS_TO_TMPFS(mp); sbp->f_iosize = PAGE_SIZE; sbp->f_bsize = PAGE_SIZE; - sbp->f_blocks = TMPFS_PAGES_MAX(tmp); - sbp->f_bavail = sbp->f_bfree = TMPFS_PAGES_AVAIL(tmp); - - freenodes = MIN(tmp->tm_nodes_max - tmp->tm_nodes_inuse, - TMPFS_PAGES_AVAIL(tmp) * PAGE_SIZE / sizeof(struct tmpfs_node)); - - sbp->f_files = freenodes + tmp->tm_nodes_inuse; - sbp->f_ffree = freenodes; + used = tmpfs_pages_used(tmp); + if (tmp->tm_pages_max != SIZE_MAX) + sbp->f_blocks = tmp->tm_pages_max; + else + sbp->f_blocks = used + tmpfs_mem_avail(); + if (sbp->f_blocks <= used) + sbp->f_bavail = 0; + else + sbp->f_bavail = sbp->f_blocks - used; + sbp->f_bfree = sbp->f_bavail; + sbp->f_files = tmp->tm_nodes_max; + sbp->f_ffree = tmp->tm_nodes_max - tmp->tm_nodes_inuse; /* sbp->f_owner = tmp->tn_uid; */ return 0; --tThc/1wpZn/ma/RB--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120405074304.GA1048>