From owner-svn-src-all@freebsd.org Mon Oct 12 22:19:14 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 273FE429D0C; Mon, 12 Oct 2020 22:19:14 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C9Cl60VxYz3VYY; Mon, 12 Oct 2020 22:19:14 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E3D0F13795; Mon, 12 Oct 2020 22:19:13 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 09CMJDqp098255; Mon, 12 Oct 2020 22:19:13 GMT (envelope-from imp@FreeBSD.org) Received: (from imp@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 09CMJ8JQ098226; Mon, 12 Oct 2020 22:19:08 GMT (envelope-from imp@FreeBSD.org) Message-Id: <202010122219.09CMJ8JQ098226@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: imp set sender to imp@FreeBSD.org using -f From: Warner Losh Date: Mon, 12 Oct 2020 22:19:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r366657 - in head: stand/efi/boot1 stand/libsa/zfs sys/cddl/boot/zfs sys/cddl/contrib/opensolaris/uts/common/os sys/contrib/openzfs/include/os/freebsd/linux sys/contrib/openzfs/include/... X-SVN-Group: head X-SVN-Commit-Author: imp X-SVN-Commit-Paths: in head: stand/efi/boot1 stand/libsa/zfs sys/cddl/boot/zfs sys/cddl/contrib/opensolaris/uts/common/os sys/contrib/openzfs/include/os/freebsd/linux sys/contrib/openzfs/include/os/freebsd/spl/rpc sys/co... X-SVN-Commit-Revision: 366657 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Oct 2020 22:19:14 -0000 Author: imp Date: Mon Oct 12 22:19:07 2020 New Revision: 366657 URL: https://svnweb.freebsd.org/changeset/base/366657 Log: Add zstd support to the boot loader. Add support to the _STANDALONE environment enough bits of the kernel that we can compile it. We still have a small zstd_shim.c since there were 3 items that were a bit hard to nail down and may be cleaned up in the future. These go hand in hand with a number of commits to sys/sys in the past weeks, should this need be MFCd. Discussed with: mmacy (in review and on IRC/Slack) Reviewed by: freqlabs (on openzfs repo) Differential Revision: https://reviews.freebsd.org/D26218 Added: head/stand/libsa/zfs/zstd_shim.c (contents, props changed) - copied, changed from r366656, head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem_cache.h Deleted: head/sys/cddl/contrib/opensolaris/uts/common/os/list.c Modified: head/stand/efi/boot1/Makefile head/stand/libsa/zfs/Makefile.inc head/stand/libsa/zfs/zfsimpl.c head/sys/cddl/boot/zfs/zfsimpl.h head/sys/cddl/boot/zfs/zfssubr.c head/sys/contrib/openzfs/include/os/freebsd/linux/compiler.h head/sys/contrib/openzfs/include/os/freebsd/spl/rpc/xdr.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/atomic.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/byteorder.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/ccompile.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/cmn_err.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/condvar.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem_cache.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kstat.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/proc.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/procfs_list.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sig.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sysmacros.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/taskq.h head/sys/contrib/openzfs/include/os/freebsd/spl/sys/uio.h head/sys/contrib/openzfs/include/sys/nvpair.h head/sys/contrib/openzfs/include/sys/zfs_context.h head/sys/contrib/openzfs/lib/libspl/include/os/freebsd/sys/param.h head/sys/contrib/openzfs/module/os/freebsd/spl/list.c Modified: head/stand/efi/boot1/Makefile ============================================================================== --- head/stand/efi/boot1/Makefile Mon Oct 12 22:07:44 2020 (r366656) +++ head/stand/efi/boot1/Makefile Mon Oct 12 22:19:07 2020 (r366657) @@ -40,6 +40,7 @@ CFLAGS.zfs_module.c+= -I${SYSDIR}/contrib/openzfs/incl CFLAGS.zfs_module.c+= -I${SYSDIR}/contrib/openzfs/include/os/freebsd/spl CFLAGS.zfs_module.c+= -I${SYSDIR}/contrib/openzfs/include/os/freebsd/zfs CFLAGS.zfs_module.c+= -I${SYSDIR}/cddl/contrib/opensolaris/common/lz4 +CFLAGS.zfs_module.c+= -include ${ZFSOSINC}/spl/sys/ccompile.h CFLAGS+= -DEFI_ZFS_BOOT .endif Modified: head/stand/libsa/zfs/Makefile.inc ============================================================================== --- head/stand/libsa/zfs/Makefile.inc Mon Oct 12 22:07:44 2020 (r366656) +++ head/stand/libsa/zfs/Makefile.inc Mon Oct 12 22:19:07 2020 (r366657) @@ -1,17 +1,37 @@ # $FreeBSD$ .PATH: ${ZFSSRC} -SRCS+= zfs.c nvlist.c skein.c skein_block.c list.c .PATH: ${SYSDIR}/crypto/skein -.PATH: ${SYSDIR}/cddl/contrib/opensolaris/uts/common/os +.PATH: ${ZFSOSSRC}/spl +.PATH: ${OZFS}/module/zstd/lib +ZFSSRC= zfs.c nvlist.c skein.c skein_block.c list.c zstd_shim.c zstd.c +SRCS+= ${ZFSSRC} CFLAGS+= -I${LDRSRC} CFLAGS+= -I${SYSDIR}/cddl/boot/zfs -CFLAGS+= -I${SYSDIR}/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SYSDIR}/crypto/skein +ZFS_EARLY= -I${ZFSOSINC} \ + -I${ZFSOSINC}/spl \ + -I${ZFSOSINC}/zfs + +.for i in ${ZFSSRC} +CFLAGS.$i+= -include ${ZFSOSINC}/spl/sys/ccompile.h +.endfor + +CFLAGS_EARLY.list.c+= ${ZFS_EARLY} +CFLAGS_EARLY.zstd_shim.c+= ${ZFS_EARLY} + +# Can't use the early flags because there's two conflicting definitions of boolean_t in +# the zfs code that need to be unified. +CFLAGS.nvlist.c+= -I${ZFSOSINC}/spl +CFLAGS.zfs.c+= -I${ZFSOSINC}/spl \ + -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/lz4 +CFLAGS.zstd_shim.c+= -DIN_BASE -I${OZFS}/include + # Do not unroll skein loops, reduce code size CFLAGS.skein_block.c+= -DSKEIN_LOOP=111 + CFLAGS+= -I${SYSDIR}/contrib/openzfs/include CFLAGS+= -I${SYSDIR}/contrib/openzfs/include/os/freebsd/zfs CFLAGS.zfs.c+= -I${SYSDIR}/cddl/contrib/opensolaris/common/lz4 Modified: head/stand/libsa/zfs/zfsimpl.c ============================================================================== --- head/stand/libsa/zfs/zfsimpl.c Mon Oct 12 22:07:44 2020 (r366656) +++ head/stand/libsa/zfs/zfsimpl.c Mon Oct 12 22:19:07 2020 (r366657) @@ -42,6 +42,7 @@ __FBSDID("$FreeBSD$"); #include "zfsimpl.h" #include "zfssubr.c" +extern int zstd_init(void); struct zfsmount { const spa_t *spa; @@ -170,6 +171,7 @@ zfs_init(void) dnode_cache_buf = malloc(SPA_MAXBLOCKSIZE); zfs_init_crc(); + zstd_init(); } static int Copied and modified: head/stand/libsa/zfs/zstd_shim.c (from r366656, head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem_cache.h) ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem_cache.h Mon Oct 12 22:07:44 2020 (r366656, copy source) +++ head/stand/libsa/zfs/zstd_shim.c Mon Oct 12 22:19:07 2020 (r366657) @@ -1,6 +1,5 @@ -/* - * Copyright (c) 2020 iXsystems, Inc. - * All rights reserved. +/*- + * Copyright (c) 2020 M. Warner Losh * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -11,10 +10,10 @@ * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * - * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) @@ -23,27 +22,19 @@ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * - * $FreeBSD$ */ +#include +__FBSDID("$FreeBSD$"); +/* + * Small amount of shim code needed to get zfs_zstd.c to compile. These items + * here should all be defined in the SPL or as part of libstand somewhere, but + * aren't for reasons that haven't been tracked down yet. Ideally, they would + * all go away and we'd compile zfs_zstd.c directly. Based on an original by + * Matt Macey, but only the #include remains untouched from that. + */ -#ifndef _SPL_KMEM_CACHE_H -#define _SPL_KMEM_CACHE_H +#define ZFS_MODULE_PARAM_ARGS void +typedef int boolean_t; /* This one may be tough to get rid of */ -#include - -/* kmem move callback return values */ -typedef enum kmem_cbrc { - KMEM_CBRC_YES = 0, /* Object moved */ - KMEM_CBRC_NO = 1, /* Object not moved */ - KMEM_CBRC_LATER = 2, /* Object not moved, try again later */ - KMEM_CBRC_DONT_NEED = 3, /* Neither object is needed */ - KMEM_CBRC_DONT_KNOW = 4, /* Object unknown */ -} kmem_cbrc_t; - -extern void spl_kmem_cache_set_move(kmem_cache_t *, - kmem_cbrc_t (*)(void *, void *, size_t, void *)); - -#define kmem_cache_set_move(skc, move) spl_kmem_cache_set_move(skc, move) - -#endif +#include Modified: head/sys/cddl/boot/zfs/zfsimpl.h ============================================================================== --- head/sys/cddl/boot/zfs/zfsimpl.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/cddl/boot/zfs/zfsimpl.h Mon Oct 12 22:19:07 2020 (r366657) @@ -611,7 +611,61 @@ enum zio_compress { ZIO_COMPRESS_GZIP_9, ZIO_COMPRESS_ZLE, ZIO_COMPRESS_LZ4, + ZIO_COMPRESS_ZSTD, ZIO_COMPRESS_FUNCTIONS +}; + +enum zio_zstd_levels { + ZIO_ZSTD_LEVEL_INHERIT = 0, + ZIO_ZSTD_LEVEL_1, +#define ZIO_ZSTD_LEVEL_MIN ZIO_ZSTD_LEVEL_1 + ZIO_ZSTD_LEVEL_2, + ZIO_ZSTD_LEVEL_3, +#define ZIO_ZSTD_LEVEL_DEFAULT ZIO_ZSTD_LEVEL_3 + ZIO_ZSTD_LEVEL_4, + ZIO_ZSTD_LEVEL_5, + ZIO_ZSTD_LEVEL_6, + ZIO_ZSTD_LEVEL_7, + ZIO_ZSTD_LEVEL_8, + ZIO_ZSTD_LEVEL_9, + ZIO_ZSTD_LEVEL_10, + ZIO_ZSTD_LEVEL_11, + ZIO_ZSTD_LEVEL_12, + ZIO_ZSTD_LEVEL_13, + ZIO_ZSTD_LEVEL_14, + ZIO_ZSTD_LEVEL_15, + ZIO_ZSTD_LEVEL_16, + ZIO_ZSTD_LEVEL_17, + ZIO_ZSTD_LEVEL_18, + ZIO_ZSTD_LEVEL_19, +#define ZIO_ZSTD_LEVEL_MAX ZIO_ZSTD_LEVEL_19 + ZIO_ZSTD_LEVEL_RESERVE = 101, /* Leave room for new positive levels */ + ZIO_ZSTD_LEVEL_FAST, /* Fast levels are negative */ + ZIO_ZSTD_LEVEL_FAST_1, +#define ZIO_ZSTD_LEVEL_FAST_DEFAULT ZIO_ZSTD_LEVEL_FAST_1 + ZIO_ZSTD_LEVEL_FAST_2, + ZIO_ZSTD_LEVEL_FAST_3, + ZIO_ZSTD_LEVEL_FAST_4, + ZIO_ZSTD_LEVEL_FAST_5, + ZIO_ZSTD_LEVEL_FAST_6, + ZIO_ZSTD_LEVEL_FAST_7, + ZIO_ZSTD_LEVEL_FAST_8, + ZIO_ZSTD_LEVEL_FAST_9, + ZIO_ZSTD_LEVEL_FAST_10, + ZIO_ZSTD_LEVEL_FAST_20, + ZIO_ZSTD_LEVEL_FAST_30, + ZIO_ZSTD_LEVEL_FAST_40, + ZIO_ZSTD_LEVEL_FAST_50, + ZIO_ZSTD_LEVEL_FAST_60, + ZIO_ZSTD_LEVEL_FAST_70, + ZIO_ZSTD_LEVEL_FAST_80, + ZIO_ZSTD_LEVEL_FAST_90, + ZIO_ZSTD_LEVEL_FAST_100, + ZIO_ZSTD_LEVEL_FAST_500, + ZIO_ZSTD_LEVEL_FAST_1000, +#define ZIO_ZSTD_LEVEL_FAST_MAX ZIO_ZSTD_LEVEL_FAST_1000 + ZIO_ZSTD_LEVEL_AUTO = 251, /* Reserved for future use */ + ZIO_ZSTD_LEVEL_LEVELS }; #define ZIO_COMPRESS_ON_VALUE ZIO_COMPRESS_LZJB Modified: head/sys/cddl/boot/zfs/zfssubr.c ============================================================================== --- head/sys/cddl/boot/zfs/zfssubr.c Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/cddl/boot/zfs/zfssubr.c Mon Oct 12 22:19:07 2020 (r366657) @@ -30,8 +30,6 @@ __FBSDID("$FreeBSD$"); static uint64_t zfs_crc64_table[256]; -#define ECKSUM 666 - #define ASSERT3S(x, y, z) ((void)0) #define ASSERT3U(x, y, z) ((void)0) #define ASSERT3P(x, y, z) ((void)0) @@ -107,6 +105,10 @@ typedef struct zio_checksum_info { #include "sha256.c" #include "skein_zfs.c" +extern int zfs_zstd_decompress(void *s_start, void *d_start, size_t s_len, + size_t d_len, int n); + + static zio_checksum_info_t zio_checksum_table[ZIO_CHECKSUM_FUNCTIONS] = { {{NULL, NULL}, NULL, NULL, 0, "inherit"}, {{NULL, NULL}, NULL, NULL, 0, "on"}, @@ -181,6 +183,7 @@ static zio_compress_info_t zio_compress_table[ZIO_COMP {NULL, NULL, 9, "gzip-9"}, {NULL, zle_decompress, 64, "zle"}, {NULL, lz4_decompress, 0, "lz4"}, + {NULL, zfs_zstd_decompress, ZIO_ZSTD_LEVEL_DEFAULT, "zstd"} }; static void Modified: head/sys/contrib/openzfs/include/os/freebsd/linux/compiler.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/linux/compiler.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/linux/compiler.h Mon Oct 12 22:19:07 2020 (r366657) @@ -68,7 +68,7 @@ #define noinline __noinline #define ____cacheline_aligned __aligned(CACHE_LINE_SIZE) -#ifndef _KERNEL +#if !defined(_KERNEL) && !defined(_STANDALONE) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #endif Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/rpc/xdr.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/rpc/xdr.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/rpc/xdr.h Mon Oct 12 22:19:07 2020 (r366657) @@ -33,7 +33,7 @@ #include #include_next -#ifndef _KERNEL +#if !defined(_KERNEL) && !defined(_STANDALONE) #include @@ -66,6 +66,6 @@ xdrmem_control(XDR *xdrs, int request, void *info) xdrmem_control((xdrs), (req), (op)) : \ (*(xdrs)->x_ops->x_control)(xdrs, req, op)) -#endif /* !_KERNEL */ +#endif /* !_KERNEL && !_STANDALONE */ #endif /* !_OPENSOLARIS_RPC_XDR_H_ */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/atomic.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/atomic.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/atomic.h Mon Oct 12 22:19:07 2020 (r366657) @@ -29,6 +29,8 @@ #ifndef _OPENSOLARIS_SYS_ATOMIC_H_ #define _OPENSOLARIS_SYS_ATOMIC_H_ +#ifndef _STANDALONE + #include #include @@ -178,5 +180,14 @@ atomic_cas_ptr(volatile void *target, void *cmp, void (uint32_t)cmp, (uint32_t)newval)); } #endif /* !defined(COMPAT_32BIT) && defined(__LP64__) */ + +#else /* _STANDALONE */ +/* + * sometimes atomic_add_64 is defined, sometimes not, but the + * following is always right for the boot loader. + */ +#undef atomic_add_64 +#define atomic_add_64(ptr, val) *(ptr) += val +#endif /* !_STANDALONE */ #endif /* !_OPENSOLARIS_SYS_ATOMIC_H_ */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/byteorder.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/byteorder.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/byteorder.h Mon Oct 12 22:19:07 2020 (r366657) @@ -80,10 +80,11 @@ #define BE_64(x) BSWAP_64(x) #endif +#if !defined(_STANDALONE) #if BYTE_ORDER == _BIG_ENDIAN #define htonll(x) BMASK_64(x) #define ntohll(x) BMASK_64(x) -#else +#else /* BYTE_ORDER == _LITTLE_ENDIAN */ #ifndef __LP64__ static __inline__ uint64_t htonll(uint64_t n) @@ -96,11 +97,12 @@ ntohll(uint64_t n) { return ((((uint64_t)ntohl(n)) << 32) + ntohl(n >> 32)); } -#else +#else /* !__LP64__ */ #define htonll(x) BSWAP_64(x) #define ntohll(x) BSWAP_64(x) -#endif -#endif +#endif /* __LP64__ */ +#endif /* BYTE_ORDER */ +#endif /* _STANDALONE */ #define BE_IN32(xa) htonl(*((uint32_t *)(void *)(xa))) Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/ccompile.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/ccompile.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/ccompile.h Mon Oct 12 22:19:07 2020 (r366657) @@ -113,9 +113,9 @@ extern "C" { #define __VPRINTFLIKE(__n) __sun_attr__((__VPRINTFLIKE__(__n))) #define __KPRINTFLIKE(__n) __sun_attr__((__KPRINTFLIKE__(__n))) #define __KVPRINTFLIKE(__n) __sun_attr__((__KVPRINTFLIKE__(__n))) -#ifdef _KERNEL +#if defined(_KERNEL) || defined(_STANDALONE) #define __NORETURN __sun_attr__((__noreturn__)) -#endif +#endif /* _KERNEL || _STANDALONE */ #define __CONST __sun_attr__((__const__)) #define __PURE __sun_attr__((__pure__)) @@ -174,7 +174,7 @@ typedef int enum_t; #define __exit #endif -#ifdef _KERNEL +#if defined(_KERNEL) || defined(_STANDALONE) #define param_set_charp(a, b) (0) #define ATTR_UID AT_UID #define ATTR_GID AT_GID @@ -183,9 +183,15 @@ typedef int enum_t; #define ATTR_CTIME AT_CTIME #define ATTR_MTIME AT_MTIME #define ATTR_ATIME AT_ATIME +#if defined(_STANDALONE) +#define vmem_free kmem_free +#define vmem_zalloc kmem_zalloc +#define vmem_alloc kmem_zalloc +#else #define vmem_free zfs_kmem_free #define vmem_zalloc(size, flags) zfs_kmem_alloc(size, flags | M_ZERO) #define vmem_alloc zfs_kmem_alloc +#endif #define MUTEX_NOLOCKDEP 0 #define RW_NOLOCKDEP 0 Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/cmn_err.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/cmn_err.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/cmn_err.h Mon Oct 12 22:19:07 2020 (r366657) @@ -52,42 +52,33 @@ extern "C" { /*PRINTFLIKE2*/ extern void cmn_err(int, const char *, ...) __KPRINTFLIKE(2); -#pragma rarely_called(cmn_err) extern void vzcmn_err(zoneid_t, int, const char *, __va_list) __KVPRINTFLIKE(3); -#pragma rarely_called(vzcmn_err) extern void vcmn_err(int, const char *, __va_list) __KVPRINTFLIKE(2); -#pragma rarely_called(vcmn_err) /*PRINTFLIKE3*/ extern void zcmn_err(zoneid_t, int, const char *, ...) __KPRINTFLIKE(3); -#pragma rarely_called(zcmn_err) extern void vzprintf(zoneid_t, const char *, __va_list) __KVPRINTFLIKE(2); -#pragma rarely_called(vzprintf) /*PRINTFLIKE2*/ extern void zprintf(zoneid_t, const char *, ...) __KPRINTFLIKE(2); -#pragma rarely_called(zprintf) extern void vuprintf(const char *, __va_list) __KVPRINTFLIKE(1); -#pragma rarely_called(vuprintf) /*PRINTFLIKE1*/ extern void panic(const char *, ...) __KPRINTFLIKE(1) __NORETURN; -#pragma rarely_called(panic) extern void vpanic(const char *, __va_list) __KVPRINTFLIKE(1) __NORETURN; -#pragma rarely_called(vpanic) #endif /* !_ASM */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/condvar.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/condvar.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/condvar.h Mon Oct 12 22:19:07 2020 (r366657) @@ -36,6 +36,7 @@ #include #include #include +#include /* * cv_timedwait() is similar to cv_wait() except that it additionally expects Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem.h Mon Oct 12 22:19:07 2020 (r366657) @@ -29,6 +29,7 @@ #ifndef _OPENSOLARIS_SYS_KMEM_H_ #define _OPENSOLARIS_SYS_KMEM_H_ +#ifdef _KERNEL #include #include #include @@ -93,5 +94,14 @@ void *calloc(size_t n, size_t s); zfs_kmem_alloc((size), (kmflags) | M_ZERO) #define kmem_free(buf, size) zfs_kmem_free((buf), (size)) +#endif /* _KERNEL */ + +#ifdef _STANDALONE +/* + * At the moment, we just need it for the type. We redirect the alloc/free + * routines to the usual Free and Malloc in that environment. + */ +typedef int kmem_cache_t; +#endif /* _STANDALONE */ #endif /* _OPENSOLARIS_SYS_KMEM_H_ */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem_cache.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem_cache.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kmem_cache.h Mon Oct 12 22:19:07 2020 (r366657) @@ -30,6 +30,7 @@ #ifndef _SPL_KMEM_CACHE_H #define _SPL_KMEM_CACHE_H +#ifdef _KERNEL #include /* kmem move callback return values */ @@ -45,5 +46,7 @@ extern void spl_kmem_cache_set_move(kmem_cache_t *, kmem_cbrc_t (*)(void *, void *, size_t, void *)); #define kmem_cache_set_move(skc, move) spl_kmem_cache_set_move(skc, move) + +#endif /* _KERNEL */ #endif Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kstat.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kstat.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/kstat.h Mon Oct 12 22:19:07 2020 (r366657) @@ -24,8 +24,11 @@ #ifndef _SPL_KSTAT_H #define _SPL_KSTAT_H + #include +#ifndef _STANDALONE #include +#endif struct list_head {}; #include #include @@ -129,9 +132,10 @@ struct kstat_s { kstat_raw_ops_t ks_raw_ops; /* ops table for raw type */ char *ks_raw_buf; /* buf used for raw ops */ size_t ks_raw_bufsize; /* size of raw ops buffer */ +#ifndef _STANDALONE struct sysctl_ctx_list ks_sysctl_ctx; struct sysctl_oid *ks_sysctl_root; - +#endif /* _STANDALONE */ }; typedef struct kstat_named_s { @@ -216,10 +220,16 @@ extern void kstat_runq_exit(kstat_io_t *); __kstat_set_seq_raw_ops(k, h, d, a) #define kstat_set_raw_ops(k, h, d, a) \ __kstat_set_raw_ops(k, h, d, a) +#ifndef _STANDALONE #define kstat_create(m, i, n, c, t, s, f) \ __kstat_create(m, i, n, c, t, s, f) #define kstat_install(k) __kstat_install(k) #define kstat_delete(k) __kstat_delete(k) +#else +#define kstat_create(m, i, n, c, t, s, f) ((kstat_t *)0) +#define kstat_install(k) +#define kstat_delete(k) +#endif #endif /* _SPL_KSTAT_H */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/proc.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/proc.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/proc.h Mon Oct 12 22:19:07 2020 (r366657) @@ -41,7 +41,7 @@ #include #include - +#ifdef _KERNEL #define CPU curcpu #define minclsyspri PRIBIO #define defclsyspri minclsyspri @@ -111,4 +111,5 @@ zfs_proc_is_caller(proc_t *p) return (p == curproc); } +#endif /* _KERNEL */ #endif /* _OPENSOLARIS_SYS_PROC_H_ */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/procfs_list.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/procfs_list.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/procfs_list.h Mon Oct 12 22:19:07 2020 (r366657) @@ -25,6 +25,8 @@ #ifndef _SPL_PROCFS_LIST_H #define _SPL_PROCFS_LIST_H +#ifndef _STANDALONE + #include #include @@ -63,5 +65,9 @@ void procfs_list_install(const char *module, void procfs_list_uninstall(procfs_list_t *procfs_list); void procfs_list_destroy(procfs_list_t *procfs_list); void procfs_list_add(procfs_list_t *procfs_list, void *p); + +#else +typedef int procfs_list_t; +#endif /* !_STANDALONE */ #endif /* _SPL_PROCFS_LIST_H */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sig.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sig.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sig.h Mon Oct 12 22:19:07 2020 (r366657) @@ -29,6 +29,8 @@ #ifndef _OPENSOLARIS_SYS_SIG_H_ #define _OPENSOLARIS_SYS_SIG_H_ +#ifndef _STANDALONE + #include_next #include #include @@ -62,4 +64,7 @@ issig(int why) } return (0); } + +#endif /* !_STANDALONE */ + #endif /* _OPENSOLARIS_SYS_SIG_H_ */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sysmacros.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sysmacros.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/sysmacros.h Mon Oct 12 22:19:07 2020 (r366657) @@ -31,6 +31,7 @@ #define _SYS_SYSMACROS_H #include +#include #include #include #include @@ -71,7 +72,11 @@ extern "C" { #define DIV_ROUND_UP(n, d) (((n) + (d) - 1) / (d)) #endif +#ifdef _STANDALONE +#define boot_ncpus 1 +#else /* _STANDALONE */ #define boot_ncpus mp_ncpus +#endif /* _STANDALONE */ #define kpreempt_disable() critical_enter() #define kpreempt_enable() critical_exit() #define CPU_SEQID curcpu @@ -319,7 +324,7 @@ extern unsigned char bcd_to_byte[256]; /* avoid any possibility of clashing with version */ -#define offsetof(s, m) ((size_t)(&(((s *)0)->m))) +#define offsetof(type, field) __offsetof(type, field) #endif /* Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/taskq.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/taskq.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/taskq.h Mon Oct 12 22:19:07 2020 (r366657) @@ -26,6 +26,8 @@ #ifndef _SYS_TASKQ_H #define _SYS_TASKQ_H +#ifdef _KERNEL + #include #include #include @@ -111,5 +113,12 @@ void taskq_resume(taskq_t *); #ifdef __cplusplus } #endif + +#endif /* _KERNEL */ + +#ifdef _STANDALONE +typedef int taskq_ent_t; +#define taskq_init_ent(x) +#endif /* _STANDALONE */ #endif /* _SYS_TASKQ_H */ Modified: head/sys/contrib/openzfs/include/os/freebsd/spl/sys/uio.h ============================================================================== --- head/sys/contrib/openzfs/include/os/freebsd/spl/sys/uio.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/os/freebsd/spl/sys/uio.h Mon Oct 12 22:19:07 2020 (r366657) @@ -29,6 +29,8 @@ #ifndef _OPENSOLARIS_SYS_UIO_H_ #define _OPENSOLARIS_SYS_UIO_H_ +#ifndef _STANDALONE + #include_next #include #include @@ -106,5 +108,7 @@ uio_index_at_offset(uio_t *uio, offset_t off, uint_t * return (off); } + +#endif /* !_STANDALONE */ #endif /* !_OPENSOLARIS_SYS_UIO_H_ */ Modified: head/sys/contrib/openzfs/include/sys/nvpair.h ============================================================================== --- head/sys/contrib/openzfs/include/sys/nvpair.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/sys/nvpair.h Mon Oct 12 22:19:07 2020 (r366657) @@ -62,7 +62,7 @@ typedef enum { DATA_TYPE_UINT8, DATA_TYPE_BOOLEAN_ARRAY, DATA_TYPE_INT8_ARRAY, -#if !defined(_KERNEL) +#if !defined(_KERNEL) && !defined(_STANDALONE) DATA_TYPE_UINT8_ARRAY, DATA_TYPE_DOUBLE #else @@ -191,7 +191,7 @@ int nvlist_add_uint64_array(nvlist_t *, const char *, int nvlist_add_string_array(nvlist_t *, const char *, char *const *, uint_t); int nvlist_add_nvlist_array(nvlist_t *, const char *, nvlist_t **, uint_t); int nvlist_add_hrtime(nvlist_t *, const char *, hrtime_t); -#if !defined(_KERNEL) +#if !defined(_KERNEL) && !defined(_STANDALONE) int nvlist_add_double(nvlist_t *, const char *, double); #endif @@ -228,7 +228,7 @@ int nvlist_lookup_nvlist_array(nvlist_t *, const char nvlist_t ***, uint_t *); int nvlist_lookup_hrtime(nvlist_t *, const char *, hrtime_t *); int nvlist_lookup_pairs(nvlist_t *, int, ...); -#if !defined(_KERNEL) +#if !defined(_KERNEL) && !defined(_STANDALONE) int nvlist_lookup_double(nvlist_t *, const char *, double *); #endif @@ -269,7 +269,7 @@ int nvpair_value_uint64_array(nvpair_t *, uint64_t **, int nvpair_value_string_array(nvpair_t *, char ***, uint_t *); int nvpair_value_nvlist_array(nvpair_t *, nvlist_t ***, uint_t *); int nvpair_value_hrtime(nvpair_t *, hrtime_t *); -#if !defined(_KERNEL) +#if !defined(_KERNEL) && !defined(_STANDALONE) int nvpair_value_double(nvpair_t *, double *); #endif Modified: head/sys/contrib/openzfs/include/sys/zfs_context.h ============================================================================== --- head/sys/contrib/openzfs/include/sys/zfs_context.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/include/sys/zfs_context.h Mon Oct 12 22:19:07 2020 (r366657) @@ -32,7 +32,15 @@ extern "C" { #endif -#ifdef __KERNEL__ +/* + * This code compiles in three different contexts. When __KERNEL__ is defined, + * the code uses "unix-like" kernel interfaces. When _STANDALONE is defined, the + * code is running in a reduced capacity environment of the boot loader which is + * generally a subset of both POSIX and kernel interfaces (with a few unique + * interfaces too). When neither are defined, it's in a userland POSIX or + * similar environment. + */ +#if defined(__KERNEL__) || defined(_STANDALONE) #include #include #include @@ -65,7 +73,7 @@ extern "C" { #include #include #include -#else /* _KERNEL */ +#else /* _KERNEL || _STANDALONE */ #define _SYS_MUTEX_H #define _SYS_RWLOCK_H @@ -759,7 +767,7 @@ extern int kmem_cache_reap_active(void); #define __init #define __exit -#endif /* _KERNEL */ +#endif /* _KERNEL || _STANDALONE */ #ifdef __cplusplus }; Modified: head/sys/contrib/openzfs/lib/libspl/include/os/freebsd/sys/param.h ============================================================================== --- head/sys/contrib/openzfs/lib/libspl/include/os/freebsd/sys/param.h Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/lib/libspl/include/os/freebsd/sys/param.h Mon Oct 12 22:19:07 2020 (r366657) @@ -45,10 +45,6 @@ */ #define MAXNAMELEN 256 -#ifndef IN_BASE -#define UID_NOBODY 60001 /* user ID no body */ -#define GID_NOBODY UID_NOBODY -#endif #define UID_NOACCESS 60002 /* user ID no access */ #define MAXUID UINT32_MAX /* max user id */ Modified: head/sys/contrib/openzfs/module/os/freebsd/spl/list.c ============================================================================== --- head/sys/contrib/openzfs/module/os/freebsd/spl/list.c Mon Oct 12 22:07:44 2020 (r366656) +++ head/sys/contrib/openzfs/module/os/freebsd/spl/list.c Mon Oct 12 22:19:07 2020 (r366657) @@ -27,10 +27,10 @@ * Generic doubly-linked list implementation */ +#include #include #include #include -#include #include #define list_d2l(a, obj) ((list_node_t *)(((char *)obj) + (a)->list_offset))