From owner-svn-src-all@freebsd.org Mon Jan 8 20:14:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38458E5E836; Mon, 8 Jan 2018 20:14:18 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 124A7718B0; Mon, 8 Jan 2018 20:14:18 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4FC6D4B9D; Mon, 8 Jan 2018 20:14:17 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w08KEHVf071636; Mon, 8 Jan 2018 20:14:17 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w08KEGv8071627; Mon, 8 Jan 2018 20:14:16 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201801082014.w08KEGv8071627@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: Conrad Meyer Date: Mon, 8 Jan 2018 20:14:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r327706 - in head/sys: conf contrib/zstd/lib/freebsd X-SVN-Group: head X-SVN-Commit-Author: cem X-SVN-Commit-Paths: in head/sys: conf contrib/zstd/lib/freebsd X-SVN-Commit-Revision: 327706 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jan 2018 20:14:18 -0000 Author: cem Date: Mon Jan 8 20:14:16 2018 New Revision: 327706 URL: https://svnweb.freebsd.org/changeset/base/327706 Log: Integrate zstd into the kernel Mock userspace headers and include mocked headers first in compilation command to inject kernel headers and override e.g., malloc(3) with malloc(9). Submitted by: allanjude Reviewed by: imp (earlier version), bapt (earlier version) Differential Revision: https://reviews.freebsd.org/D10407 Added: head/sys/contrib/zstd/lib/freebsd/ head/sys/contrib/zstd/lib/freebsd/stddef.h (contents, props changed) head/sys/contrib/zstd/lib/freebsd/stdint.h (contents, props changed) head/sys/contrib/zstd/lib/freebsd/stdio.h (contents, props changed) head/sys/contrib/zstd/lib/freebsd/stdlib.h (contents, props changed) head/sys/contrib/zstd/lib/freebsd/string.h (contents, props changed) head/sys/contrib/zstd/lib/freebsd/zstd_kfreebsd.h (contents, props changed) head/sys/contrib/zstd/lib/freebsd/zstd_kmalloc.c (contents, props changed) Modified: head/sys/conf/files head/sys/conf/kern.pre.mk Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Mon Jan 8 18:47:35 2018 (r327705) +++ head/sys/conf/files Mon Jan 8 20:14:16 2018 (r327706) @@ -626,6 +626,23 @@ contrib/ngatm/netnatm/sig/sig_unimsgcpy.c optional nga compile-with "${NORMAL_C} -I$S/contrib/ngatm" contrib/ngatm/netnatm/sig/sig_verify.c optional ngatm_uni \ compile-with "${NORMAL_C} -I$S/contrib/ngatm" +# Zstd +contrib/zstd/lib/freebsd/zstd_kmalloc.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/common/zstd_common.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/common/fse_decompress.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/common/entropy_common.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/common/error_private.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/common/xxhash.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/zstd_compress.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/fse_compress.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/huf_compress.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/zstd_double_fast.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/zstd_fast.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/zstd_lazy.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/zstd_ldm.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/compress/zstd_opt.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/decompress/zstd_decompress.c standard compile-with ${ZSTD_C} +contrib/zstd/lib/decompress/huf_decompress.c standard compile-with ${ZSTD_C} crypto/blowfish/bf_ecb.c optional ipsec | ipsec_support crypto/blowfish/bf_skey.c optional crypto | ipsec | ipsec_support crypto/camellia/camellia.c optional crypto | ipsec | ipsec_support Modified: head/sys/conf/kern.pre.mk ============================================================================== --- head/sys/conf/kern.pre.mk Mon Jan 8 18:47:35 2018 (r327705) +++ head/sys/conf/kern.pre.mk Mon Jan 8 20:14:16 2018 (r327706) @@ -132,6 +132,9 @@ NORMAL_FW= uudecode -o ${.TARGET} ${.ALLSRC} NORMAL_FWO= ${LD} -b binary --no-warn-mismatch -d -warn-common -r \ -m ${LD_EMULATION} -o ${.TARGET} ${.ALLSRC:M*.fw} +# for ZSTD in the kernel (include zstd/lib/freebsd before other CFLAGS) +ZSTD_C= ${CC} -c -DZSTD_HEAPMODE=1 -I$S/contrib/zstd/lib/freebsd ${CFLAGS} -I$S/contrib/zstd/lib -I$S/contrib/zstd/lib/common ${WERROR} -Wno-missing-prototypes ${PROF} ${.IMPSRC} + # Common for dtrace / zfs CDDL_CFLAGS= -DFREEBSD_NAMECACHE -nostdinc -I$S/cddl/compat/opensolaris -I$S/cddl/contrib/opensolaris/uts/common -I$S -I$S/cddl/contrib/opensolaris/common ${CFLAGS} -Wno-unknown-pragmas -Wno-missing-prototypes -Wno-undef -Wno-strict-prototypes -Wno-cast-qual -Wno-parentheses -Wno-redundant-decls -Wno-missing-braces -Wno-uninitialized -Wno-unused -Wno-inline -Wno-switch -Wno-pointer-arith -Wno-unknown-pragmas CDDL_CFLAGS+= -include $S/cddl/compat/opensolaris/sys/debug_compat.h Added: head/sys/contrib/zstd/lib/freebsd/stddef.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/contrib/zstd/lib/freebsd/stddef.h Mon Jan 8 20:14:16 2018 (r327706) @@ -0,0 +1,3 @@ +/* This file is in the public domain */ +/* $FreeBSD$ */ +#include "zstd_kfreebsd.h" Added: head/sys/contrib/zstd/lib/freebsd/stdint.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/contrib/zstd/lib/freebsd/stdint.h Mon Jan 8 20:14:16 2018 (r327706) @@ -0,0 +1,3 @@ +/* This file is in the public domain */ +/* $FreeBSD$ */ +#include "zstd_kfreebsd.h" Added: head/sys/contrib/zstd/lib/freebsd/stdio.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/contrib/zstd/lib/freebsd/stdio.h Mon Jan 8 20:14:16 2018 (r327706) @@ -0,0 +1,3 @@ +/* This file is in the public domain */ +/* $FreeBSD$ */ +#include "zstd_kfreebsd.h" Added: head/sys/contrib/zstd/lib/freebsd/stdlib.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/contrib/zstd/lib/freebsd/stdlib.h Mon Jan 8 20:14:16 2018 (r327706) @@ -0,0 +1,3 @@ +/* This file is in the public domain */ +/* $FreeBSD$ */ +#include "zstd_kfreebsd.h" Added: head/sys/contrib/zstd/lib/freebsd/string.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/contrib/zstd/lib/freebsd/string.h Mon Jan 8 20:14:16 2018 (r327706) @@ -0,0 +1,3 @@ +/* This file is in the public domain */ +/* $FreeBSD$ */ +#include "zstd_kfreebsd.h" Added: head/sys/contrib/zstd/lib/freebsd/zstd_kfreebsd.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/contrib/zstd/lib/freebsd/zstd_kfreebsd.h Mon Jan 8 20:14:16 2018 (r327706) @@ -0,0 +1,56 @@ +/*- + * Copyright (c) 2014-2015 Allan Jude + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef ZSTD_KFREEBSD_H +#define ZSTD_KFREEBSD_H + +#ifdef __cplusplus +extern "C" { +#endif + +#ifdef _KERNEL +#include /* size_t */ +#include /* memcpy, memset */ +#ifndef BUILDING_ZFS +#include /* ptrdiff_t */ +#endif +#include + +MALLOC_DECLARE(M_ZSTD); + +#define malloc(x) (malloc)((x), M_ZSTD, M_WAITOK) +#define free(x) (free)((x), M_ZSTD) +/* in zstd's use of calloc, a is always 1 */ +#define calloc(a,b) (malloc)((a)*(b), M_ZSTD, M_WAITOK | M_ZERO) +#endif + +#ifdef __cplusplus +} +#endif + +#endif /* ZSTD_KFREEBSD_H */ Added: head/sys/contrib/zstd/lib/freebsd/zstd_kmalloc.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/contrib/zstd/lib/freebsd/zstd_kmalloc.c Mon Jan 8 20:14:16 2018 (r327706) @@ -0,0 +1,33 @@ +/*- + * Copyright (c) 2016-2017 Allan Jude + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +#include +#include + +MALLOC_DEFINE(M_ZSTD, "zstd", "ZSTD Compressor");