From owner-svn-src-all@freebsd.org Sun Jul 29 00:12:22 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F1A210628AA; Sun, 29 Jul 2018 00:12:21 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4958486DCE; Sun, 29 Jul 2018 00:12:21 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2A35162DD; Sun, 29 Jul 2018 00:12:21 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T0CLTT048906; Sun, 29 Jul 2018 00:12:21 GMT (envelope-from mm@FreeBSD.org) Received: (from mm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T0CHFH046735; Sun, 29 Jul 2018 00:12:17 GMT (envelope-from mm@FreeBSD.org) Message-Id: <201807290012.w6T0CHFH046735@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mm set sender to mm@FreeBSD.org using -f From: Martin Matuska Date: Sun, 29 Jul 2018 00:12:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336854 - in head: contrib/libarchive/cat contrib/libarchive/cpio contrib/libarchive/libarchive contrib/libarchive/libarchive/test contrib/libarchive/tar contrib/libarchive/tar/test con... X-SVN-Group: head X-SVN-Commit-Author: mm X-SVN-Commit-Paths: in head: contrib/libarchive/cat contrib/libarchive/cpio contrib/libarchive/libarchive contrib/libarchive/libarchive/test contrib/libarchive/tar contrib/libarchive/tar/test contrib/libarchive/test_util... X-SVN-Commit-Revision: 336854 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 00:12:22 -0000 Author: mm Date: Sun Jul 29 00:12:16 2018 New Revision: 336854 URL: https://svnweb.freebsd.org/changeset/base/336854 Log: MFV r336851: Update vendor/libarchive/dist to git 2c8c83b9731ff822fad6cc8c670ea5519c366a14 Important vendor changes: PR #993: Chdir to -C directory for metalog processing OSS-Fuzz #4969: Check size of the extended time field in zip archives PR #973: Record informational compression level in gzip header MFC after: 1 week Added: head/contrib/libarchive/tar/test/test_option_C_mtree.c - copied unchanged from r336851, vendor/libarchive/dist/tar/test/test_option_C_mtree.c Modified: head/contrib/libarchive/cat/bsdcat.c head/contrib/libarchive/cpio/cpio.c head/contrib/libarchive/libarchive/archive_entry.c head/contrib/libarchive/libarchive/archive_entry.h head/contrib/libarchive/libarchive/archive_openssl_hmac_private.h head/contrib/libarchive/libarchive/archive_read.c head/contrib/libarchive/libarchive/archive_read_support_format_mtree.c head/contrib/libarchive/libarchive/archive_read_support_format_zip.c head/contrib/libarchive/libarchive/archive_virtual.c head/contrib/libarchive/libarchive/archive_write_add_filter_gzip.c head/contrib/libarchive/libarchive/archive_write_set_format_ar.c head/contrib/libarchive/libarchive/test/test_sparse_basic.c head/contrib/libarchive/libarchive/test/test_write_filter_gzip.c head/contrib/libarchive/libarchive/test/test_write_filter_zstd.c head/contrib/libarchive/tar/bsdtar.c head/contrib/libarchive/tar/test/test_option_fflags.c head/contrib/libarchive/tar/write.c head/contrib/libarchive/test_utils/test_main.c head/usr.bin/tar/tests/Makefile Directory Properties: head/contrib/libarchive/ (props changed) Modified: head/contrib/libarchive/cat/bsdcat.c ============================================================================== --- head/contrib/libarchive/cat/bsdcat.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/cat/bsdcat.c Sun Jul 29 00:12:16 2018 (r336854) @@ -61,7 +61,7 @@ usage(FILE *stream, int eval) static void version(void) { - printf("bsdcat %s - %s\n", + printf("bsdcat %s - %s \n", BSDCAT_VERSION_STRING, archive_version_details()); exit(0); Modified: head/contrib/libarchive/cpio/cpio.c ============================================================================== --- head/contrib/libarchive/cpio/cpio.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/cpio/cpio.c Sun Jul 29 00:12:16 2018 (r336854) @@ -499,7 +499,7 @@ long_help(void) static void version(void) { - fprintf(stdout,"bsdcpio %s - %s\n", + fprintf(stdout,"bsdcpio %s - %s \n", BSDCPIO_VERSION_STRING, archive_version_details()); exit(0); Modified: head/contrib/libarchive/libarchive/archive_entry.c ============================================================================== --- head/contrib/libarchive/libarchive/archive_entry.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_entry.c Sun Jul 29 00:12:16 2018 (r336854) @@ -1491,7 +1491,7 @@ archive_entry_acl_next(struct archive_entry *entry, in * the style of the generated ACL. */ wchar_t * -archive_entry_acl_to_text_w(struct archive_entry *entry, ssize_t *len, +archive_entry_acl_to_text_w(struct archive_entry *entry, la_ssize_t *len, int flags) { return (archive_acl_to_text_w(&entry->acl, len, flags, @@ -1499,7 +1499,7 @@ archive_entry_acl_to_text_w(struct archive_entry *entr } char * -archive_entry_acl_to_text(struct archive_entry *entry, ssize_t *len, +archive_entry_acl_to_text(struct archive_entry *entry, la_ssize_t *len, int flags) { return (archive_acl_to_text_l(&entry->acl, len, flags, NULL)); Modified: head/contrib/libarchive/libarchive/archive_entry.h ============================================================================== --- head/contrib/libarchive/libarchive/archive_entry.h Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_entry.h Sun Jul 29 00:12:16 2018 (r336854) @@ -42,6 +42,7 @@ #include #include /* for wchar_t */ +#include #include #if defined(_WIN32) && !defined(__CYGWIN__) Modified: head/contrib/libarchive/libarchive/archive_openssl_hmac_private.h ============================================================================== --- head/contrib/libarchive/libarchive/archive_openssl_hmac_private.h Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_openssl_hmac_private.h Sun Jul 29 00:12:16 2018 (r336854) @@ -28,7 +28,8 @@ #include #include -#if OPENSSL_VERSION_NUMBER < 0x10100000L || defined(LIBRESSL_VERSION_NUMBER) +#if OPENSSL_VERSION_NUMBER < 0x10100000L || \ + (defined(LIBRESSL_VERSION_NUMBER) && LIBRESSL_VERSION_NUMBER < 0x20700000L) #include /* malloc, free */ #include /* memset */ static inline HMAC_CTX *HMAC_CTX_new(void) Modified: head/contrib/libarchive/libarchive/archive_read.c ============================================================================== --- head/contrib/libarchive/libarchive/archive_read.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_read.c Sun Jul 29 00:12:16 2018 (r336854) @@ -821,7 +821,7 @@ archive_read_format_capabilities(struct archive *_a) * DO NOT intermingle calls to this function and archive_read_data_block * to read a single entry body. */ -ssize_t +la_ssize_t archive_read_data(struct archive *_a, void *buff, size_t s) { struct archive *a = (struct archive *)_a; Modified: head/contrib/libarchive/libarchive/archive_read_support_format_mtree.c ============================================================================== --- head/contrib/libarchive/libarchive/archive_read_support_format_mtree.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_read_support_format_mtree.c Sun Jul 29 00:12:16 2018 (r336854) @@ -49,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include "archive.h" #include "archive_entry.h" #include "archive_private.h" +#include "archive_rb.h" #include "archive_read_private.h" #include "archive_string.h" #include "archive_pack_dev.h" @@ -75,8 +76,6 @@ __FBSDID("$FreeBSD$"); #define MTREE_HAS_OPTIONAL 0x0800 #define MTREE_HAS_NOCHANGE 0x1000 /* FreeBSD specific */ -#define MTREE_HASHTABLE_SIZE 1024 - #define MAX_LINE_LEN (1024 * 1024) struct mtree_option { @@ -85,13 +84,13 @@ struct mtree_option { }; struct mtree_entry { + struct archive_rb_node rbnode; + struct mtree_entry *next_dup; struct mtree_entry *next; struct mtree_option *options; char *name; char full; char used; - unsigned int name_hash; - struct mtree_entry *hashtable_next; }; struct mtree { @@ -104,11 +103,12 @@ struct mtree { const char *archive_format_name; struct mtree_entry *entries; struct mtree_entry *this_entry; - struct mtree_entry *entry_hashtable[MTREE_HASHTABLE_SIZE]; + struct archive_rb_tree entry_rbtree; struct archive_string current_dir; struct archive_string contents_name; struct archive_entry_linkresolver *resolver; + struct archive_rb_tree rbtree; int64_t cur_size; char checkfs; @@ -117,7 +117,6 @@ struct mtree { static int bid_keycmp(const char *, const char *, ssize_t); static int cleanup(struct archive_read *); static int detect_form(struct archive_read *, int *); -static unsigned int hash(const char *); static int mtree_bid(struct archive_read *, int); static int parse_file(struct archive_read *, struct archive_entry *, struct mtree *, struct mtree_entry *, int *); @@ -219,9 +218,30 @@ free_options(struct mtree_option *head) } } +static int +mtree_cmp_node(const struct archive_rb_node *n1, + const struct archive_rb_node *n2) +{ + const struct mtree_entry *e1 = (const struct mtree_entry *)n1; + const struct mtree_entry *e2 = (const struct mtree_entry *)n2; + + return (strcmp(e1->name, e2->name)); +} + +static int +mtree_cmp_key(const struct archive_rb_node *n, const void *key) +{ + const struct mtree_entry *e = (const struct mtree_entry *)n; + + return (strcmp(e->name, key)); +} + int archive_read_support_format_mtree(struct archive *_a) { + static const struct archive_rb_tree_ops rb_ops = { + mtree_cmp_node, mtree_cmp_key, + }; struct archive_read *a = (struct archive_read *)_a; struct mtree *mtree; int r; @@ -237,6 +257,8 @@ archive_read_support_format_mtree(struct archive *_a) } mtree->fd = -1; + __archive_rb_tree_init(&mtree->rbtree, &rb_ops); + r = __archive_read_register_format(a, mtree, "mtree", mtree_bid, archive_read_format_mtree_options, read_header, read_data, skip, NULL, cleanup, NULL, NULL); @@ -875,12 +897,11 @@ process_add_entry(struct archive_read *a, struct mtree struct mtree_option **global, const char *line, ssize_t line_len, struct mtree_entry **last_entry, int is_form_d) { - struct mtree_entry *entry, *ht_iter; + struct mtree_entry *entry; struct mtree_option *iter; const char *next, *eq, *name, *end; size_t name_len, len; int r, i; - unsigned int ht_idx; if ((entry = malloc(sizeof(*entry))) == NULL) { archive_set_error(&a->archive, errno, "Can't allocate memory"); @@ -891,8 +912,6 @@ process_add_entry(struct archive_read *a, struct mtree entry->name = NULL; entry->used = 0; entry->full = 0; - entry->name_hash = 0; - entry->hashtable_next = NULL; /* Add this entry to list. */ if (*last_entry == NULL) @@ -945,15 +964,17 @@ process_add_entry(struct archive_read *a, struct mtree memcpy(entry->name, name, name_len); entry->name[name_len] = '\0'; parse_escapes(entry->name, entry); - entry->name_hash = hash(entry->name); - ht_idx = entry->name_hash % MTREE_HASHTABLE_SIZE; - if ((ht_iter = mtree->entry_hashtable[ht_idx]) != NULL) { - while (ht_iter->hashtable_next) - ht_iter = ht_iter->hashtable_next; - ht_iter->hashtable_next = entry; - } else { - mtree->entry_hashtable[ht_idx] = entry; + entry->next_dup = NULL; + if (entry->full) { + if (!__archive_rb_tree_insert_node(&mtree->rbtree, &entry->rbnode)) { + struct mtree_entry *alt; + alt = (struct mtree_entry *)__archive_rb_tree_find_node( + &mtree->rbtree, entry->name); + while (alt->next_dup) + alt = alt->next_dup; + alt->next_dup = entry; + } } for (iter = *global; iter != NULL; iter = iter->next) { @@ -1148,14 +1169,13 @@ parse_file(struct archive_read *a, struct archive_entr * with pathname canonicalization, which is a very * tricky subject.) */ - for (mp = mentry->hashtable_next; mp != NULL; mp = mp->hashtable_next) { - if (mp->full && !mp->used - && mentry->name_hash == mp->name_hash - && strcmp(mentry->name, mp->name) == 0) { + mp = (struct mtree_entry *)__archive_rb_tree_find_node( + &mtree->rbtree, mentry->name); + for (; mp; mp = mp->next_dup) { + if (mp->full && !mp->used) { /* Later lines override earlier ones. */ mp->used = 1; - r1 = parse_line(a, entry, mtree, mp, - &parsed_kws); + r1 = parse_line(a, entry, mtree, mp, &parsed_kws); if (r1 < r) r = r1; } @@ -1988,20 +2008,4 @@ readline(struct archive_read *a, struct mtree *mtree, } find_off = u - mtree->line.s; } -} - -static unsigned int -hash(const char *p) -{ - /* A 32-bit version of Peter Weinberger's (PJW) hash algorithm, - as used by ELF for hashing function names. */ - unsigned g, h = 0; - while (*p != '\0') { - h = (h << 4) + *p++; - if ((g = h & 0xF0000000) != 0) { - h ^= g >> 24; - h &= 0x0FFFFFFF; - } - } - return h; } Modified: head/contrib/libarchive/libarchive/archive_read_support_format_zip.c ============================================================================== --- head/contrib/libarchive/libarchive/archive_read_support_format_zip.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_read_support_format_zip.c Sun Jul 29 00:12:16 2018 (r336854) @@ -511,7 +511,13 @@ process_extra(struct archive_read *a, const char *p, s case 0x5455: { /* Extended time field "UT". */ - int flags = p[offset]; + int flags; + if (datasize == 0) { + archive_set_error(&a->archive, ARCHIVE_ERRNO_FILE_FORMAT, + "Incomplete extended time field"); + return ARCHIVE_FAILED; + } + flags = p[offset]; offset++; datasize--; /* Flag bits indicate which dates are present. */ Modified: head/contrib/libarchive/libarchive/archive_virtual.c ============================================================================== --- head/contrib/libarchive/libarchive/archive_virtual.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_virtual.c Sun Jul 29 00:12:16 2018 (r336854) @@ -124,13 +124,13 @@ archive_write_finish_entry(struct archive *a) return ((a->vtable->archive_write_finish_entry)(a)); } -ssize_t +la_ssize_t archive_write_data(struct archive *a, const void *buff, size_t s) { return ((a->vtable->archive_write_data)(a, buff, s)); } -ssize_t +la_ssize_t archive_write_data_block(struct archive *a, const void *buff, size_t s, la_int64_t o) { Modified: head/contrib/libarchive/libarchive/archive_write_add_filter_gzip.c ============================================================================== --- head/contrib/libarchive/libarchive/archive_write_add_filter_gzip.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_write_add_filter_gzip.c Sun Jul 29 00:12:16 2018 (r336854) @@ -226,7 +226,12 @@ archive_compressor_gzip_open(struct archive_write_filt data->compressed[7] = (uint8_t)(t>>24)&0xff; } else memset(&data->compressed[4], 0, 4); - data->compressed[8] = 0; /* No deflate options */ + if (data->compression_level == 9) + data->compressed[8] = 2; + else if(data->compression_level == 1) + data->compressed[8] = 4; + else + data->compressed[8] = 0; data->compressed[9] = 3; /* OS=Unix */ data->stream.next_out += 10; data->stream.avail_out -= 10; Modified: head/contrib/libarchive/libarchive/archive_write_set_format_ar.c ============================================================================== --- head/contrib/libarchive/libarchive/archive_write_set_format_ar.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/archive_write_set_format_ar.c Sun Jul 29 00:12:16 2018 (r336854) @@ -180,7 +180,7 @@ archive_write_ar_header(struct archive_write *a, struc } memset(buff, ' ', 60); - strncpy(&buff[AR_fmag_offset], "`\n", 2); + memcpy(&buff[AR_fmag_offset], "`\n", 2); if (strcmp(pathname, "/") == 0 ) { /* Entry is archive symbol table in GNU format */ @@ -189,7 +189,7 @@ archive_write_ar_header(struct archive_write *a, struc } if (strcmp(pathname, "__.SYMDEF") == 0) { /* Entry is archive symbol table in BSD format */ - strncpy(buff + AR_name_offset, "__.SYMDEF", 9); + memcpy(buff + AR_name_offset, "__.SYMDEF", 9); goto stat; } if (strcmp(pathname, "//") == 0) { @@ -225,7 +225,7 @@ archive_write_ar_header(struct archive_write *a, struc * actually 15 bytes. */ if (strlen(filename) <= 15) { - strncpy(&buff[AR_name_offset], + memcpy(&buff[AR_name_offset], filename, strlen(filename)); buff[AR_name_offset + strlen(filename)] = '/'; } else { @@ -248,7 +248,7 @@ archive_write_ar_header(struct archive_write *a, struc return (ARCHIVE_FATAL); } - strncpy(se, filename, strlen(filename)); + memcpy(se, filename, strlen(filename)); strcpy(se + strlen(filename), "/\n"); ss = strstr(ar->strtab, se); @@ -285,11 +285,11 @@ archive_write_ar_header(struct archive_write *a, struc * archive header. */ if (strlen(filename) <= 16 && strchr(filename, ' ') == NULL) { - strncpy(&buff[AR_name_offset], filename, strlen(filename)); + memcpy(&buff[AR_name_offset], filename, strlen(filename)); buff[AR_name_offset + strlen(filename)] = ' '; } else { - strncpy(buff + AR_name_offset, "#1/", 3); + memcpy(buff + AR_name_offset, "#1/", 3); if (format_decimal(strlen(filename), buff + AR_name_offset + 3, AR_name_size - 3)) { Modified: head/contrib/libarchive/libarchive/test/test_sparse_basic.c ============================================================================== --- head/contrib/libarchive/libarchive/test/test_sparse_basic.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/test/test_sparse_basic.c Sun Jul 29 00:12:16 2018 (r336854) @@ -118,13 +118,26 @@ create_sparse_file(const char *path, const struct spar assert(handle != INVALID_HANDLE_VALUE); assert(DeviceIoControl(handle, FSCTL_SET_SPARSE, NULL, 0, NULL, 0, &dmy, NULL) != 0); + + size_t offsetSoFar = 0; + while (s->type != END) { if (s->type == HOLE) { - LARGE_INTEGER distance; + LARGE_INTEGER fileOffset, beyondOffset, distanceToMove; + fileOffset.QuadPart = offsetSoFar; + beyondOffset.QuadPart = offsetSoFar + s->size; + distanceToMove.QuadPart = s->size; - distance.QuadPart = s->size; - assert(SetFilePointerEx(handle, distance, - NULL, FILE_CURRENT) != 0); + FILE_ZERO_DATA_INFORMATION zeroInformation; + zeroInformation.FileOffset = fileOffset; + zeroInformation.BeyondFinalZero = beyondOffset; + + DWORD bytesReturned; + assert(SetFilePointerEx(handle, distanceToMove, + NULL, FILE_CURRENT) != 0); + assert(SetEndOfFile(handle) != 0); + assert(DeviceIoControl(handle, FSCTL_SET_ZERO_DATA, &zeroInformation, + sizeof(FILE_ZERO_DATA_INFORMATION), NULL, 0, &bytesReturned, NULL) != 0); } else { DWORD w, wr; size_t size; @@ -139,6 +152,7 @@ create_sparse_file(const char *path, const struct spar size -= wr; } } + offsetSoFar += s->size; s++; } assertEqualInt(CloseHandle(handle), 1); @@ -484,10 +498,15 @@ DEFINE_TEST(test_sparse_basic) * on all platform. */ const struct sparse sparse_file0[] = { + // 0 // 1024 { DATA, 1024 }, { HOLE, 2048000 }, + // 2049024 // 2051072 { DATA, 2048 }, { HOLE, 2048000 }, + // 4099072 // 4103168 { DATA, 4096 }, { HOLE, 20480000 }, + // 24583168 // 24591360 { DATA, 8192 }, { HOLE, 204800000 }, + // 229391360 // 229391361 { DATA, 1 }, { END, 0 } }; const struct sparse sparse_file1[] = { Modified: head/contrib/libarchive/libarchive/test/test_write_filter_gzip.c ============================================================================== --- head/contrib/libarchive/libarchive/test/test_write_filter_gzip.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/test/test_write_filter_gzip.c Sun Jul 29 00:12:16 2018 (r336854) @@ -39,6 +39,7 @@ DEFINE_TEST(test_write_filter_gzip) struct archive* a; char *buff, *data; size_t buffsize, datasize; + unsigned char *rbuff; char path[16]; size_t used1, used2; int i, r, use_prog = 0; @@ -58,6 +59,7 @@ DEFINE_TEST(test_write_filter_gzip) /* * Write a 100 files and read them all back. + * Use default compression level (6). */ assert((a = archive_write_new()) != NULL); assertEqualIntA(a, ARCHIVE_OK, archive_write_set_format_ustar(a)); @@ -93,6 +95,14 @@ DEFINE_TEST(test_write_filter_gzip) assertEqualIntA(a, ARCHIVE_OK, archive_write_close(a)); assertEqualInt(ARCHIVE_OK, archive_write_free(a)); + /* Basic gzip header tests */ + rbuff = (unsigned char *)buff; + assertEqualInt(rbuff[0], 0x1f); + assertEqualInt(rbuff[1], 0x8b); + assertEqualInt(rbuff[2], 0x08); + assertEqualInt(rbuff[3], 0x00); + assertEqualInt(rbuff[8], 0); /* RFC 1952 flag for compression level 6 */ + assert((a = archive_read_new()) != NULL); assertEqualIntA(a, ARCHIVE_OK, archive_read_support_format_all(a)); r = archive_read_support_filter_gzip(a); @@ -116,7 +126,7 @@ DEFINE_TEST(test_write_filter_gzip) /* * Repeat the cycle again, this time setting some compression - * options. + * options. Compression level is 9. */ assert((a = archive_write_new()) != NULL); assertEqualIntA(a, ARCHIVE_OK, archive_write_set_format_ustar(a)); @@ -150,6 +160,14 @@ DEFINE_TEST(test_write_filter_gzip) assertEqualIntA(a, ARCHIVE_OK, archive_write_close(a)); assertEqualInt(ARCHIVE_OK, archive_write_free(a)); + /* Basic gzip header tests */ + rbuff = (unsigned char *)buff; + assertEqualInt(rbuff[0], 0x1f); + assertEqualInt(rbuff[1], 0x8b); + assertEqualInt(rbuff[2], 0x08); + assertEqualInt(rbuff[3], 0x00); + assertEqualInt(rbuff[8], 2); /* RFC 1952 flag for compression level 9 */ + /* Curiously, this test fails; the test data above compresses * better at default compression than at level 9. */ /* @@ -181,7 +199,7 @@ DEFINE_TEST(test_write_filter_gzip) assertEqualInt(ARCHIVE_OK, archive_read_free(a)); /* - * Repeat again, with much lower compression. + * Repeat again, with compression level 1 */ assert((a = archive_write_new()) != NULL); assertEqualIntA(a, ARCHIVE_OK, archive_write_set_format_ustar(a)); @@ -212,6 +230,14 @@ DEFINE_TEST(test_write_filter_gzip) failure("Compression-level=1 wrote %d bytes; default wrote %d bytes", (int)used2, (int)used1); assert(used2 > used1); + + /* Basic gzip header tests */ + rbuff = (unsigned char *)buff; + assertEqualInt(rbuff[0], 0x1f); + assertEqualInt(rbuff[1], 0x8b); + assertEqualInt(rbuff[2], 0x08); + assertEqualInt(rbuff[3], 0x00); + assertEqualInt(rbuff[8], 4); /* RFC 1952 flag for compression level 1 */ assert((a = archive_read_new()) != NULL); assertEqualIntA(a, ARCHIVE_OK, archive_read_support_format_all(a)); Modified: head/contrib/libarchive/libarchive/test/test_write_filter_zstd.c ============================================================================== --- head/contrib/libarchive/libarchive/test/test_write_filter_zstd.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/libarchive/test/test_write_filter_zstd.c Sun Jul 29 00:12:16 2018 (r336854) @@ -125,7 +125,7 @@ DEFINE_TEST(test_write_filter_zstd) assertEqualIntA(a, ARCHIVE_OK, archive_write_set_filter_option(a, NULL, "compression-level", "9")); assertEqualIntA(a, ARCHIVE_OK, - archive_write_set_filter_option(a, NULL, "compression-level", "15")); + archive_write_set_filter_option(a, NULL, "compression-level", "6")); assertEqualIntA(a, ARCHIVE_OK, archive_write_open_memory(a, buff, buffsize, &used2)); for (i = 0; i < 100; i++) { sprintf(path, "file%03d", i); @@ -140,7 +140,7 @@ DEFINE_TEST(test_write_filter_zstd) assertEqualIntA(a, ARCHIVE_OK, archive_write_close(a)); assertEqualInt(ARCHIVE_OK, archive_write_free(a)); - failure("compression-level=15 wrote %d bytes, default wrote %d bytes", + failure("compression-level=6 wrote %d bytes, default wrote %d bytes", (int)used2, (int)used1); assert(used2 < used1); Modified: head/contrib/libarchive/tar/bsdtar.c ============================================================================== --- head/contrib/libarchive/tar/bsdtar.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/tar/bsdtar.c Sun Jul 29 00:12:16 2018 (r336854) @@ -920,7 +920,7 @@ usage(void) static void version(void) { - printf("bsdtar %s - %s\n", + printf("bsdtar %s - %s \n", BSDTAR_VERSION_STRING, archive_version_details()); exit(0); Copied: head/contrib/libarchive/tar/test/test_option_C_mtree.c (from r336851, vendor/libarchive/dist/tar/test/test_option_C_mtree.c) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/contrib/libarchive/tar/test/test_option_C_mtree.c Sun Jul 29 00:12:16 2018 (r336854, copy of r336851, vendor/libarchive/dist/tar/test/test_option_C_mtree.c) @@ -0,0 +1,73 @@ +/*- + * Copyright (c) 2018 The FreeBSD Foundation + * All rights reserved. + * + * This software was developed by Arshan Khanifar + * under sponsorship from the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR(S) ``AS IS'' AND ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. + * IN NO EVENT SHALL THE AUTHOR(S) BE LIABLE FOR ANY DIRECT, INDIRECT, + * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF + * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +#include "test.h" +__FBSDID("$FreeBSD$"); + +DEFINE_TEST(test_option_C_mtree) +{ + char *p0; + size_t s; + int r; + p0 = NULL; + char *content = "./foo type=file uname=root gname=root mode=0755\n"; + char *filename = "output.tar"; + + /* an absolute path to mtree file */ + char *mtree_file = "/METALOG.mtree"; + char *absolute_path = malloc(strlen(testworkdir) + strlen(mtree_file) + 1); + strcpy(absolute_path, testworkdir); + strcat(absolute_path, mtree_file ); + + /* Create an archive using an mtree file. */ + assertMakeFile(absolute_path, 0777, content); + assertMakeDir("bar", 0775); + assertMakeFile("bar/foo", 0777, "abc"); + + r = systemf("%s -cf %s -C bar \"@%s\" >step1.out 2>step1.err", testprog, filename, absolute_path); + + failure("Error invoking %s -cf %s -C bar @%s", testprog, filename, absolute_path); + assertEqualInt(r, 0); + assertEmptyFile("step1.out"); + assertEmptyFile("step1.err"); + + /* Do validation of the constructed archive. */ + + p0 = slurpfile(&s, "output.tar"); + if (!assert(p0 != NULL)) + goto done; + if (!assert(s >= 2048)) + goto done; + assertEqualMem(p0 + 0, "./foo", 5); + assertEqualMem(p0 + 512, "abc", 3); + assertEqualMem(p0 + 1024, "\0\0\0\0\0\0\0\0", 8); + assertEqualMem(p0 + 1536, "\0\0\0\0\0\0\0\0", 8); +done: + free(p0); +} + + Modified: head/contrib/libarchive/tar/test/test_option_fflags.c ============================================================================== --- head/contrib/libarchive/tar/test/test_option_fflags.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/tar/test/test_option_fflags.c Sun Jul 29 00:12:16 2018 (r336854) @@ -25,6 +25,10 @@ #include "test.h" __FBSDID("$FreeBSD$"); +#if defined(_WIN32) && !defined(__CYGWIN__) && !defined(__BORLANDC__) +#define chmod _chmod +#endif + static void clear_fflags(const char *pathname) { Modified: head/contrib/libarchive/tar/write.c ============================================================================== --- head/contrib/libarchive/tar/write.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/tar/write.c Sun Jul 29 00:12:16 2018 (r336854) @@ -503,7 +503,7 @@ write_archive(struct archive *a, struct bsdtar *bsdtar } set_chdir(bsdtar, arg); } else { - if (*arg != '/' && (arg[0] != '@' || arg[1] != '/')) + if (*arg != '/') do_chdir(bsdtar); /* Handle a deferred -C */ if (*arg == '@') { if (append_archive_filename(bsdtar, a, Modified: head/contrib/libarchive/test_utils/test_main.c ============================================================================== --- head/contrib/libarchive/test_utils/test_main.c Sat Jul 28 23:59:36 2018 (r336853) +++ head/contrib/libarchive/test_utils/test_main.c Sun Jul 29 00:12:16 2018 (r336854) @@ -2115,7 +2115,7 @@ void assertVersion(const char *prog, const char *base) int r; char *p, *q; size_t s; - unsigned int prog_len = strlen(base); + size_t prog_len = strlen(base); r = systemf("%s --version >version.stdout 2>version.stderr", prog); if (r != 0) Modified: head/usr.bin/tar/tests/Makefile ============================================================================== --- head/usr.bin/tar/tests/Makefile Sat Jul 28 23:59:36 2018 (r336853) +++ head/usr.bin/tar/tests/Makefile Sun Jul 29 00:12:16 2018 (r336854) @@ -40,6 +40,7 @@ TESTS_SRCS= \ test_help.c \ test_leading_slash.c \ test_missing_file.c \ + test_option_C_mtree.c \ test_option_C_upper.c \ test_option_H_upper.c \ test_option_L_upper.c \ From owner-svn-src-all@freebsd.org Sun Jul 29 00:30:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F245A1062CFD; Sun, 29 Jul 2018 00:30:07 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A43CC8742E; Sun, 29 Jul 2018 00:30:07 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 811A4649A; Sun, 29 Jul 2018 00:30:07 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T0U79K055128; Sun, 29 Jul 2018 00:30:07 GMT (envelope-from truckman@FreeBSD.org) Received: (from truckman@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T0U74H055127; Sun, 29 Jul 2018 00:30:07 GMT (envelope-from truckman@FreeBSD.org) Message-Id: <201807290030.w6T0U74H055127@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: truckman set sender to truckman@FreeBSD.org using -f From: Don Lewis Date: Sun, 29 Jul 2018 00:30:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336855 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: truckman X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 336855 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 00:30:08 -0000 Author: truckman Date: Sun Jul 29 00:30:06 2018 New Revision: 336855 URL: https://svnweb.freebsd.org/changeset/base/336855 Log: Fix the long term ULE load balancer so that it actually works. The initial call to sched_balance() during startup is meant to initialize balance_ticks, but does not actually do that since smp_started is still zero at that time. Since balance_ticks does not get set, there are no further calls to sched_balance(). Fix this by setting balance_ticks in sched_initticks() since we know the value of balance_interval at that time, and eliminate the useless startup call to sched_balance(). We don't need to randomize the intial value of balance_ticks. Since there is now only one call to sched_balance(), we can hoist the tests at the top of this function out to the caller and avoid the overhead of the function call when running a SMP kernel on UP hardware. PR: 223914 Reviewed by: kib MFC after: 2 weeks Modified: head/sys/kern/sched_ule.c Modified: head/sys/kern/sched_ule.c ============================================================================== --- head/sys/kern/sched_ule.c Sun Jul 29 00:12:16 2018 (r336854) +++ head/sys/kern/sched_ule.c Sun Jul 29 00:30:06 2018 (r336855) @@ -884,9 +884,6 @@ sched_balance(void) { struct tdq *tdq; - if (smp_started == 0 || rebalance == 0) - return; - balance_ticks = max(balance_interval / 2, 1) + (sched_random() % balance_interval); tdq = TDQ_SELF(); @@ -1413,7 +1410,6 @@ sched_setup_smp(void) panic("Can't find cpu group for %d\n", i); } balance_tdq = TDQ_SELF(); - sched_balance(); } #endif @@ -1474,6 +1470,7 @@ sched_initticks(void *dummy) * what realstathz is. */ balance_interval = realstathz; + balance_ticks = balance_interval; affinity = SCHED_AFFINITY_DEFAULT; #endif if (sched_idlespinthresh < 0) @@ -2382,7 +2379,7 @@ sched_clock(struct thread *td) /* * We run the long term load balancer infrequently on the first cpu. */ - if (balance_tdq == tdq) { + if (balance_tdq == tdq && smp_started != 0 && rebalance != 0) { if (balance_ticks && --balance_ticks == 0) sched_balance(); } From owner-svn-src-all@freebsd.org Sun Jul 29 01:48:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 456F81064DC2; Sun, 29 Jul 2018 01:48:46 +0000 (UTC) (envelope-from seanc@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E96FE8A250; Sun, 29 Jul 2018 01:48:45 +0000 (UTC) (envelope-from seanc@FreeBSD.org) Received: from localhost (50-46-198-89.evrt.wa.frontiernet.net [50.46.198.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: seanc) by smtp.freebsd.org (Postfix) with ESMTPSA id 5045B1D0C3; Sun, 29 Jul 2018 01:48:45 +0000 (UTC) (envelope-from seanc@FreeBSD.org) Date: Sat, 28 Jul 2018 18:48:43 -0700 From: Sean Chittenden To: rgrimes@freebsd.org, cem@freebsd.org, Eitan Adler Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers , core@FreeBSD.org Subject: Re: svn commit: r336757 - in head: share/man/man4 share/man/man7 share/misc sys/dev/firewire sys/dev/hwpmc sys/dev/sk sys/dev/sound/pci sys/dev/sound/pcm sys/fs/nfsclient Message-ID: <20180729014843.ea4vbqiyu5zl5kj2@FreeBSD.org> References: <201807271912.w6RJCJbs052385@pdx.rh.CN85.dnsmgr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="whsoynb3c7yqymt5" Content-Disposition: inline In-Reply-To: <201807271912.w6RJCJbs052385@pdx.rh.CN85.dnsmgr.net> User-Agent: NeoMutt/20180622 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 01:48:46 -0000 --whsoynb3c7yqymt5 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable > > This may intersect badly with our current policy of not shipping any CA= s in > > base. > > I objected to the conversion of http -> https in base when it started. I= saw > no good reason for it, and for the very reason you site, https is totally > useless in base until you have installed CA's. The inclusion of public CAs is a source of active debate by core@. In adva= nce of a final decision on that subject, we want to get ahead of some of this discussion. The FreeBSD Project's place on the interwebs is secured via HTTPS (with lim= ited exception). Referring to material hosted by the Project using HTTPS is sou= nd best practice that help us collectively improve our security posture. The links where the scheme was changed from http to https are all in documentation or comments, and are NOT used at runtime by developers, opera= tors, or any meaningful automation (i.e. this isn't something pkg(1) or fetch(1) uses). While this process of updating http links to https does cause a bit= of necessary churn, updating http links in documentation and comments is a reasonable activity that help us keep the project current with modern stand= ards. Maintenance activities that enhance our trust with the community is not glamorous and comes in the form of many similar incremental improvements. = Like many things in technology, the definition of what's relevant, competitive, = and modern changes over time (including hardware, protocols, performance primit= ives, developer productivity, and security best practices). Moving to HTTPS for non-runtime links is a sensible example of an incremental improvement that should not be considered avant-garde in this day and age. Regardless of the outcome of core@'s decision to include and maintain publi= c CAs in base (or change a default in the installer to install a port), modernizi= ng docs or other maintenance activities that improve our security posture is a= +1 activity from core@'s perspective. -sc (on behalf of core@) --=20 Sean Chittenden --whsoynb3c7yqymt5 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEE74y44SUGZ4YNR0/x11Nmqtx5Ry0FAltdHPdfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVG OENCOEUxMjUwNjY3ODYwRDQ3NEZGMUQ3NTM2NkFBREM3OTQ3MkQACgkQ11Nmqtx5 Ry1Z4ggAmxpMgrxkAz9q7CQbzLxLg0yHrq8FoCLkDvNMyL4+dCNiTVTRR2NFthBX cQ0CoerpeDroyuO08lh/+789XY5Eqv2ch53RNjDzrtc9zV2K5FTZ27or0tG6sXSA 3y2OtEkY3e/Gd1KB/i4ftuJWWjLiG/a/cgUJ93vNzHM0D3sTNULHUBleSO/ASU3k d2epBStviclW9hWqHlCIOlvFtG56TNAts4Xu8iT7PS4rEACzeo7hLRwOb0ZVlQIP cDbNctZ+0Ncq4AHjNIUESg8N756m38vTVp9HDgmJzz+fIrM8c8reR+VSeeZyMm1O NI6+Cyzfpyz6aQf7WNgqG4C7JloGZg== =YFcd -----END PGP SIGNATURE----- --whsoynb3c7yqymt5-- From owner-svn-src-all@freebsd.org Sun Jul 29 01:44:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C280B1064CEB for ; Sun, 29 Jul 2018 01:44:41 +0000 (UTC) (envelope-from will@firepipe.net) Received: from mail-qk0-x22f.google.com (mail-qk0-x22f.google.com [IPv6:2607:f8b0:400d:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B5428A133 for ; Sun, 29 Jul 2018 01:44:41 +0000 (UTC) (envelope-from will@firepipe.net) Received: by mail-qk0-x22f.google.com with SMTP id d22-v6so5712841qkc.8 for ; Sat, 28 Jul 2018 18:44:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=firepipe-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=7FqNfMivF9kSwi16eD6D4SUetLh9MI7avmLICFbiPGM=; b=1gXL4ZC4zurUpBaVwjMjgsri1Zo+qXGx4BT1ykDH119KQVvqEgK/1bhJMkNwsZSQrf KvymfV2cDy6DLaIqh3TISUtfhQFN0eVyHZkeLbDptLjf/HYZl4NUxgu8KzlQtEk6JRLR M+65C6Z9GrJPnA2kvn2R9qQ37dxuEusnnsk2KpvKvHOBYGC985Sy4CRqY8kS3+0WJCbI pwAQ/Cz4i0Jn3iswyfVCQ9vmycizVQ3fg73pkE97ZmiQYMfIeSyKpqtP4rdvPHy57ht0 FalH5FAP9iOs46LOy28De82ajWnr6Tr/UUjwg316uaRCnBzBET08PhBDAE5xjxw1GdbV dlUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=7FqNfMivF9kSwi16eD6D4SUetLh9MI7avmLICFbiPGM=; b=GXiUQnqNZ0gWBpEZl9qP/Gmb/KCLpSbW4CSu/nF/5sqZjGJk0qQ59YDsVB+r3Es+4M yGak5h0hn/QxHnoOnKEQ76fVk8LU/oueiiYyYBrdQwaIIJFI0aLWTm5GAC3CaZxr1Syw j6RFVdkPhQlXLnfruTBVX+10sN6xNCI+ytvcnc9bcbr4+PgEdtGJEUJu4S2XPhGiaxT4 Iyn+xxREZFpaoWjxhR7YRR2+/MAW+hAVRdE8oNoDk5m93HTFsdwCGjR0EkecY1oh8xJh 7xoWgNmNSIYLCo7E2D55eQ10eF5xR2PJvIhidK7tXqf9qwnej4F2vzQLJVdXsb+PLnuT N+yw== X-Gm-Message-State: AOUpUlFsnS/XvRbSPLl4/DNzXWeEKw6eTyA3HMI0eXXKF1MZGDt1Nefi MGjvsf/hJjhYIn/3QuZBqGJqUKGzT+lm7ZxYjC1CRg== X-Google-Smtp-Source: AAOMgpc4jY1Mmg2lmKz+LvzGhBe9eVmUIFhZZdYEuN3NC6KWe2HbYyH/pHwz6O26r0Mha7b1JM0eHDAijPJpwqjD2iQ= X-Received: by 2002:a37:6249:: with SMTP id w70-v6mr11090564qkb.312.1532828680350; Sat, 28 Jul 2018 18:44:40 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:aed:3809:0:0:0:0:0 with HTTP; Sat, 28 Jul 2018 18:44:39 -0700 (PDT) In-Reply-To: <0178BAB4-FEFB-473B-9245-3F11ABDA8C09@freebsd.org> References: <201807200044.w6K0i4QQ079894@repo.freebsd.org> <1532108303.1344.57.camel@freebsd.org> <67239C31-11DA-43E0-B282-5AE894C2B0CB@panasas.com> <0178BAB4-FEFB-473B-9245-3F11ABDA8C09@freebsd.org> From: Will Andrews Date: Sat, 28 Jul 2018 20:44:39 -0500 Message-ID: Subject: Re: svn commit: r336526 - head To: Ravi Pokala Cc: Ian Lepore , "src-committers@FreeBSD.org" , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 01:44:42 -0000 Hi Ravi, Thanks for taking an initial stab, I applied your patch and quickly realized the problem is deeper than it looks. This is because installworld and the second stage of mergemaster/etcupdate require the "pre" phase to take effect in their view of the system (specifically, the "ntpd" user must exist from their /etc/passwd, etc). For traditional upgrades, this occurs by simply rebooting. I've committed a revised version of your original patch, and have upgraded across this barrier using beinstall. Please give it a try and let me know how it goes. Thanks! --Will. On Wed, Jul 25, 2018 at 11:28 PM, Ravi Pokala wrote: > -----Original Message----- > From: Ravi Pokala > Date: 2018-07-20, Friday at 10:53 > To: Ian Lepore , , < > svn-src-all@freebsd.org>, , Will Andrews > > Subject: Re: svn commit: r336526 - head > > > -----Original Message----- > > From: on behalf of Ian Lepore < > ian@freebsd.org> > > Date: 2018-07-20, Friday at 10:38 > > To: Ravi Pokala , , < > svn-src-all@freebsd.org>, > > Subject: Re: svn commit: r336526 - head > > > >> On Fri, 2018-07-20 at 09:56 -0700, Ravi Pokala wrote: > >>> Hi Ian, > >>> > >>> -----Original Message----- > >>> From: on behalf of Ian Lepore > >>> Date: 2018-07-19, Thursday at 17:44 > >>> To: , , > >>> Subject: svn commit: r336526 - head > >>> > >>>> > >>>> Author: ian > >>>> Date: Fri Jul 20 00:44:04 2018 > >>>> New Revision: 336526 > >>>> URL: https://svnweb.freebsd.org/changeset/base/336526 > >>>> > >>>> Log: > >>>> Add ntpd to the list of users/groups to check before installing. > >>>> > >>>> Modified: > >>>> head/Makefile.inc1 > >>> When trying to installworld: > >>> > >>> ERROR: Required ntpd user is missing, see /usr/src/UPDATING. > >>> > >>> But there is no UPDATING entry about this. (Also, shouldn't it be > ${SRCTOP}/UPDATING?) > >>> > >>> Thanks, > >>> > >>> Ravi (rpokala@) > >> > >> I wondered about that too, when I accidentally stumbled across the > >> CHECK_UIDS thing and realized I should add ntpd to it. I searched in > >> UPDATING for some of the other things in the CHECK_UIDS list to see > >> what had been said about such changes in the past, even searching > >> through old-old svn history, and found nothing. > >> > >> I finally settled on the thought that the reference to UPDATING was to > >> the stuff at the bottom that never changes. In particular to the > >> section around line 1595 that documents the instalkernel, reboot, > >> mergemaster, installworld sequence. The mergemaster part should take > >> care of the new userid, I guess. > >> > >> -- Ian > > > > Interesting. I was attempting the install via the `beinstall.sh' script. > CCing will@, since he wrote it. > > > > Thanks, > > > > Ravi (rpokala@) > > After some discussion on IRC, I created this patch to fix `beinstall.sh'. > > https://people.freebsd.org/~rpokala/beinstall.patch > > However, I'm a bit afraid of testing it, until someone who actually knows > `etcupdate' and `mergemaster' can confirm that it won't blow up my box. :-) > > Can anyone take a look and assuage my fears? > > Thanks, > > Ravi (rpokala@) > > > From owner-svn-src-all@freebsd.org Sun Jul 29 01:44:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 728A51064CDE; Sun, 29 Jul 2018 01:44:27 +0000 (UTC) (envelope-from will@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 165AD8A04E; Sun, 29 Jul 2018 01:44:27 +0000 (UTC) (envelope-from will@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CCD4C727B; Sun, 29 Jul 2018 01:44:26 +0000 (UTC) (envelope-from will@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T1iQ4P095291; Sun, 29 Jul 2018 01:44:26 GMT (envelope-from will@FreeBSD.org) Received: (from will@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T1iQrI095290; Sun, 29 Jul 2018 01:44:26 GMT (envelope-from will@FreeBSD.org) Message-Id: <201807290144.w6T1iQrI095290@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: will set sender to will@FreeBSD.org using -f From: Will Andrews Date: Sun, 29 Jul 2018 01:44:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336856 - head/tools/build X-SVN-Group: head X-SVN-Commit-Author: will X-SVN-Commit-Paths: head/tools/build X-SVN-Commit-Revision: 336856 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 01:44:27 -0000 Author: will Date: Sun Jul 29 01:44:26 2018 New Revision: 336856 URL: https://svnweb.freebsd.org/changeset/base/336856 Log: beinstall: perform pre-installworld steps. Since all post-installkernel steps are assumed to operate in the updated installation, it's necessary to chroot all of the followup steps in the new boot environment. Set up and mount the source and object directories at the same paths inside the BE root, and clean up to the extent changes were made. This commit fixes upgrading using beinstall past the new ntpd user change. Improve testability of changes to this script while I'm here. Reported by: rpokala (earlier patch) Modified: head/tools/build/beinstall.sh Modified: head/tools/build/beinstall.sh ============================================================================== --- head/tools/build/beinstall.sh Sun Jul 29 00:30:06 2018 (r336855) +++ head/tools/build/beinstall.sh Sun Jul 29 01:44:26 2018 (r336856) @@ -78,26 +78,92 @@ rmdir_be() { rm -rf ${BE_MNTPT} } +unmount_be() { + mount | grep " on ${BE_MNTPT}" | awk '{print $3}' | sort -r | xargs -t umount -f +} + cleanup_be() { + # Before destroying, unmount any child filesystems that may have + # been mounted under the boot environment. Sort them in reverse + # order so children are unmounted first. + unmount_be + # Finally, clean up any directories that were created by the + # operation, via cleanup_be_dirs(). + if [ -n "${created_be_dirs}" ]; then + chroot ${BE_MNTPT} /bin/rm -rf ${created_be_dirs} + fi beadm destroy -F ${BENAME} } +create_be_dirs() { + echo "${BE_MNTPT}: Inspecting dirs $*" + for dir in $*; do + curdir="$dir" + topdir="$dir" + while :; do + [ -e "${BE_MNTPT}${curdir}" ] && break + topdir=$curdir + curdir=$(dirname ${curdir}) + done + [ "$curdir" = "$dir" ] && continue + + # Add the top-most nonexistent directory to the list, then + # mkdir -p the innermost directory specified by the argument. + # This way the least number of directories are rm'd directly. + created_be_dirs="${topdir} ${created_be_dirs}" + echo "${BE_MNTPT}: Created ${dir}" + mkdir -p ${BE_MNTPT}${dir} || return $? + done + return 0 +} + +update_mergemaster_pre() { + mergemaster -p -m ${srcdir} -D ${BE_MNTPT} -t ${BE_MM_ROOT} ${MERGEMASTER_FLAGS} +} + update_mergemaster() { - mergemaster -m $(pwd) -D ${BE_MNTPT} -t ${BE_MM_ROOT} ${MERGEMASTER_FLAGS} + chroot ${BE_MNTPT} \ + mergemaster -m ${srcdir} -t ${BE_MM_ROOT} ${MERGEMASTER_FLAGS} } +update_etcupdate_pre() { + etcupdate -p -s ${srcdir} -D ${BE_MNTPT} ${ETCUPDATE_FLAGS} || return $? + etcupdate resolve -D ${BE_MNTPT} || return $? +} + update_etcupdate() { - etcupdate -s $(pwd) -D ${BE_MNTPT} ${ETCUPDATE_FLAGS} || return $? - etcupdate resolve -D ${BE_MNTPT} + chroot ${BE_MNTPT} \ + etcupdate -s ${srcdir} ${ETCUPDATE_FLAGS} || return $? + chroot ${BE_MNTPT} etcupdate resolve } +# Special command-line subcommand that can be used to do a full cleanup +# after a manual post-mortem has been completed. +postmortem() { + [ -n "${BENAME}" ] || errx "Must specify BENAME" + [ -n "${BE_MNTPT}" ] || errx "Must specify BE_MNTPT" + echo "Performing post-mortem on BE ${BENAME} at ${BE_MNTPT} ..." + unmount_be + rmdir_be + echo "Post-mortem cleanup complete." + echo "To destroy the BE (recommended), run: beadm destroy ${BENAME}" + echo "To instead continue with the BE, run: beadm activate ${BENAME}" +} + +if [ -n "$BEINSTALL_CMD" ]; then + ${BEINSTALL_CMD} $* + exit $? +fi + + cleanup_commands="" trap 'errx "Interrupt caught"' HUP INT TERM [ "$(whoami)" != "root" ] && errx "Must be run as root" [ ! -f "Makefile.inc1" ] && errx "Must be in FreeBSD source tree" +srcdir=$(pwd) objdir=$(make -V .OBJDIR 2>/dev/null) [ ! -d "${objdir}" ] && errx "Must have built FreeBSD from source tree" @@ -139,11 +205,24 @@ beadm mount ${BENAME} ${BE_TMP}/mnt || errx "Unable to echo "Mounted ${BENAME} to ${BE_MNTPT}, performing install/update ..." make "$@" DESTDIR=${BE_MNTPT} installkernel || errx "Installkernel failed!" -make "$@" DESTDIR=${BE_MNTPT} installworld || errx "Installworld failed!" +if [ -n "${CONFIG_UPDATER}" ]; then + "update_${CONFIG_UPDATER}_pre" + [ $? -ne 0 ] && errx "${CONFIG_UPDATER} (pre-world) failed!" +fi +# Mount the source and object tree within the BE in order to account for any +# changes applied by the pre-installworld updater. Cleanup any directories +# created if they didn't exist previously. +create_be_dirs "${srcdir}" "${objdir}" || errx "Unable to create BE dirs" +mount -t nullfs "${srcdir}" "${BE_MNTPT}${srcdir}" || errx "Unable to mount src" +mount -t nullfs "${objdir}" "${BE_MNTPT}${objdir}" || errx "Unable to mount obj" + +chroot ${BE_MNTPT} make "$@" -C ${srcdir} installworld || \ + errx "Installworld failed!" + if [ -n "${CONFIG_UPDATER}" ]; then "update_${CONFIG_UPDATER}" - [ $? -ne 0 ] && errx "${CONFIG_UPDATER} failed!" + [ $? -ne 0 ] && errx "${CONFIG_UPDATER} (post-world) failed!" fi BE_PKG="chroot ${BE_MNTPT} env ASSUME_ALWAYS_YES=true pkg" @@ -152,8 +231,15 @@ if [ -z "${NO_PKG_UPGRADE}" ]; then ${BE_PKG} upgrade || errx "Unable to upgrade pkgs" fi -beadm unmount ${BENAME} || errx "Unable to unmount BE" -rmdir_be +if [ -n "$NO_CLEANUP_BE" ]; then + echo "Boot Environment ${BENAME} may be examined in ${BE_MNTPT}." + echo "Afterwards, run this to cleanup:" + echo " env BENAME=${BENAME} BE_MNTPT=${BE_MNTPT} BEINSTALL_CMD=postmortem $0" + exit 0 +fi + +unmount_be || errx "Unable to unmount BE" +rmdir_be || errx "Unable to cleanup BE" beadm activate ${BENAME} || errx "Unable to activate BE" echo beadm list From owner-svn-src-all@freebsd.org Sun Jul 29 03:20:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 367A11046B04; Sun, 29 Jul 2018 03:20:06 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E14F28E096; Sun, 29 Jul 2018 03:20:05 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A851910217; Sun, 29 Jul 2018 03:20:05 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T3K5Ax042495; Sun, 29 Jul 2018 03:20:05 GMT (envelope-from brd@FreeBSD.org) Received: (from brd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T3K5dI042494; Sun, 29 Jul 2018 03:20:05 GMT (envelope-from brd@FreeBSD.org) Message-Id: <201807290320.w6T3K5dI042494@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: brd set sender to brd@FreeBSD.org using -f From: Brad Davis Date: Sun, 29 Jul 2018 03:20:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336857 - head/sbin/init/rc.d X-SVN-Group: head X-SVN-Commit-Author: brd X-SVN-Commit-Paths: head/sbin/init/rc.d X-SVN-Commit-Revision: 336857 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 03:20:06 -0000 Author: brd Date: Sun Jul 29 03:20:05 2018 New Revision: 336857 URL: https://svnweb.freebsd.org/changeset/base/336857 Log: Fix a typo that prevented some rc scripts from being installed. Approved by: will (mentor) Differential Revision: https://reviews.freebsd.org/D16495 Modified: head/sbin/init/rc.d/Makefile Modified: head/sbin/init/rc.d/Makefile ============================================================================== --- head/sbin/init/rc.d/Makefile Sun Jul 29 01:44:26 2018 (r336856) +++ head/sbin/init/rc.d/Makefile Sun Jul 29 03:20:05 2018 (r336857) @@ -3,7 +3,7 @@ .include CONFSDIR= /etc/rc.d -CONFSGROUPS= CONFS +CONFGROUPS= CONFS CONFS= DAEMON \ FILESYSTEMS \ @@ -130,13 +130,13 @@ CONFS+= ypbind \ .endif .if ${MK_ACCT} != "no" -CONFSGROUPS+= ACCT +CONFGROUPS+= ACCT ACCT+= accounting ACCTPACKAGE= acct .endif .if ${MK_ACPI} != "no" -CONFSGROUPS+= ACPI +CONFGROUPS+= ACPI ACPI= power_profile ACPIPACKAGE= acpi .endif @@ -146,13 +146,13 @@ CONFS+= powerd .endif .if ${MK_AMD} != "no" -CONFSGROUPS+= AMD +CONFGROUPS+= AMD AMD+= amd AMDPACKAGE= amd .endif .if ${MK_APM} != "no" -CONFSGROUPS+= APM +CONFGROUPS+= APM APM+= apm .if ${MACHINE} == "i386" APM+= apmd @@ -184,7 +184,7 @@ CONFS+= bootparams .endif .if ${MK_BSNMP} != "no" -CONFSGROUPS+= BSNMP +CONFGROUPS+= BSNMP BSNMP+= bsnmpd BSNMPPACKAGE= bsnmp .endif @@ -198,7 +198,7 @@ CONFS+= ftpd .endif .if ${MK_HAST} != "no" -CONFSGROUPS+= HAST +CONFGROUPS+= HAST HAST= hastd HASTPACKAGE= hast .endif @@ -220,7 +220,7 @@ CONFS+= iscsid .endif .if ${MK_JAIL} != "no" -CONFSGROUPS+= JAIL +CONFGROUPS+= JAIL JAIL+= jail JAILPACKAGE= jail .endif @@ -268,7 +268,7 @@ CONFS+= keyserv .endif .if ${MK_OPENSSH} != "no" -CONFSGROUPS+= SSH +CONFGROUPS+= SSH SSH= sshd SSHPACKAGE= ssh .endif @@ -282,7 +282,7 @@ CONFS+= routed .endif .if ${MK_SENDMAIL} != "no" -CONFSGROUPS+= SMRCD +CONFGROUPS+= SMRCD SMRCD= sendmail SMRCDPACKAGE= sendmail .endif @@ -292,7 +292,7 @@ CONFS+= timed .endif .if ${MK_UNBOUND} != "no" -CONFSGROUPS+= UNBOUND +CONFGROUPS+= UNBOUND UNBOUND+= local_unbound UNBOUNDPACKAGE= unbound .endif @@ -302,7 +302,7 @@ _utx= utx .endif .if ${MK_VI} != "no" -CONFSGROUPS+= VI +CONFGROUPS+= VI VI+= virecover VIPACKAGE= vi .endif @@ -313,7 +313,7 @@ CONFS+= wpa_supplicant .endif .if ${MK_ZFS} != "no" -CONFSGROUPS+= ZFS +CONFGROUPS+= ZFS ZFS+= zfs ZFS+= zfsbe ZFS+= zfsd @@ -324,7 +324,7 @@ ETC_ZFS= /etc/zfs ETC_ZFSPACKAGE= zfs .endif -.for fg in ${CONFSGROUPS} +.for fg in ${CONFGROUPS} ${fg}MODE?= ${BINMODE} .endfor From owner-svn-src-all@freebsd.org Sun Jul 29 05:12:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA370104AF4C for ; Sun, 29 Jul 2018 05:12:52 +0000 (UTC) (envelope-from eadler@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FECF7172D for ; Sun, 29 Jul 2018 05:12:52 +0000 (UTC) (envelope-from eadler@freebsd.org) Received: from mail-yb0-f169.google.com (mail-yb0-f169.google.com [209.85.213.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) (Authenticated sender: eadler) by smtp.freebsd.org (Postfix) with ESMTPSA id 4AE5F1E60A for ; Sun, 29 Jul 2018 05:12:52 +0000 (UTC) (envelope-from eadler@freebsd.org) Received: by mail-yb0-f169.google.com with SMTP id s8-v6so3567123ybe.8 for ; Sat, 28 Jul 2018 22:12:52 -0700 (PDT) X-Gm-Message-State: AOUpUlEvYOVjDZ54dvnUxp32CyI9KvWpc3rmq281EjOHk7JpWGwlnLXx ymGT7kcGwq6hel+apcrYQkgf4C7ogGINhIq1uVYQ6w== X-Google-Smtp-Source: AAOMgpedkrHCYgxiBUB1mPMtGPwfW0mh71GtudECmje0uhQCAiMJRtXL4HVoyJmZNvkXpOzjrqZhNsyYD/6ytH9Wc7w= X-Received: by 2002:a25:22c1:: with SMTP id i184-v6mr6677906ybi.479.1532841171700; Sat, 28 Jul 2018 22:12:51 -0700 (PDT) MIME-Version: 1.0 References: <201807280737.w6S7b12F025452@repo.freebsd.org> <20180728233405.GD40119@kib.kiev.ua> In-Reply-To: <20180728233405.GD40119@kib.kiev.ua> From: Eitan Adler Date: Sat, 28 Jul 2018 22:12:25 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r336805 - in stable/11: share/man/man4 sys/dev/vt/hw/vga To: Kostik Belousov Cc: src-committers , svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 05:12:53 -0000 On Sat, 28 Jul 2018 at 16:34, Konstantin Belousov wrote: > > On Sat, Jul 28, 2018 at 07:37:01AM +0000, Eitan Adler wrote: > > Author: eadler > > Date: Sat Jul 28 07:37:01 2018 > > New Revision: 336805 > > URL: https://svnweb.freebsd.org/changeset/base/336805 > > > > Log: > > MFC r335629: > > > > vt: add option to ignore NO_VGA flag in ACPI > > > > To workaround buggy firmware that sets this flag when there's actually > > a VGA present. > > > > Ref D16003 > > PR: 229235 > > > > Modified: > > stable/11/share/man/man4/vt.4 > > stable/11/sys/dev/vt/hw/vga/vt_vga.c > > Directory Properties: > > stable/11/ (props changed) > > > > Modified: stable/11/share/man/man4/vt.4 > > ============================================================================== > > --- stable/11/share/man/man4/vt.4 Sat Jul 28 06:46:10 2018 (r336804) > > +++ stable/11/share/man/man4/vt.4 Sat Jul 28 07:37:01 2018 (r336805) > > @@ -44,6 +44,7 @@ > > In > > .Xr loader.conf 5 : > > .Cd hw.vga.textmode=1 > > +.Cd hw.vga.acpi_ignore_no_vga=1 > > .Cd kern.vty=vt > > .Cd kern.vt.color..rgb="" > > .Cd kern.vt.fb.default_mode="x" > > @@ -196,6 +197,10 @@ prompt or in > > Set to 1 to use virtual terminals in text mode instead of graphics mode. > > Features that require graphics mode, like loadable fonts, will be > > disabled. > > +.It Va hw.vga.acpi_ignore_no_vga > > +Set to 1 to force the usage of the VGA driver regardless of whether > > +ACPI IAPC_BOOT_ARCH signals no VGA support. > > +Can be used to workaround firmware bugs in the ACPI tables. > > .It Va kern.vty > > Set this value to > > .Ql vt > > > > Modified: stable/11/sys/dev/vt/hw/vga/vt_vga.c > > ============================================================================== > > --- stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 06:46:10 2018 (r336804) > > +++ stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 07:37:01 2018 (r336805) > > @@ -1211,6 +1211,12 @@ vga_acpi_disabled(void) > > ACPI_TABLE_FADT *fadt; > > vm_paddr_t physaddr; > > uint16_t flags; > > + int ignore; > > + > > + TUNABLE_INT_FETCH("hw.vga.acpi_ignore_no_vga", &ignore); > > + > > + if (ignore) > > + return (false); > If the tunable is not set, this code evaluates the stack garbage. > Why did you merged this without asking the author and not looking > at the followups ? Please assume some level of good faith. I MFCed this because I was asked to here https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229235 over a month ago. During that time no one mentioned or linked any followups. I did look in my email for mentions of r335629 as well - of which there are none. Now that you mention it, I looked at the svn log and found r335631 which I will MFC as well. -- Eitan Adler Source, Ports, Doc committer Bugmeister, Ports Security teams From owner-svn-src-all@freebsd.org Sun Jul 29 05:14:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7852D104B009; Sun, 29 Jul 2018 05:14:27 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E01B718A9; Sun, 29 Jul 2018 05:14:27 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0BBD9115DC; Sun, 29 Jul 2018 05:14:27 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T5EQtW004083; Sun, 29 Jul 2018 05:14:26 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T5EQPq004082; Sun, 29 Jul 2018 05:14:26 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290514.w6T5EQPq004082@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 05:14:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336858 - stable/11/sys/dev/vt/hw/vga X-SVN-Group: stable-11 X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: stable/11/sys/dev/vt/hw/vga X-SVN-Commit-Revision: 336858 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 05:14:27 -0000 Author: eadler Date: Sun Jul 29 05:14:26 2018 New Revision: 336858 URL: https://svnweb.freebsd.org/changeset/base/336858 Log: MFC r335631: Always initialize the ignore local variable. PR: 229235 Modified: stable/11/sys/dev/vt/hw/vga/vt_vga.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/vt/hw/vga/vt_vga.c ============================================================================== --- stable/11/sys/dev/vt/hw/vga/vt_vga.c Sun Jul 29 03:20:05 2018 (r336857) +++ stable/11/sys/dev/vt/hw/vga/vt_vga.c Sun Jul 29 05:14:26 2018 (r336858) @@ -1213,6 +1213,7 @@ vga_acpi_disabled(void) uint16_t flags; int ignore; + ignore = 0; TUNABLE_INT_FETCH("hw.vga.acpi_ignore_no_vga", &ignore); if (ignore) From owner-svn-src-all@freebsd.org Sun Jul 29 05:42:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 875AB104BEBD; Sun, 29 Jul 2018 05:42:08 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D84272556; Sun, 29 Jul 2018 05:42:08 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1FBFA11A9B; Sun, 29 Jul 2018 05:42:08 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T5g80E018139; Sun, 29 Jul 2018 05:42:08 GMT (envelope-from truckman@FreeBSD.org) Received: (from truckman@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T5g8rt018138; Sun, 29 Jul 2018 05:42:08 GMT (envelope-from truckman@FreeBSD.org) Message-Id: <201807290542.w6T5g8rt018138@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: truckman set sender to truckman@FreeBSD.org using -f From: Don Lewis Date: Sun, 29 Jul 2018 05:42:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336859 - head/sbin/init/rc.d X-SVN-Group: head X-SVN-Commit-Author: truckman X-SVN-Commit-Paths: head/sbin/init/rc.d X-SVN-Commit-Revision: 336859 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 05:42:08 -0000 Author: truckman Date: Sun Jul 29 05:42:07 2018 New Revision: 336859 URL: https://svnweb.freebsd.org/changeset/base/336859 Log: Fix a variable name typo in r336845 that prevented the rc.d scripts from being installed in the correct directory. Resurrect a few rc.d scripts that were prematurely deleted from the Makefile by r336845. Reviewed by: brd Modified: head/sbin/init/rc.d/Makefile Modified: head/sbin/init/rc.d/Makefile ============================================================================== --- head/sbin/init/rc.d/Makefile Sun Jul 29 05:14:26 2018 (r336858) +++ head/sbin/init/rc.d/Makefile Sun Jul 29 05:42:07 2018 (r336859) @@ -2,7 +2,7 @@ .include -CONFSDIR= /etc/rc.d +CONFDIR= /etc/rc.d CONFGROUPS= CONFS CONFS= DAEMON \ @@ -75,6 +75,7 @@ CONFS= DAEMON \ netif \ netoptions \ netwait \ + newsyslog \ nfsclient \ nfscbd \ nfsd \ @@ -112,6 +113,8 @@ CONFS= DAEMON \ stf \ swap \ swaplate \ + sysctl \ + syslogd \ tmp \ ${_ubthidhci} \ ugidfw \ From owner-svn-src-all@freebsd.org Sun Jul 29 05:49:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE7A8104C154; Sun, 29 Jul 2018 05:49:49 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D0B2727E2; Sun, 29 Jul 2018 05:49:49 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3244A11ABF; Sun, 29 Jul 2018 05:49:49 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T5nmSv019202; Sun, 29 Jul 2018 05:49:48 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T5nmwZ019197; Sun, 29 Jul 2018 05:49:48 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290549.w6T5nmwZ019197@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 05:49:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336860 - in vendor/file/5.34: . doc m4 magic magic/Magdir python src tests X-SVN-Group: vendor X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: in vendor/file/5.34: . doc m4 magic magic/Magdir python src tests X-SVN-Commit-Revision: 336860 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 05:49:50 -0000 Author: eadler Date: Sun Jul 29 05:49:48 2018 New Revision: 336860 URL: https://svnweb.freebsd.org/changeset/base/336860 Log: vendor/file: add 5.34 gpg: Good signature from "Christos Zoulas (personal key) " [unknown] gpg: aka "Christos Zoulas (NetBSD key) " [unknown] Obtained from: ftp://ftp.astron.com/pub/file/ Added: vendor/file/5.34/ vendor/file/5.34/AUTHORS vendor/file/5.34/COPYING vendor/file/5.34/ChangeLog vendor/file/5.34/INSTALL vendor/file/5.34/MAINT vendor/file/5.34/Makefile.am (contents, props changed) vendor/file/5.34/Makefile.in (contents, props changed) vendor/file/5.34/NEWS vendor/file/5.34/README vendor/file/5.34/TODO vendor/file/5.34/acinclude.m4 vendor/file/5.34/aclocal.m4 vendor/file/5.34/compile (contents, props changed) vendor/file/5.34/config.guess (contents, props changed) vendor/file/5.34/config.h.in (contents, props changed) vendor/file/5.34/config.sub (contents, props changed) vendor/file/5.34/configure (contents, props changed) vendor/file/5.34/configure.ac vendor/file/5.34/depcomp (contents, props changed) vendor/file/5.34/doc/ vendor/file/5.34/doc/Makefile.am (contents, props changed) vendor/file/5.34/doc/Makefile.in (contents, props changed) vendor/file/5.34/doc/file.man vendor/file/5.34/doc/libmagic.man vendor/file/5.34/doc/magic.man vendor/file/5.34/install-sh (contents, props changed) vendor/file/5.34/ltmain.sh (contents, props changed) vendor/file/5.34/m4/ vendor/file/5.34/m4/libtool.m4 vendor/file/5.34/m4/ltoptions.m4 vendor/file/5.34/m4/ltsugar.m4 vendor/file/5.34/m4/ltversion.m4 vendor/file/5.34/m4/lt~obsolete.m4 vendor/file/5.34/magic/ vendor/file/5.34/magic/Header vendor/file/5.34/magic/Localstuff vendor/file/5.34/magic/Magdir/ vendor/file/5.34/magic/Magdir/acorn vendor/file/5.34/magic/Magdir/adi vendor/file/5.34/magic/Magdir/adventure vendor/file/5.34/magic/Magdir/algol68 vendor/file/5.34/magic/Magdir/allegro vendor/file/5.34/magic/Magdir/alliant vendor/file/5.34/magic/Magdir/amanda vendor/file/5.34/magic/Magdir/amigaos vendor/file/5.34/magic/Magdir/android vendor/file/5.34/magic/Magdir/animation vendor/file/5.34/magic/Magdir/aout vendor/file/5.34/magic/Magdir/apache (contents, props changed) vendor/file/5.34/magic/Magdir/apl vendor/file/5.34/magic/Magdir/apple (contents, props changed) vendor/file/5.34/magic/Magdir/application vendor/file/5.34/magic/Magdir/applix vendor/file/5.34/magic/Magdir/apt vendor/file/5.34/magic/Magdir/archive vendor/file/5.34/magic/Magdir/assembler vendor/file/5.34/magic/Magdir/asterix vendor/file/5.34/magic/Magdir/att3b vendor/file/5.34/magic/Magdir/audio vendor/file/5.34/magic/Magdir/basis vendor/file/5.34/magic/Magdir/beetle vendor/file/5.34/magic/Magdir/ber vendor/file/5.34/magic/Magdir/bflt vendor/file/5.34/magic/Magdir/bhl vendor/file/5.34/magic/Magdir/bioinformatics vendor/file/5.34/magic/Magdir/blackberry vendor/file/5.34/magic/Magdir/blcr vendor/file/5.34/magic/Magdir/blender vendor/file/5.34/magic/Magdir/blit vendor/file/5.34/magic/Magdir/bout vendor/file/5.34/magic/Magdir/bsdi vendor/file/5.34/magic/Magdir/bsi vendor/file/5.34/magic/Magdir/btsnoop vendor/file/5.34/magic/Magdir/c-lang vendor/file/5.34/magic/Magdir/c64 vendor/file/5.34/magic/Magdir/cad vendor/file/5.34/magic/Magdir/cafebabe vendor/file/5.34/magic/Magdir/cbor vendor/file/5.34/magic/Magdir/cddb vendor/file/5.34/magic/Magdir/chord vendor/file/5.34/magic/Magdir/cisco vendor/file/5.34/magic/Magdir/citrus vendor/file/5.34/magic/Magdir/clarion vendor/file/5.34/magic/Magdir/claris vendor/file/5.34/magic/Magdir/clipper vendor/file/5.34/magic/Magdir/coff vendor/file/5.34/magic/Magdir/commands vendor/file/5.34/magic/Magdir/communications vendor/file/5.34/magic/Magdir/compress vendor/file/5.34/magic/Magdir/console vendor/file/5.34/magic/Magdir/convex vendor/file/5.34/magic/Magdir/coverage vendor/file/5.34/magic/Magdir/cracklib vendor/file/5.34/magic/Magdir/ctags vendor/file/5.34/magic/Magdir/ctf vendor/file/5.34/magic/Magdir/cubemap vendor/file/5.34/magic/Magdir/cups vendor/file/5.34/magic/Magdir/dact vendor/file/5.34/magic/Magdir/database vendor/file/5.34/magic/Magdir/dataone vendor/file/5.34/magic/Magdir/dbpf vendor/file/5.34/magic/Magdir/der vendor/file/5.34/magic/Magdir/diamond vendor/file/5.34/magic/Magdir/diff vendor/file/5.34/magic/Magdir/digital vendor/file/5.34/magic/Magdir/dolby vendor/file/5.34/magic/Magdir/dump vendor/file/5.34/magic/Magdir/dyadic vendor/file/5.34/magic/Magdir/ebml vendor/file/5.34/magic/Magdir/editors vendor/file/5.34/magic/Magdir/efi vendor/file/5.34/magic/Magdir/elf vendor/file/5.34/magic/Magdir/encore vendor/file/5.34/magic/Magdir/epoc vendor/file/5.34/magic/Magdir/erlang vendor/file/5.34/magic/Magdir/esri vendor/file/5.34/magic/Magdir/fcs vendor/file/5.34/magic/Magdir/filesystems vendor/file/5.34/magic/Magdir/finger vendor/file/5.34/magic/Magdir/flash vendor/file/5.34/magic/Magdir/flif vendor/file/5.34/magic/Magdir/fonts vendor/file/5.34/magic/Magdir/fortran vendor/file/5.34/magic/Magdir/frame vendor/file/5.34/magic/Magdir/freebsd vendor/file/5.34/magic/Magdir/fsav vendor/file/5.34/magic/Magdir/fusecompress vendor/file/5.34/magic/Magdir/games vendor/file/5.34/magic/Magdir/gcc vendor/file/5.34/magic/Magdir/gconv vendor/file/5.34/magic/Magdir/geo vendor/file/5.34/magic/Magdir/geos vendor/file/5.34/magic/Magdir/gimp vendor/file/5.34/magic/Magdir/gnome vendor/file/5.34/magic/Magdir/gnu vendor/file/5.34/magic/Magdir/gnumeric vendor/file/5.34/magic/Magdir/gpt vendor/file/5.34/magic/Magdir/gpu vendor/file/5.34/magic/Magdir/grace vendor/file/5.34/magic/Magdir/graphviz vendor/file/5.34/magic/Magdir/gringotts vendor/file/5.34/magic/Magdir/guile vendor/file/5.34/magic/Magdir/hitachi-sh vendor/file/5.34/magic/Magdir/hp vendor/file/5.34/magic/Magdir/human68k vendor/file/5.34/magic/Magdir/ibm370 vendor/file/5.34/magic/Magdir/ibm6000 vendor/file/5.34/magic/Magdir/icc vendor/file/5.34/magic/Magdir/iff vendor/file/5.34/magic/Magdir/images vendor/file/5.34/magic/Magdir/inform vendor/file/5.34/magic/Magdir/intel vendor/file/5.34/magic/Magdir/interleaf vendor/file/5.34/magic/Magdir/island vendor/file/5.34/magic/Magdir/ispell vendor/file/5.34/magic/Magdir/isz vendor/file/5.34/magic/Magdir/java vendor/file/5.34/magic/Magdir/javascript vendor/file/5.34/magic/Magdir/jpeg vendor/file/5.34/magic/Magdir/karma vendor/file/5.34/magic/Magdir/kde vendor/file/5.34/magic/Magdir/keepass vendor/file/5.34/magic/Magdir/kerberos vendor/file/5.34/magic/Magdir/kml vendor/file/5.34/magic/Magdir/lecter vendor/file/5.34/magic/Magdir/lex vendor/file/5.34/magic/Magdir/lif vendor/file/5.34/magic/Magdir/linux vendor/file/5.34/magic/Magdir/lisp vendor/file/5.34/magic/Magdir/llvm vendor/file/5.34/magic/Magdir/lua vendor/file/5.34/magic/Magdir/luks vendor/file/5.34/magic/Magdir/m4 vendor/file/5.34/magic/Magdir/mach vendor/file/5.34/magic/Magdir/macintosh vendor/file/5.34/magic/Magdir/macos vendor/file/5.34/magic/Magdir/magic vendor/file/5.34/magic/Magdir/mail.news vendor/file/5.34/magic/Magdir/make vendor/file/5.34/magic/Magdir/map vendor/file/5.34/magic/Magdir/maple vendor/file/5.34/magic/Magdir/marc21 vendor/file/5.34/magic/Magdir/mathcad vendor/file/5.34/magic/Magdir/mathematica vendor/file/5.34/magic/Magdir/matroska vendor/file/5.34/magic/Magdir/mcrypt vendor/file/5.34/magic/Magdir/measure vendor/file/5.34/magic/Magdir/mercurial vendor/file/5.34/magic/Magdir/metastore vendor/file/5.34/magic/Magdir/meteorological vendor/file/5.34/magic/Magdir/microfocus vendor/file/5.34/magic/Magdir/mime vendor/file/5.34/magic/Magdir/mips vendor/file/5.34/magic/Magdir/mirage vendor/file/5.34/magic/Magdir/misctools vendor/file/5.34/magic/Magdir/mkid vendor/file/5.34/magic/Magdir/mlssa vendor/file/5.34/magic/Magdir/mmdf vendor/file/5.34/magic/Magdir/modem vendor/file/5.34/magic/Magdir/motorola vendor/file/5.34/magic/Magdir/mozilla vendor/file/5.34/magic/Magdir/msdos vendor/file/5.34/magic/Magdir/msooxml vendor/file/5.34/magic/Magdir/msvc vendor/file/5.34/magic/Magdir/msx vendor/file/5.34/magic/Magdir/mup vendor/file/5.34/magic/Magdir/music vendor/file/5.34/magic/Magdir/nasa vendor/file/5.34/magic/Magdir/natinst vendor/file/5.34/magic/Magdir/ncr vendor/file/5.34/magic/Magdir/neko vendor/file/5.34/magic/Magdir/netbsd vendor/file/5.34/magic/Magdir/netscape vendor/file/5.34/magic/Magdir/netware vendor/file/5.34/magic/Magdir/news vendor/file/5.34/magic/Magdir/nitpicker vendor/file/5.34/magic/Magdir/oasis vendor/file/5.34/magic/Magdir/ocaml vendor/file/5.34/magic/Magdir/octave vendor/file/5.34/magic/Magdir/ole2compounddocs vendor/file/5.34/magic/Magdir/olf vendor/file/5.34/magic/Magdir/os2 vendor/file/5.34/magic/Magdir/os400 vendor/file/5.34/magic/Magdir/os9 vendor/file/5.34/magic/Magdir/osf1 vendor/file/5.34/magic/Magdir/palm vendor/file/5.34/magic/Magdir/parix vendor/file/5.34/magic/Magdir/parrot vendor/file/5.34/magic/Magdir/pascal vendor/file/5.34/magic/Magdir/pbf vendor/file/5.34/magic/Magdir/pbm vendor/file/5.34/magic/Magdir/pc88 vendor/file/5.34/magic/Magdir/pc98 vendor/file/5.34/magic/Magdir/pdf (contents, props changed) vendor/file/5.34/magic/Magdir/pdp vendor/file/5.34/magic/Magdir/perl vendor/file/5.34/magic/Magdir/pgf vendor/file/5.34/magic/Magdir/pgp vendor/file/5.34/magic/Magdir/pkgadd vendor/file/5.34/magic/Magdir/plan9 vendor/file/5.34/magic/Magdir/plus5 vendor/file/5.34/magic/Magdir/polyml vendor/file/5.34/magic/Magdir/printer vendor/file/5.34/magic/Magdir/project vendor/file/5.34/magic/Magdir/psdbms vendor/file/5.34/magic/Magdir/psl vendor/file/5.34/magic/Magdir/pulsar vendor/file/5.34/magic/Magdir/pwsafe vendor/file/5.34/magic/Magdir/pyramid vendor/file/5.34/magic/Magdir/python vendor/file/5.34/magic/Magdir/qt vendor/file/5.34/magic/Magdir/revision vendor/file/5.34/magic/Magdir/riff vendor/file/5.34/magic/Magdir/rpi vendor/file/5.34/magic/Magdir/rpm vendor/file/5.34/magic/Magdir/rtf vendor/file/5.34/magic/Magdir/ruby vendor/file/5.34/magic/Magdir/sc vendor/file/5.34/magic/Magdir/sccs vendor/file/5.34/magic/Magdir/scientific vendor/file/5.34/magic/Magdir/securitycerts vendor/file/5.34/magic/Magdir/selinux vendor/file/5.34/magic/Magdir/sendmail vendor/file/5.34/magic/Magdir/sequent vendor/file/5.34/magic/Magdir/sereal vendor/file/5.34/magic/Magdir/sgi vendor/file/5.34/magic/Magdir/sgml vendor/file/5.34/magic/Magdir/sharc vendor/file/5.34/magic/Magdir/sinclair vendor/file/5.34/magic/Magdir/sisu vendor/file/5.34/magic/Magdir/sketch vendor/file/5.34/magic/Magdir/smalltalk vendor/file/5.34/magic/Magdir/smile vendor/file/5.34/magic/Magdir/sniffer vendor/file/5.34/magic/Magdir/softquad vendor/file/5.34/magic/Magdir/spec vendor/file/5.34/magic/Magdir/spectrum vendor/file/5.34/magic/Magdir/sql vendor/file/5.34/magic/Magdir/ssh vendor/file/5.34/magic/Magdir/ssl vendor/file/5.34/magic/Magdir/sun vendor/file/5.34/magic/Magdir/symbos vendor/file/5.34/magic/Magdir/sysex vendor/file/5.34/magic/Magdir/tcl vendor/file/5.34/magic/Magdir/teapot vendor/file/5.34/magic/Magdir/terminfo vendor/file/5.34/magic/Magdir/tex vendor/file/5.34/magic/Magdir/tgif vendor/file/5.34/magic/Magdir/ti-8x vendor/file/5.34/magic/Magdir/timezone vendor/file/5.34/magic/Magdir/tplink vendor/file/5.34/magic/Magdir/troff vendor/file/5.34/magic/Magdir/tuxedo vendor/file/5.34/magic/Magdir/typeset vendor/file/5.34/magic/Magdir/unicode vendor/file/5.34/magic/Magdir/unknown vendor/file/5.34/magic/Magdir/uterus vendor/file/5.34/magic/Magdir/uuencode vendor/file/5.34/magic/Magdir/vacuum-cleaner vendor/file/5.34/magic/Magdir/varied.out vendor/file/5.34/magic/Magdir/varied.script vendor/file/5.34/magic/Magdir/vax vendor/file/5.34/magic/Magdir/vicar vendor/file/5.34/magic/Magdir/virtual vendor/file/5.34/magic/Magdir/virtutech vendor/file/5.34/magic/Magdir/visx vendor/file/5.34/magic/Magdir/vms vendor/file/5.34/magic/Magdir/vmware vendor/file/5.34/magic/Magdir/vorbis vendor/file/5.34/magic/Magdir/vxl vendor/file/5.34/magic/Magdir/warc vendor/file/5.34/magic/Magdir/weak vendor/file/5.34/magic/Magdir/webassembly vendor/file/5.34/magic/Magdir/windows vendor/file/5.34/magic/Magdir/wireless vendor/file/5.34/magic/Magdir/wordprocessors vendor/file/5.34/magic/Magdir/wsdl vendor/file/5.34/magic/Magdir/x68000 vendor/file/5.34/magic/Magdir/xdelta vendor/file/5.34/magic/Magdir/xenix vendor/file/5.34/magic/Magdir/xilinx vendor/file/5.34/magic/Magdir/xo65 vendor/file/5.34/magic/Magdir/xwindows vendor/file/5.34/magic/Magdir/yara vendor/file/5.34/magic/Magdir/zfs vendor/file/5.34/magic/Magdir/zilog vendor/file/5.34/magic/Magdir/zip vendor/file/5.34/magic/Magdir/zyxel vendor/file/5.34/magic/Makefile.am (contents, props changed) vendor/file/5.34/magic/Makefile.in (contents, props changed) vendor/file/5.34/missing (contents, props changed) vendor/file/5.34/python/ vendor/file/5.34/python/CHANGELOG.md vendor/file/5.34/python/LICENSE vendor/file/5.34/python/Makefile.am (contents, props changed) vendor/file/5.34/python/Makefile.in (contents, props changed) vendor/file/5.34/python/README.md vendor/file/5.34/python/example.py (contents, props changed) vendor/file/5.34/python/magic.py (contents, props changed) vendor/file/5.34/python/setup.py (contents, props changed) vendor/file/5.34/python/tests.py (contents, props changed) vendor/file/5.34/src/ vendor/file/5.34/src/Makefile.am (contents, props changed) vendor/file/5.34/src/Makefile.in (contents, props changed) vendor/file/5.34/src/apprentice.c (contents, props changed) vendor/file/5.34/src/apptype.c (contents, props changed) vendor/file/5.34/src/ascmagic.c (contents, props changed) vendor/file/5.34/src/asctime_r.c (contents, props changed) vendor/file/5.34/src/asprintf.c (contents, props changed) vendor/file/5.34/src/buffer.c (contents, props changed) vendor/file/5.34/src/cdf.c (contents, props changed) vendor/file/5.34/src/cdf.h (contents, props changed) vendor/file/5.34/src/cdf_time.c (contents, props changed) vendor/file/5.34/src/compress.c (contents, props changed) vendor/file/5.34/src/ctime_r.c (contents, props changed) vendor/file/5.34/src/der.c (contents, props changed) vendor/file/5.34/src/der.h (contents, props changed) vendor/file/5.34/src/dprintf.c (contents, props changed) vendor/file/5.34/src/elfclass.h (contents, props changed) vendor/file/5.34/src/encoding.c (contents, props changed) vendor/file/5.34/src/file.c (contents, props changed) vendor/file/5.34/src/file.h (contents, props changed) vendor/file/5.34/src/file_opts.h (contents, props changed) vendor/file/5.34/src/fmtcheck.c (contents, props changed) vendor/file/5.34/src/fsmagic.c (contents, props changed) vendor/file/5.34/src/funcs.c (contents, props changed) vendor/file/5.34/src/getline.c (contents, props changed) vendor/file/5.34/src/getopt_long.c (contents, props changed) vendor/file/5.34/src/gmtime_r.c (contents, props changed) vendor/file/5.34/src/is_tar.c (contents, props changed) vendor/file/5.34/src/localtime_r.c (contents, props changed) vendor/file/5.34/src/magic.c (contents, props changed) vendor/file/5.34/src/magic.h.in (contents, props changed) vendor/file/5.34/src/mygetopt.h (contents, props changed) vendor/file/5.34/src/pread.c (contents, props changed) vendor/file/5.34/src/print.c (contents, props changed) vendor/file/5.34/src/readcdf.c (contents, props changed) vendor/file/5.34/src/readelf.c (contents, props changed) vendor/file/5.34/src/readelf.h (contents, props changed) vendor/file/5.34/src/seccomp.c (contents, props changed) vendor/file/5.34/src/softmagic.c (contents, props changed) vendor/file/5.34/src/strcasestr.c (contents, props changed) vendor/file/5.34/src/strlcat.c (contents, props changed) vendor/file/5.34/src/strlcpy.c (contents, props changed) vendor/file/5.34/src/tar.h (contents, props changed) vendor/file/5.34/src/vasprintf.c (contents, props changed) vendor/file/5.34/tests/ vendor/file/5.34/tests/JW07022A.mp3.result vendor/file/5.34/tests/JW07022A.mp3.testfile (contents, props changed) vendor/file/5.34/tests/Makefile.am (contents, props changed) vendor/file/5.34/tests/Makefile.in (contents, props changed) vendor/file/5.34/tests/README vendor/file/5.34/tests/escapevel.result vendor/file/5.34/tests/escapevel.testfile (contents, props changed) vendor/file/5.34/tests/gedcom.result vendor/file/5.34/tests/gedcom.testfile vendor/file/5.34/tests/hddrawcopytool.result vendor/file/5.34/tests/hddrawcopytool.testfile (contents, props changed) vendor/file/5.34/tests/issue311docx.result vendor/file/5.34/tests/issue311docx.testfile (contents, props changed) vendor/file/5.34/tests/test.c (contents, props changed) Added: vendor/file/5.34/AUTHORS ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/file/5.34/AUTHORS Sun Jul 29 05:49:48 2018 (r336860) @@ -0,0 +1 @@ +See COPYING. \ No newline at end of file Added: vendor/file/5.34/COPYING ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/file/5.34/COPYING Sun Jul 29 05:49:48 2018 (r336860) @@ -0,0 +1,29 @@ +$File: COPYING,v 1.1 2008/02/05 19:08:11 christos Exp $ +Copyright (c) Ian F. Darwin 1986, 1987, 1989, 1990, 1991, 1992, 1994, 1995. +Software written by Ian F. Darwin and others; +maintained 1994- Christos Zoulas. + +This software is not subject to any export provision of the United States +Department of Commerce, and may be exported to any country or planet. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +1. Redistributions of source code must retain the above copyright + notice immediately at the beginning of the file, without modification, + this list of conditions, and the following disclaimer. +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR +ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. Added: vendor/file/5.34/ChangeLog ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/file/5.34/ChangeLog Sun Jul 29 05:49:48 2018 (r336860) @@ -0,0 +1,1753 @@ +2018-07-25 8:50 Christos Zoulas + + * release 5.34 + +2018-06-22 16:38 Christos Zoulas + + * Add Quad indirect offsets + +2018-05-24 14:10 Christos Zoulas + + * Enable parsing of ELF dynamic sections to handle PIE better + +2018-04-15 14:52 Christos Zoulas + + * release 5.33 + +2018-02-24 14:50 Christos Zoulas + + * extend the support for ${x?:} expansions for magic descriptions + +2018-02-21 16:25 Christos Zoulas + + * add support for ${x?:} in mime types to handle + pie binaries. + +2017-11-03 9:23 Christos Zoulas + + * add support for negative offsets (offsets from the end of file) + +2017-09-26 8:22 Christos Zoulas + + * close the file on error when writing magic (Steve Grubb) + +2017-09-24 12:02 Christos Zoulas + + * seccomp support (Paul Moore) + +2017-09-02 11:53 Christos Zoulas + + * release 5.32 + +2017-08-28 16:37 Christos Zoulas + + * Always reset state in {file,buffer}_apprentice (Krzysztof Wilczynski) + +2017-08-27 03:55 Christos Zoulas + + * Fix always true condition (Thomas Jarosch) + +2017-05-24 17:30 Christos Zoulas + + * pickier parsing of numeric values in magic files. + +2017-05-23 17:55 Christos Zoulas + + * PR/615 add magic_getflags() + +2017-05-23 13:55 Christos Zoulas + + * release 5.31 + +2017-03-17 20:32 Christos Zoulas + + * remove trailing spaces from magic files + * refactor is_tar + * better bounds checks for cdf + +2017-02-10 12:24 Christos Zoulas + + * release 5.30 + +2017-02-07 23:27 Christos Zoulas + + * If we exceeded the offset in a search return no match + (Christoph Biedl) + * Be more lenient on corrupt CDF files (Christoph Biedl) + +2017-02-04 16:46 Christos Zoulas + + * pacify ubsan sign extension (oss-fuzz/524) + +2017-02-01 12:42 Christos Zoulas + + * off by one in cdf parsing (PR/593) + * report debugging sections in elf (PR/591) + +2016-11-06 10:52 Christos Zoulas + + * Allow @@@ in extensions + * Add missing overflow check in der magic (Jonas Wagner) + +2016-10-25 10:40 Christos Zoulas + + * release 5.29 + +2016-10-24 11:20 Christos Zoulas + + * der getlength overflow (Jonas Wagner) + * multiple magic file load failure (Christoph Biedl) + +2016-10-17 11:26 Christos Zoulas + + * CDF parsing improvements (Guy Helmer) + +2016-07-20 7:26 Christos Zoulas + + * Add support for signed indirect offsets + +2016-07-18 7:41 Christos Zoulas + + * cat /dev/null | file - should print empty (Christoph Biedl) + +2016-07-05 15:20 Christos Zoulas + + * Bump string size from 64 to 96. + +2016-06-13 20:20 Christos Zoulas + + * PR/556: Fix separators on annotations. + +2016-06-13 19:40 Christos Zoulas + + * release 5.28 + * fix leak on allocation failure + +2016-06-01 1:20 Christos Zoulas + + * PR/555: Avoid overflow for offset > nbytes + * PR/550: Segv on DER parsing: + - use the correct variable for length + - set offset to 0 on failure. + +2016-05-13 12:00 Christos Zoulas + + * release 5.27 + +2016-04-18 9:35 Christos Zoulas + + * Errors comparing DER entries or computing offsets + are just indications of malformed non-DER files. + Don't print them. + * Offset comparison was off-by-one. + * Fix compression code (Werner Fink) + * Put new bytes constant in the right file (not the generated one) + +2016-04-16 18:34 Christos Zoulas + + * release 5.26 + +2016-03-31 13:50 Christos Zoulas + + * make the number of bytes read from files configurable. + +2016-03-21 13:40 Christos Zoulas + + * Add bounds checks for DER code (discovered by Thomas Jarosch) + * Change indirect recursion limit to indirect use count and + bump from 15 to 50 to prevent abuse. + +2016-03-13 20:39 Christos Zoulas + + * Add -00 which prints filename\0description\0 + +2016-03-01 13:28 Christos Zoulas + + * Fix ID3 indirect parsing + +2016-01-19 10:18 Christos Zoulas + + * add DER parsing capability + +2015-11-13 10:35 Christos Zoulas + + * provide dprintf(3) for the OS's that don't have it. + +2015-11-11 16:25 Christos Zoulas + + * redo the compression code report decompression errors + +2015-11-10 23:25 Christos Zoulas + + * REG_STARTEND code is not working as expected, delete it. + +2015-11-09 16:05 Christos Zoulas + + * Add zlib support if we have it. + +2015-11-05 11:22 Christos Zoulas + + * PR/492: compression forking was broken with magic_buffer. + +2015-09-16 9:50 Christos Zoulas + + * release 5.25 + +2015-09-11 13:25 Christos Zoulas + + * add a limit to the length of regex searches + +2015-09-08 9:50 Christos Zoulas + + * fix problems with --parameter (Christoph Biedl) + +2015-07-11 10:35 Christos Zoulas + + * Windows fixes PR/466 (Jason Hood) + +2015-07-09 10:35 Christos Zoulas + + * release 5.24 + +2015-06-11 8:52 Christos Zoulas + + * redo long option encoding to fix off-by-one in 5.23 + +2015-06-10 13:50 Christos Zoulas + + * release 5.23 + +2015-06-09 16:10 Christos Zoulas + + * Fix issue with regex range for magic with offset + * Always return true from mget with USE (success to mget not match + indication). Fixes mime evaluation after USE magic + * PR/459: Don't insert magic entries to the list if there are parsing + errors for them. + +2015-06-03 16:00 Christos Zoulas + + * PR/455: Add utf-7 encoding + +2015-06-03 14:30 Christos Zoulas + + * PR/455: Implement -Z, look inside, but don't report on compression + * PR/454: Fix allocation error on bad magic. + +2015-05-29 10:30 Christos Zoulas + + * handle MAGIC_CONTINUE everywhere, not just in softmagic + +2015-05-21 14:30 Christos Zoulas + + * don't print descriptions for NAME types when mime. + +2015-04-09 15:59 Christos Zoulas + + * Add --extension to list the known extensions for this file type + Idea by Andrew J Roazen + +2015-02-14 12:23 Christos Zoulas + + * Bump file search buffer size to 1M. + +2015-01-09 14:35 Christos Zoulas + + * Fix multiple issues with date formats reported by Christoph Biedl: + - T_LOCAL meaning was reversed + - Arithmetic did not work + Also stop adjusting daylight savings for gmt printing. + +2015-01-05 13:00 Christos Zoulas + + * PR/411: Fix memory corruption from corrupt cdf file. + +2015-01-02 15:15 Christos Zoulas + + * release 5.22 + +2015-01-01 12:01 Christos Zoulas + + * add indirect relative for TIFF/Exif + +2014-12-16 18:10 Christos Zoulas + + * restructure elf note printing to avoid repeated messages + * add note limit, suggested by Alexander Cherepanov + +2014-12-16 16:53 Christos Zoulas + + * Bail out on partial pread()'s (Alexander Cherepanov) + * Fix incorrect bounds check in file_printable (Alexander Cherepanov) + +2014-12-11 20:01 Christos Zoulas + + * PR/405: ignore SIGPIPE from uncompress programs + * change printable -> file_printable and use it in + more places for safety + * in ELF, instead of "(uses dynamic libraries)" when PT_INTERP + is present print the interpreter name. + +2014-12-10 20:01 Christos Zoulas + + * release 5.21 + +2014-11-27 18:40 Christos Zoulas + + * Allow setting more parameters from the command line. + * Split name/use and indirect magic recursion limits. + +2014-11-27 11:12 Christos Zoulas + + * Adjust ELF parameters and the default recursion + level. + * Allow setting the recursion level dynamically. + +2014-11-24 8:55 Christos Zoulas + + * The following fixes resulted from Thomas Jarosch's fuzzing + tests that revealed severe performance issues on pathological + input: + - limit number of elf program and sections processing + - abort elf note processing quickly + - reduce the number of recursion levels from 20 to 10 + - preserve error messages in indirect magic handling + + This is tracked as CVE-2014-8116 and CVE-2014-8117 + +2014-11-12 10:30 Christos Zoulas + + * fix bogus free in the user buffer case. + +2014-11-11 12:35 Christos Zoulas + + * fix out of bounds read for pascal strings + * fix memory leak (not freeing the head of each mlist) + +2014-11-07 10:25 Christos Zoulas + + * When printing strings from a file, convert them to printable + on a byte by byte basis, so that we don't get issues with + locale's trying to interpret random byte streams as UTF-8 and + having printf error out with EILSEQ. + +2014-10-17 11:48 Christos Zoulas + + * fix bounds in note reading (Francisco Alonso / Red Hat) + +2014-10-11 15:02 Christos Zoulas + + * fix autoconf glue for setlocale and locale_t; some OS's + have locale_t in xlocale.h + +2014-10-10 15:01 Christos Zoulas + + * release 5.20 + +2014-08-17 10:01 Christos Zoulas + + * recognize encrypted CDF documents + +2014-08-04 9:18 Christos Zoulas + + * add magic_load_buffers from Brooks Davis + +2014-07-24 16:40 Christos Zoulas + + * add thumbs.db support + +2014-06-12 12:28 Christos Zoulas + + * release 5.19 + +2014-06-09 9:04 Christos Zoulas + + * Misc buffer overruns and missing buffer size tests in cdf parsing + (Francisco Alonso, Jan Kaluza) + +2014-06-02 14:50 Christos Zoulas + + * Enforce limit of 8K on regex searches that have no limits + * Allow the l modifier for regex to mean line count. Default + to byte count. If line count is specified, assume a max + of 80 characters per line to limit the byte count. + * Don't allow conversions to be used for dates, allowing + the mask field to be used as an offset. + +2014-05-30 12:51 Christos Zoulas + + * Make the range operator limit the length of the + regex search. + +2014-05-14 19:23 Christos Zoulas + + * PR/347: Windows fixes + * PR/352: Hangul word processor recognition + * PR/354: Encoding irregularities in text files + +2014-05-06 6:12 Christos Zoulas + + * Fix uninitialized title in CDF files (Jan Kaluza) + +2014-05-04 14:55 Christos Zoulas + + * PR/351: Fix compilation of empty files + +2014-04-30 17:39 Christos Zoulas + + * Fix integer formats: We don't specify 'l' or + 'h' and 'hh' specifiers anymore, only 'll' for + quads and nothing for the rest. This is so that + magic writing is simpler. + +2014-04-01 15:25 Christos Zoulas + + * PR/341: Jan Kaluza, fix memory leak + * PR/342: Jan Kaluza, fix out of bounds read + +2014-03-28 15:25 Christos Zoulas + + * Fix issue with long formats not matching fmtcheck + +2014-03-26 11:25 Christos Zoulas + + * release 5.18 + +2014-03-15 17:45 Christos Zoulas + + * add fmtcheck(3) for those who don't have it + +2014-03-14 15:12 Christos Zoulas + + * prevent mime entries from being attached to magic + entries with no descriptions + + * adjust magic strength for regex type + + * remove superfluous ascmagic with encoding test + +2014-03-06 12:01 Christos Zoulas + + * fix regression fix echo -ne "\012\013\014" | file -i - + which printed "binary" instead of "application/octet-stream" + + * add size_t overflow check for magic file size + +2014-02-27 16:01 Christos Zoulas + + * experimental support for matching with CFD CLSID + +2014-02-18 13:04 Kimmo Suominen (kimmo@suominen.com) + + * Cache old LC_CTYPE locale before setting it to "C", so + we can use it to restore LC_CTYPE instead of asking + setlocale() to scan the environment variables. + +2014-02-12 18:21 Christos Zoulas + + * Count recursion levels through indirect magic + +2014-02-11 10:40 Christos Zoulas + + * Prevent infinite recursion on files with indirect offsets of 0 + +2014-01-30 21:00 Christos Zoulas + + * Add -E flag that makes file print filesystem errors to stderr + and exit. + +2014-01-08 17:20 Christos Zoulas + + * mime printing could print results from multiple magic entries + if there were multiple matches. + * in some cases overflow was not detected when computing offsets + in softmagic. + +2013-12-05 12:00 Christos Zoulas + + * use strcasestr() to for cdf strings + * reset to the "C" locale while doing regex operations, or case + insensitive comparisons; this is provisional + +2013-11-19 20:10 Christos Zoulas + + * always leave magic file loaded, don't unload for magic_check, etc. + * fix default encoding to binary instead of unknown which broke recently + * handle empty and one byte files, less specially so that + --mime-encoding does not break completely. + ` +2013-11-06 14:40 Christos Zoulas + + * fix erroneous non-zero exit code from non-existent file and message + +2013-10-29 14:25 Christos Zoulas + + * add CDF MSI file detection (Guy Helmer) + +2013-09-03 11:56 Christos Zoulas + + * Don't mix errors and regular output if there was an error + * in magic_descriptor() don't close the file and try to restore + its position + +2013-05-30 17:25 Christos Zoulas + + * Don't treat magic as an error if offset was past EOF (Christoph Biedl) + +2013-05-28 17:25 Christos Zoulas + + * Fix spacing issues in softmagic and elf (Jan Kaluza) + +2013-05-02 18:00 Christos Zoulas + + * Fix segmentation fault with multiple magic_load commands. + +2013-04-22 11:20 Christos Zoulas + + * The way "default" was implemented was not very useful + because the "if something was printed at that level" + was not easily controlled by the user, and the format + was bound to a string which is too restrictive. Add + a "clear" for that level keyword and make "default" + void. This way one can do: + + >>13 clear x + >>13 lelong 1 foo + >>13 lelong 2 bar + >>13 default x + >>>13 lelong x unknown %x + +2013-03-25 13:20 Christos Zoulas + + * disallow strength setting in "name" entries + +2013-03-06 21:24 Christos Zoulas + + * fix recursive magic separator printing + +2013-02-26 19:28 Christos Zoulas + + * limit recursion level for mget + * fix pread() related breakage in cdf + * handle offsets properly in recursive "use" + +2013-02-18 10:39 Christos Zoulas + + * add elf reading of debug info to determine if file is stripped + (Jan Kaluza) + * use pread() + +2013-01-25 18:05 Christos Zoulas + + * change mime description size from 64 to 80 to accommodate OOXML. + +2013-01-11 14:50 Christos Zoulas + + * Warn about inconsistent continuation levels. + * Change fsmagic to add a space after it prints. + +2013-01-10 21:00 Christos Zoulas + + * Make getline public so that file can link against it. + Perhaps it is better to rename it, or hide it differently. + Fixes builds on platforms that do not provide it. + +2013-01-07 16:30 Christos Zoulas + + * Add SuS d{,1,2,4,8}, u{,1,2,4,8} and document + what long, int, short, etc is (Guy Harris) + +2013-01-06 11:20 Christos Zoulas + + * add magic_version function and constant + * Redo memory allocation and de-allocation. + (prevents double frees on non mmap platforms) + * Fix bug with name/use having to do with passing + found state from the parent to the child and back. + +2012-12-19 8:47 Christos Zoulas + + * Only print elf capabilities for archs we know (Jan Kaluza) + +2012-10-30 19:14 Christos Zoulas + + * Add "name" and "use" file types in order to look + inside mach-o files. + +2012-09-06 10:40 Christos Zoulas + + * make --version exit 0 (Matthew Schultz) + * add string/T (Jan Kaluza) + +2012-08-09 2:15 Christos Zoulas + + * add z and t modifiers for our own vasprintf + * search for $HOME/.magic.mgc if it is there first + * fix reads from a pipe, and preserve errno + +2012-05-15 13:12 Christos Zoulas + + * use ctime_r, asctime_r + +2012-04-06 17:18 Christos Zoulas + + * Fixes for indirect offsets to handle apple disk formats + +2012-04-03 18:26 Christos Zoulas + + * Add windows date field types + * More info for windows shortcuts (incomplete) + +2012-02-20 17:33 Christos Zoulas + + * Fix CDF parsing issues found by CERT's fuzzing tool (Will Dormann) + +2011-12-15 12:17 Chris Metcalf + + * Support Tilera architectures (tile64, tilepro, tilegx). + +2011-12-16 16:33 Reuben Thomas + + * Add magic for /usr/bin/env Perl scripts + * Weaken generic script magic to avoid clashing with + language-specific magic. + +2011-12-08 13:37 Reuben Thomas + + * Simplify if (p) free(p) to free(p). + +2011-12-08 13:07 Reuben Thomas + + * Remove hardwired token finding (names.h), turning it into soft + magic. Patterns are either anchored regexs or search/8192. English + language detection and PL/1 detection have been removed as they + were too fragile. -e tokens is still accepted for backwards + compatibility. + * Move 3ds patterns (which are commented out anyway) into autodesk + (they were, oddly, in c-lang). + +2011-12-06 00:16 Reuben Thomas + + * Tweak strength of generic hash-bang detectors to be less than + specific ones. + * Make an inconsistent description of Python scripts consistent. + +2011-12-05 23:58 Reuben Thomas + + * Fix minor error in file(1). + +2011-11-05 00:00 Reuben Thomas + + * Fix issue #150 (I hope). + +2011-09-22 12:57 Christos Zoulas + + * Python3 binding fixes from Kelly Anderson + +2011-09-20 11:32 Christos Zoulas + + * If a string type magic entry is marked as text or binary + only match text files against text entries and binary + files against binary entries. + +2011-09-01 12:12 Christos Zoulas + + * Don't wait for any subprocess, just the one we forked. + +2011-08-26 16:40 Christos Zoulas + + * If the application name is not set in a cdf file, try to see + if it has a directory with the application name on it. + +2011-08-17 14:32 Christos Zoulas + + * Fix ELF lseek(2) madness. Inspired by PR/134 by Jan Kaluza + +2011-08-14 09:03 Christos Zoulas + + * Don't use variable string formats. + +2011-07-12 12:32 Reuben Thomas + + * Fix detection of Zip files (Mantis #128). + * Make some minor improvements to file(1). + * Rename MIME types for filesystem objects for consistency with + xdg-utils. Typically this means that application/x-foo becomes + inode/foo, but some names also change slightly, e.g. + application/x-character-device becomes inode/chardevice. + +2011-05-10 20:57 Christos Zoulas + + * fix mingw compilation (Abradoks) + +2011-05-10 20:57 Christos Zoulas + + * remove patchlevel.h + * Fix read past allocated memory caused by double-incrementing + a pointer in a loop (reported by Roberto Maar) + +2011-03-30 15:45 Christos Zoulas + + * Fix cdf string buffer setting (Sven Anders) + +2011-03-20 16:35 Christos Zoulas + + * Eliminate MAXPATHLEN and use dynamic allocation for + path and file buffers. + +2011-03-15 18:15 Christos Zoulas + + * binary tests on magic entries with masks could spuriously + get converted to ascii. + +2011-03-12 18:06 Reuben Thomas + + * Improve file.man (remove BUGS, present email addresses consistently). + +2011-03-07 19:38 Christos Zoulas + + * add lrzip support (from Ville Skytta) + +2011-02-10 16:36 Christos Zoulas + + * fix CDF bounds checking (Guy Helmer) + +2011-02-10 12:03 Christos Zoulas + + * add cdf_ctime() that prints a meaningful error when time cannot + be converted. + +2011-02-02 20:40 Christos Zoulas + + * help and version output to stdout. + + * When matching softmagic for ascii files, don't just print + the softmagic classification, keep going and print the + text classification too. This fixes broken troff files when + we moved them from keyword recognition to softmagic + (they stopped printing "with CRLF" etc.) + Reported by Doug McIlroy. + +2011-01-16 19:31 Reuben Thomas + + * Fix two potential buffer overruns in apprentice_list. + +2011-01-14 22:33 Reuben Thomas + + * New Python binding in pure Python. + * Update libmagic(3). + +2011-01-06 21:40 Reuben Thomas + + * Fix Python bindings (including recent Python 3 compatibility + update). + +2011-01-04 18:43 Reuben Thomas + + * magic/Makefile.am: make it easier to recover from magic build failures. + * Fix pstring length specifier parsing to avoid generating invalid + magic files. + * Add pstring length "J" (for "JPEG") to specify that the length + include itself. + * Fix JPEG comment parsing at last using pstring/HJ! + * Ignore section 5 man pages in doc/.cvsignore. + +2010-12-22 13:12 Christos Zoulas + + * Add pstring/BHhLl to specify the type of the length of pascal + strings. + +2010-11-26 18:39 Reuben Thomas + + * Fix "-e soft": it was ignored when softmagic was called + during asciimagic. + * Improve comments and use "unsigned char" in tar.h/is_tar.c. + +2010-11-05 17:26 Reuben Thomas + + * Make bug reporting addresses more visible. + +2010-11-01 18:35 Reuben Thomas + + * Add tcl magic from Gustaf Neumann + +2010-10-24 10:42 Christos Zoulas + + * Fix the whitespace comparing code (Christopher Chittleborough) + +2010-10-06 21:05 Christos Zoulas + + * allow string/t to work (Jan Kaluza) + +2010-09-20 22:11 Reuben Thomas + + * Apply some patches from Ubuntu and Fedora. + +2010-09-20 21:16 Reuben Thomas + + * Apply all patches from Debian package 5.04-6 which have not + already been applied and are not Debian-specific. + +2010-09-20 15:24 Reuben Thomas + + * Minor security fix to softmagic.c (don't use untrusted + string as printf format). + +2010-07-21 12:20 Christos Zoulas + + * MINGW32 portability from LRN + + * Don't warn about escaping magic regex chars when we are in a regex. + +2010-07-19 10:55 Christos Zoulas + + * Only try to print prpsinfo for core files. (Jan Kaluza) + +2010-04-22 12:55 Christos Zoulas + + * Try more elf offsets for Debian core files. (Arnaud Giersch) + +2010-02-20 15:18 Reuben Thomas + + * Clarify which sort of CDF we mean. + +2010-02-14 22:58 Reuben Thomas + + * Re-jig Zip file type magic so that unsupported special + Zip types (those with "mimetype" at offset 30) can be + recognized. + +2010-02-02 21:50 Reuben Thomas + + * Add support for OCF (EPUB) files (application/epub+zip) + +2010-01-28 18:25 Christos Zoulas + + * Fix core-dump from unbound loop: + https://bugzilla.redhat.com/show_bug.cgi?id=533245 + +2010-01-22 15:45 Christos Zoulas + + * print proper mime for crystal reports file + + * print the last summary information of a cdf document, not the + first so that nested documents print the right info + +2010-01-16 18:42 Charles Longeau + + * bring back some fixes from OpenBSD: + - make gcc2 builds file + - fix typos in a magic file comment + +2009-11-17 18:35 Christos Zoulas + + * ctime/asctime can return NULL on some OS's although + they should not (Toshit Antani) + +2009-09-14 13:49 Christos Zoulas + + * Centralize magic path handling routines and remove the + special-casing from file.c so that the python module for + example comes up with the same magic path (Fixes ~/.magic + handling) (from Gab) + +2009-09-11 23:38 Reuben Thomas + + * When magic argument is a directory, read the files in + strcmp-sorted order (fixes Debian bug #488562 and our own FIXME). + +2009-09-11 13:11 Reuben Thomas + + * Combine overlapping epoc and psion magic files into one (epoc). + + * Add some more EPOC MIME types. + +2009-08-19 15:55 Christos Zoulas + + * Fix 3 bugs (From Ian Darwin): + - file_showstr could move one past the end of the array + - parse_apple did not nul terminate the string in the overflow case + - parse_mime truncated the wrong string in the overflow case + +2009-08-12 12:28 Robert Byrnes + + * Include Localstuff when compiling magic. + +2009-07-15 10:05 Christos Zoulas + + * Fix logic for including mygetopts.h + + * Make cdf.c compile again with debugging + + * Add the necessary field handling for crystal reports files to work + +2009-06-23 01:34 Reuben Thomas + + * Stop "(if" identifying Lisp files, that's plain dumb! + +2009-06-09 22:13 Reuben Thomas + + * Add a couple of missing MP3 MIME types. + +2009-05-27 23:00 Reuben Thomas + + * Add full range of hash-bang tests for Python and Ruby. + + * Add MIME types for Python and Ruby scripts. + +2009-05-13 10:44 Christos Zoulas + + * off by one in parsing hw capabilities in elf + (Cheng Renquan) + +2009-05-08 13:40 Christos Zoulas + + * lint fixes and more from NetBSD + +2009-05-06 10:25 Christos Zoulas + + * Avoid null dereference in cdf code (Drew Yao) + + * More cdf bounds checks and overflow checks + +2009-05-01 18:37 Christos Zoulas + + * Buffer overflow fixes from Drew Yao + +2009-04-30 17:10 Christos Zoulas + + * Fix more cdf lossage. All the documents I have + right now print the correct information. + +2009-03-27 18:43 Christos Zoulas + + * don't print \012- separators in the same magic entry + if it consists of multiple magic printing lines. + +2009-03-23 10:20 Christos Zoulas + + * Avoid file descriptor leak in compress code from + (Daniel Novotny) + +2009-03-18 16:50 Christos Zoulas + + * Allow escaping of relation characters, so that we can say \^[A-Z] + and the ^ is not eaten as a relation char. + + * Fix troff and fortran to their previous glory using + regex. This was broken since their removel from ascmagic. + +2009-03-10 16:50 Christos Zoulas + + * don't use strlen in strndup() (Toby Peterson) + +2009-03-10 7:45 Christos Zoulas + + * avoid c99 syntax. + +2009-02-23 15:45 Christos Zoulas + + * make the cdf code use the buffer first if available, + and then the fd code. + +2009-02-13 13:45 Christos Zoulas + + * look for struct option to determine if getopt.h is usable for IRIX. + + * sanitize cdf document strings + +2009-02-04 13:25 Christos Zoulas + *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Sun Jul 29 05:51:15 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 549DE104C358; Sun, 29 Jul 2018 05:51:15 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A86B729A6; Sun, 29 Jul 2018 05:51:15 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C5C9311AF0; Sun, 29 Jul 2018 05:51:14 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T5pEfl020106; Sun, 29 Jul 2018 05:51:14 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T5pE7K020105; Sun, 29 Jul 2018 05:51:14 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290551.w6T5pE7K020105@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 05:51:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336861 - vendor/file/5.34 X-SVN-Group: vendor X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: vendor/file/5.34 X-SVN-Commit-Revision: 336861 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 05:51:15 -0000 Author: eadler Date: Sun Jul 29 05:51:14 2018 New Revision: 336861 URL: https://svnweb.freebsd.org/changeset/base/336861 Log: Remove 5.34 I inverted the order of the import. Deleted: vendor/file/5.34/ From owner-svn-src-all@freebsd.org Sun Jul 29 05:53:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DB5C104C4BF; Sun, 29 Jul 2018 05:53:51 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F11372D72; Sun, 29 Jul 2018 05:53:51 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 209B811C53; Sun, 29 Jul 2018 05:53:51 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T5rpdu024291; Sun, 29 Jul 2018 05:53:51 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T5roAQ024285; Sun, 29 Jul 2018 05:53:50 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290553.w6T5roAQ024285@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 05:53:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336862 - in vendor/file/dist: . doc magic magic/Magdir python src tests X-SVN-Group: vendor X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: in vendor/file/dist: . doc magic magic/Magdir python src tests X-SVN-Commit-Revision: 336862 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 05:53:52 -0000 Author: eadler Date: Sun Jul 29 05:53:49 2018 New Revision: 336862 URL: https://svnweb.freebsd.org/changeset/base/336862 Log: vendor/file: add 5.34 gpg: Good signature from "Christos Zoulas (personal key) " [unknown] gpg: aka "Christos Zoulas (NetBSD key) " [unknown] Obtained from: ftp://ftp.astron.com/pub/file/ Added: vendor/file/dist/magic/Magdir/dataone vendor/file/dist/python/CHANGELOG.md vendor/file/dist/python/LICENSE vendor/file/dist/python/README.md vendor/file/dist/python/tests.py (contents, props changed) vendor/file/dist/tests/JW07022A.mp3.result vendor/file/dist/tests/JW07022A.mp3.testfile (contents, props changed) Deleted: vendor/file/dist/python/README Modified: vendor/file/dist/ChangeLog vendor/file/dist/Makefile.in vendor/file/dist/README vendor/file/dist/aclocal.m4 vendor/file/dist/compile vendor/file/dist/config.guess vendor/file/dist/config.h.in vendor/file/dist/config.sub vendor/file/dist/configure vendor/file/dist/configure.ac vendor/file/dist/depcomp vendor/file/dist/doc/Makefile.in vendor/file/dist/doc/file.man vendor/file/dist/doc/magic.man vendor/file/dist/install-sh vendor/file/dist/ltmain.sh vendor/file/dist/magic/Magdir/animation vendor/file/dist/magic/Magdir/archive vendor/file/dist/magic/Magdir/audio vendor/file/dist/magic/Magdir/cad vendor/file/dist/magic/Magdir/console vendor/file/dist/magic/Magdir/database vendor/file/dist/magic/Magdir/dbpf vendor/file/dist/magic/Magdir/dump vendor/file/dist/magic/Magdir/elf vendor/file/dist/magic/Magdir/fsav vendor/file/dist/magic/Magdir/images vendor/file/dist/magic/Magdir/linux vendor/file/dist/magic/Magdir/make vendor/file/dist/magic/Magdir/measure vendor/file/dist/magic/Magdir/msdos vendor/file/dist/magic/Magdir/msooxml vendor/file/dist/magic/Magdir/pdf vendor/file/dist/magic/Magdir/sgi vendor/file/dist/magic/Makefile.am vendor/file/dist/magic/Makefile.in vendor/file/dist/missing vendor/file/dist/python/Makefile.am vendor/file/dist/python/Makefile.in vendor/file/dist/python/magic.py vendor/file/dist/python/setup.py vendor/file/dist/src/Makefile.in vendor/file/dist/src/apprentice.c vendor/file/dist/src/compress.c vendor/file/dist/src/der.c vendor/file/dist/src/file.h vendor/file/dist/src/funcs.c vendor/file/dist/src/readcdf.c vendor/file/dist/src/readelf.c vendor/file/dist/src/readelf.h vendor/file/dist/src/seccomp.c vendor/file/dist/src/softmagic.c vendor/file/dist/tests/Makefile.am vendor/file/dist/tests/Makefile.in Modified: vendor/file/dist/ChangeLog ============================================================================== --- vendor/file/dist/ChangeLog Sun Jul 29 05:51:14 2018 (r336861) +++ vendor/file/dist/ChangeLog Sun Jul 29 05:53:49 2018 (r336862) @@ -1,3 +1,15 @@ +2018-07-25 8:50 Christos Zoulas + + * release 5.34 + +2018-06-22 16:38 Christos Zoulas + + * Add Quad indirect offsets + +2018-05-24 14:10 Christos Zoulas + + * Enable parsing of ELF dynamic sections to handle PIE better + 2018-04-15 14:52 Christos Zoulas * release 5.33 Modified: vendor/file/dist/Makefile.in ============================================================================== --- vendor/file/dist/Makefile.in Sun Jul 29 05:51:14 2018 (r336861) +++ vendor/file/dist/Makefile.in Sun Jul 29 05:53:49 2018 (r336862) @@ -1,7 +1,7 @@ -# Makefile.in generated by automake 1.13.1 from Makefile.am. +# Makefile.in generated by automake 1.15 from Makefile.am. # @configure_input@ -# Copyright (C) 1994-2012 Free Software Foundation, Inc. +# Copyright (C) 1994-2014 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -14,23 +14,61 @@ @SET_MAKE@ VPATH = @srcdir@ -am__make_dryrun = \ - { \ - am__dry=no; \ +am__is_gnu_make = { \ + if test -z '$(MAKELEVEL)'; then \ + false; \ + elif test -n '$(MAKE_HOST)'; then \ + true; \ + elif test -n '$(MAKE_VERSION)' && test -n '$(CURDIR)'; then \ + true; \ + else \ + false; \ + fi; \ +} +am__make_running_with_option = \ + case $${target_option-} in \ + ?) ;; \ + *) echo "am__make_running_with_option: internal error: invalid" \ + "target option '$${target_option-}' specified" >&2; \ + exit 1;; \ + esac; \ + has_opt=no; \ + sane_makeflags=$$MAKEFLAGS; \ + if $(am__is_gnu_make); then \ + sane_makeflags=$$MFLAGS; \ + else \ case $$MAKEFLAGS in \ *\\[\ \ ]*) \ - echo 'am--echo: ; @echo "AM" OK' | $(MAKE) -f - 2>/dev/null \ - | grep '^AM OK$$' >/dev/null || am__dry=yes;; \ - *) \ - for am__flg in $$MAKEFLAGS; do \ - case $$am__flg in \ - *=*|--*) ;; \ - *n*) am__dry=yes; break;; \ - esac; \ - done;; \ + bs=\\; \ + sane_makeflags=`printf '%s\n' "$$MAKEFLAGS" \ + | sed "s/$$bs$$bs[$$bs $$bs ]*//g"`;; \ esac; \ - test $$am__dry = yes; \ - } + fi; \ + skip_next=no; \ + strip_trailopt () \ + { \ + flg=`printf '%s\n' "$$flg" | sed "s/$$1.*$$//"`; \ + }; \ + for flg in $$sane_makeflags; do \ + test $$skip_next = yes && { skip_next=no; continue; }; \ + case $$flg in \ + *=*|--*) continue;; \ + -*I) strip_trailopt 'I'; skip_next=yes;; \ + -*I?*) strip_trailopt 'I';; \ + -*O) strip_trailopt 'O'; skip_next=yes;; \ + -*O?*) strip_trailopt 'O';; \ + -*l) strip_trailopt 'l'; skip_next=yes;; \ + -*l?*) strip_trailopt 'l';; \ + -[dEDm]) skip_next=yes;; \ + -[JT]) skip_next=yes;; \ + esac; \ + case $$flg in \ + *$$target_option*) has_opt=yes; break;; \ + esac; \ + done; \ + test $$has_opt = yes +am__make_dryrun = (target_option=n; $(am__make_running_with_option)) +am__make_keepgoing = (target_option=k; $(am__make_running_with_option)) pkgincludedir = $(includedir)/@PACKAGE@ pkglibdir = $(libdir)/@PACKAGE@ pkglibexecdir = $(libexecdir)/@PACKAGE@ @@ -49,11 +87,6 @@ POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ subdir = . -DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am \ - $(top_srcdir)/configure $(am__configure_deps) \ - $(srcdir)/config.h.in AUTHORS COPYING ChangeLog INSTALL NEWS \ - README TODO compile config.guess config.sub depcomp install-sh \ - missing ltmain.sh ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \ $(top_srcdir)/m4/ltoptions.m4 $(top_srcdir)/m4/ltsugar.m4 \ @@ -61,6 +94,8 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \ $(top_srcdir)/acinclude.m4 $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ $(ACLOCAL_M4) +DIST_COMMON = $(srcdir)/Makefile.am $(top_srcdir)/configure \ + $(am__configure_deps) $(am__DIST_COMMON) am__CONFIG_DISTCLEAN_FILES = config.status config.cache config.log \ configure.lineno config.status.lineno mkinstalldirs = $(install_sh) -d @@ -124,6 +159,9 @@ ETAGS = etags CTAGS = ctags CSCOPE = cscope DIST_SUBDIRS = $(SUBDIRS) +am__DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/config.h.in AUTHORS \ + COPYING ChangeLog INSTALL NEWS README TODO compile \ + config.guess config.sub depcomp install-sh ltmain.sh missing DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) distdir = $(PACKAGE)-$(VERSION) top_distdir = $(distdir) @@ -178,6 +216,7 @@ AWK = @AWK@ CC = @CC@ CCDEPMODE = @CCDEPMODE@ CFLAGS = @CFLAGS@ +CFLAG_VISIBILITY = @CFLAG_VISIBILITY@ CPP = @CPP@ CPPFLAGS = @CPPFLAGS@ CYGPATH_W = @CYGPATH_W@ @@ -193,6 +232,7 @@ EGREP = @EGREP@ EXEEXT = @EXEEXT@ FGREP = @FGREP@ GREP = @GREP@ +HAVE_VISIBILITY = @HAVE_VISIBILITY@ INSTALL = @INSTALL@ INSTALL_DATA = @INSTALL_DATA@ INSTALL_PROGRAM = @INSTALL_PROGRAM@ @@ -306,7 +346,6 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__c echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign Makefile'; \ $(am__cd) $(top_srcdir) && \ $(AUTOMAKE) --foreign Makefile -.PRECIOUS: Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ @@ -327,8 +366,8 @@ $(ACLOCAL_M4): $(am__aclocal_m4_deps) $(am__aclocal_m4_deps): config.h: stamp-h1 - @if test ! -f $@; then rm -f stamp-h1; else :; fi - @if test ! -f $@; then $(MAKE) $(AM_MAKEFLAGS) stamp-h1; else :; fi + @test -f $@ || rm -f stamp-h1 + @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) stamp-h1 stamp-h1: $(srcdir)/config.h.in $(top_builddir)/config.status @rm -f stamp-h1 @@ -357,13 +396,12 @@ distclean-libtool: # (which will cause the Makefiles to be regenerated when you run 'make'); # (2) otherwise, pass the desired values on the 'make' command line. $(am__recursive_targets): - @fail= failcom='exit 1'; \ - for f in x $$MAKEFLAGS; do \ - case $$f in \ - *=* | --[!k]*);; \ - *k*) failcom='fail=yes';; \ - esac; \ - done; \ + @fail=; \ + if $(am__make_keepgoing); then \ + failcom='fail=yes'; \ + else \ + failcom='exit 1'; \ + fi; \ dot_seen=no; \ target=`echo $@ | sed s/-recursive//`; \ case "$@" in \ @@ -538,10 +576,16 @@ dist-xz: distdir $(am__post_remove_distdir) dist-tarZ: distdir + @echo WARNING: "Support for distribution archives compressed with" \ + "legacy program 'compress' is deprecated." >&2 + @echo WARNING: "It will be removed altogether in Automake 2.0" >&2 tardir=$(distdir) && $(am__tar) | compress -c >$(distdir).tar.Z $(am__post_remove_distdir) dist-shar: distdir + @echo WARNING: "Support for shar distribution archives is" \ + "deprecated." >&2 + @echo WARNING: "It will be removed altogether in Automake 2.0" >&2 shar $(distdir) | GZIP=$(GZIP_ENV) gzip -c >$(distdir).shar.gz $(am__post_remove_distdir) @@ -576,16 +620,17 @@ distcheck: dist esac chmod -R a-w $(distdir) chmod u+w $(distdir) - mkdir $(distdir)/_build $(distdir)/_inst + mkdir $(distdir)/_build $(distdir)/_build/sub $(distdir)/_inst chmod a-w $(distdir) test -d $(distdir)/_build || exit 0; \ dc_install_base=`$(am__cd) $(distdir)/_inst && pwd | sed -e 's,^[^:\\/]:[\\/],/,'` \ && dc_destdir="$${TMPDIR-/tmp}/am-dc-$$$$/" \ && am__cwd=`pwd` \ - && $(am__cd) $(distdir)/_build \ - && ../configure --srcdir=.. --prefix="$$dc_install_base" \ + && $(am__cd) $(distdir)/_build/sub \ + && ../../configure \ $(AM_DISTCHECK_CONFIGURE_FLAGS) \ $(DISTCHECK_CONFIGURE_FLAGS) \ + --srcdir=../.. --prefix="$$dc_install_base" \ && $(MAKE) $(AM_MAKEFLAGS) \ && $(MAKE) $(AM_MAKEFLAGS) dvi \ && $(MAKE) $(AM_MAKEFLAGS) check \ @@ -761,6 +806,8 @@ uninstall-am: maintainer-clean-generic mostlyclean mostlyclean-generic \ mostlyclean-libtool pdf pdf-am ps ps-am tags tags-am uninstall \ uninstall-am + +.PRECIOUS: Makefile # Tell versions [3.59,3.63) of GNU make to not export all variables. Modified: vendor/file/dist/README ============================================================================== --- vendor/file/dist/README Sun Jul 29 05:51:14 2018 (r336861) +++ vendor/file/dist/README Sun Jul 29 05:53:49 2018 (r336862) @@ -1,10 +1,10 @@ ## README for file(1) Command ## - @(#) $File: README,v 1.53 2018/03/11 13:06:47 glen Exp $ + @(#) $File: README,v 1.54 2018/05/30 03:06:56 christos Exp $ -Mailing List: file@mx.gw.com [currently down] -Mailing List archives: http://mx.gw.com/pipermail/file/ [currently down] -Bug tracker: http://bugs.gw.com/ [currently down] +Mailing List: file@astron.com +Mailing List archives: http://mailman.astron.com/pipermail/file/ +Bug tracker: http://bugs.astron.com/ E-mail: christos@astron.com Build Status: https://travis-ci.org/file/file Modified: vendor/file/dist/aclocal.m4 ============================================================================== --- vendor/file/dist/aclocal.m4 Sun Jul 29 05:51:14 2018 (r336861) +++ vendor/file/dist/aclocal.m4 Sun Jul 29 05:53:49 2018 (r336862) @@ -1,6 +1,6 @@ -# generated automatically by aclocal 1.13.1 -*- Autoconf -*- +# generated automatically by aclocal 1.15 -*- Autoconf -*- -# Copyright (C) 1996-2012 Free Software Foundation, Inc. +# Copyright (C) 1996-2014 Free Software Foundation, Inc. # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -20,7 +20,85 @@ You have another version of autoconf. It may work, bu If you have problems, you may need to regenerate the build system entirely. To do so, use the procedure documented by the package, typically 'autoreconf'.])]) -# Copyright (C) 2002-2013 Free Software Foundation, Inc. +# visibility.m4 serial 5 (gettext-0.18.2) +dnl Copyright (C) 2005, 2008, 2010-2016 Free Software Foundation, Inc. +dnl This file is free software; the Free Software Foundation +dnl gives unlimited permission to copy and/or distribute it, +dnl with or without modifications, as long as this notice is preserved. + +dnl From Bruno Haible. + +dnl Tests whether the compiler supports the command-line option +dnl -fvisibility=hidden and the function and variable attributes +dnl __attribute__((__visibility__("hidden"))) and +dnl __attribute__((__visibility__("default"))). +dnl Does *not* test for __visibility__("protected") - which has tricky +dnl semantics (see the 'vismain' test in glibc) and does not exist e.g. on +dnl Mac OS X. +dnl Does *not* test for __visibility__("internal") - which has processor +dnl dependent semantics. +dnl Does *not* test for #pragma GCC visibility push(hidden) - which is +dnl "really only recommended for legacy code". +dnl Set the variable CFLAG_VISIBILITY. +dnl Defines and sets the variable HAVE_VISIBILITY. + +AC_DEFUN([gl_VISIBILITY], +[ + AC_REQUIRE([AC_PROG_CC]) + CFLAG_VISIBILITY= + HAVE_VISIBILITY=0 + if test -n "$GCC"; then + dnl First, check whether -Werror can be added to the command line, or + dnl whether it leads to an error because of some other option that the + dnl user has put into $CC $CFLAGS $CPPFLAGS. + AC_MSG_CHECKING([whether the -Werror option is usable]) + AC_CACHE_VAL([gl_cv_cc_vis_werror], [ + gl_save_CFLAGS="$CFLAGS" + CFLAGS="$CFLAGS -Werror" + AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM([[]], [[]])], + [gl_cv_cc_vis_werror=yes], + [gl_cv_cc_vis_werror=no]) + CFLAGS="$gl_save_CFLAGS"]) + AC_MSG_RESULT([$gl_cv_cc_vis_werror]) + dnl Now check whether visibility declarations are supported. + AC_MSG_CHECKING([for simple visibility declarations]) + AC_CACHE_VAL([gl_cv_cc_visibility], [ + gl_save_CFLAGS="$CFLAGS" + CFLAGS="$CFLAGS -fvisibility=hidden" + dnl We use the option -Werror and a function dummyfunc, because on some + dnl platforms (Cygwin 1.7) the use of -fvisibility triggers a warning + dnl "visibility attribute not supported in this configuration; ignored" + dnl at the first function definition in every compilation unit, and we + dnl don't want to use the option in this case. + if test $gl_cv_cc_vis_werror = yes; then + CFLAGS="$CFLAGS -Werror" + fi + AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM( + [[extern __attribute__((__visibility__("hidden"))) int hiddenvar; + extern __attribute__((__visibility__("default"))) int exportedvar; + extern __attribute__((__visibility__("hidden"))) int hiddenfunc (void); + extern __attribute__((__visibility__("default"))) int exportedfunc (void); + void dummyfunc (void) {} + ]], + [[]])], + [gl_cv_cc_visibility=yes], + [gl_cv_cc_visibility=no]) + CFLAGS="$gl_save_CFLAGS"]) + AC_MSG_RESULT([$gl_cv_cc_visibility]) + if test $gl_cv_cc_visibility = yes; then + CFLAG_VISIBILITY="-fvisibility=hidden" + HAVE_VISIBILITY=1 + fi + fi + AC_SUBST([CFLAG_VISIBILITY]) + AC_SUBST([HAVE_VISIBILITY]) + AC_DEFINE_UNQUOTED([HAVE_VISIBILITY], [$HAVE_VISIBILITY], + [Define to 1 or 0, depending whether the compiler supports simple visibility declarations.]) +]) + +# Copyright (C) 2002-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -32,10 +110,10 @@ To do so, use the procedure documented by the package, # generated from the m4 files accompanying Automake X.Y. # (This private macro should not be called outside this file.) AC_DEFUN([AM_AUTOMAKE_VERSION], -[am__api_version='1.13' +[am__api_version='1.15' dnl Some users find AM_AUTOMAKE_VERSION and mistake it for a way to dnl require some minimum version. Point them to the right macro. -m4_if([$1], [1.13.1], [], +m4_if([$1], [1.15], [], [AC_FATAL([Do not call $0, use AM_INIT_AUTOMAKE([$1]).])])dnl ]) @@ -51,14 +129,14 @@ m4_define([_AM_AUTOCONF_VERSION], []) # Call AM_AUTOMAKE_VERSION and AM_AUTOMAKE_VERSION so they can be traced. # This function is AC_REQUIREd by AM_INIT_AUTOMAKE. AC_DEFUN([AM_SET_CURRENT_AUTOMAKE_VERSION], -[AM_AUTOMAKE_VERSION([1.13.1])dnl +[AM_AUTOMAKE_VERSION([1.15])dnl m4_ifndef([AC_AUTOCONF_VERSION], [m4_copy([m4_PACKAGE_VERSION], [AC_AUTOCONF_VERSION])])dnl _AM_AUTOCONF_VERSION(m4_defn([AC_AUTOCONF_VERSION]))]) # AM_AUX_DIR_EXPAND -*- Autoconf -*- -# Copyright (C) 2001-2013 Free Software Foundation, Inc. +# Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -103,15 +181,14 @@ _AM_AUTOCONF_VERSION(m4_defn([AC_AUTOCONF_VERSION]))]) # configured tree to be moved without reconfiguration. AC_DEFUN([AM_AUX_DIR_EXPAND], -[dnl Rely on autoconf to set up CDPATH properly. -AC_PREREQ([2.50])dnl -# expand $ac_aux_dir to an absolute path -am_aux_dir=`cd $ac_aux_dir && pwd` +[AC_REQUIRE([AC_CONFIG_AUX_DIR_DEFAULT])dnl +# Expand $ac_aux_dir to an absolute path. +am_aux_dir=`cd "$ac_aux_dir" && pwd` ]) # AM_CONDITIONAL -*- Autoconf -*- -# Copyright (C) 1997-2013 Free Software Foundation, Inc. +# Copyright (C) 1997-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -142,7 +219,7 @@ AC_CONFIG_COMMANDS_PRE( Usually this means the macro was only invoked conditionally.]]) fi])]) -# Copyright (C) 1999-2013 Free Software Foundation, Inc. +# Copyright (C) 1999-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -333,7 +410,7 @@ _AM_SUBST_NOTMAKE([am__nodep])dnl # Generate code to set up dependency tracking. -*- Autoconf -*- -# Copyright (C) 1999-2013 Free Software Foundation, Inc. +# Copyright (C) 1999-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -373,7 +450,7 @@ AC_DEFUN([_AM_OUTPUT_DEPENDENCY_COMMANDS], DEPDIR=`sed -n 's/^DEPDIR = //p' < "$mf"` test -z "$DEPDIR" && continue am__include=`sed -n 's/^am__include = //p' < "$mf"` - test -z "am__include" && continue + test -z "$am__include" && continue am__quote=`sed -n 's/^am__quote = //p' < "$mf"` # Find all dependency output files, they are included files with # $(DEPDIR) in their names. We invoke sed twice because it is the @@ -409,7 +486,7 @@ AC_DEFUN([AM_OUTPUT_DEPENDENCY_COMMANDS], # Do all the work for Automake. -*- Autoconf -*- -# Copyright (C) 1996-2013 Free Software Foundation, Inc. +# Copyright (C) 1996-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -418,6 +495,12 @@ AC_DEFUN([AM_OUTPUT_DEPENDENCY_COMMANDS], # This macro actually does too much. Some checks are only needed if # your package does certain things. But this isn't really a big deal. +dnl Redefine AC_PROG_CC to automatically invoke _AM_PROG_CC_C_O. +m4_define([AC_PROG_CC], +m4_defn([AC_PROG_CC]) +[_AM_PROG_CC_C_O +]) + # AM_INIT_AUTOMAKE(PACKAGE, VERSION, [NO-DEFINE]) # AM_INIT_AUTOMAKE([OPTIONS]) # ----------------------------------------------- @@ -493,8 +576,8 @@ AC_REQUIRE([AC_PROG_MKDIR_P])dnl # # AC_SUBST([mkdir_p], ['$(MKDIR_P)']) -# We need awk for the "check" target. The system "awk" is bad on -# some platforms. +# We need awk for the "check" target (and possibly the TAP driver). The +# system "awk" is bad on some platforms. AC_REQUIRE([AC_PROG_AWK])dnl AC_REQUIRE([AC_PROG_MAKE_SET])dnl AC_REQUIRE([AM_SET_LEADING_DOT])dnl @@ -526,6 +609,51 @@ dnl macro is hooked onto _AC_COMPILER_EXEEXT early, se AC_CONFIG_COMMANDS_PRE(dnl [m4_provide_if([_AM_COMPILER_EXEEXT], [AM_CONDITIONAL([am__EXEEXT], [test -n "$EXEEXT"])])])dnl + +# POSIX will say in a future version that running "rm -f" with no argument +# is OK; and we want to be able to make that assumption in our Makefile +# recipes. So use an aggressive probe to check that the usage we want is +# actually supported "in the wild" to an acceptable degree. +# See automake bug#10828. +# To make any issue more visible, cause the running configure to be aborted +# by default if the 'rm' program in use doesn't match our expectations; the +# user can still override this though. +if rm -f && rm -fr && rm -rf; then : OK; else + cat >&2 <<'END' +Oops! + +Your 'rm' program seems unable to run without file operands specified +on the command line, even when the '-f' option is present. This is contrary +to the behaviour of most rm programs out there, and not conforming with +the upcoming POSIX standard: + +Please tell bug-automake@gnu.org about your system, including the value +of your $PATH and any error possibly output before this message. This +can help us improve future automake versions. + +END + if test x"$ACCEPT_INFERIOR_RM_PROGRAM" = x"yes"; then + echo 'Configuration will proceed anyway, since you have set the' >&2 + echo 'ACCEPT_INFERIOR_RM_PROGRAM variable to "yes"' >&2 + echo >&2 + else + cat >&2 <<'END' +Aborting the configuration process, to ensure you take notice of the issue. + +You can download and install GNU coreutils to get an 'rm' implementation +that behaves properly: . + +If you want to complete the configuration process using your problematic +'rm' anyway, export the environment variable ACCEPT_INFERIOR_RM_PROGRAM +to "yes", and re-run configure. + +END + AC_MSG_ERROR([Your 'rm' program is bad, sorry.]) + fi +fi +dnl The trailing newline in this macro's definition is deliberate, for +dnl backward compatibility and to allow trailing 'dnl'-style comments +dnl after the AM_INIT_AUTOMAKE invocation. See automake bug#16841. ]) dnl Hook into '_AC_COMPILER_EXEEXT' early to learn its expansion. Do not @@ -534,7 +662,6 @@ dnl mangled by Autoconf and run in a shell conditional m4_define([_AC_COMPILER_EXEEXT], m4_defn([_AC_COMPILER_EXEEXT])[m4_provide([_AM_COMPILER_EXEEXT])]) - # When config.status generates a header, we must update the stamp-h file. # This file resides in the same directory as the config header # that is generated. The stamp files are numbered to have different names. @@ -556,7 +683,7 @@ for _am_header in $config_headers :; do done echo "timestamp for $_am_arg" >`AS_DIRNAME(["$_am_arg"])`/stamp-h[]$_am_stamp_count]) -# Copyright (C) 2001-2013 Free Software Foundation, Inc. +# Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -567,7 +694,7 @@ echo "timestamp for $_am_arg" >`AS_DIRNAME(["$_am_arg" # Define $install_sh. AC_DEFUN([AM_PROG_INSTALL_SH], [AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl -if test x"${install_sh}" != xset; then +if test x"${install_sh+set}" != xset; then case $am_aux_dir in *\ * | *\ *) install_sh="\${SHELL} '$am_aux_dir/install-sh'" ;; @@ -577,7 +704,7 @@ if test x"${install_sh}" != xset; then fi AC_SUBST([install_sh])]) -# Copyright (C) 2003-2013 Free Software Foundation, Inc. +# Copyright (C) 2003-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -598,7 +725,7 @@ AC_SUBST([am__leading_dot])]) # Check to see how 'make' treats includes. -*- Autoconf -*- -# Copyright (C) 2001-2013 Free Software Foundation, Inc. +# Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -646,41 +773,9 @@ AC_MSG_RESULT([$_am_result]) rm -f confinc confmf ]) -# Copyright (C) 1999-2013 Free Software Foundation, Inc. -# -# This file is free software; the Free Software Foundation -# gives unlimited permission to copy and/or distribute it, -# with or without modifications, as long as this notice is preserved. - -# AM_PROG_CC_C_O -# -------------- -# Like AC_PROG_CC_C_O, but changed for automake. -AC_DEFUN([AM_PROG_CC_C_O], -[AC_REQUIRE([AC_PROG_CC_C_O])dnl -AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl -AC_REQUIRE_AUX_FILE([compile])dnl -# FIXME: we rely on the cache variable name because -# there is no other way. -set dummy $CC -am_cc=`echo $[2] | sed ['s/[^a-zA-Z0-9_]/_/g;s/^[0-9]/_/']` -eval am_t=\$ac_cv_prog_cc_${am_cc}_c_o -if test "$am_t" != yes; then - # Losing compiler, so override with the script. - # FIXME: It is wrong to rewrite CC. - # But if we don't then we get into trouble of one sort or another. - # A longer-term fix would be to have automake use am__CC in this case, - # and then we could set am__CC="\$(top_srcdir)/compile \$(CC)" - CC="$am_aux_dir/compile $CC" -fi -dnl Make sure AC_PROG_CC is never called again, or it will override our -dnl setting of CC. -m4_define([AC_PROG_CC], - [m4_fatal([AC_PROG_CC cannot be called after AM_PROG_CC_C_O])]) -]) - # Fake the existence of programs that GNU maintainers use. -*- Autoconf -*- -# Copyright (C) 1997-2013 Free Software Foundation, Inc. +# Copyright (C) 1997-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -719,7 +814,7 @@ fi # Helper functions for option handling. -*- Autoconf -*- -# Copyright (C) 2001-2013 Free Software Foundation, Inc. +# Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -748,9 +843,73 @@ AC_DEFUN([_AM_SET_OPTIONS], AC_DEFUN([_AM_IF_OPTION], [m4_ifset(_AM_MANGLE_OPTION([$1]), [$2], [$3])]) +# Copyright (C) 1999-2014 Free Software Foundation, Inc. +# +# This file is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# _AM_PROG_CC_C_O +# --------------- +# Like AC_PROG_CC_C_O, but changed for automake. We rewrite AC_PROG_CC +# to automatically call this. +AC_DEFUN([_AM_PROG_CC_C_O], +[AC_REQUIRE([AM_AUX_DIR_EXPAND])dnl +AC_REQUIRE_AUX_FILE([compile])dnl +AC_LANG_PUSH([C])dnl +AC_CACHE_CHECK( + [whether $CC understands -c and -o together], + [am_cv_prog_cc_c_o], + [AC_LANG_CONFTEST([AC_LANG_PROGRAM([])]) + # Make sure it works both with $CC and with simple cc. + # Following AC_PROG_CC_C_O, we do the test twice because some + # compilers refuse to overwrite an existing .o file with -o, + # though they will create one. + am_cv_prog_cc_c_o=yes + for am_i in 1 2; do + if AM_RUN_LOG([$CC -c conftest.$ac_ext -o conftest2.$ac_objext]) \ + && test -f conftest2.$ac_objext; then + : OK + else + am_cv_prog_cc_c_o=no + break + fi + done + rm -f core conftest* + unset am_i]) +if test "$am_cv_prog_cc_c_o" != yes; then + # Losing compiler, so override with the script. + # FIXME: It is wrong to rewrite CC. + # But if we don't then we get into trouble of one sort or another. + # A longer-term fix would be to have automake use am__CC in this case, + # and then we could set am__CC="\$(top_srcdir)/compile \$(CC)" + CC="$am_aux_dir/compile $CC" +fi +AC_LANG_POP([C])]) + +# For backward compatibility. +AC_DEFUN_ONCE([AM_PROG_CC_C_O], [AC_REQUIRE([AC_PROG_CC])]) + +# Copyright (C) 2001-2014 Free Software Foundation, Inc. +# +# This file is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# AM_RUN_LOG(COMMAND) +# ------------------- +# Run COMMAND, save the exit status in ac_status, and log it. +# (This has been adapted from Autoconf's _AC_RUN_LOG macro.) +AC_DEFUN([AM_RUN_LOG], +[{ echo "$as_me:$LINENO: $1" >&AS_MESSAGE_LOG_FD + ($1) >&AS_MESSAGE_LOG_FD 2>&AS_MESSAGE_LOG_FD + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&AS_MESSAGE_LOG_FD + (exit $ac_status); }]) + # Check to make sure that the build environment is sane. -*- Autoconf -*- -# Copyright (C) 1996-2013 Free Software Foundation, Inc. +# Copyright (C) 1996-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -831,7 +990,7 @@ AC_CONFIG_COMMANDS_PRE( rm -f conftest.file ]) -# Copyright (C) 2009-2013 Free Software Foundation, Inc. +# Copyright (C) 2009-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -891,7 +1050,7 @@ AC_SUBST([AM_BACKSLASH])dnl _AM_SUBST_NOTMAKE([AM_BACKSLASH])dnl ]) -# Copyright (C) 2001-2013 Free Software Foundation, Inc. +# Copyright (C) 2001-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -919,7 +1078,7 @@ fi INSTALL_STRIP_PROGRAM="\$(install_sh) -c -s" AC_SUBST([INSTALL_STRIP_PROGRAM])]) -# Copyright (C) 2006-2013 Free Software Foundation, Inc. +# Copyright (C) 2006-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -938,7 +1097,7 @@ AC_DEFUN([AM_SUBST_NOTMAKE], [_AM_SUBST_NOTMAKE($@)]) # Check how to create a tarball. -*- Autoconf -*- -# Copyright (C) 2004-2013 Free Software Foundation, Inc. +# Copyright (C) 2004-2014 Free Software Foundation, Inc. # # This file is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -957,76 +1116,114 @@ AC_DEFUN([AM_SUBST_NOTMAKE], [_AM_SUBST_NOTMAKE($@)]) # Substitute a variable $(am__untar) that extract such # a tarball read from stdin. # $(am__untar) < result.tar +# AC_DEFUN([_AM_PROG_TAR], [# Always define AMTAR for backward compatibility. Yes, it's still used # in the wild :-( We should find a proper way to deprecate it ... AC_SUBST([AMTAR], ['$${TAR-tar}']) -m4_if([$1], [v7], - [am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -'], - [m4_case([$1], [ustar],, [pax],, - [m4_fatal([Unknown tar format])]) -AC_MSG_CHECKING([how to create a $1 tar archive]) -# Loop over all known methods to create a tar archive until one works. + +# We'll loop over all known methods to create a tar archive until one works. _am_tools='gnutar m4_if([$1], [ustar], [plaintar]) pax cpio none' -_am_tools=${am_cv_prog_tar_$1-$_am_tools} -# Do not fold the above two line into one, because Tru64 sh and -# Solaris sh will not grok spaces in the rhs of '-'. -for _am_tool in $_am_tools -do - case $_am_tool in - gnutar) - for _am_tar in tar gnutar gtar; - do - AM_RUN_LOG([$_am_tar --version]) && break - done - am__tar="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$$tardir"' - am__tar_="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$tardir"' - am__untar="$_am_tar -xf -" - ;; - plaintar) - # Must skip GNU tar: if it does not support --format= it doesn't create - # ustar tarball either. - (tar --version) >/dev/null 2>&1 && continue - am__tar='tar chf - "$$tardir"' - am__tar_='tar chf - "$tardir"' - am__untar='tar xf -' - ;; - pax) - am__tar='pax -L -x $1 -w "$$tardir"' - am__tar_='pax -L -x $1 -w "$tardir"' - am__untar='pax -r' - ;; - cpio) - am__tar='find "$$tardir" -print | cpio -o -H $1 -L' - am__tar_='find "$tardir" -print | cpio -o -H $1 -L' - am__untar='cpio -i -H $1 -d' - ;; - none) - am__tar=false - am__tar_=false - am__untar=false - ;; - esac - # If the value was cached, stop now. We just wanted to have am__tar - # and am__untar set. - test -n "${am_cv_prog_tar_$1}" && break +m4_if([$1], [v7], + [am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -'], - # tar/untar a dummy directory, and stop if the command works + [m4_case([$1], + [ustar], + [# The POSIX 1988 'ustar' format is defined with fixed-size fields. + # There is notably a 21 bits limit for the UID and the GID. In fact, + # the 'pax' utility can hang on bigger UID/GID (see automake bug#8343 + # and bug#13588). + am_max_uid=2097151 # 2^21 - 1 + am_max_gid=$am_max_uid + # The $UID and $GID variables are not portable, so we need to resort + # to the POSIX-mandated id(1) utility. Errors in the 'id' calls + # below are definitely unexpected, so allow the users to see them + # (that is, avoid stderr redirection). + am_uid=`id -u || echo unknown` + am_gid=`id -g || echo unknown` + AC_MSG_CHECKING([whether UID '$am_uid' is supported by ustar format]) + if test $am_uid -le $am_max_uid; then + AC_MSG_RESULT([yes]) + else + AC_MSG_RESULT([no]) + _am_tools=none + fi + AC_MSG_CHECKING([whether GID '$am_gid' is supported by ustar format]) + if test $am_gid -le $am_max_gid; then + AC_MSG_RESULT([yes]) + else + AC_MSG_RESULT([no]) + _am_tools=none + fi], + + [pax], + [], + + [m4_fatal([Unknown tar format])]) + + AC_MSG_CHECKING([how to create a $1 tar archive]) + + # Go ahead even if we have the value already cached. We do so because we + # need to set the values for the 'am__tar' and 'am__untar' variables. + _am_tools=${am_cv_prog_tar_$1-$_am_tools} + + for _am_tool in $_am_tools; do + case $_am_tool in + gnutar) + for _am_tar in tar gnutar gtar; do + AM_RUN_LOG([$_am_tar --version]) && break + done + am__tar="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$$tardir"' + am__tar_="$_am_tar --format=m4_if([$1], [pax], [posix], [$1]) -chf - "'"$tardir"' + am__untar="$_am_tar -xf -" + ;; + plaintar) + # Must skip GNU tar: if it does not support --format= it doesn't create + # ustar tarball either. + (tar --version) >/dev/null 2>&1 && continue + am__tar='tar chf - "$$tardir"' + am__tar_='tar chf - "$tardir"' + am__untar='tar xf -' + ;; + pax) + am__tar='pax -L -x $1 -w "$$tardir"' + am__tar_='pax -L -x $1 -w "$tardir"' + am__untar='pax -r' + ;; + cpio) + am__tar='find "$$tardir" -print | cpio -o -H $1 -L' + am__tar_='find "$tardir" -print | cpio -o -H $1 -L' + am__untar='cpio -i -H $1 -d' + ;; + none) + am__tar=false + am__tar_=false + am__untar=false + ;; + esac + + # If the value was cached, stop now. We just wanted to have am__tar + # and am__untar set. + test -n "${am_cv_prog_tar_$1}" && break + + # tar/untar a dummy directory, and stop if the command works. + rm -rf conftest.dir + mkdir conftest.dir + echo GrepMe > conftest.dir/file + AM_RUN_LOG([tardir=conftest.dir && eval $am__tar_ >conftest.tar]) + rm -rf conftest.dir + if test -s conftest.tar; then + AM_RUN_LOG([$am__untar /dev/null 2>&1 && break + fi + done rm -rf conftest.dir - mkdir conftest.dir - echo GrepMe > conftest.dir/file - AM_RUN_LOG([tardir=conftest.dir && eval $am__tar_ >conftest.tar]) - rm -rf conftest.dir - if test -s conftest.tar; then - AM_RUN_LOG([$am__untar /dev/null 2>&1 && break - fi -done -rm -rf conftest.dir -AC_CACHE_VAL([am_cv_prog_tar_$1], [am_cv_prog_tar_$1=$_am_tool]) -AC_MSG_RESULT([$am_cv_prog_tar_$1])]) + AC_CACHE_VAL([am_cv_prog_tar_$1], [am_cv_prog_tar_$1=$_am_tool]) + AC_MSG_RESULT([$am_cv_prog_tar_$1])]) + AC_SUBST([am__tar]) AC_SUBST([am__untar]) ]) # _AM_PROG_TAR Modified: vendor/file/dist/compile ============================================================================== --- vendor/file/dist/compile Sun Jul 29 05:51:14 2018 (r336861) +++ vendor/file/dist/compile Sun Jul 29 05:53:49 2018 (r336862) @@ -3,7 +3,7 @@ scriptversion=2012-10-14.11; # UTC -# Copyright (C) 1999-2013 Free Software Foundation, Inc. +# Copyright (C) 1999-2014 Free Software Foundation, Inc. # Written by Tom Tromey . # # This program is free software; you can redistribute it and/or modify Modified: vendor/file/dist/config.guess ============================================================================== --- vendor/file/dist/config.guess Sun Jul 29 05:51:14 2018 (r336861) +++ vendor/file/dist/config.guess Sun Jul 29 05:53:49 2018 (r336862) @@ -1,10 +1,8 @@ #! /bin/sh # Attempt to guess a canonical system name. -# Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, -# 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, -# 2011, 2012, 2013 Free Software Foundation, Inc. +# Copyright 1992-2017 Free Software Foundation, Inc. -timestamp='2012-12-29' +timestamp='2017-01-01' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by @@ -26,12 +24,12 @@ timestamp='2012-12-29' # program. This Exception is an additional permission under section 7 # of the GNU General Public License, version 3 ("GPLv3"). # -# Originally written by Per Bothner. +# Originally written by Per Bothner; maintained since 2000 by Ben Elliston. # # You can get the latest version of this script from: -# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD +# http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess # -# Please send patches with a ChangeLog entry to config-patches@gnu.org. +# Please send patches to . me=`echo "$0" | sed -e 's,.*/,,'` @@ -52,9 +50,7 @@ version="\ GNU config.guess ($timestamp) Originally written by Per Bothner. -Copyright (C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, -2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, -2012, 2013 Free Software Foundation, Inc. +Copyright 1992-2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." @@ -136,6 +132,27 @@ UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEAS UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown +case "${UNAME_SYSTEM}" in +Linux|GNU|GNU/*) + # If the system lacks a compiler, then just pick glibc. + # We could probably try harder. + LIBC=gnu + + eval $set_cc_for_build + cat <<-EOF > $dummy.c + #include + #if defined(__UCLIBC__) + LIBC=uclibc + #elif defined(__dietlibc__) + LIBC=dietlibc + #else + LIBC=gnu + #endif + EOF + eval `$CC_FOR_BUILD -E $dummy.c 2>/dev/null | grep '^LIBC' | sed 's, ,,g'` + ;; +esac + # Note: order is significant - the case branches are not exclusive. case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}" in @@ -151,19 +168,29 @@ case "${UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE # Note: NetBSD doesn't particularly care about the vendor # portion of the name. We always set it to "unknown". sysctl="sysctl -n hw.machine_arch" - UNAME_MACHINE_ARCH=`(/sbin/$sysctl 2>/dev/null || \ - /usr/sbin/$sysctl 2>/dev/null || echo unknown)` + UNAME_MACHINE_ARCH=`(uname -p 2>/dev/null || \ + /sbin/$sysctl 2>/dev/null || \ + /usr/sbin/$sysctl 2>/dev/null || \ + echo unknown)` case "${UNAME_MACHINE_ARCH}" in armeb) machine=armeb-unknown ;; arm*) machine=arm-unknown ;; sh3el) machine=shl-unknown ;; sh3eb) machine=sh-unknown ;; sh5el) machine=sh5le-unknown ;; + earmv*) + arch=`echo ${UNAME_MACHINE_ARCH} | sed -e 's,^e\(armv[0-9]\).*$,\1,'` + endian=`echo ${UNAME_MACHINE_ARCH} | sed -ne 's,^.*\(eb\)$,\1,p'` + machine=${arch}${endian}-unknown *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Sun Jul 29 05:54:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4C46104C574; Sun, 29 Jul 2018 05:54:55 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 598E072ECF; Sun, 29 Jul 2018 05:54:55 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3A70711C55; Sun, 29 Jul 2018 05:54:55 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T5stN7024374; Sun, 29 Jul 2018 05:54:55 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T5stTq024372; Sun, 29 Jul 2018 05:54:55 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290554.w6T5stTq024372@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 05:54:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336863 - in vendor/file/5.34: . doc magic magic/Magdir python src tests X-SVN-Group: vendor X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: in vendor/file/5.34: . doc magic magic/Magdir python src tests X-SVN-Commit-Revision: 336863 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 05:54:56 -0000 Author: eadler Date: Sun Jul 29 05:54:54 2018 New Revision: 336863 URL: https://svnweb.freebsd.org/changeset/base/336863 Log: vendor/file: tag 5.34 Added: vendor/file/5.34/ - copied from r336859, vendor/file/dist/ vendor/file/5.34/magic/Magdir/dataone - copied unchanged from r336862, vendor/file/dist/magic/Magdir/dataone vendor/file/5.34/python/CHANGELOG.md - copied unchanged from r336862, vendor/file/dist/python/CHANGELOG.md vendor/file/5.34/python/LICENSE - copied unchanged from r336862, vendor/file/dist/python/LICENSE vendor/file/5.34/python/README.md - copied unchanged from r336862, vendor/file/dist/python/README.md vendor/file/5.34/python/tests.py - copied unchanged from r336862, vendor/file/dist/python/tests.py vendor/file/5.34/tests/JW07022A.mp3.result - copied unchanged from r336862, vendor/file/dist/tests/JW07022A.mp3.result vendor/file/5.34/tests/JW07022A.mp3.testfile - copied unchanged from r336862, vendor/file/dist/tests/JW07022A.mp3.testfile Replaced: vendor/file/5.34/ChangeLog - copied unchanged from r336862, vendor/file/dist/ChangeLog vendor/file/5.34/Makefile.in - copied unchanged from r336862, vendor/file/dist/Makefile.in vendor/file/5.34/README - copied unchanged from r336862, vendor/file/dist/README vendor/file/5.34/aclocal.m4 - copied unchanged from r336862, vendor/file/dist/aclocal.m4 vendor/file/5.34/compile - copied unchanged from r336862, vendor/file/dist/compile vendor/file/5.34/config.guess - copied unchanged from r336862, vendor/file/dist/config.guess vendor/file/5.34/config.h.in - copied unchanged from r336862, vendor/file/dist/config.h.in vendor/file/5.34/config.sub - copied unchanged from r336862, vendor/file/dist/config.sub vendor/file/5.34/configure - copied unchanged from r336862, vendor/file/dist/configure vendor/file/5.34/configure.ac - copied unchanged from r336862, vendor/file/dist/configure.ac vendor/file/5.34/depcomp - copied unchanged from r336862, vendor/file/dist/depcomp vendor/file/5.34/doc/Makefile.in - copied unchanged from r336862, vendor/file/dist/doc/Makefile.in vendor/file/5.34/doc/file.man - copied unchanged from r336862, vendor/file/dist/doc/file.man vendor/file/5.34/doc/magic.man - copied unchanged from r336862, vendor/file/dist/doc/magic.man vendor/file/5.34/install-sh - copied unchanged from r336862, vendor/file/dist/install-sh vendor/file/5.34/ltmain.sh - copied unchanged from r336862, vendor/file/dist/ltmain.sh vendor/file/5.34/magic/Magdir/animation - copied unchanged from r336862, vendor/file/dist/magic/Magdir/animation vendor/file/5.34/magic/Magdir/archive - copied unchanged from r336862, vendor/file/dist/magic/Magdir/archive vendor/file/5.34/magic/Magdir/audio - copied unchanged from r336862, vendor/file/dist/magic/Magdir/audio vendor/file/5.34/magic/Magdir/cad - copied unchanged from r336862, vendor/file/dist/magic/Magdir/cad vendor/file/5.34/magic/Magdir/console - copied unchanged from r336862, vendor/file/dist/magic/Magdir/console vendor/file/5.34/magic/Magdir/database - copied unchanged from r336862, vendor/file/dist/magic/Magdir/database vendor/file/5.34/magic/Magdir/dbpf - copied unchanged from r336862, vendor/file/dist/magic/Magdir/dbpf vendor/file/5.34/magic/Magdir/dump - copied unchanged from r336862, vendor/file/dist/magic/Magdir/dump vendor/file/5.34/magic/Magdir/elf - copied unchanged from r336862, vendor/file/dist/magic/Magdir/elf vendor/file/5.34/magic/Magdir/fsav - copied unchanged from r336862, vendor/file/dist/magic/Magdir/fsav vendor/file/5.34/magic/Magdir/images - copied unchanged from r336862, vendor/file/dist/magic/Magdir/images vendor/file/5.34/magic/Magdir/linux - copied unchanged from r336862, vendor/file/dist/magic/Magdir/linux vendor/file/5.34/magic/Magdir/make - copied unchanged from r336862, vendor/file/dist/magic/Magdir/make vendor/file/5.34/magic/Magdir/measure - copied unchanged from r336862, vendor/file/dist/magic/Magdir/measure vendor/file/5.34/magic/Magdir/msdos - copied unchanged from r336862, vendor/file/dist/magic/Magdir/msdos vendor/file/5.34/magic/Magdir/msooxml - copied unchanged from r336862, vendor/file/dist/magic/Magdir/msooxml vendor/file/5.34/magic/Magdir/pdf - copied unchanged from r336862, vendor/file/dist/magic/Magdir/pdf vendor/file/5.34/magic/Magdir/sgi - copied unchanged from r336862, vendor/file/dist/magic/Magdir/sgi vendor/file/5.34/magic/Makefile.am - copied unchanged from r336862, vendor/file/dist/magic/Makefile.am vendor/file/5.34/magic/Makefile.in - copied unchanged from r336862, vendor/file/dist/magic/Makefile.in vendor/file/5.34/missing - copied unchanged from r336862, vendor/file/dist/missing vendor/file/5.34/python/Makefile.am - copied unchanged from r336862, vendor/file/dist/python/Makefile.am vendor/file/5.34/python/Makefile.in - copied unchanged from r336862, vendor/file/dist/python/Makefile.in vendor/file/5.34/python/magic.py - copied unchanged from r336862, vendor/file/dist/python/magic.py vendor/file/5.34/python/setup.py - copied unchanged from r336862, vendor/file/dist/python/setup.py vendor/file/5.34/src/Makefile.in - copied unchanged from r336862, vendor/file/dist/src/Makefile.in vendor/file/5.34/src/apprentice.c - copied unchanged from r336862, vendor/file/dist/src/apprentice.c vendor/file/5.34/src/compress.c - copied unchanged from r336862, vendor/file/dist/src/compress.c vendor/file/5.34/src/der.c - copied unchanged from r336862, vendor/file/dist/src/der.c vendor/file/5.34/src/file.h - copied unchanged from r336862, vendor/file/dist/src/file.h vendor/file/5.34/src/funcs.c - copied unchanged from r336862, vendor/file/dist/src/funcs.c vendor/file/5.34/src/readcdf.c - copied unchanged from r336862, vendor/file/dist/src/readcdf.c vendor/file/5.34/src/readelf.c - copied unchanged from r336862, vendor/file/dist/src/readelf.c vendor/file/5.34/src/readelf.h - copied unchanged from r336862, vendor/file/dist/src/readelf.h vendor/file/5.34/src/seccomp.c - copied unchanged from r336862, vendor/file/dist/src/seccomp.c vendor/file/5.34/src/softmagic.c - copied unchanged from r336862, vendor/file/dist/src/softmagic.c vendor/file/5.34/tests/Makefile.am - copied unchanged from r336862, vendor/file/dist/tests/Makefile.am vendor/file/5.34/tests/Makefile.in - copied unchanged from r336862, vendor/file/dist/tests/Makefile.in Deleted: vendor/file/5.34/python/README Copied: vendor/file/5.34/ChangeLog (from r336862, vendor/file/dist/ChangeLog) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/file/5.34/ChangeLog Sun Jul 29 05:54:54 2018 (r336863, copy of r336862, vendor/file/dist/ChangeLog) @@ -0,0 +1,1753 @@ +2018-07-25 8:50 Christos Zoulas + + * release 5.34 + +2018-06-22 16:38 Christos Zoulas + + * Add Quad indirect offsets + +2018-05-24 14:10 Christos Zoulas + + * Enable parsing of ELF dynamic sections to handle PIE better + +2018-04-15 14:52 Christos Zoulas + + * release 5.33 + +2018-02-24 14:50 Christos Zoulas + + * extend the support for ${x?:} expansions for magic descriptions + +2018-02-21 16:25 Christos Zoulas + + * add support for ${x?:} in mime types to handle + pie binaries. + +2017-11-03 9:23 Christos Zoulas + + * add support for negative offsets (offsets from the end of file) + +2017-09-26 8:22 Christos Zoulas + + * close the file on error when writing magic (Steve Grubb) + +2017-09-24 12:02 Christos Zoulas + + * seccomp support (Paul Moore) + +2017-09-02 11:53 Christos Zoulas + + * release 5.32 + +2017-08-28 16:37 Christos Zoulas + + * Always reset state in {file,buffer}_apprentice (Krzysztof Wilczynski) + +2017-08-27 03:55 Christos Zoulas + + * Fix always true condition (Thomas Jarosch) + +2017-05-24 17:30 Christos Zoulas + + * pickier parsing of numeric values in magic files. + +2017-05-23 17:55 Christos Zoulas + + * PR/615 add magic_getflags() + +2017-05-23 13:55 Christos Zoulas + + * release 5.31 + +2017-03-17 20:32 Christos Zoulas + + * remove trailing spaces from magic files + * refactor is_tar + * better bounds checks for cdf + +2017-02-10 12:24 Christos Zoulas + + * release 5.30 + +2017-02-07 23:27 Christos Zoulas + + * If we exceeded the offset in a search return no match + (Christoph Biedl) + * Be more lenient on corrupt CDF files (Christoph Biedl) + +2017-02-04 16:46 Christos Zoulas + + * pacify ubsan sign extension (oss-fuzz/524) + +2017-02-01 12:42 Christos Zoulas + + * off by one in cdf parsing (PR/593) + * report debugging sections in elf (PR/591) + +2016-11-06 10:52 Christos Zoulas + + * Allow @@@ in extensions + * Add missing overflow check in der magic (Jonas Wagner) + +2016-10-25 10:40 Christos Zoulas + + * release 5.29 + +2016-10-24 11:20 Christos Zoulas + + * der getlength overflow (Jonas Wagner) + * multiple magic file load failure (Christoph Biedl) + +2016-10-17 11:26 Christos Zoulas + + * CDF parsing improvements (Guy Helmer) + +2016-07-20 7:26 Christos Zoulas + + * Add support for signed indirect offsets + +2016-07-18 7:41 Christos Zoulas + + * cat /dev/null | file - should print empty (Christoph Biedl) + +2016-07-05 15:20 Christos Zoulas + + * Bump string size from 64 to 96. + +2016-06-13 20:20 Christos Zoulas + + * PR/556: Fix separators on annotations. + +2016-06-13 19:40 Christos Zoulas + + * release 5.28 + * fix leak on allocation failure + +2016-06-01 1:20 Christos Zoulas + + * PR/555: Avoid overflow for offset > nbytes + * PR/550: Segv on DER parsing: + - use the correct variable for length + - set offset to 0 on failure. + +2016-05-13 12:00 Christos Zoulas + + * release 5.27 + +2016-04-18 9:35 Christos Zoulas + + * Errors comparing DER entries or computing offsets + are just indications of malformed non-DER files. + Don't print them. + * Offset comparison was off-by-one. + * Fix compression code (Werner Fink) + * Put new bytes constant in the right file (not the generated one) + +2016-04-16 18:34 Christos Zoulas + + * release 5.26 + +2016-03-31 13:50 Christos Zoulas + + * make the number of bytes read from files configurable. + +2016-03-21 13:40 Christos Zoulas + + * Add bounds checks for DER code (discovered by Thomas Jarosch) + * Change indirect recursion limit to indirect use count and + bump from 15 to 50 to prevent abuse. + +2016-03-13 20:39 Christos Zoulas + + * Add -00 which prints filename\0description\0 + +2016-03-01 13:28 Christos Zoulas + + * Fix ID3 indirect parsing + +2016-01-19 10:18 Christos Zoulas + + * add DER parsing capability + +2015-11-13 10:35 Christos Zoulas + + * provide dprintf(3) for the OS's that don't have it. + +2015-11-11 16:25 Christos Zoulas + + * redo the compression code report decompression errors + +2015-11-10 23:25 Christos Zoulas + + * REG_STARTEND code is not working as expected, delete it. + +2015-11-09 16:05 Christos Zoulas + + * Add zlib support if we have it. + +2015-11-05 11:22 Christos Zoulas + + * PR/492: compression forking was broken with magic_buffer. + +2015-09-16 9:50 Christos Zoulas + + * release 5.25 + +2015-09-11 13:25 Christos Zoulas + + * add a limit to the length of regex searches + +2015-09-08 9:50 Christos Zoulas + + * fix problems with --parameter (Christoph Biedl) + +2015-07-11 10:35 Christos Zoulas + + * Windows fixes PR/466 (Jason Hood) + +2015-07-09 10:35 Christos Zoulas + + * release 5.24 + +2015-06-11 8:52 Christos Zoulas + + * redo long option encoding to fix off-by-one in 5.23 + +2015-06-10 13:50 Christos Zoulas + + * release 5.23 + +2015-06-09 16:10 Christos Zoulas + + * Fix issue with regex range for magic with offset + * Always return true from mget with USE (success to mget not match + indication). Fixes mime evaluation after USE magic + * PR/459: Don't insert magic entries to the list if there are parsing + errors for them. + +2015-06-03 16:00 Christos Zoulas + + * PR/455: Add utf-7 encoding + +2015-06-03 14:30 Christos Zoulas + + * PR/455: Implement -Z, look inside, but don't report on compression + * PR/454: Fix allocation error on bad magic. + +2015-05-29 10:30 Christos Zoulas + + * handle MAGIC_CONTINUE everywhere, not just in softmagic + +2015-05-21 14:30 Christos Zoulas + + * don't print descriptions for NAME types when mime. + +2015-04-09 15:59 Christos Zoulas + + * Add --extension to list the known extensions for this file type + Idea by Andrew J Roazen + +2015-02-14 12:23 Christos Zoulas + + * Bump file search buffer size to 1M. + +2015-01-09 14:35 Christos Zoulas + + * Fix multiple issues with date formats reported by Christoph Biedl: + - T_LOCAL meaning was reversed + - Arithmetic did not work + Also stop adjusting daylight savings for gmt printing. + +2015-01-05 13:00 Christos Zoulas + + * PR/411: Fix memory corruption from corrupt cdf file. + +2015-01-02 15:15 Christos Zoulas + + * release 5.22 + +2015-01-01 12:01 Christos Zoulas + + * add indirect relative for TIFF/Exif + +2014-12-16 18:10 Christos Zoulas + + * restructure elf note printing to avoid repeated messages + * add note limit, suggested by Alexander Cherepanov + +2014-12-16 16:53 Christos Zoulas + + * Bail out on partial pread()'s (Alexander Cherepanov) + * Fix incorrect bounds check in file_printable (Alexander Cherepanov) + +2014-12-11 20:01 Christos Zoulas + + * PR/405: ignore SIGPIPE from uncompress programs + * change printable -> file_printable and use it in + more places for safety + * in ELF, instead of "(uses dynamic libraries)" when PT_INTERP + is present print the interpreter name. + +2014-12-10 20:01 Christos Zoulas + + * release 5.21 + +2014-11-27 18:40 Christos Zoulas + + * Allow setting more parameters from the command line. + * Split name/use and indirect magic recursion limits. + +2014-11-27 11:12 Christos Zoulas + + * Adjust ELF parameters and the default recursion + level. + * Allow setting the recursion level dynamically. + +2014-11-24 8:55 Christos Zoulas + + * The following fixes resulted from Thomas Jarosch's fuzzing + tests that revealed severe performance issues on pathological + input: + - limit number of elf program and sections processing + - abort elf note processing quickly + - reduce the number of recursion levels from 20 to 10 + - preserve error messages in indirect magic handling + + This is tracked as CVE-2014-8116 and CVE-2014-8117 + +2014-11-12 10:30 Christos Zoulas + + * fix bogus free in the user buffer case. + +2014-11-11 12:35 Christos Zoulas + + * fix out of bounds read for pascal strings + * fix memory leak (not freeing the head of each mlist) + +2014-11-07 10:25 Christos Zoulas + + * When printing strings from a file, convert them to printable + on a byte by byte basis, so that we don't get issues with + locale's trying to interpret random byte streams as UTF-8 and + having printf error out with EILSEQ. + +2014-10-17 11:48 Christos Zoulas + + * fix bounds in note reading (Francisco Alonso / Red Hat) + +2014-10-11 15:02 Christos Zoulas + + * fix autoconf glue for setlocale and locale_t; some OS's + have locale_t in xlocale.h + +2014-10-10 15:01 Christos Zoulas + + * release 5.20 + +2014-08-17 10:01 Christos Zoulas + + * recognize encrypted CDF documents + +2014-08-04 9:18 Christos Zoulas + + * add magic_load_buffers from Brooks Davis + +2014-07-24 16:40 Christos Zoulas + + * add thumbs.db support + +2014-06-12 12:28 Christos Zoulas + + * release 5.19 + +2014-06-09 9:04 Christos Zoulas + + * Misc buffer overruns and missing buffer size tests in cdf parsing + (Francisco Alonso, Jan Kaluza) + +2014-06-02 14:50 Christos Zoulas + + * Enforce limit of 8K on regex searches that have no limits + * Allow the l modifier for regex to mean line count. Default + to byte count. If line count is specified, assume a max + of 80 characters per line to limit the byte count. + * Don't allow conversions to be used for dates, allowing + the mask field to be used as an offset. + +2014-05-30 12:51 Christos Zoulas + + * Make the range operator limit the length of the + regex search. + +2014-05-14 19:23 Christos Zoulas + + * PR/347: Windows fixes + * PR/352: Hangul word processor recognition + * PR/354: Encoding irregularities in text files + +2014-05-06 6:12 Christos Zoulas + + * Fix uninitialized title in CDF files (Jan Kaluza) + +2014-05-04 14:55 Christos Zoulas + + * PR/351: Fix compilation of empty files + +2014-04-30 17:39 Christos Zoulas + + * Fix integer formats: We don't specify 'l' or + 'h' and 'hh' specifiers anymore, only 'll' for + quads and nothing for the rest. This is so that + magic writing is simpler. + +2014-04-01 15:25 Christos Zoulas + + * PR/341: Jan Kaluza, fix memory leak + * PR/342: Jan Kaluza, fix out of bounds read + +2014-03-28 15:25 Christos Zoulas + + * Fix issue with long formats not matching fmtcheck + +2014-03-26 11:25 Christos Zoulas + + * release 5.18 + +2014-03-15 17:45 Christos Zoulas + + * add fmtcheck(3) for those who don't have it + +2014-03-14 15:12 Christos Zoulas + + * prevent mime entries from being attached to magic + entries with no descriptions + + * adjust magic strength for regex type + + * remove superfluous ascmagic with encoding test + +2014-03-06 12:01 Christos Zoulas + + * fix regression fix echo -ne "\012\013\014" | file -i - + which printed "binary" instead of "application/octet-stream" + + * add size_t overflow check for magic file size + +2014-02-27 16:01 Christos Zoulas + + * experimental support for matching with CFD CLSID + +2014-02-18 13:04 Kimmo Suominen (kimmo@suominen.com) + + * Cache old LC_CTYPE locale before setting it to "C", so + we can use it to restore LC_CTYPE instead of asking + setlocale() to scan the environment variables. + +2014-02-12 18:21 Christos Zoulas + + * Count recursion levels through indirect magic + +2014-02-11 10:40 Christos Zoulas + + * Prevent infinite recursion on files with indirect offsets of 0 + +2014-01-30 21:00 Christos Zoulas + + * Add -E flag that makes file print filesystem errors to stderr + and exit. + +2014-01-08 17:20 Christos Zoulas + + * mime printing could print results from multiple magic entries + if there were multiple matches. + * in some cases overflow was not detected when computing offsets + in softmagic. + +2013-12-05 12:00 Christos Zoulas + + * use strcasestr() to for cdf strings + * reset to the "C" locale while doing regex operations, or case + insensitive comparisons; this is provisional + +2013-11-19 20:10 Christos Zoulas + + * always leave magic file loaded, don't unload for magic_check, etc. + * fix default encoding to binary instead of unknown which broke recently + * handle empty and one byte files, less specially so that + --mime-encoding does not break completely. + ` +2013-11-06 14:40 Christos Zoulas + + * fix erroneous non-zero exit code from non-existent file and message + +2013-10-29 14:25 Christos Zoulas + + * add CDF MSI file detection (Guy Helmer) + +2013-09-03 11:56 Christos Zoulas + + * Don't mix errors and regular output if there was an error + * in magic_descriptor() don't close the file and try to restore + its position + +2013-05-30 17:25 Christos Zoulas + + * Don't treat magic as an error if offset was past EOF (Christoph Biedl) + +2013-05-28 17:25 Christos Zoulas + + * Fix spacing issues in softmagic and elf (Jan Kaluza) + +2013-05-02 18:00 Christos Zoulas + + * Fix segmentation fault with multiple magic_load commands. + +2013-04-22 11:20 Christos Zoulas + + * The way "default" was implemented was not very useful + because the "if something was printed at that level" + was not easily controlled by the user, and the format + was bound to a string which is too restrictive. Add + a "clear" for that level keyword and make "default" + void. This way one can do: + + >>13 clear x + >>13 lelong 1 foo + >>13 lelong 2 bar + >>13 default x + >>>13 lelong x unknown %x + +2013-03-25 13:20 Christos Zoulas + + * disallow strength setting in "name" entries + +2013-03-06 21:24 Christos Zoulas + + * fix recursive magic separator printing + +2013-02-26 19:28 Christos Zoulas + + * limit recursion level for mget + * fix pread() related breakage in cdf + * handle offsets properly in recursive "use" + +2013-02-18 10:39 Christos Zoulas + + * add elf reading of debug info to determine if file is stripped + (Jan Kaluza) + * use pread() + +2013-01-25 18:05 Christos Zoulas + + * change mime description size from 64 to 80 to accommodate OOXML. + +2013-01-11 14:50 Christos Zoulas + + * Warn about inconsistent continuation levels. + * Change fsmagic to add a space after it prints. + +2013-01-10 21:00 Christos Zoulas + + * Make getline public so that file can link against it. + Perhaps it is better to rename it, or hide it differently. + Fixes builds on platforms that do not provide it. + +2013-01-07 16:30 Christos Zoulas + + * Add SuS d{,1,2,4,8}, u{,1,2,4,8} and document + what long, int, short, etc is (Guy Harris) + +2013-01-06 11:20 Christos Zoulas + + * add magic_version function and constant + * Redo memory allocation and de-allocation. + (prevents double frees on non mmap platforms) + * Fix bug with name/use having to do with passing + found state from the parent to the child and back. + +2012-12-19 8:47 Christos Zoulas + + * Only print elf capabilities for archs we know (Jan Kaluza) + +2012-10-30 19:14 Christos Zoulas + + * Add "name" and "use" file types in order to look + inside mach-o files. + +2012-09-06 10:40 Christos Zoulas + + * make --version exit 0 (Matthew Schultz) + * add string/T (Jan Kaluza) + +2012-08-09 2:15 Christos Zoulas + + * add z and t modifiers for our own vasprintf + * search for $HOME/.magic.mgc if it is there first + * fix reads from a pipe, and preserve errno + +2012-05-15 13:12 Christos Zoulas + + * use ctime_r, asctime_r + +2012-04-06 17:18 Christos Zoulas + + * Fixes for indirect offsets to handle apple disk formats + +2012-04-03 18:26 Christos Zoulas + + * Add windows date field types + * More info for windows shortcuts (incomplete) + +2012-02-20 17:33 Christos Zoulas + + * Fix CDF parsing issues found by CERT's fuzzing tool (Will Dormann) + +2011-12-15 12:17 Chris Metcalf + + * Support Tilera architectures (tile64, tilepro, tilegx). + +2011-12-16 16:33 Reuben Thomas + + * Add magic for /usr/bin/env Perl scripts + * Weaken generic script magic to avoid clashing with + language-specific magic. + +2011-12-08 13:37 Reuben Thomas + + * Simplify if (p) free(p) to free(p). + +2011-12-08 13:07 Reuben Thomas + + * Remove hardwired token finding (names.h), turning it into soft + magic. Patterns are either anchored regexs or search/8192. English + language detection and PL/1 detection have been removed as they + were too fragile. -e tokens is still accepted for backwards + compatibility. + * Move 3ds patterns (which are commented out anyway) into autodesk + (they were, oddly, in c-lang). + +2011-12-06 00:16 Reuben Thomas + + * Tweak strength of generic hash-bang detectors to be less than + specific ones. + * Make an inconsistent description of Python scripts consistent. + +2011-12-05 23:58 Reuben Thomas + + * Fix minor error in file(1). + +2011-11-05 00:00 Reuben Thomas + + * Fix issue #150 (I hope). + +2011-09-22 12:57 Christos Zoulas + + * Python3 binding fixes from Kelly Anderson + +2011-09-20 11:32 Christos Zoulas + + * If a string type magic entry is marked as text or binary + only match text files against text entries and binary + files against binary entries. + +2011-09-01 12:12 Christos Zoulas + + * Don't wait for any subprocess, just the one we forked. + +2011-08-26 16:40 Christos Zoulas + + * If the application name is not set in a cdf file, try to see + if it has a directory with the application name on it. + +2011-08-17 14:32 Christos Zoulas + + * Fix ELF lseek(2) madness. Inspired by PR/134 by Jan Kaluza + +2011-08-14 09:03 Christos Zoulas + + * Don't use variable string formats. + +2011-07-12 12:32 Reuben Thomas + + * Fix detection of Zip files (Mantis #128). + * Make some minor improvements to file(1). + * Rename MIME types for filesystem objects for consistency with + xdg-utils. Typically this means that application/x-foo becomes + inode/foo, but some names also change slightly, e.g. + application/x-character-device becomes inode/chardevice. + +2011-05-10 20:57 Christos Zoulas + + * fix mingw compilation (Abradoks) + +2011-05-10 20:57 Christos Zoulas + + * remove patchlevel.h + * Fix read past allocated memory caused by double-incrementing + a pointer in a loop (reported by Roberto Maar) + +2011-03-30 15:45 Christos Zoulas + + * Fix cdf string buffer setting (Sven Anders) + +2011-03-20 16:35 Christos Zoulas + + * Eliminate MAXPATHLEN and use dynamic allocation for + path and file buffers. + +2011-03-15 18:15 Christos Zoulas + + * binary tests on magic entries with masks could spuriously + get converted to ascii. + +2011-03-12 18:06 Reuben Thomas + + * Improve file.man (remove BUGS, present email addresses consistently). + +2011-03-07 19:38 Christos Zoulas + + * add lrzip support (from Ville Skytta) + +2011-02-10 16:36 Christos Zoulas + + * fix CDF bounds checking (Guy Helmer) + +2011-02-10 12:03 Christos Zoulas + + * add cdf_ctime() that prints a meaningful error when time cannot + be converted. + +2011-02-02 20:40 Christos Zoulas + + * help and version output to stdout. + + * When matching softmagic for ascii files, don't just print + the softmagic classification, keep going and print the + text classification too. This fixes broken troff files when + we moved them from keyword recognition to softmagic + (they stopped printing "with CRLF" etc.) + Reported by Doug McIlroy. + +2011-01-16 19:31 Reuben Thomas + + * Fix two potential buffer overruns in apprentice_list. + +2011-01-14 22:33 Reuben Thomas + + * New Python binding in pure Python. + * Update libmagic(3). + +2011-01-06 21:40 Reuben Thomas + + * Fix Python bindings (including recent Python 3 compatibility + update). + +2011-01-04 18:43 Reuben Thomas + + * magic/Makefile.am: make it easier to recover from magic build failures. + * Fix pstring length specifier parsing to avoid generating invalid + magic files. + * Add pstring length "J" (for "JPEG") to specify that the length + include itself. + * Fix JPEG comment parsing at last using pstring/HJ! + * Ignore section 5 man pages in doc/.cvsignore. + +2010-12-22 13:12 Christos Zoulas + + * Add pstring/BHhLl to specify the type of the length of pascal + strings. + +2010-11-26 18:39 Reuben Thomas + + * Fix "-e soft": it was ignored when softmagic was called + during asciimagic. + * Improve comments and use "unsigned char" in tar.h/is_tar.c. + +2010-11-05 17:26 Reuben Thomas + + * Make bug reporting addresses more visible. + +2010-11-01 18:35 Reuben Thomas + + * Add tcl magic from Gustaf Neumann + +2010-10-24 10:42 Christos Zoulas + + * Fix the whitespace comparing code (Christopher Chittleborough) + +2010-10-06 21:05 Christos Zoulas + + * allow string/t to work (Jan Kaluza) + +2010-09-20 22:11 Reuben Thomas + + * Apply some patches from Ubuntu and Fedora. + +2010-09-20 21:16 Reuben Thomas + + * Apply all patches from Debian package 5.04-6 which have not + already been applied and are not Debian-specific. + +2010-09-20 15:24 Reuben Thomas + + * Minor security fix to softmagic.c (don't use untrusted + string as printf format). + +2010-07-21 12:20 Christos Zoulas + + * MINGW32 portability from LRN + + * Don't warn about escaping magic regex chars when we are in a regex. + +2010-07-19 10:55 Christos Zoulas + + * Only try to print prpsinfo for core files. (Jan Kaluza) + +2010-04-22 12:55 Christos Zoulas + + * Try more elf offsets for Debian core files. (Arnaud Giersch) + +2010-02-20 15:18 Reuben Thomas + + * Clarify which sort of CDF we mean. + +2010-02-14 22:58 Reuben Thomas + + * Re-jig Zip file type magic so that unsupported special + Zip types (those with "mimetype" at offset 30) can be + recognized. + +2010-02-02 21:50 Reuben Thomas + + * Add support for OCF (EPUB) files (application/epub+zip) + +2010-01-28 18:25 Christos Zoulas + + * Fix core-dump from unbound loop: + https://bugzilla.redhat.com/show_bug.cgi?id=533245 + +2010-01-22 15:45 Christos Zoulas + + * print proper mime for crystal reports file + + * print the last summary information of a cdf document, not the + first so that nested documents print the right info + +2010-01-16 18:42 Charles Longeau + + * bring back some fixes from OpenBSD: + - make gcc2 builds file + - fix typos in a magic file comment + +2009-11-17 18:35 Christos Zoulas + + * ctime/asctime can return NULL on some OS's although + they should not (Toshit Antani) + +2009-09-14 13:49 Christos Zoulas + + * Centralize magic path handling routines and remove the + special-casing from file.c so that the python module for + example comes up with the same magic path (Fixes ~/.magic + handling) (from Gab) + +2009-09-11 23:38 Reuben Thomas + + * When magic argument is a directory, read the files in + strcmp-sorted order (fixes Debian bug #488562 and our own FIXME). + +2009-09-11 13:11 Reuben Thomas + + * Combine overlapping epoc and psion magic files into one (epoc). + + * Add some more EPOC MIME types. + +2009-08-19 15:55 Christos Zoulas + + * Fix 3 bugs (From Ian Darwin): + - file_showstr could move one past the end of the array + - parse_apple did not nul terminate the string in the overflow case + - parse_mime truncated the wrong string in the overflow case + +2009-08-12 12:28 Robert Byrnes + + * Include Localstuff when compiling magic. + +2009-07-15 10:05 Christos Zoulas + + * Fix logic for including mygetopts.h + + * Make cdf.c compile again with debugging + + * Add the necessary field handling for crystal reports files to work + +2009-06-23 01:34 Reuben Thomas + + * Stop "(if" identifying Lisp files, that's plain dumb! + +2009-06-09 22:13 Reuben Thomas + + * Add a couple of missing MP3 MIME types. + +2009-05-27 23:00 Reuben Thomas + + * Add full range of hash-bang tests for Python and Ruby. + + * Add MIME types for Python and Ruby scripts. + +2009-05-13 10:44 Christos Zoulas + + * off by one in parsing hw capabilities in elf + (Cheng Renquan) + +2009-05-08 13:40 Christos Zoulas + + * lint fixes and more from NetBSD + +2009-05-06 10:25 Christos Zoulas + + * Avoid null dereference in cdf code (Drew Yao) + + * More cdf bounds checks and overflow checks + +2009-05-01 18:37 Christos Zoulas + + * Buffer overflow fixes from Drew Yao + +2009-04-30 17:10 Christos Zoulas + + * Fix more cdf lossage. All the documents I have + right now print the correct information. + +2009-03-27 18:43 Christos Zoulas + + * don't print \012- separators in the same magic entry + if it consists of multiple magic printing lines. + +2009-03-23 10:20 Christos Zoulas + + * Avoid file descriptor leak in compress code from + (Daniel Novotny) + +2009-03-18 16:50 Christos Zoulas + + * Allow escaping of relation characters, so that we can say \^[A-Z] + and the ^ is not eaten as a relation char. + + * Fix troff and fortran to their previous glory using + regex. This was broken since their removel from ascmagic. + +2009-03-10 16:50 Christos Zoulas + + * don't use strlen in strndup() (Toby Peterson) + +2009-03-10 7:45 Christos Zoulas + + * avoid c99 syntax. + +2009-02-23 15:45 Christos Zoulas + + * make the cdf code use the buffer first if available, + and then the fd code. + +2009-02-13 13:45 Christos Zoulas + + * look for struct option to determine if getopt.h is usable for IRIX. + + * sanitize cdf document strings + +2009-02-04 13:25 Christos Zoulas + + * fix OS/2 warnings. + +2008-12-12 15:50 Christos Zoulas + + * fix initial offset calculation for non 4K sector files + + * add loop limits to avoid DoS attacks by constructing + looping sector references. + +2008-12-03 13:05 Christos Zoulas + + * fix memory botches on cdf file parsing. + + * exit with non-zero value for any error, not just for the last + file processed. + +2008-11-09 20:42 Charles Longeau + + * Replace all str{cpy,cat} functions with strl{cpy,cat} + * Ensure that strl{cpy,cat} are included in libmagic, + as needed. + +2008-11-06 18:18 Christos Zoulas + + * Handle ID3 format files. + +2008-11-06 23:00 Reuben Thomas + + * Fix --mime, --mime-type and --mime-encoding under new scheme. + + * Rename "ascii" to "text" and add "encoding" test. + + * Return a precise ("utf-16le" or "utf-16be") MIME charset for + UTF-16. + + * Fix error in comment caused by automatic indentation adding + words! *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Sun Jul 29 07:51:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED02E104ED14; Sun, 29 Jul 2018 07:51:42 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A3BCD75B5D; Sun, 29 Jul 2018 07:51:42 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8502D12E99; Sun, 29 Jul 2018 07:51:42 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T7pg51083029; Sun, 29 Jul 2018 07:51:42 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T7pgRS083028; Sun, 29 Jul 2018 07:51:42 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290751.w6T7pgRS083028@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 07:51:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336865 - head/usr.bin/truncate X-SVN-Group: head X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: head/usr.bin/truncate X-SVN-Commit-Revision: 336865 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 07:51:43 -0000 Author: eadler Date: Sun Jul 29 07:51:42 2018 New Revision: 336865 URL: https://svnweb.freebsd.org/changeset/base/336865 Log: truncate: delintify Modified: head/usr.bin/truncate/truncate.c Modified: head/usr.bin/truncate/truncate.c ============================================================================== --- head/usr.bin/truncate/truncate.c Sun Jul 29 07:51:40 2018 (r336864) +++ head/usr.bin/truncate/truncate.c Sun Jul 29 07:51:42 2018 (r336865) @@ -27,10 +27,8 @@ * */ -#ifndef lint static const char rcsid[] = "$FreeBSD$"; -#endif #include From owner-svn-src-all@freebsd.org Sun Jul 29 07:51:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D893104ED05; Sun, 29 Jul 2018 07:51:41 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B81C475B54; Sun, 29 Jul 2018 07:51:40 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9AA6312E95; Sun, 29 Jul 2018 07:51:40 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T7peHa082986; Sun, 29 Jul 2018 07:51:40 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T7pe0e082985; Sun, 29 Jul 2018 07:51:40 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290751.w6T7pe0e082985@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 07:51:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336864 - head/usr.bin/truncate X-SVN-Group: head X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: head/usr.bin/truncate X-SVN-Commit-Revision: 336864 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 07:51:41 -0000 Author: eadler Date: Sun Jul 29 07:51:40 2018 New Revision: 336864 URL: https://svnweb.freebsd.org/changeset/base/336864 Log: truncate: use better type for 'round' Modified: head/usr.bin/truncate/truncate.c Modified: head/usr.bin/truncate/truncate.c ============================================================================== --- head/usr.bin/truncate/truncate.c Sun Jul 29 05:54:54 2018 (r336863) +++ head/usr.bin/truncate/truncate.c Sun Jul 29 07:51:40 2018 (r336864) @@ -51,7 +51,7 @@ main(int argc, char **argv) { struct stat sb; mode_t omode; - off_t oflow, rsize, sz, tsize; + off_t oflow, rsize, sz, tsize, round; uint64_t usz; int ch, error, fd, oflags; int no_create; @@ -59,7 +59,6 @@ main(int argc, char **argv) int do_round; int do_refer; int got_size; - int round; char *fname, *rname; fd = -1; From owner-svn-src-all@freebsd.org Sun Jul 29 08:43:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18D521050625; Sun, 29 Jul 2018 08:43:06 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C02FD7744D; Sun, 29 Jul 2018 08:43:05 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A0F34137E1; Sun, 29 Jul 2018 08:43:05 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T8h5wf010669; Sun, 29 Jul 2018 08:43:05 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T8h503010668; Sun, 29 Jul 2018 08:43:05 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290843.w6T8h503010668@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 08:43:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336866 - head/sbin/devd X-SVN-Group: head X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: head/sbin/devd X-SVN-Commit-Revision: 336866 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 08:43:06 -0000 Author: eadler Date: Sun Jul 29 08:43:05 2018 New Revision: 336866 URL: https://svnweb.freebsd.org/changeset/base/336866 Log: devd: use static where possible Modified: head/sbin/devd/devd.cc Modified: head/sbin/devd/devd.cc ============================================================================== --- head/sbin/devd/devd.cc Sun Jul 29 07:51:42 2018 (r336865) +++ head/sbin/devd/devd.cc Sun Jul 29 08:43:05 2018 (r336866) @@ -904,7 +904,7 @@ process_event(char *buffer) cfg.pop_var_table(); } -int +static int create_socket(const char *name, int socktype) { int fd, slen; @@ -934,7 +934,7 @@ static unsigned int num_clients; static list clients; -void +static void notify_clients(const char *data, int len) { list::iterator i; @@ -964,7 +964,7 @@ notify_clients(const char *data, int len) } } -void +static void check_clients(void) { int s; @@ -994,7 +994,7 @@ check_clients(void) } } -void +static void new_client(int fd, int socktype) { client_t s; From owner-svn-src-all@freebsd.org Sun Jul 29 08:43:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0931A1050640; Sun, 29 Jul 2018 08:43:10 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 75D4F77485; Sun, 29 Jul 2018 08:43:09 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 10FC7137E2; Sun, 29 Jul 2018 08:43:09 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6T8h89F010713; Sun, 29 Jul 2018 08:43:08 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6T8h8jY010712; Sun, 29 Jul 2018 08:43:08 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807290843.w6T8h8jY010712@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 08:43:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336867 - head/sbin/devd X-SVN-Group: head X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: head/sbin/devd X-SVN-Commit-Revision: 336867 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 08:43:10 -0000 Author: eadler Date: Sun Jul 29 08:43:08 2018 New Revision: 336867 URL: https://svnweb.freebsd.org/changeset/base/336867 Log: devd: use C++ style casts Modified: head/sbin/devd/devd.cc Modified: head/sbin/devd/devd.cc ============================================================================== --- head/sbin/devd/devd.cc Sun Jul 29 08:43:05 2018 (r336866) +++ head/sbin/devd/devd.cc Sun Jul 29 08:43:08 2018 (r336867) @@ -262,12 +262,12 @@ my_system(const char *command) */ cfg.close_pidfile(); ::closefrom(3); - ::execl(_PATH_BSHELL, "sh", "-c", command, (char *)NULL); + ::execl(_PATH_BSHELL, "sh", "-c", command, nullptr); ::_exit(127); default: /* parent */ savedpid = pid; do { - pid = ::wait4(savedpid, &pstat, 0, (struct rusage *)0); + pid = ::wait4(savedpid, &pstat, 0, nullptr); } while (pid == -1 && errno == EINTR); break; } @@ -374,7 +374,7 @@ media::do_match(config &c) memset(&ifmr, 0, sizeof(ifmr)); strlcpy(ifmr.ifm_name, value.c_str(), sizeof(ifmr.ifm_name)); - if (ioctl(s, SIOCGIFMEDIA, (caddr_t)&ifmr) >= 0 && + if (ioctl(s, SIOCGIFMEDIA, &ifmr) >= 0 && ifmr.ifm_status & IFM_AVALID) { devdlog(LOG_DEBUG, "%s has media type 0x%x\n", value.c_str(), IFM_TYPE(ifmr.ifm_active)); @@ -527,7 +527,7 @@ config::open_pidfile() pfh = pidfile_open(_pidfile.c_str(), 0600, &otherpid); if (pfh == NULL) { if (errno == EEXIST) - errx(1, "devd already running, pid: %d", (int)otherpid); + errx(1, "devd already running, pid: %d", static_cast(otherpid)); warn("cannot open pid file"); } } @@ -854,7 +854,7 @@ process_event(char *buffer) // Save the time this happened (as approximated by when we got // around to processing it). gettimeofday(&tv, NULL); - asprintf(×tr, "%jd.%06ld", (uintmax_t)tv.tv_sec, tv.tv_usec); + asprintf(×tr, "%jd.%06ld", static_cast(tv.tv_sec), tv.tv_usec); cfg.set_variable("timestamp", timestr); free(timestr); @@ -919,7 +919,7 @@ create_socket(const char *name, int socktype) unlink(name); if (fcntl(fd, F_SETFL, O_NONBLOCK) < 0) err(1, "fcntl"); - if (::bind(fd, (struct sockaddr *) & sun, slen) < 0) + if (::bind(fd, reinterpret_cast(&sun), slen) < 0) err(1, "bind"); listen(fd, 4); if (chown(name, 0, 0)) /* XXX - root.wheel */ From owner-svn-src-all@freebsd.org Sun Jul 29 12:41:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B361210566B1; Sun, 29 Jul 2018 12:41:57 +0000 (UTC) (envelope-from antoine@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69D6E8000D; Sun, 29 Jul 2018 12:41:57 +0000 (UTC) (envelope-from antoine@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4C6CE15F22; Sun, 29 Jul 2018 12:41:57 +0000 (UTC) (envelope-from antoine@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TCfv3s033872; Sun, 29 Jul 2018 12:41:57 GMT (envelope-from antoine@FreeBSD.org) Received: (from antoine@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TCfuvE033869; Sun, 29 Jul 2018 12:41:56 GMT (envelope-from antoine@FreeBSD.org) Message-Id: <201807291241.w6TCfuvE033869@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: antoine set sender to antoine@FreeBSD.org using -f From: Antoine Brodin Date: Sun, 29 Jul 2018 12:41:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336868 - in head: sys/kern sys/sys usr.sbin/jail X-SVN-Group: head X-SVN-Commit-Author: antoine X-SVN-Commit-Paths: in head: sys/kern sys/sys usr.sbin/jail X-SVN-Commit-Revision: 336868 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 12:41:57 -0000 Author: antoine Date: Sun Jul 29 12:41:56 2018 New Revision: 336868 URL: https://svnweb.freebsd.org/changeset/base/336868 Log: Add allow.mlock to jail parameters It allows locking or unlocking physical pages in memory within a jail This allows running elasticsearch with "bootstrap.memory_lock" inside a jail Reviewed by: jamie@ Differential Revision: https://reviews.freebsd.org/D16342 Modified: head/sys/kern/kern_jail.c head/sys/sys/jail.h head/usr.sbin/jail/jail.8 Modified: head/sys/kern/kern_jail.c ============================================================================== --- head/sys/kern/kern_jail.c Sun Jul 29 08:43:08 2018 (r336867) +++ head/sys/kern/kern_jail.c Sun Jul 29 12:41:56 2018 (r336868) @@ -190,6 +190,7 @@ static struct bool_flags pr_flag_allow[NBBY * NBPW] = {"allow.mount", "allow.nomount", PR_ALLOW_MOUNT}, {"allow.quotas", "allow.noquotas", PR_ALLOW_QUOTAS}, {"allow.socket_af", "allow.nosocket_af", PR_ALLOW_SOCKET_AF}, + {"allow.mlock", "allow.nomlock", PR_ALLOW_MLOCK}, {"allow.reserved_ports", "allow.noreserved_ports", PR_ALLOW_RESERVED_PORTS}, }; @@ -3293,6 +3294,17 @@ prison_priv_check(struct ucred *cred, int priv) return (EPERM); /* + * Conditionnaly allow locking (unlocking) physical pages + * in memory. + */ + case PRIV_VM_MLOCK: + case PRIV_VM_MUNLOCK: + if (cred->cr_prison->pr_allow & PR_ALLOW_MLOCK) + return (0); + else + return (EPERM); + + /* * Conditionally allow jailed root to bind reserved ports. */ case PRIV_NETINET_RESERVEDPORT: @@ -3752,6 +3764,8 @@ SYSCTL_JAIL_PARAM(_allow, quotas, CTLTYPE_INT | CTLFLA "B", "Jail may set file quotas"); SYSCTL_JAIL_PARAM(_allow, socket_af, CTLTYPE_INT | CTLFLAG_RW, "B", "Jail may create sockets other than just UNIX/IPv4/IPv6/route"); +SYSCTL_JAIL_PARAM(_allow, mlock, CTLTYPE_INT | CTLFLAG_RW, + "B", "Jail may lock (unlock) physical pages in memory"); SYSCTL_JAIL_PARAM(_allow, reserved_ports, CTLTYPE_INT | CTLFLAG_RW, "B", "Jail may bind sockets to reserved ports"); Modified: head/sys/sys/jail.h ============================================================================== --- head/sys/sys/jail.h Sun Jul 29 08:43:08 2018 (r336867) +++ head/sys/sys/jail.h Sun Jul 29 12:41:56 2018 (r336868) @@ -227,9 +227,10 @@ struct prison_racct { #define PR_ALLOW_MOUNT 0x00000010 #define PR_ALLOW_QUOTAS 0x00000020 #define PR_ALLOW_SOCKET_AF 0x00000040 +#define PR_ALLOW_MLOCK 0x00000080 #define PR_ALLOW_RESERVED_PORTS 0x00008000 #define PR_ALLOW_KMEM_ACCESS 0x00010000 /* reserved, not used yet */ -#define PR_ALLOW_ALL_STATIC 0x0001807f +#define PR_ALLOW_ALL_STATIC 0x000180ff /* * OSD methods Modified: head/usr.sbin/jail/jail.8 ============================================================================== --- head/usr.sbin/jail/jail.8 Sun Jul 29 08:43:08 2018 (r336867) +++ head/usr.sbin/jail/jail.8 Sun Jul 29 12:41:56 2018 (r336868) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 20, 2018 +.Dd July 29, 2018 .Dt JAIL 8 .Os .Sh NAME @@ -553,6 +553,16 @@ with non-jailed parts of the system. Sockets within a jail are normally restricted to IPv4, IPv6, local (UNIX), and route. This allows access to other protocol stacks that have not had jail functionality added to them. +.It Va allow.mlock +Locking or unlocking physical pages in memory are normally not available +within a jail. +When this parameter is set, users may +.Xr mlock 2 +or +.Xr munlock 2 +memory subject to +.Va security.bsd.unprivileged_mlock +and resource limits. .It Va allow.reserved_ports The jail root may bind to ports lower than 1024. .El From owner-svn-src-all@freebsd.org Sun Jul 29 14:16:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D30CB1057EF5 for ; Sun, 29 Jul 2018 14:16:37 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5589582431 for ; Sun, 29 Jul 2018 14:16:37 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-RoutePath: aGlwcGll X-MHO-User: fbc23a03-9339-11e8-904b-1d2e466b3c59 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.ore.mailhop.org (Halon) with ESMTPSA id fbc23a03-9339-11e8-904b-1d2e466b3c59; Sun, 29 Jul 2018 14:16:28 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w6TEGR8m029447; Sun, 29 Jul 2018 08:16:27 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1532873787.61594.107.camel@freebsd.org> Subject: Re: svn commit: r336859 - head/sbin/init/rc.d From: Ian Lepore To: Don Lewis , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Date: Sun, 29 Jul 2018 08:16:27 -0600 In-Reply-To: <201807290542.w6T5g8rt018138@repo.freebsd.org> References: <201807290542.w6T5g8rt018138@repo.freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 14:16:38 -0000 On Sun, 2018-07-29 at 05:42 +0000, Don Lewis wrote: > Author: truckman > Date: Sun Jul 29 05:42:07 2018 > New Revision: 336859 > URL: https://svnweb.freebsd.org/changeset/base/336859 > > Log: >   Fix a variable name typo in r336845 that prevented the rc.d scripts >   from being installed in the correct directory. >    >   Resurrect a few rc.d scripts that were prematurely deleted from the >   Makefile by r336845. >    >   Reviewed by: brd > > Modified: >   head/sbin/init/rc.d/Makefile > > Modified: head/sbin/init/rc.d/Makefile > ===================================================================== > ========= > --- head/sbin/init/rc.d/Makefile Sun Jul 29 05:14:26 2018 > (r336858) > +++ head/sbin/init/rc.d/Makefile Sun Jul 29 05:42:07 2018 > (r336859) > @@ -2,7 +2,7 @@ >   >  .include >   > -CONFSDIR= /etc/rc.d > +CONFDIR= /etc/rc.d >  CONFGROUPS= CONFS >   I'm pretty sure CONFSDIR was right here. You are now trapped in a maze of small twisty variable names, all alike. CONFGROUPS=CONFS is the default set by bsd.conf.mk, and it's the only one that needs CONF to be singular. Perhaps if we establish the idiom of not re-specifying the default value in all the individual makefiles, that'll leave everything as CONFS* and it'll be a bit less confusing? -- Ian >  CONFS= DAEMON \ > @@ -75,6 +75,7 @@ CONFS= DAEMON \ >   netif \ >   netoptions \ >   netwait \ > + newsyslog \ >   nfsclient \ >   nfscbd \ >   nfsd \ > @@ -112,6 +113,8 @@ CONFS= DAEMON \ >   stf \ >   swap \ >   swaplate \ > + sysctl \ > + syslogd \ >   tmp \ >   ${_ubthidhci} \ >   ugidfw \ > From owner-svn-src-all@freebsd.org Sun Jul 29 15:34:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F483105998C; Sun, 29 Jul 2018 15:34:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 29161844D4; Sun, 29 Jul 2018 15:34:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w6TFYhYX072968 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 29 Jul 2018 18:34:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w6TFYhYX072968 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w6TFYhjK072967; Sun, 29 Jul 2018 18:34:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Jul 2018 18:34:43 +0300 From: Konstantin Belousov To: Eitan Adler Cc: src-committers , svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: Re: svn commit: r336805 - in stable/11: share/man/man4 sys/dev/vt/hw/vga Message-ID: <20180729153443.GF40119@kib.kiev.ua> References: <201807280737.w6S7b12F025452@repo.freebsd.org> <20180728233405.GD40119@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 15:34:54 -0000 On Sat, Jul 28, 2018 at 10:12:25PM -0700, Eitan Adler wrote: > On Sat, 28 Jul 2018 at 16:34, Konstantin Belousov wrote: > > > > On Sat, Jul 28, 2018 at 07:37:01AM +0000, Eitan Adler wrote: > > > Author: eadler > > > Date: Sat Jul 28 07:37:01 2018 > > > New Revision: 336805 > > > URL: https://svnweb.freebsd.org/changeset/base/336805 > > > > > > Log: > > > MFC r335629: > > > > > > vt: add option to ignore NO_VGA flag in ACPI > > > > > > To workaround buggy firmware that sets this flag when there's actually > > > a VGA present. > > > > > > Ref D16003 > > > PR: 229235 > > > > > > Modified: > > > stable/11/share/man/man4/vt.4 > > > stable/11/sys/dev/vt/hw/vga/vt_vga.c > > > Directory Properties: > > > stable/11/ (props changed) > > > > > > Modified: stable/11/share/man/man4/vt.4 > > > ============================================================================== > > > --- stable/11/share/man/man4/vt.4 Sat Jul 28 06:46:10 2018 (r336804) > > > +++ stable/11/share/man/man4/vt.4 Sat Jul 28 07:37:01 2018 (r336805) > > > @@ -44,6 +44,7 @@ > > > In > > > .Xr loader.conf 5 : > > > .Cd hw.vga.textmode=1 > > > +.Cd hw.vga.acpi_ignore_no_vga=1 > > > .Cd kern.vty=vt > > > .Cd kern.vt.color..rgb="" > > > .Cd kern.vt.fb.default_mode="x" > > > @@ -196,6 +197,10 @@ prompt or in > > > Set to 1 to use virtual terminals in text mode instead of graphics mode. > > > Features that require graphics mode, like loadable fonts, will be > > > disabled. > > > +.It Va hw.vga.acpi_ignore_no_vga > > > +Set to 1 to force the usage of the VGA driver regardless of whether > > > +ACPI IAPC_BOOT_ARCH signals no VGA support. > > > +Can be used to workaround firmware bugs in the ACPI tables. > > > .It Va kern.vty > > > Set this value to > > > .Ql vt > > > > > > Modified: stable/11/sys/dev/vt/hw/vga/vt_vga.c > > > ============================================================================== > > > --- stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 06:46:10 2018 (r336804) > > > +++ stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 07:37:01 2018 (r336805) > > > @@ -1211,6 +1211,12 @@ vga_acpi_disabled(void) > > > ACPI_TABLE_FADT *fadt; > > > vm_paddr_t physaddr; > > > uint16_t flags; > > > + int ignore; > > > + > > > + TUNABLE_INT_FETCH("hw.vga.acpi_ignore_no_vga", &ignore); > > > + > > > + if (ignore) > > > + return (false); > > > If the tunable is not set, this code evaluates the stack garbage. > > Why did you merged this without asking the author and not looking > > at the followups ? > > Please assume some level of good faith. > > I MFCed this because I was asked to here > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229235 over a month > ago. During that time no one mentioned or linked any followups. I did > look in my email for mentions of r335629 as well - of which there are > none. > > Now that you mention it, I looked at the svn log and found r335631 > which I will MFC as well. What if I did not mentioned it ? Good faith has nothing to do with not asking the author and not looking at the followups. From owner-svn-src-all@freebsd.org Sun Jul 29 15:43:00 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28B7A1059C25; Sun, 29 Jul 2018 15:43:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 926F58494E; Sun, 29 Jul 2018 15:42:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w6TFgnLG075219 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 29 Jul 2018 18:42:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w6TFgnLG075219 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w6TFgnSf075218; Sun, 29 Jul 2018 18:42:49 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Jul 2018 18:42:49 +0300 From: Konstantin Belousov To: Brad Davis Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336845 - in head: etc etc/defaults etc/rc.d sbin/init sbin/init/rc.d Message-ID: <20180729154249.GG40119@kib.kiev.ua> References: <201807282036.w6SKaNSi037944@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201807282036.w6SKaNSi037944@repo.freebsd.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 15:43:00 -0000 On Sat, Jul 28, 2018 at 08:36:23PM +0000, Brad Davis wrote: > Author: brd > Date: Sat Jul 28 20:36:23 2018 > New Revision: 336845 > URL: https://svnweb.freebsd.org/changeset/base/336845 > > Log: > Move rc startup scripts from etc/ to sbin/init/ Why all these moves are useful ? No commit message explained what is the goal of the moves, and why it is better that way comparing the existing src/etc collection. Moving startup scripts to sbin/init is at least dubious, IMO. Startup scripts has nothing to do with init(8), but this is the second-order question. From owner-svn-src-all@freebsd.org Sun Jul 29 16:55:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E41F3105B169; Sun, 29 Jul 2018 16:55:29 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B22A867B5; Sun, 29 Jul 2018 16:55:29 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 66B9C1872F; Sun, 29 Jul 2018 16:55:29 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TGtT4p064112; Sun, 29 Jul 2018 16:55:29 GMT (envelope-from ian@FreeBSD.org) Received: (from ian@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TGtTGh064110; Sun, 29 Jul 2018 16:55:29 GMT (envelope-from ian@FreeBSD.org) Message-Id: <201807291655.w6TGtTGh064110@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: ian set sender to ian@FreeBSD.org using -f From: Ian Lepore Date: Sun, 29 Jul 2018 16:55:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336869 - head/sys/dev/flash X-SVN-Group: head X-SVN-Commit-Author: ian X-SVN-Commit-Paths: head/sys/dev/flash X-SVN-Commit-Revision: 336869 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 16:55:30 -0000 Author: ian Date: Sun Jul 29 16:55:28 2018 New Revision: 336869 URL: https://svnweb.freebsd.org/changeset/base/336869 Log: The device ID tables are used only within the driver, make them static so that both of these drivers can exist in the same kernel. Modified: head/sys/dev/flash/mx25l.c head/sys/dev/flash/n25q.c Modified: head/sys/dev/flash/mx25l.c ============================================================================== --- head/sys/dev/flash/mx25l.c Sun Jul 29 12:41:56 2018 (r336868) +++ head/sys/dev/flash/mx25l.c Sun Jul 29 16:55:28 2018 (r336869) @@ -116,7 +116,7 @@ static void mx25l_strategy(struct bio *bp); static int mx25l_getattr(struct bio *bp); static void mx25l_task(void *arg); -struct mx25l_flash_ident flash_devices[] = { +static struct mx25l_flash_ident flash_devices[] = { { "en25f32", 0x1c, 0x3116, 64 * 1024, 64, FL_NONE }, { "en25p32", 0x1c, 0x2016, 64 * 1024, 64, FL_NONE }, { "en25p64", 0x1c, 0x2017, 64 * 1024, 128, FL_NONE }, Modified: head/sys/dev/flash/n25q.c ============================================================================== --- head/sys/dev/flash/n25q.c Sun Jul 29 12:41:56 2018 (r336868) +++ head/sys/dev/flash/n25q.c Sun Jul 29 16:55:28 2018 (r336869) @@ -139,7 +139,7 @@ static void n25q_strategy(struct bio *bp); static int n25q_getattr(struct bio *bp); static void n25q_task(void *arg); -struct n25q_flash_ident flash_devices[] = { +static struct n25q_flash_ident flash_devices[] = { { "n25q00", 0x20, 0xbb21, (64 * 1024), 2048, FL_ENABLE_4B_ADDR}, }; From owner-svn-src-all@freebsd.org Sun Jul 29 16:59:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDE89105B31A; Sun, 29 Jul 2018 16:59:08 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B26B86B03; Sun, 29 Jul 2018 16:59:08 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (unknown [76.212.85.177]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: truckman) by smtp.freebsd.org (Postfix) with ESMTPSA id B500222CCD; Sun, 29 Jul 2018 16:59:07 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Date: Sun, 29 Jul 2018 09:59:05 -0700 (PDT) From: Don Lewis Subject: Re: svn commit: r336859 - head/sbin/init/rc.d To: Ian Lepore cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org In-Reply-To: <1532873787.61594.107.camel@freebsd.org> Message-ID: References: <201807290542.w6T5g8rt018138@repo.freebsd.org> <1532873787.61594.107.camel@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Content-Disposition: INLINE X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 16:59:09 -0000 On 29 Jul, Ian Lepore wrote: > On Sun, 2018-07-29 at 05:42 +0000, Don Lewis wrote: >> Author: truckman >> Date: Sun Jul 29 05:42:07 2018 >> New Revision: 336859 >> URL: https://svnweb.freebsd.org/changeset/base/336859 >>=20 >> Log: >> =A0 Fix a variable name typo in r336845 that prevented the rc.d scripts >> =A0 from being installed in the correct directory. >> =A0=A0 >> =A0 Resurrect a few rc.d scripts that were prematurely deleted from the >> =A0 Makefile by r336845. >> =A0=A0 >> =A0 Reviewed by:=09brd >>=20 >> Modified: >> =A0 head/sbin/init/rc.d/Makefile >>=20 >> Modified: head/sbin/init/rc.d/Makefile >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> =3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- head/sbin/init/rc.d/Makefile=09Sun Jul 29 05:14:26 2018=09 >> (r336858) >> +++ head/sbin/init/rc.d/Makefile=09Sun Jul 29 05:42:07 2018=09 >> (r336859) >> @@ -2,7 +2,7 @@ >> =A0 >> =A0.include >> =A0 >> -CONFSDIR=3D=09/etc/rc.d >> +CONFDIR=3D=09/etc/rc.d >> =A0CONFGROUPS=3D=09CONFS >> =A0 >=20 > I'm pretty sure CONFSDIR was right here. You are now trapped in a maze > of small twisty variable names, all alike. Without this change, the rc.d scripts get installed directly under /var/tmp/temproot/etc/ and mergemaster wants to delete the scripts under /etc/rc.d. Answering yes to that mergemaster question results in an unbootable system. I see the following in /usr/share/mk/bsd.confs.mk: ${group}DIR?=3D ${CONFDIR} but I don't see ${CONFSDIR} anywhere. After this commit, mergemaster works properly again. > CONFGROUPS=3DCONFS is the default set by bsd.conf.mk, and it's the only > one that needs CONF to be singular. Perhaps if we establish the idiom > of not re-specifying the default value in all the individual makefiles, > that'll leave everything as CONFS* and it'll be a bit less confusing? >=20 > -- Ian From owner-svn-src-all@freebsd.org Sun Jul 29 17:01:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DCC32105B67A; Sun, 29 Jul 2018 17:01:28 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E17386E1A; Sun, 29 Jul 2018 17:01:28 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id E2A582156E; Sun, 29 Jul 2018 13:01:27 -0400 (EDT) Received: from web6 ([10.202.2.216]) by compute5.internal (MEProxy); Sun, 29 Jul 2018 13:01:27 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=IjOULS l7L99nCSeL4IXkr51TgIwnBBnesmm1eGClHyU=; b=vVX8mihjEiH69bGU9OKuTY 6iwi2/X805nOsWCf7qjYUA36U1OrWUQLQTXlYm3aBzApemAlZ+lNKPHUZH6BNkgZ BociJLwz9pulkvNZxC9VJneTbM+kxJ7KgdDCUUcHeRAGAIP/bEYLV1x1ucEeu3cK WbNxzlRDzUAtMNegp0ooxHUoC//JD/8fnWvoAsXXvmXufcLvIsgdK2yHqVMu0Zbm D9qbovLZm670DC0s0QRkscM9dJQxDyrpZo4qzNykeiwKv9U5z52hHt40QOy0L7gL xGAQ/cOZwpGBxtiGrr+5ku9v2vra0b0vVSINTEPXeOaZ4k7piHnFOr6pq8wsBfFQ == X-ME-Proxy: X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 7FCE141EB; Sun, 29 Jul 2018 13:01:27 -0400 (EDT) Message-Id: <1532883687.1202573.1456550816.74531130@webmail.messagingengine.com> From: Brad Davis To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-0843ff3e Subject: Re: svn commit: r336845 - in head: etc etc/defaults etc/rc.d sbin/init sbin/init/rc.d References: <201807282036.w6SKaNSi037944@repo.freebsd.org> <20180729154249.GG40119@kib.kiev.ua> Date: Sun, 29 Jul 2018 11:01:27 -0600 In-Reply-To: <20180729154249.GG40119@kib.kiev.ua> X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 17:01:29 -0000 On Sun, Jul 29, 2018, at 9:42 AM, Konstantin Belousov wrote: > On Sat, Jul 28, 2018 at 08:36:23PM +0000, Brad Davis wrote: > > Author: brd > > Date: Sat Jul 28 20:36:23 2018 > > New Revision: 336845 > > URL: https://svnweb.freebsd.org/changeset/base/336845 > > > > Log: > > Move rc startup scripts from etc/ to sbin/init/ > Why all these moves are useful ? > > No commit message explained what is the goal of the moves, and why > it is better that way comparing the existing src/etc collection. For pkg base we need to distill things out of etc/ and into places where they fit better. This enables them to be pkged up with the parts of the system them belong with and the CONFS= tags them as config files so that pkg will know about them and be able to 3-way merge them. This keeps the flexibility we have today with the many WITH_*/WITHOUT_* knobs. Though it is not without trade offs, keeping all of one part of the system together is simpler and makes some maintenance easier. > Moving startup scripts to sbin/init is at least dubious, IMO. > Startup scripts has nothing to do with init(8), but this is the > second-order question. Do you have a better idea of a place for them? This is the most logical place I could think of. Regards, Brad Davis From owner-svn-src-all@freebsd.org Sun Jul 29 17:36:00 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3FC4105BF86 for ; Sun, 29 Jul 2018 17:36:00 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4A18B885F1 for ; Sun, 29 Jul 2018 17:36:00 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-RoutePath: aGlwcGll X-MHO-User: d92369b2-9355-11e8-904b-1d2e466b3c59 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.ore.mailhop.org (Halon) with ESMTPSA id d92369b2-9355-11e8-904b-1d2e466b3c59; Sun, 29 Jul 2018 17:35:56 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w6THZtmg029792; Sun, 29 Jul 2018 11:35:55 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1532885755.61594.122.camel@freebsd.org> Subject: Re: svn commit: r336859 - head/sbin/init/rc.d From: Ian Lepore To: Don Lewis Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Date: Sun, 29 Jul 2018 11:35:55 -0600 In-Reply-To: References: <201807290542.w6T5g8rt018138@repo.freebsd.org> <1532873787.61594.107.camel@freebsd.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 17:36:00 -0000 On Sun, 2018-07-29 at 09:59 -0700, Don Lewis wrote: > On 29 Jul, Ian Lepore wrote: > > On Sun, 2018-07-29 at 05:42 +0000, Don Lewis wrote: > >> Author: truckman > >> Date: Sun Jul 29 05:42:07 2018 > >> New Revision: 336859 > >> URL: https://svnweb.freebsd.org/changeset/base/336859 > >>  > >> Log: > >>   Fix a variable name typo in r336845 that prevented the rc.d scripts > >>   from being installed in the correct directory. > >>    > >>   Resurrect a few rc.d scripts that were prematurely deleted from the > >>   Makefile by r336845. > >>    > >>   Reviewed by:       brd > >>  > >> Modified: > >>   head/sbin/init/rc.d/Makefile > >>  > >> Modified: head/sbin/init/rc.d/Makefile > >> ===================================================================== > >> ========= > >> --- head/sbin/init/rc.d/Makefile     Sun Jul 29 05:14:26 2018         > >> (r336858) > >> +++ head/sbin/init/rc.d/Makefile     Sun Jul 29 05:42:07 2018         > >> (r336859) > >> @@ -2,7 +2,7 @@ > >>   > >>  .include > >>   > >> -CONFSDIR=   /etc/rc.d > >> +CONFDIR=    /etc/rc.d > >>  CONFGROUPS= CONFS > >>   > >  > > I'm pretty sure CONFSDIR was right here. You are now trapped in a maze > > of small twisty variable names, all alike. > > Without this change, the rc.d scripts get installed directly under > /var/tmp/temproot/etc/ and mergemaster wants to delete the scripts under > /etc/rc.d.  Answering yes to that mergemaster question results in an > unbootable system. > > I see the following in /usr/share/mk/bsd.confs.mk: >   ${group}DIR?=   ${CONFDIR} > but I don't see ${CONFSDIR} anywhere. > You don't see CONFSDIR because it appears as ${group}DIR and with CONFGROUPS?= CONFS that turns into CONFSDIR. So now the question is why doesn't setting CONFSDIR work in this makefile, but it works in others? -- Ian > After this commit, mergemaster works properly again. > > > CONFGROUPS=CONFS is the default set by bsd.conf.mk, and it's the only > > one that needs CONF to be singular. Perhaps if we establish the idiom > > of not re-specifying the default value in all the individual makefiles, > > that'll leave everything as CONFS* and it'll be a bit less confusing? > >  > > -- Ian > > From owner-svn-src-all@freebsd.org Sun Jul 29 17:45:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0012105C29F; Sun, 29 Jul 2018 17:45:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5EFB688C82; Sun, 29 Jul 2018 17:45:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w6THilP3002606 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 29 Jul 2018 20:44:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w6THilP3002606 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w6THiltg002605; Sun, 29 Jul 2018 20:44:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Jul 2018 20:44:47 +0300 From: Konstantin Belousov To: Brad Davis Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336845 - in head: etc etc/defaults etc/rc.d sbin/init sbin/init/rc.d Message-ID: <20180729174447.GJ40119@kib.kiev.ua> References: <201807282036.w6SKaNSi037944@repo.freebsd.org> <20180729154249.GG40119@kib.kiev.ua> <1532883687.1202573.1456550816.74531130@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1532883687.1202573.1456550816.74531130@webmail.messagingengine.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 17:45:03 -0000 On Sun, Jul 29, 2018 at 11:01:27AM -0600, Brad Davis wrote: > On Sun, Jul 29, 2018, at 9:42 AM, Konstantin Belousov wrote: > > On Sat, Jul 28, 2018 at 08:36:23PM +0000, Brad Davis wrote: > > > Author: brd > > > Date: Sat Jul 28 20:36:23 2018 > > > New Revision: 336845 > > > URL: https://svnweb.freebsd.org/changeset/base/336845 > > > > > > Log: > > > Move rc startup scripts from etc/ to sbin/init/ > > Why all these moves are useful ? > > > > No commit message explained what is the goal of the moves, and why > > it is better that way comparing the existing src/etc collection. > > For pkg base we need to distill things out of etc/ and into places > where they fit better. This enables them to be pkged up with the > parts of the system them belong with and the CONFS= tags them as > config files so that pkg will know about them and be able to 3-way > merge them. This keeps the flexibility we have today with the many > WITH_*/WITHOUT_* knobs. Though it is not without trade offs, keeping > all of one part of the system together is simpler and makes some > maintenance easier. I do not see why do we need this for the pkgbase. There is nothing wrong with installation of the whole clean dist-supplied /etc on the fresh system. More, as you noted yourself, the ability to have all the config files installed is simpler and allows to overview and prepare configuration without requiring to have the corresponding packages installed. > > > Moving startup scripts to sbin/init is at least dubious, IMO. > > Startup scripts has nothing to do with init(8), but this is the > > second-order question. > > Do you have a better idea of a place for them? This is the most > logical place I could think of. rc scripts are definitely not configuration files. They are executables. I proposed this several times, the /etc/rc.d and /etc/rc.* are very much wrong place to contain the scripts. For instance, /libexec/rc/ could be more logical and much more operationally correct: e.g. the scripts can be updated by make installworld (or some sub-target) and removed by the ObsoleteFiles. From owner-svn-src-all@freebsd.org Sun Jul 29 17:47:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8EBE105C44B; Sun, 29 Jul 2018 17:47:43 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 761B088F90; Sun, 29 Jul 2018 17:47:43 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 3246220F02; Sun, 29 Jul 2018 13:47:43 -0400 (EDT) Received: from web6 ([10.202.2.216]) by compute5.internal (MEProxy); Sun, 29 Jul 2018 13:47:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=C6V37t lBrM4WVVUGQniLlKYjGr/a1qkUCGwa+WXK/14=; b=qkEu8cEteuIb65iWpVsEfw jtF5I2cEhDOOS6/94O4bcxpPTYRX8nsbq6djBAQ5p30p4tyeiblDtS5/LbU+FykF dd5SFADRNBrEigKZg2iwTCEQDfEK5t3x62bbEOHVawNGtlEBq5Qgj0r3wS5w9dX0 X7syG3ab5VHhOWtajzNrcs3lLmCvL/63I2LEut/gaIcdoNRCKrFfR1ci2FBR5ZQx mTVdbjmjGoNmhTtUCyMxE91bCVlSmgs1CEuRKHUBh4asgkAaDK+8euGVObylBmRw 0hm7ccMDH/NAyYGTpuQIMZc6/3+oTqf72vQ/YDUYKd2TcHzsjM56O8mb2oMsrABw == X-ME-Proxy: X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id B16E141EB; Sun, 29 Jul 2018 13:47:42 -0400 (EDT) Message-Id: <1532886462.1211531.1456577968.06E21EAE@webmail.messagingengine.com> From: Brad Davis To: Ian Lepore , Don Lewis Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-0843ff3e In-Reply-To: <1532885755.61594.122.camel@freebsd.org> Date: Sun, 29 Jul 2018 11:47:42 -0600 Subject: Re: svn commit: r336859 - head/sbin/init/rc.d References: <201807290542.w6T5g8rt018138@repo.freebsd.org> <1532873787.61594.107.camel@freebsd.org> <1532885755.61594.122.camel@freebsd.org> X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 17:47:44 -0000 On Sun, Jul 29, 2018, at 11:35 AM, Ian Lepore wrote: > On Sun, 2018-07-29 at 09:59 -0700, Don Lewis wrote: > > On 29 Jul, Ian Lepore wrote: > > > On Sun, 2018-07-29 at 05:42 +0000, Don Lewis wrote: > > >> Author: truckman > > >> Date: Sun Jul 29 05:42:07 2018 > > >> New Revision: 336859 > > >> URL: https://svnweb.freebsd.org/changeset/base/336859 > > >>=C2=A0 > > >> Log: > > >> =C2=A0 Fix a variable name typo in r336845 that prevented the rc.d s= cripts > > >> =C2=A0 from being installed in the correct directory. > > >> =C2=A0=C2=A0 > > >> =C2=A0 Resurrect a few rc.d scripts that were prematurely deleted fr= om the > > >> =C2=A0 Makefile by r336845. > > >> =C2=A0=C2=A0 > > >> =C2=A0 Reviewed by:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0brd > > >>=C2=A0 > > >> Modified: > > >> =C2=A0 head/sbin/init/rc.d/Makefile > > >>=C2=A0 > > >> Modified: head/sbin/init/rc.d/Makefile > > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D > > >> --- head/sbin/init/rc.d/Makefile=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0Sun Ju= l 29 05:14:26 2018=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 > > >> (r336858) > > >> +++ head/sbin/init/rc.d/Makefile=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0Sun Ju= l 29 05:42:07 2018=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 > > >> (r336859) > > >> @@ -2,7 +2,7 @@ > > >> =C2=A0 > > >> =C2=A0.include=20 > > >> =C2=A0 > > >> -CONFSDIR=3D=C2=A0=C2=A0=C2=A0/etc/rc.d > > >> +CONFDIR=3D=C2=A0=C2=A0=C2=A0=C2=A0/etc/rc.d > > >> =C2=A0CONFGROUPS=3D=C2=A0CONFS > > >> =C2=A0 > > >=C2=A0 > > > I'm pretty sure CONFSDIR was right here. You are now trapped in a maze > > > of small twisty variable names, all alike. > >=20 > > Without this change, the rc.d scripts get installed directly under > > /var/tmp/temproot/etc/ and mergemaster wants to delete the scripts under > > /etc/rc.d.=C2=A0 Answering yes to that mergemaster question results in = an > > unbootable system. > >=20 > > I see the following in /usr/share/mk/bsd.confs.mk: > > =C2=A0 ${group}DIR?=3D=C2=A0=C2=A0 ${CONFDIR} > > but I don't see ${CONFSDIR} anywhere. > >=20 >=20 > You don't see CONFSDIR because it appears as ${group}DIR and with > CONFGROUPS?=3D CONFS that turns into CONFSDIR. So now the question is why > doesn't setting CONFSDIR work in this makefile, but it works in others? >=20 > -- Ian >=20 > > After this commit, mergemaster works properly again. > >=20 > > > CONFGROUPS=3DCONFS is the default set by bsd.conf.mk, and it's the on= ly > > > one that needs CONF to be singular. Perhaps if we establish the idiom > > > of not re-specifying the default value in all the individual makefile= s, > > > that'll leave everything as CONFS* and it'll be a bit less confusing? I spoke with Ian on IRC about this, but for everyone else following along h= ere.. This works because CONFDIR is the default for all CONFGROUPS, where as CONF= SDIR only applied to the CONFS group. Regards, Brad Davis From owner-svn-src-all@freebsd.org Sun Jul 29 17:51:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A8A6105C629; Sun, 29 Jul 2018 17:51:33 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0233F89344; Sun, 29 Jul 2018 17:51:32 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id AE10B20CA5; Sun, 29 Jul 2018 13:51:32 -0400 (EDT) Received: from web6 ([10.202.2.216]) by compute5.internal (MEProxy); Sun, 29 Jul 2018 13:51:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=OHOqes RkyW/0gfKw8MZsUiZVQOCFGQctsOt/tJlFC98=; b=i2rMurxmBbhqTDReOevcl2 +ZHFGwDNmZTrGWb3cmaZzS0oT3/uMTC31iuhD2DT4oaCx3PzD87hd9cgqLQo53CV grUyOQCHLiNWQWCN5Ll8bRuJTse8c6FROHLR/b3JbrCcGkSAs45/H3UspyshIGYv w7ZKLEjib4g+9hVgbW74YD7ncXefpnSzd8IUiqT9fs+mRoULNasL9KOYxibt4/se Ipezq89ebAhbEYzKO54rMxCbT3pviMqUVt8H0hEfGdAOjgYXwxVBDLcrpxcBQpk7 ay6+FIEnMy5A1lS2JN04IbZ//iTgAWYjfxmsE4eaD3zDWWa8Vj1lccaGZzoMSZZg == X-ME-Proxy: X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 3E8E941EB; Sun, 29 Jul 2018 13:51:32 -0400 (EDT) Message-Id: <1532886692.1212563.1456581312.54C4C94B@webmail.messagingengine.com> From: Brad Davis To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-0843ff3e Date: Sun, 29 Jul 2018 11:51:32 -0600 In-Reply-To: <20180729174447.GJ40119@kib.kiev.ua> References: <201807282036.w6SKaNSi037944@repo.freebsd.org> <20180729154249.GG40119@kib.kiev.ua> <1532883687.1202573.1456550816.74531130@webmail.messagingengine.com> <20180729174447.GJ40119@kib.kiev.ua> Subject: Re: svn commit: r336845 - in head: etc etc/defaults etc/rc.d sbin/init sbin/init/rc.d X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 17:51:33 -0000 On Sun, Jul 29, 2018, at 11:44 AM, Konstantin Belousov wrote: > On Sun, Jul 29, 2018 at 11:01:27AM -0600, Brad Davis wrote: > > On Sun, Jul 29, 2018, at 9:42 AM, Konstantin Belousov wrote: > > > On Sat, Jul 28, 2018 at 08:36:23PM +0000, Brad Davis wrote: > > > > Author: brd > > > > Date: Sat Jul 28 20:36:23 2018 > > > > New Revision: 336845 > > > > URL: https://svnweb.freebsd.org/changeset/base/336845 > > > > > > > > Log: > > > > Move rc startup scripts from etc/ to sbin/init/ > > > Why all these moves are useful ? > > > > > > No commit message explained what is the goal of the moves, and why > > > it is better that way comparing the existing src/etc collection. > > > > For pkg base we need to distill things out of etc/ and into places > > where they fit better. This enables them to be pkged up with the > > parts of the system them belong with and the CONFS= tags them as > > config files so that pkg will know about them and be able to 3-way > > merge them. This keeps the flexibility we have today with the many > > WITH_*/WITHOUT_* knobs. Though it is not without trade offs, keeping > > all of one part of the system together is simpler and makes some > > maintenance easier. > I do not see why do we need this for the pkgbase. There is nothing > wrong with installation of the whole clean dist-supplied /etc on > the fresh system. More, as you noted yourself, the ability to have > all the config files installed is simpler and allows to overview and > prepare configuration without requiring to have the corresponding > packages installed. On a fresh system sure, but on upgrades that is a problem. I was not advocating for having all the etc files installed, even when you do not have that pkg installed. I am saying it matches what happens today if you have, for example, WITHOUT_ACPI=y set. > > > Moving startup scripts to sbin/init is at least dubious, IMO. > > > Startup scripts has nothing to do with init(8), but this is the > > > second-order question. > > > > Do you have a better idea of a place for them? This is the most > > logical place I could think of. > rc scripts are definitely not configuration files. They are executables. > > I proposed this several times, the /etc/rc.d and /etc/rc.* are very much > wrong place to contain the scripts. For instance, /libexec/rc/ could be > more logical and much more operationally correct: e.g. the scripts can > be updated by make installworld (or some sub-target) and removed by the > ObsoleteFiles. I agree with that, but was shot down as too big of a POLA violation on this list back during BSDCan. I think it is something we can tackle, but I do not want it to hold up pkgbase so I am not willing to merge the two projects. Regards, Brad Davis From owner-svn-src-all@freebsd.org Sun Jul 29 18:22:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECE51105D19A; Sun, 29 Jul 2018 18:22:26 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1ED18A45F; Sun, 29 Jul 2018 18:22:26 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8316619568; Sun, 29 Jul 2018 18:22:26 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TIMQcV010727; Sun, 29 Jul 2018 18:22:26 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TIMQhG010726; Sun, 29 Jul 2018 18:22:26 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201807291822.w6TIMQhG010726@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Sun, 29 Jul 2018 18:22:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336871 - head/sys/compat/freebsd32 X-SVN-Group: head X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: head/sys/compat/freebsd32 X-SVN-Commit-Revision: 336871 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 18:22:27 -0000 Author: asomers Date: Sun Jul 29 18:22:26 2018 New Revision: 336871 URL: https://svnweb.freebsd.org/changeset/base/336871 Log: getrusage(2): fix return value under 32-bit emulation According to the man page, getrusage(2) should return EFAULT if the rusage argument lies outside of the process's address space. But due to an oversight in r100384, that's never been the case during 32-bit emulation. Fix it. PR: 230153 Reported by: tests(7) Reviewed by: cem MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D16500 Modified: head/sys/compat/freebsd32/freebsd32_misc.c Modified: head/sys/compat/freebsd32/freebsd32_misc.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_misc.c Sun Jul 29 17:27:10 2018 (r336870) +++ head/sys/compat/freebsd32/freebsd32_misc.c Sun Jul 29 18:22:26 2018 (r336871) @@ -883,12 +883,9 @@ freebsd32_getrusage(struct thread *td, struct freebsd3 int error; error = kern_getrusage(td, uap->who, &s); - if (error) - return (error); - if (uap->rusage != NULL) { - freebsd32_rusage_out(&s, &s32); + freebsd32_rusage_out(&s, &s32); + if (error == 0) error = copyout(&s32, uap->rusage, sizeof(s32)); - } return (error); } From owner-svn-src-all@freebsd.org Sun Jul 29 18:33:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A08C105D52C; Sun, 29 Jul 2018 18:33:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F2AE8AA54; Sun, 29 Jul 2018 18:33:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w6TIX7Db014782 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 29 Jul 2018 21:33:10 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w6TIX7Db014782 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w6TIX7kf014781; Sun, 29 Jul 2018 21:33:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 29 Jul 2018 21:33:07 +0300 From: Konstantin Belousov To: Brad Davis Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336845 - in head: etc etc/defaults etc/rc.d sbin/init sbin/init/rc.d Message-ID: <20180729183307.GK40119@kib.kiev.ua> References: <201807282036.w6SKaNSi037944@repo.freebsd.org> <20180729154249.GG40119@kib.kiev.ua> <1532883687.1202573.1456550816.74531130@webmail.messagingengine.com> <20180729174447.GJ40119@kib.kiev.ua> <1532886692.1212563.1456581312.54C4C94B@webmail.messagingengine.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1532886692.1212563.1456581312.54C4C94B@webmail.messagingengine.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 18:33:18 -0000 On Sun, Jul 29, 2018 at 11:51:32AM -0600, Brad Davis wrote: > On Sun, Jul 29, 2018, at 11:44 AM, Konstantin Belousov wrote: > > On Sun, Jul 29, 2018 at 11:01:27AM -0600, Brad Davis wrote: > > > On Sun, Jul 29, 2018, at 9:42 AM, Konstantin Belousov wrote: > > > > On Sat, Jul 28, 2018 at 08:36:23PM +0000, Brad Davis wrote: > > > > > Author: brd > > > > > Date: Sat Jul 28 20:36:23 2018 > > > > > New Revision: 336845 > > > > > URL: https://svnweb.freebsd.org/changeset/base/336845 > > > > > > > > > > Log: > > > > > Move rc startup scripts from etc/ to sbin/init/ > > > > Why all these moves are useful ? > > > > > > > > No commit message explained what is the goal of the moves, and why > > > > it is better that way comparing the existing src/etc collection. > > > > > > For pkg base we need to distill things out of etc/ and into places > > > where they fit better. This enables them to be pkged up with the > > > parts of the system them belong with and the CONFS= tags them as > > > config files so that pkg will know about them and be able to 3-way > > > merge them. This keeps the flexibility we have today with the many > > > WITH_*/WITHOUT_* knobs. Though it is not without trade offs, keeping > > > all of one part of the system together is simpler and makes some > > > maintenance easier. > > I do not see why do we need this for the pkgbase. There is nothing > > wrong with installation of the whole clean dist-supplied /etc on > > the fresh system. More, as you noted yourself, the ability to have > > all the config files installed is simpler and allows to overview and > > prepare configuration without requiring to have the corresponding > > packages installed. > > On a fresh system sure, but on upgrades that is a problem. > > I was not advocating for having all the etc files installed, even when you do not have that pkg installed. I am saying it matches what happens today if you have, for example, WITHOUT_ACPI=y set. > All of this should have been written, perhaps even with more details, in the commit message. Still, sbin/init/rc is nonsence. > > > > Moving startup scripts to sbin/init is at least dubious, IMO. > > > > Startup scripts has nothing to do with init(8), but this is the > > > > second-order question. > > > > > > Do you have a better idea of a place for them? This is the most > > > logical place I could think of. > > rc scripts are definitely not configuration files. They are executables. > > > > I proposed this several times, the /etc/rc.d and /etc/rc.* are very much > > wrong place to contain the scripts. For instance, /libexec/rc/ could be > > more logical and much more operationally correct: e.g. the scripts can > > be updated by make installworld (or some sub-target) and removed by the > > ObsoleteFiles. > > I agree with that, but was shot down as too big of a POLA violation on this list back during BSDCan. > > I think it is something we can tackle, but I do not want it to hold up pkgbase so I am not willing to merge the two projects. > > > Regards, > Brad Davis From owner-svn-src-all@freebsd.org Sun Jul 29 19:11:24 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B166105E916; Sun, 29 Jul 2018 19:11:24 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF02B8D7A1; Sun, 29 Jul 2018 19:11:22 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BA95D19C5B; Sun, 29 Jul 2018 19:11:21 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TJBL12038368; Sun, 29 Jul 2018 19:11:21 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TJBLr5038367; Sun, 29 Jul 2018 19:11:21 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201807291911.w6TJBLr5038367@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Sun, 29 Jul 2018 19:11:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336872 - stable/11/lib/libproc/tests X-SVN-Group: stable-11 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: stable/11/lib/libproc/tests X-SVN-Commit-Revision: 336872 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 19:11:24 -0000 Author: markj Date: Sun Jul 29 19:11:21 2018 New Revision: 336872 URL: https://svnweb.freebsd.org/changeset/base/336872 Log: MFC r336591: Disable optimization of the libproc test program. Modified: stable/11/lib/libproc/tests/Makefile Directory Properties: stable/11/ (props changed) Modified: stable/11/lib/libproc/tests/Makefile ============================================================================== --- stable/11/lib/libproc/tests/Makefile Sun Jul 29 18:22:26 2018 (r336871) +++ stable/11/lib/libproc/tests/Makefile Sun Jul 29 19:11:21 2018 (r336872) @@ -12,5 +12,6 @@ LIBADD= elf proc rtld_db util # Ensure that symbols aren't stripped from the test program, as they're needed # for testing symbol lookup. STRIP= +CFLAGS.target_prog.c+= -O0 .include From owner-svn-src-all@freebsd.org Sun Jul 29 19:12:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1846A105EAE2; Sun, 29 Jul 2018 19:12:08 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF1888E07F; Sun, 29 Jul 2018 19:12:07 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CB06D19D97; Sun, 29 Jul 2018 19:12:06 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TJC67H042635; Sun, 29 Jul 2018 19:12:06 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TJC65n042634; Sun, 29 Jul 2018 19:12:06 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201807291912.w6TJC65n042634@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Sun, 29 Jul 2018 19:12:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336873 - stable/11/tests/sys/kern X-SVN-Group: stable-11 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: stable/11/tests/sys/kern X-SVN-Commit-Revision: 336873 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 19:12:08 -0000 Author: markj Date: Sun Jul 29 19:12:06 2018 New Revision: 336873 URL: https://svnweb.freebsd.org/changeset/base/336873 Log: MFC r336614: Add a regression test for PR 131876. PR: 131876 Modified: stable/11/tests/sys/kern/unix_passfd_test.c Directory Properties: stable/11/ (props changed) Modified: stable/11/tests/sys/kern/unix_passfd_test.c ============================================================================== --- stable/11/tests/sys/kern/unix_passfd_test.c Sun Jul 29 19:11:21 2018 (r336872) +++ stable/11/tests/sys/kern/unix_passfd_test.c Sun Jul 29 19:12:06 2018 (r336873) @@ -23,10 +23,11 @@ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. - * - * $FreeBSD$ */ +#include +__FBSDID("$FreeBSD$"); + #include #include #include @@ -100,6 +101,23 @@ dofstat(int fd, struct stat *sb) "fstat failed: %s", strerror(errno)); } +static int +getnfds(void) +{ + size_t len; + int mib[4], n, rc; + + len = sizeof(n); + mib[0] = CTL_KERN; + mib[1] = KERN_PROC; + mib[2] = KERN_PROC_NFDS; + mib[3] = 0; + + rc = sysctl(mib, 4, &n, &len, NULL, 0); + ATF_REQUIRE_MSG(rc != -1, "sysctl(KERN_PROC_NFDS) failed"); + return (n); +} + static void samefile(struct stat *sb1, struct stat *sb2) { @@ -129,7 +147,7 @@ sendfd_payload(int sockfd, int send_fd, void *payload, msghdr.msg_iov = &iovec; msghdr.msg_iovlen = 1; - cmsghdr = (struct cmsghdr *)(void*)message; + cmsghdr = (struct cmsghdr *)(void *)message; cmsghdr->cmsg_len = CMSG_LEN(sizeof(int)); cmsghdr->cmsg_level = SOL_SOCKET; cmsghdr->cmsg_type = SCM_RIGHTS; @@ -380,6 +398,55 @@ ATF_TC_BODY(rights_creds_payload, tc) closesocketpair(fd); } +/* + * Test for PR 131876. Receiver uses a control message buffer that is too + * small for the incoming SCM_RIGHTS message, so the message is truncated. + * The kernel must not leak the copied right into the receiver's namespace. + */ +ATF_TC_WITHOUT_HEAD(truncated_rights); +ATF_TC_BODY(truncated_rights, tc) +{ + struct iovec iovec; + struct msghdr msghdr; + char buf[16], message[CMSG_SPACE(0)]; + ssize_t len; + int fd[2], nfds, putfd; + + atf_tc_expect_fail("PR 131876: " + "FD leak when 'control' message is truncated"); + + memset(buf, 42, sizeof(buf)); + domainsocketpair(fd); + devnull(&putfd); + nfds = getnfds(); + + sendfd_payload(fd[0], putfd, buf, sizeof(buf)); + + bzero(&msghdr, sizeof(msghdr)); + bzero(message, sizeof(message)); + + iovec.iov_base = buf; + iovec.iov_len = sizeof(buf); + msghdr.msg_control = message; + msghdr.msg_controllen = sizeof(message); + msghdr.msg_iov = &iovec; + msghdr.msg_iovlen = 1; + + len = recvmsg(fd[1], &msghdr, 0); + ATF_REQUIRE_MSG(len != -1, "recvmsg failed: %s", strerror(errno)); + ATF_REQUIRE_MSG((size_t)len == sizeof(buf), + "recvmsg: %zd bytes received; expected %zd", len, sizeof(buf)); + for (size_t i = 0; i < sizeof(buf); i++) + ATF_REQUIRE_MSG(buf[i] == 42, "unexpected buffer contents"); + + ATF_REQUIRE_MSG((msghdr.msg_flags & MSG_CTRUNC) != 0, + "MSG_CTRUNC not set after truncation"); + ATF_REQUIRE(getnfds() == nfds); + + close(putfd); + closesocketpair(fd); +} + ATF_TP_ADD_TCS(tp) { @@ -391,6 +458,7 @@ ATF_TP_ADD_TCS(tp) ATF_TP_ADD_TC(tp, bundle_cancel); ATF_TP_ADD_TC(tp, devfs_orphan); ATF_TP_ADD_TC(tp, rights_creds_payload); + ATF_TP_ADD_TC(tp, truncated_rights); return (atf_no_error()); } From owner-svn-src-all@freebsd.org Sun Jul 29 19:18:39 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9ECA4105F8ED; Sun, 29 Jul 2018 19:18:39 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3368D712AC; Sun, 29 Jul 2018 19:18:39 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id BBC8A219E6; Sun, 29 Jul 2018 15:18:38 -0400 (EDT) Received: from web6 ([10.202.2.216]) by compute5.internal (MEProxy); Sun, 29 Jul 2018 15:18:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=CIng86 wV/AGUR0I76m2sOvxN7l3UglvIxFikikTth40=; b=bt//Fx626ydt2ZZM4N+4UC Z8NwkQS1P1BANbhV7YS2xxen3NKjgw8PySJDkvtVaSYbJcuvdO+jnLadcN9E2Clp AwxALTE/ksAOtQLBa85f6h+fqJOzMYDZRWWpmgSg4XpHXmp8/PNN8116F2k0vwfM sVlPz+tbZcHCc2KZc1loHQhbtQBiDDAwA6bsUN+UTCCwAcc1C4EEthqAAms8Zc5M XX0OohXEMgfpCK2BilpSBPK4DDMneBRvIW1qOCoPHfNLSC8RVzLoeE085fBKoXs2 jN8qbEHw34wrxJPk2Fs+un9qK4sP31fg+TxuK0VvW1Y2bwhAYK7+DYWSh0nuCgmg == X-ME-Proxy: X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 54B624274; Sun, 29 Jul 2018 15:18:38 -0400 (EDT) Message-Id: <1532891918.1231266.1456630544.3F954111@webmail.messagingengine.com> From: Brad Davis To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-0843ff3e References: <201807282036.w6SKaNSi037944@repo.freebsd.org> <20180729154249.GG40119@kib.kiev.ua> <1532883687.1202573.1456550816.74531130@webmail.messagingengine.com> <20180729174447.GJ40119@kib.kiev.ua> <1532886692.1212563.1456581312.54C4C94B@webmail.messagingengine.com> <20180729183307.GK40119@kib.kiev.ua> In-Reply-To: <20180729183307.GK40119@kib.kiev.ua> Subject: Re: svn commit: r336845 - in head: etc etc/defaults etc/rc.d sbin/init sbin/init/rc.d Date: Sun, 29 Jul 2018 13:18:38 -0600 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 19:18:40 -0000 On Sun, Jul 29, 2018, at 12:33 PM, Konstantin Belousov wrote: > On Sun, Jul 29, 2018 at 11:51:32AM -0600, Brad Davis wrote: > > On Sun, Jul 29, 2018, at 11:44 AM, Konstantin Belousov wrote: > > > On Sun, Jul 29, 2018 at 11:01:27AM -0600, Brad Davis wrote: > > > > On Sun, Jul 29, 2018, at 9:42 AM, Konstantin Belousov wrote: > > > > > On Sat, Jul 28, 2018 at 08:36:23PM +0000, Brad Davis wrote: > > > > > > Author: brd > > > > > > Date: Sat Jul 28 20:36:23 2018 > > > > > > New Revision: 336845 > > > > > > URL: https://svnweb.freebsd.org/changeset/base/336845 > > > > > > > > > > > > Log: > > > > > > Move rc startup scripts from etc/ to sbin/init/ > > > > > Why all these moves are useful ? > > > > > > > > > > No commit message explained what is the goal of the moves, and why > > > > > it is better that way comparing the existing src/etc collection. > > > > > > > > For pkg base we need to distill things out of etc/ and into places > > > > where they fit better. This enables them to be pkged up with the > > > > parts of the system them belong with and the CONFS= tags them as > > > > config files so that pkg will know about them and be able to 3-way > > > > merge them. This keeps the flexibility we have today with the many > > > > WITH_*/WITHOUT_* knobs. Though it is not without trade offs, keeping > > > > all of one part of the system together is simpler and makes some > > > > maintenance easier. > > > I do not see why do we need this for the pkgbase. There is nothing > > > wrong with installation of the whole clean dist-supplied /etc on > > > the fresh system. More, as you noted yourself, the ability to have > > > all the config files installed is simpler and allows to overview and > > > prepare configuration without requiring to have the corresponding > > > packages installed. > > > > On a fresh system sure, but on upgrades that is a problem. > > > > I was not advocating for having all the etc files installed, even when you do not have that pkg installed. I am saying it matches what happens today if you have, for example, WITHOUT_ACPI=y set. > > > All of this should have been written, perhaps even with more details, > in the commit message. You are right, I will working on improving my commit messages. Regards, Brad Davis From owner-svn-src-all@freebsd.org Sun Jul 29 19:20:13 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E126A105FAF3; Sun, 29 Jul 2018 19:20:13 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95C51718ED; Sun, 29 Jul 2018 19:20:13 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7883F19DF0; Sun, 29 Jul 2018 19:20:13 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TJKDcp049992; Sun, 29 Jul 2018 19:20:13 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TJKDpK049991; Sun, 29 Jul 2018 19:20:13 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201807291920.w6TJKDpK049991@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Sun, 29 Jul 2018 19:20:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336874 - head/sys/compat/freebsd32 X-SVN-Group: head X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: head/sys/compat/freebsd32 X-SVN-Commit-Revision: 336874 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 19:20:14 -0000 Author: asomers Date: Sun Jul 29 19:20:13 2018 New Revision: 336874 URL: https://svnweb.freebsd.org/changeset/base/336874 Log: freebsd32_getrusage(2): skip freebsd32_rusage_out on error PR: 230153 Reported by: kib MFC after: 2 weeks X-MFC-With: 336871 Differential Revision: https://reviews.freebsd.org/D16500 Modified: head/sys/compat/freebsd32/freebsd32_misc.c Modified: head/sys/compat/freebsd32/freebsd32_misc.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_misc.c Sun Jul 29 19:12:06 2018 (r336873) +++ head/sys/compat/freebsd32/freebsd32_misc.c Sun Jul 29 19:20:13 2018 (r336874) @@ -883,9 +883,10 @@ freebsd32_getrusage(struct thread *td, struct freebsd3 int error; error = kern_getrusage(td, uap->who, &s); - freebsd32_rusage_out(&s, &s32); - if (error == 0) + if (error == 0) { + freebsd32_rusage_out(&s, &s32); error = copyout(&s32, uap->rusage, sizeof(s32)); + } return (error); } From owner-svn-src-all@freebsd.org Sun Jul 29 19:33:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E414910610F2; Sun, 29 Jul 2018 19:33:27 +0000 (UTC) (envelope-from rpokala@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E94776822; Sun, 29 Jul 2018 19:33:27 +0000 (UTC) (envelope-from rpokala@freebsd.org) Received: from [192.168.1.4] (c-71-198-162-232.hsd1.ca.comcast.net [71.198.162.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: rpokala) by smtp.freebsd.org (Postfix) with ESMTPSA id BAAA223D00; Sun, 29 Jul 2018 19:33:26 +0000 (UTC) (envelope-from rpokala@freebsd.org) User-Agent: Microsoft-MacOutlook/10.f.0.180709 Date: Sun, 29 Jul 2018 12:33:23 -0700 Subject: Re: svn commit: r336526 - head From: Ravi Pokala To: Will Andrews CC: Ian Lepore , "src-committers@FreeBSD.org" , , Message-ID: <4BC0DA67-495D-4029-A9CA-18407A1B9746@panasas.com> Thread-Topic: svn commit: r336526 - head References: <201807200044.w6K0i4QQ079894@repo.freebsd.org> <1532108303.1344.57.camel@freebsd.org> <67239C31-11DA-43E0-B282-5AE894C2B0CB@panasas.com> <0178BAB4-FEFB-473B-9245-3F11ABDA8C09@freebsd.org> In-Reply-To: Mime-version: 1.0 Content-type: text/plain; charset="UTF-8" Content-transfer-encoding: 7bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 19:33:28 -0000 -----Original Message----- From: Will Andrews Date: 2018-07-28, Saturday at 18:44 To: Ravi Pokala Cc: Ian Lepore , "src-committers@FreeBSD.org" , , Subject: Re: svn commit: r336526 - head > Hi Ravi, > Thanks for taking an initial stab, I applied your patch and quickly realized the problem is deeper than it looks. This is because installworld and the second stage of mergemaster/etcupdate require the "pre" phase to take effect in their view of the system (specifically, the "ntpd" user must exist from their /etc/passwd, etc). For traditional upgrades, this occurs by simply rebooting. > > I've committed a revised version of your original patch, and have upgraded across this barrier using beinstall. Please give it a try and let me know how it goes. > > Thanks! > --Will. It worked! Thanks so much for following up on this, Will! :-) -Ravi (rpokala@) On Wed, Jul 25, 2018 at 11:28 PM, Ravi Pokala wrote: -----Original Message----- From: Ravi Pokala Date: 2018-07-20, Friday at 10:53 To: Ian Lepore , , , , Will Andrews Subject: Re: svn commit: r336526 - head > -----Original Message----- > From: on behalf of Ian Lepore > Date: 2018-07-20, Friday at 10:38 > To: Ravi Pokala , , , > Subject: Re: svn commit: r336526 - head > >> On Fri, 2018-07-20 at 09:56 -0700, Ravi Pokala wrote: >>> Hi Ian, >>> >>> -----Original Message----- >>> From: on behalf of Ian Lepore >>> Date: 2018-07-19, Thursday at 17:44 >>> To: , , >>> Subject: svn commit: r336526 - head >>> >>>> >>>> Author: ian >>>> Date: Fri Jul 20 00:44:04 2018 >>>> New Revision: 336526 >>>> URL: https://svnweb.freebsd.org/changeset/base/336526 >>>> >>>> Log: >>>> Add ntpd to the list of users/groups to check before installing. >>>> >>>> Modified: >>>> head/Makefile.inc1 >>> When trying to installworld: >>> >>> ERROR: Required ntpd user is missing, see /usr/src/UPDATING. >>> >>> But there is no UPDATING entry about this. (Also, shouldn't it be ${SRCTOP}/UPDATING?) >>> >>> Thanks, >>> >>> Ravi (rpokala@) >> >> I wondered about that too, when I accidentally stumbled across the >> CHECK_UIDS thing and realized I should add ntpd to it. I searched in >> UPDATING for some of the other things in the CHECK_UIDS list to see >> what had been said about such changes in the past, even searching >> through old-old svn history, and found nothing. >> >> I finally settled on the thought that the reference to UPDATING was to >> the stuff at the bottom that never changes. In particular to the >> section around line 1595 that documents the instalkernel, reboot, >> mergemaster, installworld sequence. The mergemaster part should take >> care of the new userid, I guess. >> >> -- Ian > > Interesting. I was attempting the install via the `beinstall.sh' script. CCing will@, since he wrote it. > > Thanks, > > Ravi (rpokala@) After some discussion on IRC, I created this patch to fix `beinstall.sh'. https://people.freebsd.org/~rpokala/beinstall.patch However, I'm a bit afraid of testing it, until someone who actually knows `etcupdate' and `mergemaster' can confirm that it won't blow up my box. :-) Can anyone take a look and assuage my fears? Thanks, Ravi (rpokala@) From owner-svn-src-all@freebsd.org Sun Jul 29 20:34:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 660961062754; Sun, 29 Jul 2018 20:34:45 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 13DD2784BE; Sun, 29 Jul 2018 20:34:45 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E4C611AAFD; Sun, 29 Jul 2018 20:34:44 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TKYixn099377; Sun, 29 Jul 2018 20:34:44 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TKYitl099375; Sun, 29 Jul 2018 20:34:44 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201807292034.w6TKYitl099375@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Sun, 29 Jul 2018 20:34:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336875 - head/tests/sys/audit X-SVN-Group: head X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: head/tests/sys/audit X-SVN-Commit-Revision: 336875 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 20:34:45 -0000 Author: asomers Date: Sun Jul 29 20:34:44 2018 New Revision: 336875 URL: https://svnweb.freebsd.org/changeset/base/336875 Log: audit(4): add tests for sysctl(3) and sysarch(2) Submitted by: aniketp MFC after: 2 weeks Sponsored by: Google, Inc. (GSoC 2018) Differential Revision: https://reviews.freebsd.org/D16116 Added: head/tests/sys/audit/miscellaneous.c (contents, props changed) Modified: head/tests/sys/audit/Makefile Modified: head/tests/sys/audit/Makefile ============================================================================== --- head/tests/sys/audit/Makefile Sun Jul 29 19:20:13 2018 (r336874) +++ head/tests/sys/audit/Makefile Sun Jul 29 20:34:44 2018 (r336875) @@ -15,6 +15,7 @@ ATF_TESTS_C+= network ATF_TESTS_C+= inter-process ATF_TESTS_C+= administrative ATF_TESTS_C+= process-control +ATF_TESTS_C+= miscellaneous SRCS.file-attribute-access+= file-attribute-access.c SRCS.file-attribute-access+= utils.c @@ -42,6 +43,8 @@ SRCS.administrative+= administrative.c SRCS.administrative+= utils.c SRCS.process-control+= process-control.c SRCS.process-control+= utils.c +SRCS.miscellaneous+= miscellaneous.c +SRCS.miscellaneous+= utils.c TEST_METADATA+= timeout="30" TEST_METADATA+= required_user="root" Added: head/tests/sys/audit/miscellaneous.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/tests/sys/audit/miscellaneous.c Sun Jul 29 20:34:44 2018 (r336875) @@ -0,0 +1,232 @@ +/*- + * Copyright (c) 2018 Aniket Pandey + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +#include + +#include +#include + +#include +#include + +#include "utils.h" + +static pid_t pid; +static char miscreg[80]; +static struct pollfd fds[1]; +static const char *auclass = "ot"; + + +/* + * Success case of audit(2) is skipped for now as the behaviour is quite + * undeterministic. It will be added when the intermittency is resolved. + */ + + +ATF_TC_WITH_CLEANUP(audit_failure); +ATF_TC_HEAD(audit_failure, tc) +{ + atf_tc_set_md_var(tc, "descr", "Tests the audit of an unsuccessful " + "audit(2) call"); +} + +ATF_TC_BODY(audit_failure, tc) +{ + pid = getpid(); + snprintf(miscreg, sizeof(miscreg), "audit.*%d.*return,failure", pid); + + FILE *pipefd = setup(fds, auclass); + /* Failure reason: Invalid argument */ + ATF_REQUIRE_EQ(-1, audit(NULL, -1)); + check_audit(fds, miscreg, pipefd); +} + +ATF_TC_CLEANUP(audit_failure, tc) +{ + cleanup(); +} + + +ATF_TC_WITH_CLEANUP(sysarch_success); +ATF_TC_HEAD(sysarch_success, tc) +{ + atf_tc_set_md_var(tc, "descr", "Tests the audit of a successful " + "sysarch(2) call"); +} + +ATF_TC_BODY(sysarch_success, tc) +{ + pid = getpid(); + snprintf(miscreg, sizeof(miscreg), "sysarch.*%d.*return,success", pid); + + /* Set sysnum to the syscall corresponding to the system architecture */ +#if defined(I386_GET_IOPERM) /* i386 */ + struct i386_ioperm_args i3sysarg; + bzero(&i3sysarg, sizeof(i3sysarg)); + +#elif defined(AMD64_GET_FSBASE) /* amd64 */ + register_t amd64arg; + +#elif defined(MIPS_GET_TLS) /* MIPS */ + char *mipsarg; + +#elif defined(ARM_SYNC_ICACHE) /* ARM */ + struct arm_sync_icache_args armsysarg; + bzero(&armsysarg, sizeof(armsysarg)); + +#elif defined(SPARC_UTRAP_INSTALL) /* Sparc64 */ + struct sparc_utrap_args handler = { + .type = UT_DIVISION_BY_ZERO, + /* We don't want to change the previous handlers */ + .new_precise = (void *)UTH_NOCHANGE, + .new_deferred = (void *)UTH_NOCHANGE, + .old_precise = NULL, + .old_deferred = NULL + }; + + struct sparc_utrap_install_args sparc64arg = { + .num = ST_DIVISION_BY_ZERO, + .handlers = &handler + }; +#else + /* For PowerPC, ARM64, RISCV archs, sysarch(2) is not supported */ + atf_tc_skip("sysarch(2) is not supported for the system architecture"); +#endif + + FILE *pipefd = setup(fds, auclass); +#if defined(I386_GET_IOPERM) + ATF_REQUIRE_EQ(0, sysarch(I386_GET_IOPERM, &i3sysarg)); +#elif defined(AMD64_GET_FSBASE) + ATF_REQUIRE_EQ(0, sysarch(AMD64_GET_FSBASE, &amd64arg)); +#elif defined(MIPS_GET_TLS) + ATF_REQUIRE_EQ(0, sysarch(MIPS_GET_TLS, &mipsarg)); +#elif defined(ARM_SYNC_ICACHE) + ATF_REQUIRE_EQ(0, sysarch(ARM_SYNC_ICACHE, &armsysarg)); +#elif defined(SPARC_UTRAP_INSTALL) + ATF_REQUIRE_EQ(0, sysarch(SPARC_UTRAP_INSTALL, &sparc64arg)); +#endif + check_audit(fds, miscreg, pipefd); +} + +ATF_TC_CLEANUP(sysarch_success, tc) +{ + cleanup(); +} + + +ATF_TC_WITH_CLEANUP(sysarch_failure); +ATF_TC_HEAD(sysarch_failure, tc) +{ + atf_tc_set_md_var(tc, "descr", "Tests the audit of an unsuccessful " + "sysarch(2) call for any architecture"); +} + +ATF_TC_BODY(sysarch_failure, tc) +{ + pid = getpid(); + snprintf(miscreg, sizeof(miscreg), "sysarch.*%d.*return,failure", pid); + + FILE *pipefd = setup(fds, auclass); + /* Failure reason: Invalid argument and Bad address */ + ATF_REQUIRE_EQ(-1, sysarch(-1, NULL)); + check_audit(fds, miscreg, pipefd); +} + +ATF_TC_CLEANUP(sysarch_failure, tc) +{ + cleanup(); +} + + +ATF_TC_WITH_CLEANUP(sysctl_success); +ATF_TC_HEAD(sysctl_success, tc) +{ + atf_tc_set_md_var(tc, "descr", "Tests the audit of a successful " + "sysctl(3) call"); +} + +ATF_TC_BODY(sysctl_success, tc) +{ + int mib[2], maxproc; + size_t proclen; + + /* Set mib to retrieve the maximum number of allowed processes */ + mib[0] = CTL_KERN; + mib[1] = KERN_MAXPROC; + proclen = sizeof(maxproc); + + pid = getpid(); + snprintf(miscreg, sizeof(miscreg), "sysctl.*%d.*return,success", pid); + + FILE *pipefd = setup(fds, auclass); + ATF_REQUIRE_EQ(0, sysctl(mib, 2, &maxproc, &proclen, NULL, 0)); + check_audit(fds, miscreg, pipefd); +} + +ATF_TC_CLEANUP(sysctl_success, tc) +{ + cleanup(); +} + + +ATF_TC_WITH_CLEANUP(sysctl_failure); +ATF_TC_HEAD(sysctl_failure, tc) +{ + atf_tc_set_md_var(tc, "descr", "Tests the audit of an unsuccessful " + "sysctl(3) call"); +} + +ATF_TC_BODY(sysctl_failure, tc) +{ + pid = getpid(); + snprintf(miscreg, sizeof(miscreg), "sysctl.*%d.*return,failure", pid); + + FILE *pipefd = setup(fds, auclass); + /* Failure reason: Invalid arguments */ + ATF_REQUIRE_EQ(-1, sysctl(NULL, 0, NULL, NULL, NULL, 0)); + check_audit(fds, miscreg, pipefd); +} + +ATF_TC_CLEANUP(sysctl_failure, tc) +{ + cleanup(); +} + + +ATF_TP_ADD_TCS(tp) +{ + ATF_TP_ADD_TC(tp, audit_failure); + + ATF_TP_ADD_TC(tp, sysarch_success); + ATF_TP_ADD_TC(tp, sysarch_failure); + + ATF_TP_ADD_TC(tp, sysctl_success); + ATF_TP_ADD_TC(tp, sysctl_failure); + + return (atf_no_error()); +} From owner-svn-src-all@freebsd.org Sun Jul 29 20:47:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD5791062CAD; Sun, 29 Jul 2018 20:47:02 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 62B0678F6F; Sun, 29 Jul 2018 20:47:02 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3F85F1AC96; Sun, 29 Jul 2018 20:47:02 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TKl2Oo004701; Sun, 29 Jul 2018 20:47:02 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TKl0hV004691; Sun, 29 Jul 2018 20:47:00 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807292047.w6TKl0hV004691@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Sun, 29 Jul 2018 20:47:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336876 - in head/sys: amd64/amd64 amd64/ia32 amd64/include conf dev/hyperv/vmbus/amd64 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys: amd64/amd64 amd64/ia32 amd64/include conf dev/hyperv/vmbus/amd64 X-SVN-Commit-Revision: 336876 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 20:47:03 -0000 Author: kib Date: Sun Jul 29 20:47:00 2018 New Revision: 336876 URL: https://svnweb.freebsd.org/changeset/base/336876 Log: Use SMAP on amd64. Ifuncs selectors dispatch copyin(9) family to the suitable variant, to set rflags.AC around userspace access. Rflags.AC bit is cleared in all kernel entry points unconditionally even on machines not supporting SMAP. Reviewed by: jhb Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D13838 Added: head/sys/amd64/amd64/copyout.c (contents, props changed) Modified: head/sys/amd64/amd64/exception.S head/sys/amd64/amd64/initcpu.c head/sys/amd64/amd64/machdep.c head/sys/amd64/amd64/pmap.c head/sys/amd64/amd64/support.S head/sys/amd64/amd64/trap.c head/sys/amd64/ia32/ia32_exception.S head/sys/amd64/include/asmacros.h head/sys/conf/files.amd64 head/sys/dev/hyperv/vmbus/amd64/vmbus_vector.S Added: head/sys/amd64/amd64/copyout.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/amd64/amd64/copyout.c Sun Jul 29 20:47:00 2018 (r336876) @@ -0,0 +1,178 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD + * + * Copyright (c) 2018 The FreeBSD Foundation + * All rights reserved. + * + * This software was developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include + +#include +#include +#include + +int fubyte_nosmap(volatile const void *base); +int fubyte_smap(volatile const void *base); +DEFINE_IFUNC(, int, fubyte, (volatile const void *), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + fubyte_smap : fubyte_nosmap); +} + +int fuword16_nosmap(volatile const void *base); +int fuword16_smap(volatile const void *base); +DEFINE_IFUNC(, int, fuword16, (volatile const void *), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + fuword16_smap : fuword16_nosmap); +} + +int fueword_nosmap(volatile const void *base, long *val); +int fueword_smap(volatile const void *base, long *val); +DEFINE_IFUNC(, int, fueword, (volatile const void *, long *), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + fueword_smap : fueword_nosmap); +} +DEFINE_IFUNC(, int, fueword64, (volatile const void *, int64_t *), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + fueword_smap : fueword_nosmap); +} + +int fueword32_nosmap(volatile const void *base, int32_t *val); +int fueword32_smap(volatile const void *base, int32_t *val); +DEFINE_IFUNC(, int, fueword32, (volatile const void *, int32_t *), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + fueword32_smap : fueword32_nosmap); +} + +int subyte_nosmap(volatile void *base, int byte); +int subyte_smap(volatile void *base, int byte); +DEFINE_IFUNC(, int, subyte, (volatile void *, int), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + subyte_smap : subyte_nosmap); +} + +int suword16_nosmap(volatile void *base, int word); +int suword16_smap(volatile void *base, int word); +DEFINE_IFUNC(, int, suword16, (volatile void *, int), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + suword16_smap : suword16_nosmap); +} + +int suword32_nosmap(volatile void *base, int32_t word); +int suword32_smap(volatile void *base, int32_t word); +DEFINE_IFUNC(, int, suword32, (volatile void *, int32_t), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + suword32_smap : suword32_nosmap); +} + +int suword_nosmap(volatile void *base, long word); +int suword_smap(volatile void *base, long word); +DEFINE_IFUNC(, int, suword, (volatile void *, long), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + suword_smap : suword_nosmap); +} +DEFINE_IFUNC(, int, suword64, (volatile void *, int64_t), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + suword_smap : suword_nosmap); +} + +int casueword32_nosmap(volatile uint32_t *base, uint32_t oldval, + uint32_t *oldvalp, uint32_t newval); +int casueword32_smap(volatile uint32_t *base, uint32_t oldval, + uint32_t *oldvalp, uint32_t newval); +DEFINE_IFUNC(, int, casueword32, (volatile uint32_t *, uint32_t, uint32_t *, + uint32_t), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + casueword32_smap : casueword32_nosmap); +} + +int casueword_nosmap(volatile u_long *p, u_long oldval, u_long *oldvalp, + u_long newval); +int casueword_smap(volatile u_long *p, u_long oldval, u_long *oldvalp, + u_long newval); +DEFINE_IFUNC(, int, casueword, (volatile u_long *, u_long, u_long *, u_long), + static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + casueword_smap : casueword_nosmap); +} + +int copyinstr_nosmap(const void *udaddr, void *kaddr, size_t len, + size_t *lencopied); +int copyinstr_smap(const void *udaddr, void *kaddr, size_t len, + size_t *lencopied); +DEFINE_IFUNC(, int, copyinstr, (const void *, void *, size_t, size_t *), + static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + copyinstr_smap : copyinstr_nosmap); +} + +int copyin_nosmap(const void *udaddr, void *kaddr, size_t len); +int copyin_smap(const void *udaddr, void *kaddr, size_t len); +DEFINE_IFUNC(, int, copyin, (const void *, void *, size_t), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + copyin_smap : copyin_nosmap); +} + +int copyout_nosmap(const void *kaddr, void *udaddr, size_t len); +int copyout_smap(const void *kaddr, void *udaddr, size_t len); +DEFINE_IFUNC(, int, copyout, (const void *, void *, size_t), static) +{ + + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? + copyout_smap : copyout_nosmap); +} Modified: head/sys/amd64/amd64/exception.S ============================================================================== --- head/sys/amd64/amd64/exception.S Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/amd64/exception.S Sun Jul 29 20:47:00 2018 (r336876) @@ -43,8 +43,8 @@ #include "assym.inc" -#include #include +#include #include #include @@ -196,7 +196,9 @@ alltraps_pushregs_no_rax: movq %r14,TF_R14(%rsp) movq %r15,TF_R15(%rsp) movl $TF_HASSEGS,TF_FLAGS(%rsp) - cld + pushfq + andq $~(PSL_D | PSL_AC),(%rsp) + popfq FAKE_MCOUNT(TF_RIP(%rsp)) #ifdef KDTRACE_HOOKS /* @@ -277,7 +279,9 @@ IDTVEC(dblfault) movq %r15,TF_R15(%rsp) SAVE_SEGS movl $TF_HASSEGS,TF_FLAGS(%rsp) - cld + pushfq + andq $~(PSL_D | PSL_AC),(%rsp) + popfq testb $SEL_RPL_MASK,TF_CS(%rsp) /* Did we come from kernel? */ jz 1f /* already running with kernel GS.base */ swapgs @@ -571,7 +575,9 @@ IDTVEC(dbg) movq %r15,TF_R15(%rsp) SAVE_SEGS movl $TF_HASSEGS,TF_FLAGS(%rsp) - cld + pushfq + andq $~(PSL_D | PSL_AC),(%rsp) + popfq testb $SEL_RPL_MASK,TF_CS(%rsp) jnz dbg_fromuserspace /* @@ -704,7 +710,9 @@ IDTVEC(nmi) movq %r15,TF_R15(%rsp) SAVE_SEGS movl $TF_HASSEGS,TF_FLAGS(%rsp) - cld + pushfq + andq $~(PSL_D | PSL_AC),(%rsp) + popfq xorl %ebx,%ebx testb $SEL_RPL_MASK,TF_CS(%rsp) jnz nmi_fromuserspace @@ -793,7 +801,9 @@ nmi_calltrap: subq %rcx,%rdx movq %rdx,%rdi /* destination stack pointer */ shrq $3,%rcx /* trap frame size in long words */ - cld + pushfq + andq $~(PSL_D | PSL_AC),(%rsp) + popfq rep movsq /* copy trapframe */ movq %rdx,%rsp /* we are on the regular kstack */ @@ -902,7 +912,9 @@ IDTVEC(mchk) movq %r15,TF_R15(%rsp) SAVE_SEGS movl $TF_HASSEGS,TF_FLAGS(%rsp) - cld + pushfq + andq $~(PSL_D | PSL_AC),(%rsp) + popfq xorl %ebx,%ebx testb $SEL_RPL_MASK,TF_CS(%rsp) jnz mchk_fromuserspace Modified: head/sys/amd64/amd64/initcpu.c ============================================================================== --- head/sys/amd64/amd64/initcpu.c Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/amd64/initcpu.c Sun Jul 29 20:47:00 2018 (r336876) @@ -239,8 +239,12 @@ initializecpu(void) * to the kernel tables. The boot loader enables the U bit in * its tables. */ - if (!IS_BSP() && (cpu_stdext_feature & CPUID_STDEXT_SMEP)) - cr4 |= CR4_SMEP; + if (!IS_BSP()) { + if (cpu_stdext_feature & CPUID_STDEXT_SMEP) + cr4 |= CR4_SMEP; + if (cpu_stdext_feature & CPUID_STDEXT_SMAP) + cr4 |= CR4_SMAP; + } load_cr4(cr4); if (IS_BSP() && (amd_feature & AMDID_NX) != 0) { msr = rdmsr(MSR_EFER) | EFER_NXE; Modified: head/sys/amd64/amd64/machdep.c ============================================================================== --- head/sys/amd64/amd64/machdep.c Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/amd64/machdep.c Sun Jul 29 20:47:00 2018 (r336876) @@ -1548,7 +1548,7 @@ amd64_conf_fast_syscall(void) msr = ((u_int64_t)GSEL(GCODE_SEL, SEL_KPL) << 32) | ((u_int64_t)GSEL(GUCODE32_SEL, SEL_UPL) << 48); wrmsr(MSR_STAR, msr); - wrmsr(MSR_SF_MASK, PSL_NT | PSL_T | PSL_I | PSL_C | PSL_D); + wrmsr(MSR_SF_MASK, PSL_NT | PSL_T | PSL_I | PSL_C | PSL_D | PSL_AC); } u_int64_t Modified: head/sys/amd64/amd64/pmap.c ============================================================================== --- head/sys/amd64/amd64/pmap.c Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/amd64/pmap.c Sun Jul 29 20:47:00 2018 (r336876) @@ -1092,6 +1092,7 @@ pmap_bootstrap(vm_paddr_t *firstaddr) { vm_offset_t va; pt_entry_t *pte; + uint64_t cr4; int i; KERNend = *firstaddr; @@ -1118,11 +1119,21 @@ pmap_bootstrap(vm_paddr_t *firstaddr) virtual_end = VM_MAX_KERNEL_ADDRESS; - /* XXX do %cr0 as well */ - load_cr4(rcr4() | CR4_PGE); + /* + * Enable PG_G global pages, then switch to the kernel page + * table from the bootstrap page table. After the switch, it + * is possible to enable SMEP and SMAP since PG_U bits are + * correct now. + */ + cr4 = rcr4(); + cr4 |= CR4_PGE; + load_cr4(cr4); load_cr3(KPML4phys); if (cpu_stdext_feature & CPUID_STDEXT_SMEP) - load_cr4(rcr4() | CR4_SMEP); + cr4 |= CR4_SMEP; + if (cpu_stdext_feature & CPUID_STDEXT_SMAP) + cr4 |= CR4_SMAP; + load_cr4(cr4); /* * Initialize the kernel pmap (which is statically allocated). Modified: head/sys/amd64/amd64/support.S ============================================================================== --- head/sys/amd64/amd64/support.S Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/amd64/support.S Sun Jul 29 20:47:00 2018 (r336876) @@ -226,7 +226,7 @@ END(fillw) * copyout(from_kernel, to_user, len) * %rdi, %rsi, %rdx */ -ENTRY(copyout) +ENTRY(copyout_nosmap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rax movq $copyout_fault,PCB_ONFAULT(%rax) @@ -268,6 +268,55 @@ ENTRY(copyout) rep movsb + jmp done_copyout +END(copyout_nosmap) + +ENTRY(copyout_smap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rax + /* Trap entry clears PSL.AC */ + movq $copyout_fault,PCB_ONFAULT(%rax) + testq %rdx,%rdx /* anything to do? */ + jz done_copyout + + /* + * Check explicitly for non-user addresses. If 486 write protection + * is being used, this check is essential because we are in kernel + * mode so the h/w does not provide any protection against writing + * kernel addresses. + */ + + /* + * First, prevent address wrapping. + */ + movq %rsi,%rax + addq %rdx,%rax + jc copyout_fault +/* + * XXX STOP USING VM_MAXUSER_ADDRESS. + * It is an end address, not a max, so every time it is used correctly it + * looks like there is an off by one error, and of course it caused an off + * by one error in several places. + */ + movq $VM_MAXUSER_ADDRESS,%rcx + cmpq %rcx,%rax + ja copyout_fault + + xchgq %rdi,%rsi + /* bcopy(%rsi, %rdi, %rdx) */ + movq %rdx,%rcx + + shrq $3,%rcx + cld + stac + rep + movsq + movb %dl,%cl + andb $7,%cl + rep + movsb + clac + done_copyout: xorl %eax,%eax movq PCPU(CURPCB),%rdx @@ -288,7 +337,7 @@ END(copyout) * copyin(from_user, to_kernel, len) * %rdi, %rsi, %rdx */ -ENTRY(copyin) +ENTRY(copyin_nosmap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rax movq $copyin_fault,PCB_ONFAULT(%rax) @@ -309,13 +358,47 @@ ENTRY(copyin) movq %rdx,%rcx movb %cl,%al shrq $3,%rcx /* copy longword-wise */ + cld rep movsq movb %al,%cl andb $7,%cl /* copy remaining bytes */ + rep + movsb + + jmp done_copyin +END(copyin_nosmap) + +ENTRY(copyin_smap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rax + movq $copyin_fault,PCB_ONFAULT(%rax) + testq %rdx,%rdx /* anything to do? */ + jz done_copyin + + /* + * make sure address is valid + */ + movq %rdi,%rax + addq %rdx,%rax + jc copyin_fault + movq $VM_MAXUSER_ADDRESS,%rcx + cmpq %rcx,%rax + ja copyin_fault + + xchgq %rdi,%rsi + movq %rdx,%rcx + movb %cl,%al + shrq $3,%rcx /* copy longword-wise */ + stac + rep + movsq + movb %al,%cl + andb $7,%cl /* copy remaining bytes */ je done_copyin rep movsb + clac done_copyin: xorl %eax,%eax @@ -323,6 +406,7 @@ done_copyin: movq %rax,PCB_ONFAULT(%rdx) POP_FRAME_POINTER ret +END(copyin_smap) ALIGN_TEXT copyin_fault: @@ -331,14 +415,13 @@ copyin_fault: movq $EFAULT,%rax POP_FRAME_POINTER ret -END(copyin) /* * casueword32. Compare and set user integer. Returns -1 on fault, * 0 if access was successful. Old value is written to *oldp. * dst = %rdi, old = %esi, oldp = %rdx, new = %ecx */ -ENTRY(casueword32) +ENTRY(casueword32_nosmap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%r8 movq $fusufault,PCB_ONFAULT(%r8) @@ -370,14 +453,50 @@ ENTRY(casueword32) movl %esi,(%rdx) /* oldp = %rdx */ POP_FRAME_POINTER ret -END(casueword32) +END(casueword32_nosmap) +ENTRY(casueword32_smap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%r8 + movq $fusufault,PCB_ONFAULT(%r8) + + movq $VM_MAXUSER_ADDRESS-4,%rax + cmpq %rax,%rdi /* verify address is valid */ + ja fusufault + + movl %esi,%eax /* old */ + stac +#ifdef SMP + lock +#endif + cmpxchgl %ecx,(%rdi) /* new = %ecx */ + clac + + /* + * The old value is in %eax. If the store succeeded it will be the + * value we expected (old) from before the store, otherwise it will + * be the current value. Save %eax into %esi to prepare the return + * value. + */ + movl %eax,%esi + xorl %eax,%eax + movq %rax,PCB_ONFAULT(%r8) + + /* + * Access the oldp after the pcb_onfault is cleared, to correctly + * catch corrupted pointer. + */ + movl %esi,(%rdx) /* oldp = %rdx */ + POP_FRAME_POINTER + ret +END(casueword32_smap) + /* * casueword. Compare and set user long. Returns -1 on fault, * 0 if access was successful. Old value is written to *oldp. * dst = %rdi, old = %rsi, oldp = %rdx, new = %rcx */ -ENTRY(casueword) +ENTRY(casueword_nosmap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%r8 movq $fusufault,PCB_ONFAULT(%r8) @@ -403,16 +522,45 @@ ENTRY(casueword) movq %rsi,(%rdx) POP_FRAME_POINTER ret -END(casueword) +END(casueword_nosmap) +ENTRY(casueword_smap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%r8 + movq $fusufault,PCB_ONFAULT(%r8) + + movq $VM_MAXUSER_ADDRESS-4,%rax + cmpq %rax,%rdi /* verify address is valid */ + ja fusufault + + movq %rsi,%rax /* old */ + stac +#ifdef SMP + lock +#endif + cmpxchgq %rcx,(%rdi) /* new = %rcx */ + clac + + /* + * The old value is in %rax. If the store succeeded it will be the + * value we expected (old) from before the store, otherwise it will + * be the current value. + */ + movq %rax,%rsi + xorl %eax,%eax + movq %rax,PCB_ONFAULT(%r8) + movq %rsi,(%rdx) + POP_FRAME_POINTER + ret +END(casueword_smap) + /* * Fetch (load) a 64-bit word, a 32-bit word, a 16-bit word, or an 8-bit * byte from user memory. * addr = %rdi, valp = %rsi */ -ALTENTRY(fueword64) -ENTRY(fueword) +ENTRY(fueword_nosmap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) @@ -427,14 +575,32 @@ ENTRY(fueword) movq %r11,(%rsi) POP_FRAME_POINTER ret -END(fueword64) -END(fueword) +END(fueword64_nosmap) -ENTRY(fueword32) +ENTRY(fueword_smap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-8,%rax + cmpq %rax,%rdi /* verify address is valid */ + ja fusufault + + xorl %eax,%eax + stac + movq (%rdi),%r11 + clac + movq %rax,PCB_ONFAULT(%rcx) + movq %r11,(%rsi) + POP_FRAME_POINTER + ret +END(fueword64_smap) + +ENTRY(fueword32_nosmap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-4,%rax cmpq %rax,%rdi /* verify address is valid */ ja fusufault @@ -445,13 +611,32 @@ ENTRY(fueword32) movl %r11d,(%rsi) POP_FRAME_POINTER ret -END(fueword32) +END(fueword32_nosmap) -ENTRY(fuword16) +ENTRY(fueword32_smap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-4,%rax + cmpq %rax,%rdi /* verify address is valid */ + ja fusufault + + xorl %eax,%eax + stac + movl (%rdi),%r11d + clac + movq %rax,PCB_ONFAULT(%rcx) + movl %r11d,(%rsi) + POP_FRAME_POINTER + ret +END(fueword32_smap) + +ENTRY(fuword16_nosmap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-2,%rax cmpq %rax,%rdi ja fusufault @@ -460,13 +645,30 @@ ENTRY(fuword16) movq $0,PCB_ONFAULT(%rcx) POP_FRAME_POINTER ret -END(fuword16) +END(fuword16_nosmap) -ENTRY(fubyte) +ENTRY(fuword16_smap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-2,%rax + cmpq %rax,%rdi + ja fusufault + + stac + movzwl (%rdi),%eax + clac + movq $0,PCB_ONFAULT(%rcx) + POP_FRAME_POINTER + ret +END(fuword16_smap) + +ENTRY(fubyte_nosmap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-1,%rax cmpq %rax,%rdi ja fusufault @@ -475,9 +677,27 @@ ENTRY(fubyte) movq $0,PCB_ONFAULT(%rcx) POP_FRAME_POINTER ret -END(fubyte) +END(fubyte_nosmap) +ENTRY(fubyte_smap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + + movq $VM_MAXUSER_ADDRESS-1,%rax + cmpq %rax,%rdi + ja fusufault + + stac + movzbl (%rdi),%eax + clac + movq $0,PCB_ONFAULT(%rcx) + POP_FRAME_POINTER + ret +END(fubyte_smap) + ALIGN_TEXT + /* Fault entry clears PSL.AC */ fusufault: movq PCPU(CURPCB),%rcx xorl %eax,%eax @@ -491,8 +711,7 @@ fusufault: * user memory. * addr = %rdi, value = %rsi */ -ALTENTRY(suword64) -ENTRY(suword) +ENTRY(suword_nosmap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) @@ -507,14 +726,32 @@ ENTRY(suword) movq %rax,PCB_ONFAULT(%rcx) POP_FRAME_POINTER ret -END(suword64) -END(suword) +END(suword_nosmap) -ENTRY(suword32) +ENTRY(suword_smap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-8,%rax + cmpq %rax,%rdi /* verify address validity */ + ja fusufault + + stac + movq %rsi,(%rdi) + clac + xorl %eax,%eax + movq PCPU(CURPCB),%rcx + movq %rax,PCB_ONFAULT(%rcx) + POP_FRAME_POINTER + ret +END(suword_smap) + +ENTRY(suword32_nosmap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-4,%rax cmpq %rax,%rdi /* verify address validity */ ja fusufault @@ -525,13 +762,32 @@ ENTRY(suword32) movq %rax,PCB_ONFAULT(%rcx) POP_FRAME_POINTER ret -END(suword32) +END(suword32_nosmap) -ENTRY(suword16) +ENTRY(suword32_smap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-4,%rax + cmpq %rax,%rdi /* verify address validity */ + ja fusufault + + stac + movl %esi,(%rdi) + clac + xorl %eax,%eax + movq PCPU(CURPCB),%rcx + movq %rax,PCB_ONFAULT(%rcx) + POP_FRAME_POINTER + ret +END(suword32_smap) + +ENTRY(suword16_nosmap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-2,%rax cmpq %rax,%rdi /* verify address validity */ ja fusufault @@ -542,13 +798,32 @@ ENTRY(suword16) movq %rax,PCB_ONFAULT(%rcx) POP_FRAME_POINTER ret -END(suword16) +END(suword16_nosmap) -ENTRY(subyte) +ENTRY(suword16_smap) PUSH_FRAME_POINTER movq PCPU(CURPCB),%rcx movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-2,%rax + cmpq %rax,%rdi /* verify address validity */ + ja fusufault + + stac + movw %si,(%rdi) + clac + xorl %eax,%eax + movq PCPU(CURPCB),%rcx /* restore trashed register */ + movq %rax,PCB_ONFAULT(%rcx) + POP_FRAME_POINTER + ret +END(suword16_smap) + +ENTRY(subyte_nosmap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + movq $VM_MAXUSER_ADDRESS-1,%rax cmpq %rax,%rdi /* verify address validity */ ja fusufault @@ -560,8 +835,28 @@ ENTRY(subyte) movq %rax,PCB_ONFAULT(%rcx) POP_FRAME_POINTER ret -END(subyte) +END(subyte_nosmap) +ENTRY(subyte_smap) + PUSH_FRAME_POINTER + movq PCPU(CURPCB),%rcx + movq $fusufault,PCB_ONFAULT(%rcx) + + movq $VM_MAXUSER_ADDRESS-1,%rax + cmpq %rax,%rdi /* verify address validity */ + ja fusufault + + movl %esi,%eax + stac + movb %al,(%rdi) + clac + xorl %eax,%eax + movq PCPU(CURPCB),%rcx /* restore trashed register */ + movq %rax,PCB_ONFAULT(%rcx) + POP_FRAME_POINTER + ret +END(subyte_smap) + /* * copyinstr(from, to, maxlen, int *lencopied) * %rdi, %rsi, %rdx, %rcx @@ -571,7 +866,7 @@ END(subyte) * EFAULT on protection violations. If lencopied is non-zero, * return the actual length in *lencopied. */ -ENTRY(copyinstr) +ENTRY(copyinstr_nosmap) PUSH_FRAME_POINTER movq %rdx,%r8 /* %r8 = maxlen */ movq %rcx,%r9 /* %r9 = *len */ @@ -592,29 +887,67 @@ ENTRY(copyinstr) movq %rax,%r8 1: incq %rdx + cld 2: decq %rdx - jz 3f + jz copyinstr_toolong lodsb stosb orb %al,%al jnz 2b + jmp copyinstr_succ +END(copyinstr_nosmap) + +ENTRY(copyinstr_smap) + PUSH_FRAME_POINTER + movq %rdx,%r8 /* %r8 = maxlen */ + movq %rcx,%r9 /* %r9 = *len */ + xchgq %rdi,%rsi /* %rdi = from, %rsi = to */ + movq PCPU(CURPCB),%rcx + movq $cpystrflt,PCB_ONFAULT(%rcx) + + movq $VM_MAXUSER_ADDRESS,%rax + + /* make sure 'from' is within bounds */ + subq %rsi,%rax + jbe cpystrflt + + /* restrict maxlen to <= VM_MAXUSER_ADDRESS-from */ + cmpq %rdx,%rax + jae 1f + movq %rax,%rdx + movq %rax,%r8 +1: + incq %rdx + +2: + decq %rdx + jz copyinstr_succ + + stac + lodsb + stosb + clac + orb %al,%al + jnz 2b + +copyinstr_succ: /* Success -- 0 byte reached */ decq %rdx xorl %eax,%eax jmp cpystrflt_x -3: +copyinstr_toolong: /* rdx is zero - return ENAMETOOLONG or EFAULT */ movq $VM_MAXUSER_ADDRESS,%rax cmpq %rax,%rsi jae cpystrflt -4: movq $ENAMETOOLONG,%rax jmp cpystrflt_x + /* Fault entry clears PSL.AC */ cpystrflt: movq $EFAULT,%rax @@ -630,7 +963,7 @@ cpystrflt_x: 1: POP_FRAME_POINTER ret -END(copyinstr) +END(copyinstr_smap) /* * copystr(from, to, maxlen, int *lencopied) Modified: head/sys/amd64/amd64/trap.c ============================================================================== --- head/sys/amd64/amd64/trap.c Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/amd64/trap.c Sun Jul 29 20:47:00 2018 (r336876) @@ -673,6 +673,24 @@ trap_check(struct trapframe *frame) trap(frame); } +static bool +trap_is_smap(struct trapframe *frame) +{ + + /* + * A page fault on a userspace address is classified as + * SMAP-induced if: + * - SMAP is supported; + * - kernel mode accessed present data page; + * - rflags.AC was cleared. + * Kernel must never access user space with rflags.AC cleared + * if SMAP is enabled. + */ + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 && + (frame->tf_err & (PGEX_P | PGEX_U | PGEX_I | PGEX_RSV)) == + PGEX_P && (frame->tf_rflags & PSL_AC) == 0); +} + static int trap_pfault(struct trapframe *frame, int usermode) { @@ -750,9 +768,13 @@ trap_pfault(struct trapframe *frame, int usermode) * handling routine. Since accessing the address * without the handler is a bug, do not try to handle * it normally, and panic immediately. + * + * If SMAP is enabled, filter SMAP faults also, + * because illegal access might occur to the mapped + * user address, causing infinite loop. */ if (!usermode && (td->td_intr_nesting_level != 0 || - curpcb->pcb_onfault == NULL)) { + trap_is_smap(frame) || curpcb->pcb_onfault == NULL)) { trap_fatal(frame, eva); return (-1); } Modified: head/sys/amd64/ia32/ia32_exception.S ============================================================================== --- head/sys/amd64/ia32/ia32_exception.S Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/ia32/ia32_exception.S Sun Jul 29 20:47:00 2018 (r336876) @@ -70,7 +70,9 @@ int0x80_syscall_common: movq %r14,TF_R14(%rsp) movq %r15,TF_R15(%rsp) movl $TF_HASSEGS,TF_FLAGS(%rsp) - cld + pushfq + andq $~(PSL_D | PSL_AC),(%rsp) + popfq FAKE_MCOUNT(TF_RIP(%rsp)) movq %rsp, %rdi call ia32_syscall Modified: head/sys/amd64/include/asmacros.h ============================================================================== --- head/sys/amd64/include/asmacros.h Sun Jul 29 20:34:44 2018 (r336875) +++ head/sys/amd64/include/asmacros.h Sun Jul 29 20:47:00 2018 (r336876) @@ -255,7 +255,9 @@ X\vec_name: movq %r15,TF_R15(%rsp) SAVE_SEGS *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Sun Jul 29 21:00:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 42AD2106329A; Sun, 29 Jul 2018 21:00:19 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE9307992D; Sun, 29 Jul 2018 21:00:18 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D13371AE59; Sun, 29 Jul 2018 21:00:18 +0000 (UTC) (envelope-from eadler@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TL0IQV010310; Sun, 29 Jul 2018 21:00:18 GMT (envelope-from eadler@FreeBSD.org) Received: (from eadler@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TL0ILI010309; Sun, 29 Jul 2018 21:00:18 GMT (envelope-from eadler@FreeBSD.org) Message-Id: <201807292100.w6TL0ILI010309@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: eadler set sender to eadler@FreeBSD.org using -f From: Eitan Adler Date: Sun, 29 Jul 2018 21:00:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336877 - head/sbin/devd X-SVN-Group: head X-SVN-Commit-Author: eadler X-SVN-Commit-Paths: head/sbin/devd X-SVN-Commit-Revision: 336877 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 21:00:19 -0000 Author: eadler Date: Sun Jul 29 21:00:18 2018 New Revision: 336877 URL: https://svnweb.freebsd.org/changeset/base/336877 Log: Revert "devd: use C++ style casts" It seems that nullptr is not yet supported by all compilers used by FreeBSD. :( Modified: head/sbin/devd/devd.cc Modified: head/sbin/devd/devd.cc ============================================================================== --- head/sbin/devd/devd.cc Sun Jul 29 20:47:00 2018 (r336876) +++ head/sbin/devd/devd.cc Sun Jul 29 21:00:18 2018 (r336877) @@ -262,12 +262,12 @@ my_system(const char *command) */ cfg.close_pidfile(); ::closefrom(3); - ::execl(_PATH_BSHELL, "sh", "-c", command, nullptr); + ::execl(_PATH_BSHELL, "sh", "-c", command, (char *)NULL); ::_exit(127); default: /* parent */ savedpid = pid; do { - pid = ::wait4(savedpid, &pstat, 0, nullptr); + pid = ::wait4(savedpid, &pstat, 0, (struct rusage *)0); } while (pid == -1 && errno == EINTR); break; } @@ -374,7 +374,7 @@ media::do_match(config &c) memset(&ifmr, 0, sizeof(ifmr)); strlcpy(ifmr.ifm_name, value.c_str(), sizeof(ifmr.ifm_name)); - if (ioctl(s, SIOCGIFMEDIA, &ifmr) >= 0 && + if (ioctl(s, SIOCGIFMEDIA, (caddr_t)&ifmr) >= 0 && ifmr.ifm_status & IFM_AVALID) { devdlog(LOG_DEBUG, "%s has media type 0x%x\n", value.c_str(), IFM_TYPE(ifmr.ifm_active)); @@ -527,7 +527,7 @@ config::open_pidfile() pfh = pidfile_open(_pidfile.c_str(), 0600, &otherpid); if (pfh == NULL) { if (errno == EEXIST) - errx(1, "devd already running, pid: %d", static_cast(otherpid)); + errx(1, "devd already running, pid: %d", (int)otherpid); warn("cannot open pid file"); } } @@ -854,7 +854,7 @@ process_event(char *buffer) // Save the time this happened (as approximated by when we got // around to processing it). gettimeofday(&tv, NULL); - asprintf(×tr, "%jd.%06ld", static_cast(tv.tv_sec), tv.tv_usec); + asprintf(×tr, "%jd.%06ld", (uintmax_t)tv.tv_sec, tv.tv_usec); cfg.set_variable("timestamp", timestr); free(timestr); @@ -919,7 +919,7 @@ create_socket(const char *name, int socktype) unlink(name); if (fcntl(fd, F_SETFL, O_NONBLOCK) < 0) err(1, "fcntl"); - if (::bind(fd, reinterpret_cast(&sun), slen) < 0) + if (::bind(fd, (struct sockaddr *) & sun, slen) < 0) err(1, "bind"); listen(fd, 4); if (chown(name, 0, 0)) /* XXX - root.wheel */ From owner-svn-src-all@freebsd.org Sun Jul 29 21:51:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 674E81064F89; Sun, 29 Jul 2018 21:51:18 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1CE117BB2B; Sun, 29 Jul 2018 21:51:18 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F239C1B771; Sun, 29 Jul 2018 21:51:17 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6TLpH65035550; Sun, 29 Jul 2018 21:51:17 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6TLpHEU035547; Sun, 29 Jul 2018 21:51:17 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807292151.w6TLpHEU035547@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Sun, 29 Jul 2018 21:51:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336878 - head/sys/fs/nfsserver X-SVN-Group: head X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: head/sys/fs/nfsserver X-SVN-Commit-Revision: 336878 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 21:51:18 -0000 Author: rmacklem Date: Sun Jul 29 21:51:17 2018 New Revision: 336878 URL: https://svnweb.freebsd.org/changeset/base/336878 Log: Silence newer gcc warnings. Newer versions of gcc generate "set, but not used" warnings in the NFS server. Add __unused macros to silence these warnings. Requested by: mmacy Modified: head/sys/fs/nfsserver/nfs_nfsdport.c head/sys/fs/nfsserver/nfs_nfsdserv.c head/sys/fs/nfsserver/nfs_nfsdstate.c Modified: head/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- head/sys/fs/nfsserver/nfs_nfsdport.c Sun Jul 29 21:00:18 2018 (r336877) +++ head/sys/fs/nfsserver/nfs_nfsdport.c Sun Jul 29 21:51:17 2018 (r336878) @@ -1799,7 +1799,7 @@ nfsrvd_readdir(struct nfsrv_descript *nd, int isdgram, struct nfsvattr at; int nlen, error = 0, getret = 1; int siz, cnt, fullsiz, eofflag, ncookies; - u_int64_t off, toff, verf; + u_int64_t off, toff, verf __unused; u_long *cookies = NULL, *cookiep; struct uio io; struct iovec iv; Modified: head/sys/fs/nfsserver/nfs_nfsdserv.c ============================================================================== --- head/sys/fs/nfsserver/nfs_nfsdserv.c Sun Jul 29 21:00:18 2018 (r336877) +++ head/sys/fs/nfsserver/nfs_nfsdserv.c Sun Jul 29 21:51:17 2018 (r336878) @@ -3852,7 +3852,7 @@ nfsrvd_openattr(struct nfsrv_descript *nd, __unused in __unused NFSPROC_T *p, __unused struct nfsexstuff *exp) { u_int32_t *tl; - int error = 0, createdir; + int error = 0, createdir __unused; NFSM_DISSECT(tl, u_int32_t *, NFSX_UNSIGNED); createdir = fxdr_unsigned(int, *tl); Modified: head/sys/fs/nfsserver/nfs_nfsdstate.c ============================================================================== --- head/sys/fs/nfsserver/nfs_nfsdstate.c Sun Jul 29 21:00:18 2018 (r336877) +++ head/sys/fs/nfsserver/nfs_nfsdstate.c Sun Jul 29 21:51:17 2018 (r336878) @@ -5765,7 +5765,7 @@ nfsrv_localunlock(vnode_t vp, struct nfslockfile *lfp, uint64_t init_end, NFSPROC_T *p) { struct nfslock *lop; - uint64_t first, end, prevfirst; + uint64_t first, end, prevfirst __unused; first = init_first; end = init_end; From owner-svn-src-all@freebsd.org Sun Jul 29 21:57:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5B5B1065053 for ; Sun, 29 Jul 2018 21:57:49 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: from mail-yw0-x235.google.com (mail-yw0-x235.google.com [IPv6:2607:f8b0:4002:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3F3C87BDAE for ; Sun, 29 Jul 2018 21:57:49 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: by mail-yw0-x235.google.com with SMTP id q129-v6so3708007ywg.8 for ; Sun, 29 Jul 2018 14:57:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=F5GuB4ekEku1R6pAlHDH623+GcWSA9w4o/2Ckm3tHcE=; b=FmcyJw72z0AHNqHVgntNtXKDw1NQnt9QvR5fp6RsSoX3JCpalER8STgDJhhram0b1n psw/eNe5qI3brC7zGsEjpOq4SCkz7kfpmZuYCKDiM8cTPsTblLpv3gZf2MS8K5a68isL LAlDUeFVPHoRlV0ZE3WhwkaQ0Wb9VZkcviBBw44gBvs0NTLKrxiHujuOa9rznDHlOhyO +AO6jcFwrJWO2G+l+eYGh8P0z0yk2sTnJBA+Zysc24bEX2KQo80A4FMIkbuYXdOFYN+e eHE8X2IqlVwPgpf7Qz1qvP9LxQdanJ7yvYJcNJAyYsXMX9V6Jp4dwF2A1yRzxq4pCQul QQUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=F5GuB4ekEku1R6pAlHDH623+GcWSA9w4o/2Ckm3tHcE=; b=FvDyyUFDzPgTeryU4b4/U1yED54HRqm6cXVMCNEkvMRLq4sm4kRA9pxH3dS3SJ+ME+ 4inL31MbKSrAip4sn6TVsZ1/wN6B1h0hSMEEpKQ535kpf0FjSDjtlQD8aAv/PMEwuFiP 6ddi1/eKYV3EQBrXOQ6XSHNMUDMSSQiNR36gkqY2W6szyICcJgZt5ujrMvq+SGIqRO/z sixzQg8osOBi+Slm4d566gUi6ZAAoVJoL8eJAWEybXTX8318HAvy+zU4e9txPqtUlN5u u3LHOpXWJAdsbzy3T9XY3PXrJMz29i5Zzqp/xS1PWPf96lNFr8fhWJm9dFAS2c1vBDC6 sfVQ== X-Gm-Message-State: AOUpUlECWLXkf3XTgCYj+C1AvmdNxlfOSTbiyZE0YKWGKrUr6lUnxhTT hBy7Eruw+DRsWPNqHy1/D5Jv45eE2pviodJtTmfSoA== X-Google-Smtp-Source: AAOMgpf0owibppOYQoMeX2AS4WPwRcxRuG3lZj33nTY59/X4eh128D2YuZDxhU7T9IV/TcPlIDWbQdQF47VRzFcUSzk= X-Received: by 2002:a81:4511:: with SMTP id s17-v6mr7858036ywa.104.1532901468224; Sun, 29 Jul 2018 14:57:48 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a25:b90:0:0:0:0:0 with HTTP; Sun, 29 Jul 2018 14:57:47 -0700 (PDT) In-Reply-To: <201807292047.w6TKl0hV004691@repo.freebsd.org> References: <201807292047.w6TKl0hV004691@repo.freebsd.org> From: Oliver Pinter Date: Sun, 29 Jul 2018 23:57:47 +0200 Message-ID: Subject: Re: svn commit: r336876 - in head/sys: amd64/amd64 amd64/ia32 amd64/include conf dev/hyperv/vmbus/amd64 To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 21:57:50 -0000 On 7/29/18, Konstantin Belousov wrote: > Author: kib > Date: Sun Jul 29 20:47:00 2018 > New Revision: 336876 > URL: https://svnweb.freebsd.org/changeset/base/336876 > > Log: > Use SMAP on amd64. > > Ifuncs selectors dispatch copyin(9) family to the suitable variant, to > set rflags.AC around userspace access. Rflags.AC bit is cleared in > all kernel entry points unconditionally even on machines not > supporting SMAP. > > Reviewed by: jhb > Sponsored by: The FreeBSD Foundation > Differential revision: https://reviews.freebsd.org/D13838 > > Added: > head/sys/amd64/amd64/copyout.c (contents, props changed) > Modified: > head/sys/amd64/amd64/exception.S > head/sys/amd64/amd64/initcpu.c > head/sys/amd64/amd64/machdep.c > head/sys/amd64/amd64/pmap.c > head/sys/amd64/amd64/support.S > head/sys/amd64/amd64/trap.c > head/sys/amd64/ia32/ia32_exception.S > head/sys/amd64/include/asmacros.h > head/sys/conf/files.amd64 > head/sys/dev/hyperv/vmbus/amd64/vmbus_vector.S > > Added: head/sys/amd64/amd64/copyout.c > ============================================================================== > --- /dev/null 00:00:00 1970 (empty, because file is newly added) > +++ head/sys/amd64/amd64/copyout.c Sun Jul 29 20:47:00 2018 (r336876) > @@ -0,0 +1,178 @@ > +/*- > + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD > + * > + * Copyright (c) 2018 The FreeBSD Foundation > + * All rights reserved. > + * > + * This software was developed by Konstantin Belousov > + * under sponsorship from the FreeBSD Foundation. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in the > + * documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, > STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY > WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + */ > + > +#include > +__FBSDID("$FreeBSD$"); > + > +#include > +#include > + > +#include > +#include > +#include > + > +int fubyte_nosmap(volatile const void *base); > +int fubyte_smap(volatile const void *base); > +DEFINE_IFUNC(, int, fubyte, (volatile const void *), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + fubyte_smap : fubyte_nosmap); > +} > + > +int fuword16_nosmap(volatile const void *base); > +int fuword16_smap(volatile const void *base); > +DEFINE_IFUNC(, int, fuword16, (volatile const void *), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + fuword16_smap : fuword16_nosmap); > +} > + > +int fueword_nosmap(volatile const void *base, long *val); > +int fueword_smap(volatile const void *base, long *val); > +DEFINE_IFUNC(, int, fueword, (volatile const void *, long *), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + fueword_smap : fueword_nosmap); > +} > +DEFINE_IFUNC(, int, fueword64, (volatile const void *, int64_t *), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + fueword_smap : fueword_nosmap); > +} > + > +int fueword32_nosmap(volatile const void *base, int32_t *val); > +int fueword32_smap(volatile const void *base, int32_t *val); > +DEFINE_IFUNC(, int, fueword32, (volatile const void *, int32_t *), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + fueword32_smap : fueword32_nosmap); > +} > + > +int subyte_nosmap(volatile void *base, int byte); > +int subyte_smap(volatile void *base, int byte); > +DEFINE_IFUNC(, int, subyte, (volatile void *, int), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + subyte_smap : subyte_nosmap); > +} > + > +int suword16_nosmap(volatile void *base, int word); > +int suword16_smap(volatile void *base, int word); > +DEFINE_IFUNC(, int, suword16, (volatile void *, int), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + suword16_smap : suword16_nosmap); > +} > + > +int suword32_nosmap(volatile void *base, int32_t word); > +int suword32_smap(volatile void *base, int32_t word); > +DEFINE_IFUNC(, int, suword32, (volatile void *, int32_t), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + suword32_smap : suword32_nosmap); > +} > + > +int suword_nosmap(volatile void *base, long word); > +int suword_smap(volatile void *base, long word); > +DEFINE_IFUNC(, int, suword, (volatile void *, long), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + suword_smap : suword_nosmap); > +} > +DEFINE_IFUNC(, int, suword64, (volatile void *, int64_t), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + suword_smap : suword_nosmap); > +} > + > +int casueword32_nosmap(volatile uint32_t *base, uint32_t oldval, > + uint32_t *oldvalp, uint32_t newval); > +int casueword32_smap(volatile uint32_t *base, uint32_t oldval, > + uint32_t *oldvalp, uint32_t newval); > +DEFINE_IFUNC(, int, casueword32, (volatile uint32_t *, uint32_t, uint32_t > *, > + uint32_t), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + casueword32_smap : casueword32_nosmap); > +} > + > +int casueword_nosmap(volatile u_long *p, u_long oldval, u_long *oldvalp, > + u_long newval); > +int casueword_smap(volatile u_long *p, u_long oldval, u_long *oldvalp, > + u_long newval); > +DEFINE_IFUNC(, int, casueword, (volatile u_long *, u_long, u_long *, > u_long), > + static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + casueword_smap : casueword_nosmap); > +} > + > +int copyinstr_nosmap(const void *udaddr, void *kaddr, size_t len, > + size_t *lencopied); > +int copyinstr_smap(const void *udaddr, void *kaddr, size_t len, > + size_t *lencopied); > +DEFINE_IFUNC(, int, copyinstr, (const void *, void *, size_t, size_t *), > + static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + copyinstr_smap : copyinstr_nosmap); > +} > + > +int copyin_nosmap(const void *udaddr, void *kaddr, size_t len); > +int copyin_smap(const void *udaddr, void *kaddr, size_t len); > +DEFINE_IFUNC(, int, copyin, (const void *, void *, size_t), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + copyin_smap : copyin_nosmap); > +} > + > +int copyout_nosmap(const void *kaddr, void *udaddr, size_t len); > +int copyout_smap(const void *kaddr, void *udaddr, size_t len); > +DEFINE_IFUNC(, int, copyout, (const void *, void *, size_t), static) > +{ > + > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 ? > + copyout_smap : copyout_nosmap); > +} > > Modified: head/sys/amd64/amd64/exception.S > ============================================================================== > --- head/sys/amd64/amd64/exception.S Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/amd64/exception.S Sun Jul 29 20:47:00 2018 (r336876) > @@ -43,8 +43,8 @@ > > #include "assym.inc" > > -#include > #include > +#include > #include > #include > > @@ -196,7 +196,9 @@ alltraps_pushregs_no_rax: > movq %r14,TF_R14(%rsp) > movq %r15,TF_R15(%rsp) > movl $TF_HASSEGS,TF_FLAGS(%rsp) > - cld > + pushfq > + andq $~(PSL_D | PSL_AC),(%rsp) > + popfq > FAKE_MCOUNT(TF_RIP(%rsp)) > #ifdef KDTRACE_HOOKS > /* > @@ -277,7 +279,9 @@ IDTVEC(dblfault) > movq %r15,TF_R15(%rsp) > SAVE_SEGS > movl $TF_HASSEGS,TF_FLAGS(%rsp) > - cld > + pushfq > + andq $~(PSL_D | PSL_AC),(%rsp) > + popfq > testb $SEL_RPL_MASK,TF_CS(%rsp) /* Did we come from kernel? */ > jz 1f /* already running with kernel GS.base */ > swapgs > @@ -571,7 +575,9 @@ IDTVEC(dbg) > movq %r15,TF_R15(%rsp) > SAVE_SEGS > movl $TF_HASSEGS,TF_FLAGS(%rsp) > - cld > + pushfq > + andq $~(PSL_D | PSL_AC),(%rsp) > + popfq > testb $SEL_RPL_MASK,TF_CS(%rsp) > jnz dbg_fromuserspace > /* > @@ -704,7 +710,9 @@ IDTVEC(nmi) > movq %r15,TF_R15(%rsp) > SAVE_SEGS > movl $TF_HASSEGS,TF_FLAGS(%rsp) > - cld > + pushfq > + andq $~(PSL_D | PSL_AC),(%rsp) > + popfq > xorl %ebx,%ebx > testb $SEL_RPL_MASK,TF_CS(%rsp) > jnz nmi_fromuserspace > @@ -793,7 +801,9 @@ nmi_calltrap: > subq %rcx,%rdx > movq %rdx,%rdi /* destination stack pointer */ > shrq $3,%rcx /* trap frame size in long words */ > - cld > + pushfq > + andq $~(PSL_D | PSL_AC),(%rsp) > + popfq > rep > movsq /* copy trapframe */ > movq %rdx,%rsp /* we are on the regular kstack */ > @@ -902,7 +912,9 @@ IDTVEC(mchk) > movq %r15,TF_R15(%rsp) > SAVE_SEGS > movl $TF_HASSEGS,TF_FLAGS(%rsp) > - cld > + pushfq > + andq $~(PSL_D | PSL_AC),(%rsp) > + popfq > xorl %ebx,%ebx > testb $SEL_RPL_MASK,TF_CS(%rsp) > jnz mchk_fromuserspace > > Modified: head/sys/amd64/amd64/initcpu.c > ============================================================================== > --- head/sys/amd64/amd64/initcpu.c Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/amd64/initcpu.c Sun Jul 29 20:47:00 2018 (r336876) > @@ -239,8 +239,12 @@ initializecpu(void) > * to the kernel tables. The boot loader enables the U bit in > * its tables. > */ > - if (!IS_BSP() && (cpu_stdext_feature & CPUID_STDEXT_SMEP)) > - cr4 |= CR4_SMEP; > + if (!IS_BSP()) { > + if (cpu_stdext_feature & CPUID_STDEXT_SMEP) > + cr4 |= CR4_SMEP; > + if (cpu_stdext_feature & CPUID_STDEXT_SMAP) > + cr4 |= CR4_SMAP; > + } > load_cr4(cr4); > if (IS_BSP() && (amd_feature & AMDID_NX) != 0) { > msr = rdmsr(MSR_EFER) | EFER_NXE; > > Modified: head/sys/amd64/amd64/machdep.c > ============================================================================== > --- head/sys/amd64/amd64/machdep.c Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/amd64/machdep.c Sun Jul 29 20:47:00 2018 (r336876) > @@ -1548,7 +1548,7 @@ amd64_conf_fast_syscall(void) > msr = ((u_int64_t)GSEL(GCODE_SEL, SEL_KPL) << 32) | > ((u_int64_t)GSEL(GUCODE32_SEL, SEL_UPL) << 48); > wrmsr(MSR_STAR, msr); > - wrmsr(MSR_SF_MASK, PSL_NT | PSL_T | PSL_I | PSL_C | PSL_D); > + wrmsr(MSR_SF_MASK, PSL_NT | PSL_T | PSL_I | PSL_C | PSL_D | PSL_AC); > } > > u_int64_t > > Modified: head/sys/amd64/amd64/pmap.c > ============================================================================== > --- head/sys/amd64/amd64/pmap.c Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/amd64/pmap.c Sun Jul 29 20:47:00 2018 (r336876) > @@ -1092,6 +1092,7 @@ pmap_bootstrap(vm_paddr_t *firstaddr) > { > vm_offset_t va; > pt_entry_t *pte; > + uint64_t cr4; > int i; > > KERNend = *firstaddr; > @@ -1118,11 +1119,21 @@ pmap_bootstrap(vm_paddr_t *firstaddr) > virtual_end = VM_MAX_KERNEL_ADDRESS; > > > - /* XXX do %cr0 as well */ > - load_cr4(rcr4() | CR4_PGE); > + /* > + * Enable PG_G global pages, then switch to the kernel page > + * table from the bootstrap page table. After the switch, it > + * is possible to enable SMEP and SMAP since PG_U bits are > + * correct now. > + */ > + cr4 = rcr4(); > + cr4 |= CR4_PGE; > + load_cr4(cr4); > load_cr3(KPML4phys); > if (cpu_stdext_feature & CPUID_STDEXT_SMEP) > - load_cr4(rcr4() | CR4_SMEP); > + cr4 |= CR4_SMEP; > + if (cpu_stdext_feature & CPUID_STDEXT_SMAP) > + cr4 |= CR4_SMAP; > + load_cr4(cr4); > > /* > * Initialize the kernel pmap (which is statically allocated). > > Modified: head/sys/amd64/amd64/support.S > ============================================================================== > --- head/sys/amd64/amd64/support.S Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/amd64/support.S Sun Jul 29 20:47:00 2018 (r336876) > @@ -226,7 +226,7 @@ END(fillw) > * copyout(from_kernel, to_user, len) > * %rdi, %rsi, %rdx > */ > -ENTRY(copyout) > +ENTRY(copyout_nosmap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rax > movq $copyout_fault,PCB_ONFAULT(%rax) > @@ -268,6 +268,55 @@ ENTRY(copyout) > rep > movsb > > + jmp done_copyout > +END(copyout_nosmap) > + > +ENTRY(copyout_smap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rax > + /* Trap entry clears PSL.AC */ > + movq $copyout_fault,PCB_ONFAULT(%rax) > + testq %rdx,%rdx /* anything to do? */ > + jz done_copyout > + > + /* > + * Check explicitly for non-user addresses. If 486 write protection > + * is being used, this check is essential because we are in kernel > + * mode so the h/w does not provide any protection against writing > + * kernel addresses. > + */ > + > + /* > + * First, prevent address wrapping. > + */ > + movq %rsi,%rax > + addq %rdx,%rax > + jc copyout_fault > +/* > + * XXX STOP USING VM_MAXUSER_ADDRESS. > + * It is an end address, not a max, so every time it is used correctly it > + * looks like there is an off by one error, and of course it caused an off > + * by one error in several places. > + */ > + movq $VM_MAXUSER_ADDRESS,%rcx > + cmpq %rcx,%rax > + ja copyout_fault > + > + xchgq %rdi,%rsi > + /* bcopy(%rsi, %rdi, %rdx) */ > + movq %rdx,%rcx > + > + shrq $3,%rcx > + cld > + stac > + rep > + movsq > + movb %dl,%cl > + andb $7,%cl > + rep > + movsb > + clac > + > done_copyout: > xorl %eax,%eax > movq PCPU(CURPCB),%rdx > @@ -288,7 +337,7 @@ END(copyout) > * copyin(from_user, to_kernel, len) > * %rdi, %rsi, %rdx > */ > -ENTRY(copyin) > +ENTRY(copyin_nosmap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rax > movq $copyin_fault,PCB_ONFAULT(%rax) > @@ -309,13 +358,47 @@ ENTRY(copyin) > movq %rdx,%rcx > movb %cl,%al > shrq $3,%rcx /* copy longword-wise */ > + cld > rep > movsq > movb %al,%cl > andb $7,%cl /* copy remaining bytes */ > + rep > + movsb > + > + jmp done_copyin > +END(copyin_nosmap) > + > +ENTRY(copyin_smap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rax > + movq $copyin_fault,PCB_ONFAULT(%rax) > + testq %rdx,%rdx /* anything to do? */ > + jz done_copyin > + > + /* > + * make sure address is valid > + */ > + movq %rdi,%rax > + addq %rdx,%rax > + jc copyin_fault > + movq $VM_MAXUSER_ADDRESS,%rcx > + cmpq %rcx,%rax > + ja copyin_fault > + > + xchgq %rdi,%rsi > + movq %rdx,%rcx > + movb %cl,%al > + shrq $3,%rcx /* copy longword-wise */ missing cld from here > + stac > + rep > + movsq > + movb %al,%cl > + andb $7,%cl /* copy remaining bytes */ > je done_copyin > rep > movsb > + clac > > done_copyin: > xorl %eax,%eax > @@ -323,6 +406,7 @@ done_copyin: > movq %rax,PCB_ONFAULT(%rdx) > POP_FRAME_POINTER > ret > +END(copyin_smap) > > ALIGN_TEXT > copyin_fault: > @@ -331,14 +415,13 @@ copyin_fault: > movq $EFAULT,%rax > POP_FRAME_POINTER > ret > -END(copyin) > > /* > * casueword32. Compare and set user integer. Returns -1 on fault, > * 0 if access was successful. Old value is written to *oldp. > * dst = %rdi, old = %esi, oldp = %rdx, new = %ecx > */ > -ENTRY(casueword32) > +ENTRY(casueword32_nosmap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%r8 > movq $fusufault,PCB_ONFAULT(%r8) > @@ -370,14 +453,50 @@ ENTRY(casueword32) > movl %esi,(%rdx) /* oldp = %rdx */ > POP_FRAME_POINTER > ret > -END(casueword32) > +END(casueword32_nosmap) > > +ENTRY(casueword32_smap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%r8 > + movq $fusufault,PCB_ONFAULT(%r8) > + > + movq $VM_MAXUSER_ADDRESS-4,%rax > + cmpq %rax,%rdi /* verify address is valid */ > + ja fusufault > + > + movl %esi,%eax /* old */ > + stac > +#ifdef SMP > + lock > +#endif > + cmpxchgl %ecx,(%rdi) /* new = %ecx */ > + clac > + > + /* > + * The old value is in %eax. If the store succeeded it will be the > + * value we expected (old) from before the store, otherwise it will > + * be the current value. Save %eax into %esi to prepare the return > + * value. > + */ > + movl %eax,%esi > + xorl %eax,%eax > + movq %rax,PCB_ONFAULT(%r8) > + > + /* > + * Access the oldp after the pcb_onfault is cleared, to correctly > + * catch corrupted pointer. > + */ > + movl %esi,(%rdx) /* oldp = %rdx */ > + POP_FRAME_POINTER > + ret > +END(casueword32_smap) > + > /* > * casueword. Compare and set user long. Returns -1 on fault, > * 0 if access was successful. Old value is written to *oldp. > * dst = %rdi, old = %rsi, oldp = %rdx, new = %rcx > */ > -ENTRY(casueword) > +ENTRY(casueword_nosmap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%r8 > movq $fusufault,PCB_ONFAULT(%r8) > @@ -403,16 +522,45 @@ ENTRY(casueword) > movq %rsi,(%rdx) > POP_FRAME_POINTER > ret > -END(casueword) > +END(casueword_nosmap) > > +ENTRY(casueword_smap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%r8 > + movq $fusufault,PCB_ONFAULT(%r8) > + > + movq $VM_MAXUSER_ADDRESS-4,%rax > + cmpq %rax,%rdi /* verify address is valid */ > + ja fusufault > + > + movq %rsi,%rax /* old */ > + stac > +#ifdef SMP > + lock > +#endif > + cmpxchgq %rcx,(%rdi) /* new = %rcx */ > + clac > + > + /* > + * The old value is in %rax. If the store succeeded it will be the > + * value we expected (old) from before the store, otherwise it will > + * be the current value. > + */ > + movq %rax,%rsi > + xorl %eax,%eax > + movq %rax,PCB_ONFAULT(%r8) > + movq %rsi,(%rdx) > + POP_FRAME_POINTER > + ret > +END(casueword_smap) > + > /* > * Fetch (load) a 64-bit word, a 32-bit word, a 16-bit word, or an 8-bit > * byte from user memory. > * addr = %rdi, valp = %rsi > */ > > -ALTENTRY(fueword64) > -ENTRY(fueword) > +ENTRY(fueword_nosmap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > @@ -427,14 +575,32 @@ ENTRY(fueword) > movq %r11,(%rsi) > POP_FRAME_POINTER > ret > -END(fueword64) > -END(fueword) > +END(fueword64_nosmap) > > -ENTRY(fueword32) > +ENTRY(fueword_smap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > > + movq $VM_MAXUSER_ADDRESS-8,%rax > + cmpq %rax,%rdi /* verify address is valid */ > + ja fusufault > + > + xorl %eax,%eax > + stac > + movq (%rdi),%r11 > + clac > + movq %rax,PCB_ONFAULT(%rcx) > + movq %r11,(%rsi) > + POP_FRAME_POINTER > + ret > +END(fueword64_smap) > + > +ENTRY(fueword32_nosmap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > movq $VM_MAXUSER_ADDRESS-4,%rax > cmpq %rax,%rdi /* verify address is valid */ > ja fusufault > @@ -445,13 +611,32 @@ ENTRY(fueword32) > movl %r11d,(%rsi) > POP_FRAME_POINTER > ret > -END(fueword32) > +END(fueword32_nosmap) > > -ENTRY(fuword16) > +ENTRY(fueword32_smap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > > + movq $VM_MAXUSER_ADDRESS-4,%rax > + cmpq %rax,%rdi /* verify address is valid */ > + ja fusufault > + > + xorl %eax,%eax > + stac > + movl (%rdi),%r11d > + clac > + movq %rax,PCB_ONFAULT(%rcx) > + movl %r11d,(%rsi) > + POP_FRAME_POINTER > + ret > +END(fueword32_smap) > + > +ENTRY(fuword16_nosmap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > movq $VM_MAXUSER_ADDRESS-2,%rax > cmpq %rax,%rdi > ja fusufault > @@ -460,13 +645,30 @@ ENTRY(fuword16) > movq $0,PCB_ONFAULT(%rcx) > POP_FRAME_POINTER > ret > -END(fuword16) > +END(fuword16_nosmap) > > -ENTRY(fubyte) > +ENTRY(fuword16_smap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > > + movq $VM_MAXUSER_ADDRESS-2,%rax > + cmpq %rax,%rdi > + ja fusufault > + > + stac > + movzwl (%rdi),%eax > + clac > + movq $0,PCB_ONFAULT(%rcx) > + POP_FRAME_POINTER > + ret > +END(fuword16_smap) > + > +ENTRY(fubyte_nosmap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > movq $VM_MAXUSER_ADDRESS-1,%rax > cmpq %rax,%rdi > ja fusufault > @@ -475,9 +677,27 @@ ENTRY(fubyte) > movq $0,PCB_ONFAULT(%rcx) > POP_FRAME_POINTER > ret > -END(fubyte) > +END(fubyte_nosmap) > > +ENTRY(fubyte_smap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > + movq $VM_MAXUSER_ADDRESS-1,%rax > + cmpq %rax,%rdi > + ja fusufault > + > + stac > + movzbl (%rdi),%eax > + clac > + movq $0,PCB_ONFAULT(%rcx) > + POP_FRAME_POINTER > + ret > +END(fubyte_smap) > + > ALIGN_TEXT > + /* Fault entry clears PSL.AC */ > fusufault: > movq PCPU(CURPCB),%rcx > xorl %eax,%eax > @@ -491,8 +711,7 @@ fusufault: > * user memory. > * addr = %rdi, value = %rsi > */ > -ALTENTRY(suword64) > -ENTRY(suword) > +ENTRY(suword_nosmap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > @@ -507,14 +726,32 @@ ENTRY(suword) > movq %rax,PCB_ONFAULT(%rcx) > POP_FRAME_POINTER > ret > -END(suword64) > -END(suword) > +END(suword_nosmap) > > -ENTRY(suword32) > +ENTRY(suword_smap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > > + movq $VM_MAXUSER_ADDRESS-8,%rax > + cmpq %rax,%rdi /* verify address validity */ > + ja fusufault > + > + stac > + movq %rsi,(%rdi) > + clac > + xorl %eax,%eax > + movq PCPU(CURPCB),%rcx > + movq %rax,PCB_ONFAULT(%rcx) > + POP_FRAME_POINTER > + ret > +END(suword_smap) > + > +ENTRY(suword32_nosmap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > movq $VM_MAXUSER_ADDRESS-4,%rax > cmpq %rax,%rdi /* verify address validity */ > ja fusufault > @@ -525,13 +762,32 @@ ENTRY(suword32) > movq %rax,PCB_ONFAULT(%rcx) > POP_FRAME_POINTER > ret > -END(suword32) > +END(suword32_nosmap) > > -ENTRY(suword16) > +ENTRY(suword32_smap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > > + movq $VM_MAXUSER_ADDRESS-4,%rax > + cmpq %rax,%rdi /* verify address validity */ > + ja fusufault > + > + stac > + movl %esi,(%rdi) > + clac > + xorl %eax,%eax > + movq PCPU(CURPCB),%rcx > + movq %rax,PCB_ONFAULT(%rcx) > + POP_FRAME_POINTER > + ret > +END(suword32_smap) > + > +ENTRY(suword16_nosmap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > movq $VM_MAXUSER_ADDRESS-2,%rax > cmpq %rax,%rdi /* verify address validity */ > ja fusufault > @@ -542,13 +798,32 @@ ENTRY(suword16) > movq %rax,PCB_ONFAULT(%rcx) > POP_FRAME_POINTER > ret > -END(suword16) > +END(suword16_nosmap) > > -ENTRY(subyte) > +ENTRY(suword16_smap) > PUSH_FRAME_POINTER > movq PCPU(CURPCB),%rcx > movq $fusufault,PCB_ONFAULT(%rcx) > > + movq $VM_MAXUSER_ADDRESS-2,%rax > + cmpq %rax,%rdi /* verify address validity */ > + ja fusufault > + > + stac > + movw %si,(%rdi) > + clac > + xorl %eax,%eax > + movq PCPU(CURPCB),%rcx /* restore trashed register */ > + movq %rax,PCB_ONFAULT(%rcx) > + POP_FRAME_POINTER > + ret > +END(suword16_smap) > + > +ENTRY(subyte_nosmap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > movq $VM_MAXUSER_ADDRESS-1,%rax > cmpq %rax,%rdi /* verify address validity */ > ja fusufault > @@ -560,8 +835,28 @@ ENTRY(subyte) > movq %rax,PCB_ONFAULT(%rcx) > POP_FRAME_POINTER > ret > -END(subyte) > +END(subyte_nosmap) > > +ENTRY(subyte_smap) > + PUSH_FRAME_POINTER > + movq PCPU(CURPCB),%rcx > + movq $fusufault,PCB_ONFAULT(%rcx) > + > + movq $VM_MAXUSER_ADDRESS-1,%rax > + cmpq %rax,%rdi /* verify address validity */ > + ja fusufault > + > + movl %esi,%eax > + stac > + movb %al,(%rdi) > + clac > + xorl %eax,%eax > + movq PCPU(CURPCB),%rcx /* restore trashed register */ > + movq %rax,PCB_ONFAULT(%rcx) > + POP_FRAME_POINTER > + ret > +END(subyte_smap) > + > /* > * copyinstr(from, to, maxlen, int *lencopied) > * %rdi, %rsi, %rdx, %rcx > @@ -571,7 +866,7 @@ END(subyte) > * EFAULT on protection violations. If lencopied is non-zero, > * return the actual length in *lencopied. > */ > -ENTRY(copyinstr) > +ENTRY(copyinstr_nosmap) > PUSH_FRAME_POINTER > movq %rdx,%r8 /* %r8 = maxlen */ > movq %rcx,%r9 /* %r9 = *len */ > @@ -592,29 +887,67 @@ ENTRY(copyinstr) > movq %rax,%r8 > 1: > incq %rdx > + cld > > 2: > decq %rdx > - jz 3f > + jz copyinstr_toolong > > lodsb > stosb > orb %al,%al > jnz 2b > > + jmp copyinstr_succ > +END(copyinstr_nosmap) > + > +ENTRY(copyinstr_smap) > + PUSH_FRAME_POINTER > + movq %rdx,%r8 /* %r8 = maxlen */ > + movq %rcx,%r9 /* %r9 = *len */ > + xchgq %rdi,%rsi /* %rdi = from, %rsi = to */ > + movq PCPU(CURPCB),%rcx > + movq $cpystrflt,PCB_ONFAULT(%rcx) > + > + movq $VM_MAXUSER_ADDRESS,%rax > + > + /* make sure 'from' is within bounds */ > + subq %rsi,%rax > + jbe cpystrflt > + > + /* restrict maxlen to <= VM_MAXUSER_ADDRESS-from */ > + cmpq %rdx,%rax > + jae 1f > + movq %rax,%rdx > + movq %rax,%r8 > +1: > + incq %rdx missing cld here > + > +2: > + decq %rdx > + jz copyinstr_succ > + > + stac > + lodsb > + stosb > + clac > + orb %al,%al > + jnz 2b > + > +copyinstr_succ: > /* Success -- 0 byte reached */ > decq %rdx > xorl %eax,%eax > jmp cpystrflt_x > -3: > +copyinstr_toolong: > /* rdx is zero - return ENAMETOOLONG or EFAULT */ > movq $VM_MAXUSER_ADDRESS,%rax > cmpq %rax,%rsi > jae cpystrflt > -4: > movq $ENAMETOOLONG,%rax > jmp cpystrflt_x > > + /* Fault entry clears PSL.AC */ > cpystrflt: > movq $EFAULT,%rax > > @@ -630,7 +963,7 @@ cpystrflt_x: cpystrflt_x: /* set *lencopied and return %eax */ movq PCPU(CURPCB),%rcx movq $0,PCB_ONFAULT(%rcx) testq %r9,%r9 jz 1f subq %rdx,%r8 movq %r8,(%r9) << Here you access user-space, with cleared RFLAGS.AC from the fault handler. 1: POP_FRAME_POINTER ret > 1: > POP_FRAME_POINTER > ret > -END(copyinstr) > +END(copyinstr_smap) > > /* > * copystr(from, to, maxlen, int *lencopied) > > Modified: head/sys/amd64/amd64/trap.c > ============================================================================== > --- head/sys/amd64/amd64/trap.c Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/amd64/trap.c Sun Jul 29 20:47:00 2018 (r336876) > @@ -673,6 +673,24 @@ trap_check(struct trapframe *frame) > trap(frame); > } > > +static bool > +trap_is_smap(struct trapframe *frame) > +{ > + > + /* > + * A page fault on a userspace address is classified as > + * SMAP-induced if: > + * - SMAP is supported; > + * - kernel mode accessed present data page; > + * - rflags.AC was cleared. > + * Kernel must never access user space with rflags.AC cleared > + * if SMAP is enabled. > + */ > + return ((cpu_stdext_feature & CPUID_STDEXT_SMAP) != 0 && > + (frame->tf_err & (PGEX_P | PGEX_U | PGEX_I | PGEX_RSV)) == > + PGEX_P && (frame->tf_rflags & PSL_AC) == 0); > +} > + > static int > trap_pfault(struct trapframe *frame, int usermode) > { > @@ -750,9 +768,13 @@ trap_pfault(struct trapframe *frame, int usermode) > * handling routine. Since accessing the address > * without the handler is a bug, do not try to handle > * it normally, and panic immediately. > + * > + * If SMAP is enabled, filter SMAP faults also, > + * because illegal access might occur to the mapped > + * user address, causing infinite loop. > */ > if (!usermode && (td->td_intr_nesting_level != 0 || > - curpcb->pcb_onfault == NULL)) { > + trap_is_smap(frame) || curpcb->pcb_onfault == NULL)) { > trap_fatal(frame, eva); > return (-1); > } > > Modified: head/sys/amd64/ia32/ia32_exception.S > ============================================================================== > --- head/sys/amd64/ia32/ia32_exception.S Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/ia32/ia32_exception.S Sun Jul 29 20:47:00 2018 (r336876) > @@ -70,7 +70,9 @@ int0x80_syscall_common: > movq %r14,TF_R14(%rsp) > movq %r15,TF_R15(%rsp) > movl $TF_HASSEGS,TF_FLAGS(%rsp) > - cld > + pushfq > + andq $~(PSL_D | PSL_AC),(%rsp) > + popfq > FAKE_MCOUNT(TF_RIP(%rsp)) > movq %rsp, %rdi > call ia32_syscall > > Modified: head/sys/amd64/include/asmacros.h > ============================================================================== > --- head/sys/amd64/include/asmacros.h Sun Jul 29 20:34:44 2018 (r336875) > +++ head/sys/amd64/include/asmacros.h Sun Jul 29 20:47:00 2018 (r336876) > @@ -255,7 +255,9 @@ X\vec_name: > movq %r15,TF_R15(%rsp) > SAVE_SEGS > > *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** > _______________________________________________ > svn-src-head@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-head > To unsubscribe, send any mail to "svn-src-head-unsubscribe@freebsd.org" > From owner-svn-src-all@freebsd.org Sun Jul 29 22:32:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3825A10658C8; Sun, 29 Jul 2018 22:32:10 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9B60B7D0D5; Sun, 29 Jul 2018 22:32:09 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w6TMVwuN070784 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 30 Jul 2018 01:32:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w6TMVwuN070784 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w6TMVwsq070783; Mon, 30 Jul 2018 01:31:58 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 30 Jul 2018 01:31:58 +0300 From: Konstantin Belousov To: Oliver Pinter Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336876 - in head/sys: amd64/amd64 amd64/ia32 amd64/include conf dev/hyperv/vmbus/amd64 Message-ID: <20180729223158.GL40119@kib.kiev.ua> References: <201807292047.w6TKl0hV004691@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 22:32:10 -0000 Please trim useless content. Did I missed anything interesting in your mail ? On Sun, Jul 29, 2018 at 11:57:47PM +0200, Oliver Pinter wrote: > On 7/29/18, Konstantin Belousov wrote: > > +ENTRY(copyin_smap) > > + PUSH_FRAME_POINTER > > + movq PCPU(CURPCB),%rax > > + movq $copyin_fault,PCB_ONFAULT(%rax) > > + testq %rdx,%rdx /* anything to do? */ > > + jz done_copyin > > + > > + /* > > + * make sure address is valid > > + */ > > + movq %rdi,%rax > > + addq %rdx,%rax > > + jc copyin_fault > > + movq $VM_MAXUSER_ADDRESS,%rcx > > + cmpq %rcx,%rax > > + ja copyin_fault > > + > > + xchgq %rdi,%rsi > > + movq %rdx,%rcx > > + movb %cl,%al > > + shrq $3,%rcx /* copy longword-wise */ > > missing cld from here In fact not. It is copyin_nosmap that got unneeded cld. See r327820, apparently I mis-merged this commit into the SMAP branch. > > > + stac > > + rep > > + movsq > > + movb %al,%cl > > + andb $7,%cl /* copy remaining bytes */ > > je done_copyin > > rep > > movsb > > + clac > > +ENTRY(copyinstr_smap) > > + PUSH_FRAME_POINTER > > + movq %rdx,%r8 /* %r8 = maxlen */ > > + movq %rcx,%r9 /* %r9 = *len */ > > + xchgq %rdi,%rsi /* %rdi = from, %rsi = to */ > > + movq PCPU(CURPCB),%rcx > > + movq $cpystrflt,PCB_ONFAULT(%rcx) > > + > > + movq $VM_MAXUSER_ADDRESS,%rax > > + > > + /* make sure 'from' is within bounds */ > > + subq %rsi,%rax > > + jbe cpystrflt > > + > > + /* restrict maxlen to <= VM_MAXUSER_ADDRESS-from */ > > + cmpq %rdx,%rax > > + jae 1f > > + movq %rax,%rdx > > + movq %rax,%r8 > > +1: > > + incq %rdx > > missing cld here Same. > > > + > > +2: > > + decq %rdx > > + jz copyinstr_succ > > cpystrflt_x: > /* set *lencopied and return %eax */ > movq PCPU(CURPCB),%rcx > movq $0,PCB_ONFAULT(%rcx) > > testq %r9,%r9 > jz 1f > subq %rdx,%r8 > movq %r8,(%r9) << Here you access user-space, with cleared > RFLAGS.AC from the fault handler. How does this instruction access userspace ? I do not see. > 1: > POP_FRAME_POINTER > ret So the patch below removes unneeded (mismerged) cld's left in the support.S. diff --git a/sys/amd64/amd64/support.S b/sys/amd64/amd64/support.S index 9b8b2a40461..0aa307e6895 100644 --- a/sys/amd64/amd64/support.S +++ b/sys/amd64/amd64/support.S @@ -307,7 +307,6 @@ ENTRY(copyout_smap) movq %rdx,%rcx shrq $3,%rcx - cld stac rep movsq @@ -358,7 +357,6 @@ ENTRY(copyin_nosmap) movq %rdx,%rcx movb %cl,%al shrq $3,%rcx /* copy longword-wise */ - cld rep movsq movb %al,%cl @@ -887,7 +885,6 @@ ENTRY(copyinstr_nosmap) movq %rax,%r8 1: incq %rdx - cld 2: decq %rdx From owner-svn-src-all@freebsd.org Sun Jul 29 22:48:13 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 052FB1065DB6; Sun, 29 Jul 2018 22:48:13 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A1FE67D754; Sun, 29 Jul 2018 22:48:12 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mousie.catspoiler.org (unknown [76.212.85.177]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) (Authenticated sender: truckman) by smtp.freebsd.org (Postfix) with ESMTPSA id B8FCB25033; Sun, 29 Jul 2018 22:48:11 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Date: Sun, 29 Jul 2018 15:48:10 -0700 (PDT) From: Don Lewis Subject: Re: svn commit: r336859 - head/sbin/init/rc.d To: Ian Lepore cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org In-Reply-To: <1532885755.61594.122.camel@freebsd.org> Message-ID: References: <201807290542.w6T5g8rt018138@repo.freebsd.org> <1532873787.61594.107.camel@freebsd.org> <1532885755.61594.122.camel@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Content-Disposition: INLINE X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 22:48:13 -0000 On 29 Jul, Ian Lepore wrote: > On Sun, 2018-07-29 at 09:59 -0700, Don Lewis wrote: >> On 29 Jul, Ian Lepore wrote: >> > On Sun, 2018-07-29 at 05:42 +0000, Don Lewis wrote: >> >> Author: truckman >> >> Date: Sun Jul 29 05:42:07 2018 >> >> New Revision: 336859 >> >> URL: https://svnweb.freebsd.org/changeset/base/336859 >> >>=A0 >> >> Log: >> >> =A0 Fix a variable name typo in r336845 that prevented the rc.d scrip= ts >> >> =A0 from being installed in the correct directory. >> >> =A0=A0 >> >> =A0 Resurrect a few rc.d scripts that were prematurely deleted from t= he >> >> =A0 Makefile by r336845. >> >> =A0=A0 >> >> =A0 Reviewed by:=A0=A0=A0=A0=A0=A0=A0brd >> >>=A0 >> >> Modified: >> >> =A0 head/sbin/init/rc.d/Makefile >> >>=A0 >> >> Modified: head/sbin/init/rc.d/Makefile >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> --- head/sbin/init/rc.d/Makefile=A0=A0=A0=A0=A0Sun Jul 29 05:14:26 20= 18=A0=A0=A0=A0=A0=A0=A0=A0 >> >> (r336858) >> >> +++ head/sbin/init/rc.d/Makefile=A0=A0=A0=A0=A0Sun Jul 29 05:42:07 20= 18=A0=A0=A0=A0=A0=A0=A0=A0 >> >> (r336859) >> >> @@ -2,7 +2,7 @@ >> >> =A0 >> >> =A0.include=20 >> >> =A0 >> >> -CONFSDIR=3D=A0=A0=A0/etc/rc.d >> >> +CONFDIR=3D=A0=A0=A0=A0/etc/rc.d >> >> =A0CONFGROUPS=3D=A0CONFS >> >> =A0 >> >=A0 >> > I'm pretty sure CONFSDIR was right here. You are now trapped in a maze >> > of small twisty variable names, all alike. >>=20 >> Without this change, the rc.d scripts get installed directly under >> /var/tmp/temproot/etc/ and mergemaster wants to delete the scripts under >> /etc/rc.d.=A0 Answering yes to that mergemaster question results in an >> unbootable system. >>=20 >> I see the following in /usr/share/mk/bsd.confs.mk: >> =A0 ${group}DIR?=3D=A0=A0 ${CONFDIR} >> but I don't see ${CONFSDIR} anywhere. >>=20 >=20 > You don't see CONFSDIR because it appears as ${group}DIR and with > CONFGROUPS?=3D CONFS that turns into CONFSDIR. So now the question is why > doesn't setting CONFSDIR work in this makefile, but it works in others? I don't know, but with undoing my change with this: Index: sbin/init/rc.d/Makefile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sbin/init/rc.d/Makefile=09(revision 336859) +++ sbin/init/rc.d/Makefile=09(working copy) @@ -2,7 +2,7 @@ =20 .include =20 -CONFDIR=3D=09/etc/rc.d +CONFSDIR=3D=09/etc/rc.d CONFGROUPS=3D=09CONFS =20 CONFS=3D=09DAEMON \ gives me this in mergemaster: *** Beginning comparison *** Checking /etc/rc.d for stale files *** The following files exist in /etc/rc.d but not in /var/tmp/temproot/etc/rc.d/: accounting amd apm bsnmpd hastd jail local_unbound power_profile sendmail = sshd virecover zfs zfsbe zfsd zvol The presence of stale files in this directory can cause the dreaded unpredictable results, and therefore it is highly recommended that you delete them. *** Delete them now? [n]=20 From owner-svn-src-all@freebsd.org Sun Jul 29 23:26:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C74521066AB5 for ; Sun, 29 Jul 2018 23:26:52 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: from mail-yw0-x22b.google.com (mail-yw0-x22b.google.com [IPv6:2607:f8b0:4002:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63F267EA2D for ; Sun, 29 Jul 2018 23:26:52 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: by mail-yw0-x22b.google.com with SMTP id k18-v6so3746898ywm.11 for ; Sun, 29 Jul 2018 16:26:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=XYRuRUVmntnbYdM6jiMcV82dPI8UhKg76nxLlKYvlqI=; b=GM94xX2igvUNhfJDeKYZ9mtPjdmQD4uYqnWaWIRc/xiN4+oUCA4CmEOE8JF73GLvDm fdLjVfrBJ6YVMioaKlG7/bX2YO4KGThNp459cRh2fBc6XiPbC3BV+xkof6RUWGIZqYRI nQGI08y9xutSiuwL7BQNUGoXQTvGG7m+nRjTulOpD2Wax7DHxQrFqt5J74SaiXsd0kvN aqGNLYciTxUSAntggExjBKZp4gP8WBUdyfnkTMKmjBLAVX++W3YheqgTtZZSpH4qRJ3s aXzjvOELYL7sgXtXz4Og+0JdXJzipj78NJuqqG1gMtd3f5It4ktrGFMucXRTe2gEFVq6 5hAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=XYRuRUVmntnbYdM6jiMcV82dPI8UhKg76nxLlKYvlqI=; b=YxmYB8AV0nzywyFYT27IP987Yy7J9jVXSEg2VSUPwT98aC1WjCYxFE9m6MI35zzgEK xkRtSEM0FUWnfJQ3Z+yLQDCBi/2oSsTrTa7VZ5YJaPT/XPDH3LRI3wpsQXnL77CF0Ptr 0VNZsHxJdhg14A9OiQvIPu2/Wq3aQq3zgBhxczTDB5d67c196/FDGj8wm0+vZXq5/gGv dCpPuKHdtJw/iyn2eFqi4Qure9TC/TUVbl1Akc8cPeGglqUrJ3L9hJQog2W4hSiKTbLJ 0+Zdn0GKiCktJmKhaBhigNhQf2DViawjWmw58Tl92vM8HWmzcCM2j7t0eQlA+5YVGet+ M3Zw== X-Gm-Message-State: AOUpUlHROBQhrta5X46wLsNn7KiS4J955LmZXQPtrPQZJQ5N/PQl4xhd ugXP256qq2jlQmbigiMM3c4xeZzdaubkNiTr1VISsg== X-Google-Smtp-Source: AAOMgpcLf0MNcV6usSR8TrL3E/USvPM8l8AOZpA/j4iRHG3e8Ee4UdoMkU5loKVQR+z7XvjWNpuJpn8RpSb31BIv/eg= X-Received: by 2002:a81:83c3:: with SMTP id t186-v6mr7446412ywf.455.1532906811896; Sun, 29 Jul 2018 16:26:51 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a25:b90:0:0:0:0:0 with HTTP; Sun, 29 Jul 2018 16:26:51 -0700 (PDT) In-Reply-To: <20180729223158.GL40119@kib.kiev.ua> References: <201807292047.w6TKl0hV004691@repo.freebsd.org> <20180729223158.GL40119@kib.kiev.ua> From: Oliver Pinter Date: Mon, 30 Jul 2018 01:26:51 +0200 Message-ID: Subject: Re: svn commit: r336876 - in head/sys: amd64/amd64 amd64/ia32 amd64/include conf dev/hyperv/vmbus/amd64 To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 23:26:53 -0000 On 7/30/18, Konstantin Belousov wrote: > Please trim useless content. > Did I missed anything interesting in your mail ? > > On Sun, Jul 29, 2018 at 11:57:47PM +0200, Oliver Pinter wrote: >> On 7/29/18, Konstantin Belousov wrote: >> > +ENTRY(copyin_smap) >> > + PUSH_FRAME_POINTER >> > + movq PCPU(CURPCB),%rax >> > + movq $copyin_fault,PCB_ONFAULT(%rax) >> > + testq %rdx,%rdx /* anything to do? */ >> > + jz done_copyin >> > + >> > + /* >> > + * make sure address is valid >> > + */ >> > + movq %rdi,%rax >> > + addq %rdx,%rax >> > + jc copyin_fault >> > + movq $VM_MAXUSER_ADDRESS,%rcx >> > + cmpq %rcx,%rax >> > + ja copyin_fault >> > + >> > + xchgq %rdi,%rsi >> > + movq %rdx,%rcx >> > + movb %cl,%al >> > + shrq $3,%rcx /* copy longword-wise */ >> >> missing cld from here > In fact not. It is copyin_nosmap that got unneeded cld. > > See r327820, apparently I mis-merged this commit into the SMAP branch. > >> >> > + stac >> > + rep >> > + movsq >> > + movb %al,%cl >> > + andb $7,%cl /* copy remaining bytes */ >> > je done_copyin >> > rep >> > movsb >> > + clac > >> > +ENTRY(copyinstr_smap) >> > + PUSH_FRAME_POINTER >> > + movq %rdx,%r8 /* %r8 = maxlen */ >> > + movq %rcx,%r9 /* %r9 = *len */ >> > + xchgq %rdi,%rsi /* %rdi = from, %rsi = to */ >> > + movq PCPU(CURPCB),%rcx >> > + movq $cpystrflt,PCB_ONFAULT(%rcx) >> > + >> > + movq $VM_MAXUSER_ADDRESS,%rax >> > + >> > + /* make sure 'from' is within bounds */ >> > + subq %rsi,%rax >> > + jbe cpystrflt >> > + >> > + /* restrict maxlen to <= VM_MAXUSER_ADDRESS-from */ >> > + cmpq %rdx,%rax >> > + jae 1f >> > + movq %rax,%rdx >> > + movq %rax,%r8 >> > +1: >> > + incq %rdx >> >> missing cld here > Same. > >> >> > + >> > +2: >> > + decq %rdx >> > + jz copyinstr_succ >> > >> cpystrflt_x: >> /* set *lencopied and return %eax */ >> movq PCPU(CURPCB),%rcx >> movq $0,PCB_ONFAULT(%rcx) >> >> testq %r9,%r9 >> jz 1f >> subq %rdx,%r8 >> movq %r8,(%r9) << Here you access user-space, with cleared >> RFLAGS.AC from the fault handler. > How does this instruction access userspace ? I do not see. As far as I remember from 4 years, the r9 may contained a user-space address in 10-STABLE in the case of starting the init. I've a stac/clac pair in my internal version, but I haven't found yet the relevant commit message. For a quick grep around - http://ix.io/1fje - I haven't found yet this place, so it's looks good in your version. > >> 1: >> POP_FRAME_POINTER >> ret > > So the patch below removes unneeded (mismerged) cld's left in the > support.S. > > diff --git a/sys/amd64/amd64/support.S b/sys/amd64/amd64/support.S > index 9b8b2a40461..0aa307e6895 100644 > --- a/sys/amd64/amd64/support.S > +++ b/sys/amd64/amd64/support.S > @@ -307,7 +307,6 @@ ENTRY(copyout_smap) > movq %rdx,%rcx > > shrq $3,%rcx > - cld > stac > rep > movsq > @@ -358,7 +357,6 @@ ENTRY(copyin_nosmap) > movq %rdx,%rcx > movb %cl,%al > shrq $3,%rcx /* copy longword-wise */ > - cld > rep > movsq > movb %al,%cl > @@ -887,7 +885,6 @@ ENTRY(copyinstr_nosmap) > movq %rax,%r8 > 1: > incq %rdx > - cld > > 2: > decq %rdx Looks fine to me. > From owner-svn-src-all@freebsd.org Sun Jul 29 23:53:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C42F810670F2; Sun, 29 Jul 2018 23:53:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 16B087F958; Sun, 29 Jul 2018 23:53:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w6TNr1RI088933 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 30 Jul 2018 02:53:04 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w6TNr1RI088933 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w6TNr1GN088932; Mon, 30 Jul 2018 02:53:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 30 Jul 2018 02:53:01 +0300 From: Konstantin Belousov To: Oliver Pinter Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336876 - in head/sys: amd64/amd64 amd64/ia32 amd64/include conf dev/hyperv/vmbus/amd64 Message-ID: <20180729235301.GM40119@kib.kiev.ua> References: <201807292047.w6TKl0hV004691@repo.freebsd.org> <20180729223158.GL40119@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2018 23:53:12 -0000 On Mon, Jul 30, 2018 at 01:26:51AM +0200, Oliver Pinter wrote: > On 7/30/18, Konstantin Belousov wrote: > >> testq %r9,%r9 > >> jz 1f > >> subq %rdx,%r8 > >> movq %r8,(%r9) << Here you access user-space, with cleared > >> RFLAGS.AC from the fault handler. > > How does this instruction access userspace ? I do not see. > > As far as I remember from 4 years, the r9 may contained a user-space > address in 10-STABLE > in the case of starting the init. I've a stac/clac pair in my internal > version, but I haven't found > yet the relevant commit message. This does not make any sense at all. What could be a use for copyinstr(9) to pass a userspace pointer to return the copied string' length ? More, kernel must not directly access userspace, it needs to prepare for these accesses to fault. From owner-svn-src-all@freebsd.org Mon Jul 30 00:04:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D19A10676A3; Mon, 30 Jul 2018 00:04:43 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6A427FEF6; Mon, 30 Jul 2018 00:04:42 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AD8041CC99; Mon, 30 Jul 2018 00:04:42 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U04gNX037260; Mon, 30 Jul 2018 00:04:42 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U04ge9037259; Mon, 30 Jul 2018 00:04:42 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807300004.w6U04ge9037259@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 00:04:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336879 - head X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head X-SVN-Commit-Revision: 336879 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 00:04:43 -0000 Author: emaste Date: Mon Jul 30 00:04:42 2018 New Revision: 336879 URL: https://svnweb.freebsd.org/changeset/base/336879 Log: Correct typo and whitespace in UPDATING PR: 230124 Submitted by: Samy Mahmoudi Modified: head/UPDATING Modified: head/UPDATING ============================================================================== --- head/UPDATING Sun Jul 29 21:51:17 2018 (r336878) +++ head/UPDATING Mon Jul 30 00:04:42 2018 (r336879) @@ -45,7 +45,7 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 12.x IS SLOW: 20180720: zfsloader's functionality has now been folded into loader. - zfsloader is no longer necesasary once you've updated your + zfsloader is no longer necessary once you've updated your boot blocks. For a transition period, we will install a hardlink for zfsloader to loader to allow a smooth transition until the boot blocks can be updated (hard link because old @@ -180,7 +180,7 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 12.x IS SLOW: Support for FDDI networks has been removed. If you have device fddi or device fpa in your kernel config file they must be removed. - + 20180406: In addition to supporting RFC 3164 formatted messages, the syslogd(8) service is now capable of parsing RFC 5424 formatted From owner-svn-src-all@freebsd.org Mon Jul 30 00:08:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73D2510679BC; Mon, 30 Jul 2018 00:08:37 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A8BE802D9; Mon, 30 Jul 2018 00:08:37 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E1ADC1CCA2; Mon, 30 Jul 2018 00:08:36 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U08auX037683; Mon, 30 Jul 2018 00:08:36 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U08aK4037682; Mon, 30 Jul 2018 00:08:36 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807300008.w6U08aK4037682@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 00:08:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336880 - stable/11/contrib/llvm/tools/lld/ELF X-SVN-Group: stable-11 X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: stable/11/contrib/llvm/tools/lld/ELF X-SVN-Commit-Revision: 336880 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 00:08:37 -0000 Author: emaste Date: Mon Jul 30 00:08:36 2018 New Revision: 336880 URL: https://svnweb.freebsd.org/changeset/base/336880 Log: MFC r336664: lld: fix addends with partial linking [ELF] Update addends in non-allocatable sections for REL targets when creating a relocatable output. LLVM PR: 37735 LLVM Differential Revision: https://reviews.llvm.org/D48929 PR: 225128, 228753 Obtained from: LLVM r336799 by Igor Kudrin Modified: stable/11/contrib/llvm/tools/lld/ELF/InputSection.cpp Directory Properties: stable/11/ (props changed) Modified: stable/11/contrib/llvm/tools/lld/ELF/InputSection.cpp ============================================================================== --- stable/11/contrib/llvm/tools/lld/ELF/InputSection.cpp Mon Jul 30 00:04:42 2018 (r336879) +++ stable/11/contrib/llvm/tools/lld/ELF/InputSection.cpp Mon Jul 30 00:08:36 2018 (r336880) @@ -686,6 +686,23 @@ void InputSection::relocateNonAlloc(uint8_t *Buf, Arra } } +// This is used when '-r' is given. +// For REL targets, InputSection::copyRelocations() may store artificial +// relocations aimed to update addends. They are handled in relocateAlloc() +// for allocatable sections, and this function does the same for +// non-allocatable sections, such as sections with debug information. +static void relocateNonAllocForRelocatable(InputSection *Sec, uint8_t *Buf) { + const unsigned Bits = Config->Is64 ? 64 : 32; + + for (const Relocation &Rel : Sec->Relocations) { + // InputSection::copyRelocations() adds only R_ABS relocations. + assert(Rel.Expr == R_ABS); + uint8_t *BufLoc = Buf + Rel.Offset + Sec->OutSecOff; + uint64_t TargetVA = SignExtend64(Rel.Sym->getVA(Rel.Addend), Bits); + Target->relocateOne(BufLoc, Rel.Type, TargetVA); + } +} + template void InputSectionBase::relocate(uint8_t *Buf, uint8_t *BufEnd) { if (Flags & SHF_ALLOC) { @@ -694,7 +711,9 @@ void InputSectionBase::relocate(uint8_t *Buf, uint8_t } auto *Sec = cast(this); - if (Sec->AreRelocsRela) + if (Config->Relocatable) + relocateNonAllocForRelocatable(Sec, Buf); + else if (Sec->AreRelocsRela) Sec->relocateNonAlloc(Buf, Sec->template relas()); else Sec->relocateNonAlloc(Buf, Sec->template rels()); From owner-svn-src-all@freebsd.org Mon Jul 30 01:54:26 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80E0B1069AC1; Mon, 30 Jul 2018 01:54:26 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24D738327C; Mon, 30 Jul 2018 01:54:26 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 003D01DE25; Mon, 30 Jul 2018 01:54:26 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U1sPSJ093236; Mon, 30 Jul 2018 01:54:25 GMT (envelope-from alc@FreeBSD.org) Received: (from alc@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U1sPgO093235; Mon, 30 Jul 2018 01:54:25 GMT (envelope-from alc@FreeBSD.org) Message-Id: <201807300154.w6U1sPgO093235@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: alc set sender to alc@FreeBSD.org using -f From: Alan Cox Date: Mon, 30 Jul 2018 01:54:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336881 - head/sys/arm/arm X-SVN-Group: head X-SVN-Commit-Author: alc X-SVN-Commit-Paths: head/sys/arm/arm X-SVN-Commit-Revision: 336881 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 01:54:26 -0000 Author: alc Date: Mon Jul 30 01:54:25 2018 New Revision: 336881 URL: https://svnweb.freebsd.org/changeset/base/336881 Log: Prepare for adding psind == 1 support to armv6's pmap_enter(). Precompute the new PTE before entering the critical section. Eliminate duplication of the pmap and pv list unlock operations in pmap_enter() by implementing a single return path. Otherwise, the duplication will only increase with the upcoming support for psind == 1. Reviewed by: mmel Tested by: mmel Discussed with: kib, markj MFC after: 3 weeks Differential Revision: https://reviews.freebsd.org/D16443 Modified: head/sys/arm/arm/pmap-v6.c Modified: head/sys/arm/arm/pmap-v6.c ============================================================================== --- head/sys/arm/arm/pmap-v6.c Mon Jul 30 00:08:36 2018 (r336880) +++ head/sys/arm/arm/pmap-v6.c Mon Jul 30 01:54:25 2018 (r336881) @@ -3849,18 +3849,36 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, v pv_entry_t pv; vm_paddr_t opa, pa; vm_page_t mpte2, om; - boolean_t wired; + int rv; va = trunc_page(va); - mpte2 = NULL; - wired = (flags & PMAP_ENTER_WIRED) != 0; - KASSERT(va <= vm_max_kernel_address, ("%s: toobig", __func__)); KASSERT(va < UPT2V_MIN_ADDRESS || va >= UPT2V_MAX_ADDRESS, ("%s: invalid to pmap_enter page table pages (va: 0x%x)", __func__, va)); + KASSERT((m->oflags & VPO_UNMANAGED) != 0 || va < kmi.clean_sva || + va >= kmi.clean_eva, + ("%s: managed mapping within the clean submap", __func__)); if ((m->oflags & VPO_UNMANAGED) == 0 && !vm_page_xbusied(m)) VM_OBJECT_ASSERT_LOCKED(m->object); + KASSERT((flags & PMAP_ENTER_RESERVED) == 0, + ("%s: flags %u has reserved bits set", __func__, flags)); + pa = VM_PAGE_TO_PHYS(m); + npte2 = PTE2(pa, PTE2_A, vm_page_pte2_attr(m)); + if ((flags & VM_PROT_WRITE) == 0) + npte2 |= PTE2_NM; + if ((prot & VM_PROT_WRITE) == 0) + npte2 |= PTE2_RO; + KASSERT((npte2 & (PTE2_NM | PTE2_RO)) != PTE2_RO, + ("pmap_enter: flags includes VM_PROT_WRITE but prot doesn't")); + if ((prot & VM_PROT_EXECUTE) == 0) + npte2 |= PTE2_NX; + if ((flags & PMAP_ENTER_WIRED) != 0) + npte2 |= PTE2_W; + if (va < VM_MAXUSER_ADDRESS) + npte2 |= PTE2_U; + if (pmap != kernel_pmap) + npte2 |= PTE2_NG; rw_wlock(&pvh_global_lock); PMAP_LOCK(pmap); @@ -3875,12 +3893,11 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, v if (mpte2 == NULL) { KASSERT((flags & PMAP_ENTER_NOSLEEP) != 0, ("pmap_allocpte2 failed with sleep allowed")); - sched_unpin(); - rw_wunlock(&pvh_global_lock); - PMAP_UNLOCK(pmap); - return (KERN_RESOURCE_SHORTAGE); + rv = KERN_RESOURCE_SHORTAGE; + goto out; } - } + } else + mpte2 = NULL; pte1p = pmap_pte1(pmap, va); if (pte1_is_section(pte1_load(pte1p))) panic("%s: attempted on 1MB page", __func__); @@ -3889,7 +3906,6 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, v panic("%s: invalid L1 page table entry va=%#x", __func__, va); om = NULL; - pa = VM_PAGE_TO_PHYS(m); opte2 = pte2_load(pte2p); opa = pte2_pa(opte2); /* @@ -3902,9 +3918,9 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, v * are valid mappings in them. Hence, if a user page is wired, * the PT2 page will be also. */ - if (wired && !pte2_is_wired(opte2)) + if (pte2_is_wired(npte2) && !pte2_is_wired(opte2)) pmap->pm_stats.wired_count++; - else if (!wired && pte2_is_wired(opte2)) + else if (!pte2_is_wired(npte2) && pte2_is_wired(opte2)) pmap->pm_stats.wired_count--; /* @@ -3953,11 +3969,10 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, v * Enter on the PV list if part of our managed memory. */ if ((m->oflags & VPO_UNMANAGED) == 0) { - KASSERT(va < kmi.clean_sva || va >= kmi.clean_eva, - ("%s: managed mapping within the clean submap", __func__)); - if (pv == NULL) + if (pv == NULL) { pv = get_pv_entry(pmap, FALSE); - pv->pv_va = va; + pv->pv_va = va; + } TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next); } else if (pv != NULL) free_pv_entry(pmap, pv); @@ -3965,28 +3980,17 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, v /* * Increment counters */ - if (wired) + if (pte2_is_wired(npte2)) pmap->pm_stats.wired_count++; validate: /* * Now validate mapping with desired protection/wiring. */ - npte2 = PTE2(pa, PTE2_NM, vm_page_pte2_attr(m)); if (prot & VM_PROT_WRITE) { if (pte2_is_managed(npte2)) vm_page_aflag_set(m, PGA_WRITEABLE); } - else - npte2 |= PTE2_RO; - if ((prot & VM_PROT_EXECUTE) == 0) - npte2 |= PTE2_NX; - if (wired) - npte2 |= PTE2_W; - if (va < VM_MAXUSER_ADDRESS) - npte2 |= PTE2_U; - if (pmap != kernel_pmap) - npte2 |= PTE2_NG; /* * If the mapping or permission bits are different, we need @@ -4016,9 +4020,6 @@ validate: (opa != pa || (opte2 & PTE2_NX))) cache_icache_sync_fresh(va, pa, PAGE_SIZE); - npte2 |= PTE2_A; - if (flags & VM_PROT_WRITE) - npte2 &= ~PTE2_NM; if (opte2 & PTE2_V) { /* Change mapping with break-before-make approach. */ opte2 = pte2_load_clear(pte2p); @@ -4063,10 +4064,13 @@ validate: vm_reserv_level_iffullpop(m) == 0) pmap_promote_pte1(pmap, pte1p, va); #endif + + rv = KERN_SUCCESS; +out: sched_unpin(); rw_wunlock(&pvh_global_lock); PMAP_UNLOCK(pmap); - return (KERN_SUCCESS); + return (rv); } /* From owner-svn-src-all@freebsd.org Mon Jul 30 04:20:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF7D010497E9; Mon, 30 Jul 2018 04:20:49 +0000 (UTC) (envelope-from lidl@pix.net) Received: from hydra.pix.net (hydra.pix.net [IPv6:2001:470:e254::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.pix.net", Issuer "Pix.Com Technologies LLC CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 85D8286B86; Mon, 30 Jul 2018 04:20:49 +0000 (UTC) (envelope-from lidl@pix.net) Received: from torb.pix.net (torb.pix.net [192.168.16.32]) (authenticated bits=0) by hydra.pix.net (8.15.2/8.15.2) with ESMTPA id w6U4KmMg078444; Mon, 30 Jul 2018 00:20:48 -0400 (EDT) (envelope-from lidl@pix.net) Subject: Re: svn commit: r336465 - in head/sys/netinet: . tcp_stacks To: Maxim Konovalov , Michael Tuexen Cc: Randall Stewart , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201807182249.w6IMns6D076446@repo.freebsd.org> <7FC7DAA2-9B03-4D89-A878-7706EDE4294A@macmic.franken.de> From: Kurt Lidl Message-ID: <91467128-5e84-6518-87ce-5374d410e1af@pix.net> Date: Mon, 30 Jul 2018 00:20:48 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 04:20:50 -0000 On 7/19/18 9:18 AM, Maxim Konovalov wrote: > On Thu, 19 Jul 2018, 08:09-0400, Michael Tuexen wrote: > >>> On 19. Jul 2018, at 03:12, Maxim Konovalov wrote: >>> >>> Hi Randall, >>> >>> On Wed, 18 Jul 2018, 22:49-0000, Randall Stewart wrote: >>> >>>> Author: rrs >>>> Date: Wed Jul 18 22:49:53 2018 >>>> New Revision: 336465 >>>> URL: https://svnweb.freebsd.org/changeset/base/336465 >>>> >>>> Log: >>>> Bump the ICMP echo limits to match the RFC >>>> >>> [...] >>> >>> Just wonder, are there any practical reasons to do that? >> In case you send encapsulated packets triggering an ICMP message >> you actually need more than the 8 bytes which are currently >> reflected. > > OK, let me rephrase: why do you need more than 8 bytes? It looks like > it has been working rather well for 20+ years. Coming late to the game (I was away for vacation)... It's handy to have more than 8 bytes of returned payload for ICMP packets to allow for more sophisticated network health scanning metrics. Back when I worked at UUNET, we used the ICMP ECHO REQUEST packets to carry accurate timestamps for monitoring dispersion of multicast datagrams to select hosts. I know, ICMP ECHO REQUEST packets have required all payload to be returned since at least RFC 1712 - so it's not exactly the same as what is being change here... I imagine that a similar generic treatment of payload data for other ICMP type message might be handy too. -Kurt From owner-svn-src-all@freebsd.org Mon Jul 30 07:01:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8578B104CA01; Mon, 30 Jul 2018 07:01:01 +0000 (UTC) (envelope-from obrien@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A91E8B141; Mon, 30 Jul 2018 07:01:01 +0000 (UTC) (envelope-from obrien@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1CE2E20F2B; Mon, 30 Jul 2018 07:01:01 +0000 (UTC) (envelope-from obrien@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U710Vo047494; Mon, 30 Jul 2018 07:01:00 GMT (envelope-from obrien@FreeBSD.org) Received: (from obrien@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U710uB047483; Mon, 30 Jul 2018 07:01:00 GMT (envelope-from obrien@FreeBSD.org) Message-Id: <201807300701.w6U710uB047483@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: obrien set sender to obrien@FreeBSD.org using -f From: "David E. O'Brien" Date: Mon, 30 Jul 2018 07:01:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336882 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: obrien X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 336882 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 07:01:01 -0000 Author: obrien Date: Mon Jul 30 07:01:00 2018 New Revision: 336882 URL: https://svnweb.freebsd.org/changeset/base/336882 Log: Correct copyright dates. Modified: head/sys/kern/imgact_elf.c Modified: head/sys/kern/imgact_elf.c ============================================================================== --- head/sys/kern/imgact_elf.c Mon Jul 30 01:54:25 2018 (r336881) +++ head/sys/kern/imgact_elf.c Mon Jul 30 07:01:00 2018 (r336882) @@ -2,7 +2,7 @@ * SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 2017 Dell EMC - * Copyright (c) 2000 David O'Brien + * Copyright (c) 2000-2001, 2003 David O'Brien * Copyright (c) 1995-1996 Søren Schmidt * Copyright (c) 1996 Peter Wemm * All rights reserved. From owner-svn-src-all@freebsd.org Mon Jul 30 09:15:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36C6D104FC78; Mon, 30 Jul 2018 09:15:10 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF6D58EEFE; Mon, 30 Jul 2018 09:15:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C0A542254D; Mon, 30 Jul 2018 09:15:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9F9LF018252; Mon, 30 Jul 2018 09:15:09 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9F9x5018251; Mon, 30 Jul 2018 09:15:09 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300915.w6U9F9x5018251@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:15:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336883 - stable/11/lib/libusb X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/lib/libusb X-SVN-Commit-Revision: 336883 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:15:10 -0000 Author: hselasky Date: Mon Jul 30 09:15:09 2018 New Revision: 336883 URL: https://svnweb.freebsd.org/changeset/base/336883 Log: MFC r335669: Improve the userspace USB string reading function in LibUSB. Some USB devices does not allow a partial descriptor readout. Found by: bz @ Sponsored by: Mellanox Technologies Modified: stable/11/lib/libusb/libusb20.c Directory Properties: stable/11/ (props changed) Modified: stable/11/lib/libusb/libusb20.c ============================================================================== --- stable/11/lib/libusb/libusb20.c Mon Jul 30 07:01:00 2018 (r336882) +++ stable/11/lib/libusb/libusb20.c Mon Jul 30 09:15:09 2018 (r336883) @@ -812,6 +812,7 @@ libusb20_dev_req_string_sync(struct libusb20_device *p { struct LIBUSB20_CONTROL_SETUP_DECODED req; int error; + int flags; /* make sure memory is initialised */ memset(ptr, 0, len); @@ -838,22 +839,24 @@ libusb20_dev_req_string_sync(struct libusb20_device *p error = libusb20_dev_request_sync(pdev, &req, ptr, NULL, 1000, LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK); if (error) { - return (error); + /* try to request full string */ + req.wLength = 255; + flags = 0; + } else { + /* extract length and request full string */ + req.wLength = *(uint8_t *)ptr; + flags = LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK; } - req.wLength = *(uint8_t *)ptr; /* bytes */ if (req.wLength > len) { /* partial string read */ req.wLength = len; } - error = libusb20_dev_request_sync(pdev, &req, - ptr, NULL, 1000, LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK); - - if (error) { + error = libusb20_dev_request_sync(pdev, &req, ptr, NULL, 1000, flags); + if (error) return (error); - } - if (((uint8_t *)ptr)[1] != LIBUSB20_DT_STRING) { + + if (((uint8_t *)ptr)[1] != LIBUSB20_DT_STRING) return (LIBUSB20_ERROR_OTHER); - } return (0); /* success */ } From owner-svn-src-all@freebsd.org Mon Jul 30 09:16:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8362A104FD97; Mon, 30 Jul 2018 09:16:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 384FD8F164; Mon, 30 Jul 2018 09:16:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 196BC2254F; Mon, 30 Jul 2018 09:16:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9GlTU018493; Mon, 30 Jul 2018 09:16:47 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9Gldr018492; Mon, 30 Jul 2018 09:16:47 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300916.w6U9Gldr018492@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:16:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r336884 - stable/10/lib/libusb X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/10/lib/libusb X-SVN-Commit-Revision: 336884 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:16:48 -0000 Author: hselasky Date: Mon Jul 30 09:16:47 2018 New Revision: 336884 URL: https://svnweb.freebsd.org/changeset/base/336884 Log: MFC r335669: Improve the userspace USB string reading function in LibUSB. Some USB devices does not allow a partial descriptor readout. Found by: bz @ Sponsored by: Mellanox Technologies Modified: stable/10/lib/libusb/libusb20.c Directory Properties: stable/10/ (props changed) Modified: stable/10/lib/libusb/libusb20.c ============================================================================== --- stable/10/lib/libusb/libusb20.c Mon Jul 30 09:15:09 2018 (r336883) +++ stable/10/lib/libusb/libusb20.c Mon Jul 30 09:16:47 2018 (r336884) @@ -794,6 +794,7 @@ libusb20_dev_req_string_sync(struct libusb20_device *p { struct LIBUSB20_CONTROL_SETUP_DECODED req; int error; + int flags; /* make sure memory is initialised */ memset(ptr, 0, len); @@ -820,22 +821,24 @@ libusb20_dev_req_string_sync(struct libusb20_device *p error = libusb20_dev_request_sync(pdev, &req, ptr, NULL, 1000, LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK); if (error) { - return (error); + /* try to request full string */ + req.wLength = 255; + flags = 0; + } else { + /* extract length and request full string */ + req.wLength = *(uint8_t *)ptr; + flags = LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK; } - req.wLength = *(uint8_t *)ptr; /* bytes */ if (req.wLength > len) { /* partial string read */ req.wLength = len; } - error = libusb20_dev_request_sync(pdev, &req, - ptr, NULL, 1000, LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK); - - if (error) { + error = libusb20_dev_request_sync(pdev, &req, ptr, NULL, 1000, flags); + if (error) return (error); - } - if (((uint8_t *)ptr)[1] != LIBUSB20_DT_STRING) { + + if (((uint8_t *)ptr)[1] != LIBUSB20_DT_STRING) return (LIBUSB20_ERROR_OTHER); - } return (0); /* success */ } From owner-svn-src-all@freebsd.org Mon Jul 30 09:18:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0BD9104FF0A; Mon, 30 Jul 2018 09:18:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 563E98F395; Mon, 30 Jul 2018 09:18:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3402122550; Mon, 30 Jul 2018 09:18:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9Ik06018628; Mon, 30 Jul 2018 09:18:46 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9Ike0018627; Mon, 30 Jul 2018 09:18:46 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300918.w6U9Ike0018627@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:18:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-9@freebsd.org Subject: svn commit: r336885 - stable/9/lib/libusb X-SVN-Group: stable-9 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/9/lib/libusb X-SVN-Commit-Revision: 336885 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:18:46 -0000 Author: hselasky Date: Mon Jul 30 09:18:45 2018 New Revision: 336885 URL: https://svnweb.freebsd.org/changeset/base/336885 Log: MFC r335669: Improve the userspace USB string reading function in LibUSB. Some USB devices does not allow a partial descriptor readout. Found by: bz @ Sponsored by: Mellanox Technologies Modified: stable/9/lib/libusb/libusb20.c Directory Properties: stable/9/lib/ (props changed) stable/9/lib/libusb/ (props changed) Modified: stable/9/lib/libusb/libusb20.c ============================================================================== --- stable/9/lib/libusb/libusb20.c Mon Jul 30 09:16:47 2018 (r336884) +++ stable/9/lib/libusb/libusb20.c Mon Jul 30 09:18:45 2018 (r336885) @@ -783,6 +783,7 @@ libusb20_dev_req_string_sync(struct libusb20_device *p { struct LIBUSB20_CONTROL_SETUP_DECODED req; int error; + int flags; /* make sure memory is initialised */ memset(ptr, 0, len); @@ -809,22 +810,24 @@ libusb20_dev_req_string_sync(struct libusb20_device *p error = libusb20_dev_request_sync(pdev, &req, ptr, NULL, 1000, LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK); if (error) { - return (error); + /* try to request full string */ + req.wLength = 255; + flags = 0; + } else { + /* extract length and request full string */ + req.wLength = *(uint8_t *)ptr; + flags = LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK; } - req.wLength = *(uint8_t *)ptr; /* bytes */ if (req.wLength > len) { /* partial string read */ req.wLength = len; } - error = libusb20_dev_request_sync(pdev, &req, - ptr, NULL, 1000, LIBUSB20_TRANSFER_SINGLE_SHORT_NOT_OK); - - if (error) { + error = libusb20_dev_request_sync(pdev, &req, ptr, NULL, 1000, flags); + if (error) return (error); - } - if (((uint8_t *)ptr)[1] != LIBUSB20_DT_STRING) { + + if (((uint8_t *)ptr)[1] != LIBUSB20_DT_STRING) return (LIBUSB20_ERROR_OTHER); - } return (0); /* success */ } From owner-svn-src-all@freebsd.org Mon Jul 30 09:20:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4090910500BB; Mon, 30 Jul 2018 09:20:18 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E9E678F57F; Mon, 30 Jul 2018 09:20:17 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B1C5022555; Mon, 30 Jul 2018 09:20:17 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9KH3c018784; Mon, 30 Jul 2018 09:20:17 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9KHcc018783; Mon, 30 Jul 2018 09:20:17 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300920.w6U9KHcc018783@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:20:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336886 - stable/11/sys/dev/usb X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/usb X-SVN-Commit-Revision: 336886 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:20:18 -0000 Author: hselasky Date: Mon Jul 30 09:20:17 2018 New Revision: 336886 URL: https://svnweb.freebsd.org/changeset/base/336886 Log: MFC r335700: Improve the kernel's USB descriptor reading function. Some USB devices does not allow a partial descriptor readout. Found by: bz@ Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/usb/usb_request.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/usb/usb_request.c ============================================================================== --- stable/11/sys/dev/usb/usb_request.c Mon Jul 30 09:18:45 2018 (r336885) +++ stable/11/sys/dev/usb/usb_request.c Mon Jul 30 09:20:17 2018 (r336886) @@ -988,7 +988,7 @@ usbd_req_get_desc(struct usb_device *udev, uint8_t retries) { struct usb_device_request req; - uint8_t *buf; + uint8_t *buf = desc; usb_error_t err; DPRINTFN(4, "id=%d, type=%d, index=%d, max_len=%d\n", @@ -1010,6 +1010,32 @@ usbd_req_get_desc(struct usb_device *udev, err = usbd_do_request_flags(udev, mtx, &req, desc, 0, NULL, 500 /* ms */); + if (err != 0 && err != USB_ERR_TIMEOUT && + min_len != max_len) { + /* clear descriptor data */ + memset(desc, 0, max_len); + + /* try to read full descriptor length */ + USETW(req.wLength, max_len); + + err = usbd_do_request_flags(udev, mtx, &req, + desc, USB_SHORT_XFER_OK, NULL, 500 /* ms */); + + if (err == 0) { + /* verify length */ + if (buf[0] > max_len) + buf[0] = max_len; + else if (buf[0] < 2) + err = USB_ERR_INVAL; + + min_len = buf[0]; + + /* enforce descriptor type */ + buf[1] = type; + goto done; + } + } + if (err) { if (!retries) { goto done; @@ -1020,7 +1046,6 @@ usbd_req_get_desc(struct usb_device *udev, continue; } - buf = desc; if (min_len == max_len) { From owner-svn-src-all@freebsd.org Mon Jul 30 09:21:20 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8C921050173; Mon, 30 Jul 2018 09:21:20 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E66E8F898; Mon, 30 Jul 2018 09:21:20 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4FA4A22687; Mon, 30 Jul 2018 09:21:20 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9LKH6018885; Mon, 30 Jul 2018 09:21:20 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9LK50018884; Mon, 30 Jul 2018 09:21:20 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300921.w6U9LK50018884@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:21:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r336887 - stable/10/sys/dev/usb X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/10/sys/dev/usb X-SVN-Commit-Revision: 336887 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:21:20 -0000 Author: hselasky Date: Mon Jul 30 09:21:19 2018 New Revision: 336887 URL: https://svnweb.freebsd.org/changeset/base/336887 Log: MFC r335700: Improve the kernel's USB descriptor reading function. Some USB devices does not allow a partial descriptor readout. Found by: bz@ Sponsored by: Mellanox Technologies Modified: stable/10/sys/dev/usb/usb_request.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/dev/usb/usb_request.c ============================================================================== --- stable/10/sys/dev/usb/usb_request.c Mon Jul 30 09:20:17 2018 (r336886) +++ stable/10/sys/dev/usb/usb_request.c Mon Jul 30 09:21:19 2018 (r336887) @@ -988,7 +988,7 @@ usbd_req_get_desc(struct usb_device *udev, uint8_t retries) { struct usb_device_request req; - uint8_t *buf; + uint8_t *buf = desc; usb_error_t err; DPRINTFN(4, "id=%d, type=%d, index=%d, max_len=%d\n", @@ -1010,6 +1010,32 @@ usbd_req_get_desc(struct usb_device *udev, err = usbd_do_request_flags(udev, mtx, &req, desc, 0, NULL, 500 /* ms */); + if (err != 0 && err != USB_ERR_TIMEOUT && + min_len != max_len) { + /* clear descriptor data */ + memset(desc, 0, max_len); + + /* try to read full descriptor length */ + USETW(req.wLength, max_len); + + err = usbd_do_request_flags(udev, mtx, &req, + desc, USB_SHORT_XFER_OK, NULL, 500 /* ms */); + + if (err == 0) { + /* verify length */ + if (buf[0] > max_len) + buf[0] = max_len; + else if (buf[0] < 2) + err = USB_ERR_INVAL; + + min_len = buf[0]; + + /* enforce descriptor type */ + buf[1] = type; + goto done; + } + } + if (err) { if (!retries) { goto done; @@ -1020,7 +1046,6 @@ usbd_req_get_desc(struct usb_device *udev, continue; } - buf = desc; if (min_len == max_len) { From owner-svn-src-all@freebsd.org Mon Jul 30 09:22:22 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 508E8105035D; Mon, 30 Jul 2018 09:22:22 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 06E278FA64; Mon, 30 Jul 2018 09:22:22 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DC3B4226D3; Mon, 30 Jul 2018 09:22:21 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9MLSB021186; Mon, 30 Jul 2018 09:22:21 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9MLei021185; Mon, 30 Jul 2018 09:22:21 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300922.w6U9MLei021185@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:22:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-9@freebsd.org Subject: svn commit: r336888 - stable/9/sys/dev/usb X-SVN-Group: stable-9 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/9/sys/dev/usb X-SVN-Commit-Revision: 336888 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:22:22 -0000 Author: hselasky Date: Mon Jul 30 09:22:21 2018 New Revision: 336888 URL: https://svnweb.freebsd.org/changeset/base/336888 Log: MFC r335700: Improve the kernel's USB descriptor reading function. Some USB devices does not allow a partial descriptor readout. Found by: bz@ Sponsored by: Mellanox Technologies Modified: stable/9/sys/dev/usb/usb_request.c Directory Properties: stable/9/sys/ (props changed) Modified: stable/9/sys/dev/usb/usb_request.c ============================================================================== --- stable/9/sys/dev/usb/usb_request.c Mon Jul 30 09:21:19 2018 (r336887) +++ stable/9/sys/dev/usb/usb_request.c Mon Jul 30 09:22:21 2018 (r336888) @@ -981,7 +981,7 @@ usbd_req_get_desc(struct usb_device *udev, uint8_t retries) { struct usb_device_request req; - uint8_t *buf; + uint8_t *buf = desc; usb_error_t err; DPRINTFN(4, "id=%d, type=%d, index=%d, max_len=%d\n", @@ -1003,6 +1003,32 @@ usbd_req_get_desc(struct usb_device *udev, err = usbd_do_request_flags(udev, mtx, &req, desc, 0, NULL, 500 /* ms */); + if (err != 0 && err != USB_ERR_TIMEOUT && + min_len != max_len) { + /* clear descriptor data */ + memset(desc, 0, max_len); + + /* try to read full descriptor length */ + USETW(req.wLength, max_len); + + err = usbd_do_request_flags(udev, mtx, &req, + desc, USB_SHORT_XFER_OK, NULL, 500 /* ms */); + + if (err == 0) { + /* verify length */ + if (buf[0] > max_len) + buf[0] = max_len; + else if (buf[0] < 2) + err = USB_ERR_INVAL; + + min_len = buf[0]; + + /* enforce descriptor type */ + buf[1] = type; + goto done; + } + } + if (err) { if (!retries) { goto done; @@ -1013,7 +1039,6 @@ usbd_req_get_desc(struct usb_device *udev, continue; } - buf = desc; if (min_len == max_len) { From owner-svn-src-all@freebsd.org Mon Jul 30 09:25:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA23A10504AF; Mon, 30 Jul 2018 09:25:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FA9B8FCAF; Mon, 30 Jul 2018 09:25:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5E19D226E1; Mon, 30 Jul 2018 09:25:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9PvmV023796; Mon, 30 Jul 2018 09:25:57 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9Pvdo023795; Mon, 30 Jul 2018 09:25:57 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300925.w6U9Pvdo023795@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:25:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336889 - stable/11/sys/dev/sound/pcm X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/sound/pcm X-SVN-Commit-Revision: 336889 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:25:58 -0000 Author: hselasky Date: Mon Jul 30 09:25:56 2018 New Revision: 336889 URL: https://svnweb.freebsd.org/changeset/base/336889 Log: MFC r336632: Update modify counter when setting a mixer control. PR: 229969 Modified: stable/11/sys/dev/sound/pcm/mixer.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/sound/pcm/mixer.c ============================================================================== --- stable/11/sys/dev/sound/pcm/mixer.c Mon Jul 30 09:22:21 2018 (r336888) +++ stable/11/sys/dev/sound/pcm/mixer.c Mon Jul 30 09:25:56 2018 (r336889) @@ -322,6 +322,7 @@ mixer_set(struct snd_mixer *m, u_int dev, u_int lev) MIXER_SET_LOCK(m, dropmtx); m->level[dev] = l | (r << 8); + m->modify_counter++; return 0; } From owner-svn-src-all@freebsd.org Mon Jul 30 09:28:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 700331050581; Mon, 30 Jul 2018 09:28:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 253048FE41; Mon, 30 Jul 2018 09:28:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 06405226E5; Mon, 30 Jul 2018 09:28:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9S0oq023947; Mon, 30 Jul 2018 09:28:00 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9S00g023946; Mon, 30 Jul 2018 09:28:00 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300928.w6U9S00g023946@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:28:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r336890 - stable/10/sys/dev/sound/pcm X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/10/sys/dev/sound/pcm X-SVN-Commit-Revision: 336890 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:28:01 -0000 Author: hselasky Date: Mon Jul 30 09:28:00 2018 New Revision: 336890 URL: https://svnweb.freebsd.org/changeset/base/336890 Log: MFC r336632: Update modify counter when setting a mixer control. PR: 229969 Modified: stable/10/sys/dev/sound/pcm/mixer.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/dev/sound/pcm/mixer.c ============================================================================== --- stable/10/sys/dev/sound/pcm/mixer.c Mon Jul 30 09:25:56 2018 (r336889) +++ stable/10/sys/dev/sound/pcm/mixer.c Mon Jul 30 09:28:00 2018 (r336890) @@ -323,6 +323,7 @@ mixer_set(struct snd_mixer *m, u_int dev, u_int lev) MIXER_SET_LOCK(m, dropmtx); m->level[dev] = l | (r << 8); + m->modify_counter++; return 0; } From owner-svn-src-all@freebsd.org Mon Jul 30 09:29:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02BBD10506EC; Mon, 30 Jul 2018 09:29:33 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB858900A9; Mon, 30 Jul 2018 09:29:32 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8CA6E226E8; Mon, 30 Jul 2018 09:29:32 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9TWiY024181; Mon, 30 Jul 2018 09:29:32 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9TWUR024180; Mon, 30 Jul 2018 09:29:32 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807300929.w6U9TWUR024180@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 30 Jul 2018 09:29:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-9@freebsd.org Subject: svn commit: r336891 - stable/9/sys/dev/sound/pcm X-SVN-Group: stable-9 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/9/sys/dev/sound/pcm X-SVN-Commit-Revision: 336891 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:29:33 -0000 Author: hselasky Date: Mon Jul 30 09:29:32 2018 New Revision: 336891 URL: https://svnweb.freebsd.org/changeset/base/336891 Log: MFC r336632: Update modify counter when setting a mixer control. PR: 229969 Modified: stable/9/sys/dev/sound/pcm/mixer.c Directory Properties: stable/9/sys/ (props changed) Modified: stable/9/sys/dev/sound/pcm/mixer.c ============================================================================== --- stable/9/sys/dev/sound/pcm/mixer.c Mon Jul 30 09:28:00 2018 (r336890) +++ stable/9/sys/dev/sound/pcm/mixer.c Mon Jul 30 09:29:32 2018 (r336891) @@ -323,6 +323,7 @@ mixer_set(struct snd_mixer *m, u_int dev, u_int lev) MIXER_SET_LOCK(m, dropmtx); m->level[dev] = l | (r << 8); + m->modify_counter++; return 0; } From owner-svn-src-all@freebsd.org Mon Jul 30 09:50:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC3151050D6F; Mon, 30 Jul 2018 09:50:26 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EC0690C99; Mon, 30 Jul 2018 09:50:26 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 809A222A41; Mon, 30 Jul 2018 09:50:26 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6U9oQAM034834; Mon, 30 Jul 2018 09:50:26 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6U9oQtq034833; Mon, 30 Jul 2018 09:50:26 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807300950.w6U9oQtq034833@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Mon, 30 Jul 2018 09:50:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336892 - head/sys/arm/arm X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: head/sys/arm/arm X-SVN-Commit-Revision: 336892 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 09:50:27 -0000 Author: andrew Date: Mon Jul 30 09:50:26 2018 New Revision: 336892 URL: https://svnweb.freebsd.org/changeset/base/336892 Log: Require ARMv5 for arm. All current kernels are for ARMv5 or later, and it will allow us to clean out old ARMv4 (and earlier) specific assembly. Relnotes: yes Modified: head/sys/arm/arm/machdep.c Modified: head/sys/arm/arm/machdep.c ============================================================================== --- head/sys/arm/arm/machdep.c Mon Jul 30 09:29:32 2018 (r336891) +++ head/sys/arm/arm/machdep.c Mon Jul 30 09:50:26 2018 (r336892) @@ -77,6 +77,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -107,6 +108,10 @@ __FBSDID("$FreeBSD$"); #if __ARM_ARCH >= 6 && !defined(INTRNG) #error armv6 requires INTRNG +#endif + +#ifndef _ARM_ARCH_5E +#error FreeBSD requires ARMv5 or later #endif struct pcpu __pcpu[MAXCPU]; From owner-svn-src-all@freebsd.org Mon Jul 30 10:23:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C8DC1051B1A; Mon, 30 Jul 2018 10:23:30 +0000 (UTC) (envelope-from rrs@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BDAD592025; Mon, 30 Jul 2018 10:23:29 +0000 (UTC) (envelope-from rrs@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9CA432311A; Mon, 30 Jul 2018 10:23:29 +0000 (UTC) (envelope-from rrs@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UANTda054915; Mon, 30 Jul 2018 10:23:29 GMT (envelope-from rrs@FreeBSD.org) Received: (from rrs@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UANTbL054914; Mon, 30 Jul 2018 10:23:29 GMT (envelope-from rrs@FreeBSD.org) Message-Id: <201807301023.w6UANTbL054914@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rrs set sender to rrs@FreeBSD.org using -f From: Randall Stewart Date: Mon, 30 Jul 2018 10:23:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336893 - head/sys/netinet/tcp_stacks X-SVN-Group: head X-SVN-Commit-Author: rrs X-SVN-Commit-Paths: head/sys/netinet/tcp_stacks X-SVN-Commit-Revision: 336893 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 10:23:30 -0000 Author: rrs Date: Mon Jul 30 10:23:29 2018 New Revision: 336893 URL: https://svnweb.freebsd.org/changeset/base/336893 Log: This fixes a hole where rack could end up sending an invalid segment into the reassembly queue. This would happen if you enabled the data after close option. Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D16453 Modified: head/sys/netinet/tcp_stacks/rack.c Modified: head/sys/netinet/tcp_stacks/rack.c ============================================================================== --- head/sys/netinet/tcp_stacks/rack.c Mon Jul 30 09:50:26 2018 (r336892) +++ head/sys/netinet/tcp_stacks/rack.c Mon Jul 30 10:23:29 2018 (r336893) @@ -4657,7 +4657,6 @@ rack_process_data(struct mbuf *m, struct tcphdr *th, s rack = (struct tcp_rack *)tp->t_fb_ptr; INP_WLOCK_ASSERT(tp->t_inpcb); - nsegs = max(1, m->m_pkthdr.lro_nsegs); if ((thflags & TH_ACK) && (SEQ_LT(tp->snd_wl1, th->th_seq) || @@ -4686,6 +4685,10 @@ rack_process_data(struct mbuf *m, struct tcphdr *th, s tp->snd_nxt = tp->snd_max; /* Make sure we output to start the timer */ rack->r_wanted_output++; + } + if (tp->t_flags2 & TF2_DROP_AF_DATA) { + m_freem(m); + return (0); } /* * Process segments with URG. From owner-svn-src-all@freebsd.org Mon Jul 30 10:55:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2821A1052215; Mon, 30 Jul 2018 10:55:04 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE88293410; Mon, 30 Jul 2018 10:55:03 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 98629235FE; Mon, 30 Jul 2018 10:55:03 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UAt3XR070255; Mon, 30 Jul 2018 10:55:03 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UAt3oj070252; Mon, 30 Jul 2018 10:55:03 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807301055.w6UAt3oj070252@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Mon, 30 Jul 2018 10:55:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336894 - head/sys/arm/arm X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: head/sys/arm/arm X-SVN-Commit-Revision: 336894 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 10:55:04 -0000 Author: andrew Date: Mon Jul 30 10:55:02 2018 New Revision: 336894 URL: https://svnweb.freebsd.org/changeset/base/336894 Log: Remove teh non-INTRNG code from the ARM GIC interrupt controller driver. We don't build for the non-INTRNG case and it was makeing the code harder to read. Modified: head/sys/arm/arm/gic.c head/sys/arm/arm/gic.h head/sys/arm/arm/gic_fdt.c Modified: head/sys/arm/arm/gic.c ============================================================================== --- head/sys/arm/arm/gic.c Mon Jul 30 10:23:29 2018 (r336893) +++ head/sys/arm/arm/gic.c Mon Jul 30 10:55:02 2018 (r336894) @@ -53,9 +53,7 @@ __FBSDID("$FreeBSD$"); #include #include #include -#ifdef INTRNG #include -#endif #include #include @@ -77,10 +75,8 @@ __FBSDID("$FreeBSD$"); #include #include -#ifdef INTRNG #include "pic_if.h" #include "msi_if.h" -#endif /* We are using GICv2 register naming */ @@ -107,7 +103,6 @@ __FBSDID("$FreeBSD$"); #define GIC_DEFAULT_ICFGR_INIT 0x00000000 #endif -#ifdef INTRNG struct gic_irqsrc { struct intr_irqsrc gi_isrc; uint32_t gi_irq; @@ -130,26 +125,11 @@ static u_int sgi_first_unused = GIC_FIRST_SGI; #endif #define GIC_INTR_ISRC(sc, irq) (&sc->gic_irqs[irq].gi_isrc) -#else /* !INTRNG */ -static struct ofw_compat_data compat_data[] = { - {"arm,gic", true}, /* Non-standard, used in FreeBSD dts. */ - {"arm,gic-400", true}, - {"arm,cortex-a15-gic", true}, - {"arm,cortex-a9-gic", true}, - {"arm,cortex-a7-gic", true}, - {"arm,arm11mp-gic", true}, - {"brcm,brahma-b15-gic", true}, - {"qcom,msm-qgic2", true}, - {NULL, false} -}; -#endif static struct resource_spec arm_gic_spec[] = { { SYS_RES_MEMORY, 0, RF_ACTIVE }, /* Distributor registers */ { SYS_RES_MEMORY, 1, RF_ACTIVE }, /* CPU Interrupt Intf. registers */ -#ifdef INTRNG { SYS_RES_IRQ, 0, RF_ACTIVE | RF_OPTIONAL }, /* Parent interrupt */ -#endif { -1, 0 } }; @@ -176,13 +156,6 @@ static struct arm_gic_softc *gic_sc = NULL; #define gic_d_write_4(_sc, _reg, _val) \ bus_space_write_4((_sc)->gic_d_bst, (_sc)->gic_d_bsh, (_reg), (_val)) -#ifndef INTRNG -static int gic_config_irq(int irq, enum intr_trigger trig, - enum intr_polarity pol); -static void gic_post_filter(void *); -#endif - -#ifdef INTRNG static inline void gic_irq_unmask(struct arm_gic_softc *sc, u_int irq) { @@ -196,7 +169,6 @@ gic_irq_mask(struct arm_gic_softc *sc, u_int irq) gic_d_write_4(sc, GICD_ICENABLER(irq), GICD_I_MASK(irq)); } -#endif static uint8_t gic_cpu_mask(struct arm_gic_softc *sc) @@ -222,7 +194,6 @@ gic_cpu_mask(struct arm_gic_softc *sc) } #ifdef SMP -#ifdef INTRNG static void arm_gic_init_secondary(device_t dev) { @@ -260,105 +231,8 @@ arm_gic_init_secondary(device_t dev) if (intr_isrc_init_on_cpu(GIC_INTR_ISRC(sc, irq), cpu)) gic_irq_unmask(sc, irq); } -#else -static void -arm_gic_init_secondary(device_t dev) -{ - struct arm_gic_softc *sc = device_get_softc(dev); - int i; - - /* Set the mask so we can find this CPU to send it IPIs */ - arm_gic_map[PCPU_GET(cpuid)] = gic_cpu_mask(sc); - - for (i = 0; i < sc->nirqs; i += 4) - gic_d_write_4(sc, GICD_IPRIORITYR(i), 0); - - /* Set all the interrupts to be in Group 0 (secure) */ - for (i = 0; GIC_SUPPORT_SECEXT(sc) && i < sc->nirqs; i += 32) { - gic_d_write_4(sc, GICD_IGROUPR(i), 0); - } - - /* Enable CPU interface */ - gic_c_write_4(sc, GICC_CTLR, 1); - - /* Set priority mask register. */ - gic_c_write_4(sc, GICC_PMR, 0xff); - - /* Enable interrupt distribution */ - gic_d_write_4(sc, GICD_CTLR, 0x01); - - /* - * Activate the timer interrupts: virtual, secure, and non-secure. - */ - gic_d_write_4(sc, GICD_ISENABLER(27), GICD_I_MASK(27)); - gic_d_write_4(sc, GICD_ISENABLER(29), GICD_I_MASK(29)); - gic_d_write_4(sc, GICD_ISENABLER(30), GICD_I_MASK(30)); -} -#endif /* INTRNG */ #endif /* SMP */ -#ifndef INTRNG -int -gic_decode_fdt(phandle_t iparent, pcell_t *intr, int *interrupt, - int *trig, int *pol) -{ - static u_int num_intr_cells; - static phandle_t self; - struct ofw_compat_data *ocd; - - if (self == 0) { - for (ocd = compat_data; ocd->ocd_str != NULL; ocd++) { - if (ofw_bus_node_is_compatible(iparent, ocd->ocd_str)) { - self = iparent; - break; - } - } - } - if (self != iparent) - return (ENXIO); - - if (num_intr_cells == 0) { - if (OF_searchencprop(OF_node_from_xref(iparent), - "#interrupt-cells", &num_intr_cells, - sizeof(num_intr_cells)) == -1) { - num_intr_cells = 1; - } - } - - if (num_intr_cells == 1) { - *interrupt = fdt32_to_cpu(intr[0]); - *trig = INTR_TRIGGER_CONFORM; - *pol = INTR_POLARITY_CONFORM; - } else { - if (fdt32_to_cpu(intr[0]) == 0) - *interrupt = fdt32_to_cpu(intr[1]) + GIC_FIRST_SPI; - else - *interrupt = fdt32_to_cpu(intr[1]) + GIC_FIRST_PPI; - /* - * In intr[2], bits[3:0] are trigger type and level flags. - * 1 = low-to-high edge triggered - * 2 = high-to-low edge triggered - * 4 = active high level-sensitive - * 8 = active low level-sensitive - * The hardware only supports active-high-level or rising-edge - * for SPIs - */ - if (*interrupt >= GIC_FIRST_SPI && - fdt32_to_cpu(intr[2]) & 0x0a) { - printf("unsupported trigger/polarity configuration " - "0x%02x\n", fdt32_to_cpu(intr[2]) & 0x0f); - } - *pol = INTR_POLARITY_CONFORM; - if (fdt32_to_cpu(intr[2]) & 0x03) - *trig = INTR_TRIGGER_EDGE; - else - *trig = INTR_TRIGGER_LEVEL; - } - return (0); -} -#endif - -#ifdef INTRNG static int arm_gic_register_isrcs(struct arm_gic_softc *sc, uint32_t num) { @@ -425,7 +299,6 @@ arm_gic_reserve_msi_range(device_t dev, u_int start, u sc->gic_irqs[start + i].gi_flags |= GI_FLAG_MSI; } } -#endif int arm_gic_attach(device_t dev) @@ -465,27 +338,17 @@ arm_gic_attach(device_t dev) sc->typer = gic_d_read_4(sc, GICD_TYPER); nirqs = GICD_TYPER_I_NUM(sc->typer); -#ifdef INTRNG if (arm_gic_register_isrcs(sc, nirqs)) { device_printf(dev, "could not register irqs\n"); goto cleanup; } -#else - sc->nirqs = nirqs; - /* Set up function pointers */ - arm_post_filter = gic_post_filter; - arm_config_irq = gic_config_irq; -#endif - icciidr = gic_c_read_4(sc, GICC_IIDR); device_printf(dev, "pn 0x%x, arch 0x%x, rev 0x%x, implementer 0x%x irqs %u\n", GICD_IIDR_PROD(icciidr), GICD_IIDR_VAR(icciidr), GICD_IIDR_REV(icciidr), GICD_IIDR_IMPL(icciidr), sc->nirqs); -#ifdef INTRNG sc->gic_iidr = icciidr; -#endif /* Set all global interrupts to be level triggered, active low. */ for (i = 32; i < sc->nirqs; i += 16) { @@ -527,17 +390,14 @@ arm_gic_attach(device_t dev) gic_d_write_4(sc, GICD_CTLR, 0x01); return (0); -#ifdef INTRNG cleanup: arm_gic_detach(dev); return(ENXIO); -#endif } int arm_gic_detach(device_t dev) { -#ifdef INTRNG struct arm_gic_softc *sc; sc = device_get_softc(dev); @@ -546,12 +406,10 @@ arm_gic_detach(device_t dev) free(sc->gic_irqs, M_DEVBUF); bus_release_resources(dev, arm_gic_spec, sc->gic_res); -#endif return (0); } -#ifdef INTRNG static int arm_gic_print_child(device_t bus, device_t child) { @@ -1145,217 +1003,8 @@ arm_gic_ipi_setup(device_t dev, u_int ipi, struct intr return (0); } #endif -#else -static int -arm_gic_next_irq(struct arm_gic_softc *sc, int last_irq) -{ - uint32_t active_irq; - active_irq = gic_c_read_4(sc, GICC_IAR); - - /* - * Immediately EOIR the SGIs, because doing so requires the other - * bits (ie CPU number), not just the IRQ number, and we do not - * have this information later. - */ - if ((active_irq & 0x3ff) <= GIC_LAST_SGI) - gic_c_write_4(sc, GICC_EOIR, active_irq); - active_irq &= 0x3FF; - - if (active_irq == 0x3FF) { - if (last_irq == -1) - device_printf(sc->gic_dev, - "Spurious interrupt detected\n"); - return -1; - } - - return active_irq; -} - -static int -arm_gic_config(device_t dev, int irq, enum intr_trigger trig, - enum intr_polarity pol) -{ - struct arm_gic_softc *sc = device_get_softc(dev); - uint32_t reg; - uint32_t mask; - - /* Function is public-accessible, so validate input arguments */ - if ((irq < 0) || (irq >= sc->nirqs)) - goto invalid_args; - if ((trig != INTR_TRIGGER_EDGE) && (trig != INTR_TRIGGER_LEVEL) && - (trig != INTR_TRIGGER_CONFORM)) - goto invalid_args; - if ((pol != INTR_POLARITY_HIGH) && (pol != INTR_POLARITY_LOW) && - (pol != INTR_POLARITY_CONFORM)) - goto invalid_args; - - mtx_lock_spin(&sc->mutex); - - reg = gic_d_read_4(sc, GICD_ICFGR(irq)); - mask = (reg >> 2*(irq % 16)) & 0x3; - - if (pol == INTR_POLARITY_LOW) { - mask &= ~GICD_ICFGR_POL_MASK; - mask |= GICD_ICFGR_POL_LOW; - } else if (pol == INTR_POLARITY_HIGH) { - mask &= ~GICD_ICFGR_POL_MASK; - mask |= GICD_ICFGR_POL_HIGH; - } - - if (trig == INTR_TRIGGER_LEVEL) { - mask &= ~GICD_ICFGR_TRIG_MASK; - mask |= GICD_ICFGR_TRIG_LVL; - } else if (trig == INTR_TRIGGER_EDGE) { - mask &= ~GICD_ICFGR_TRIG_MASK; - mask |= GICD_ICFGR_TRIG_EDGE; - } - - /* Set mask */ - reg = reg & ~(0x3 << 2*(irq % 16)); - reg = reg | (mask << 2*(irq % 16)); - gic_d_write_4(sc, GICD_ICFGR(irq), reg); - - mtx_unlock_spin(&sc->mutex); - - return (0); - -invalid_args: - device_printf(dev, "gic_config_irg, invalid parameters\n"); - return (EINVAL); -} - - -static void -arm_gic_mask(device_t dev, int irq) -{ - struct arm_gic_softc *sc = device_get_softc(dev); - - gic_d_write_4(sc, GICD_ICENABLER(irq), (1UL << (irq & 0x1F))); - gic_c_write_4(sc, GICC_EOIR, irq); /* XXX - not allowed */ -} - -static void -arm_gic_unmask(device_t dev, int irq) -{ - struct arm_gic_softc *sc = device_get_softc(dev); - - if (irq > GIC_LAST_SGI) - arm_irq_memory_barrier(irq); - - gic_d_write_4(sc, GICD_ISENABLER(irq), (1UL << (irq & 0x1F))); -} - -#ifdef SMP -static void -arm_gic_ipi_send(device_t dev, cpuset_t cpus, u_int ipi) -{ - struct arm_gic_softc *sc = device_get_softc(dev); - uint32_t val = 0, i; - - for (i = 0; i < MAXCPU; i++) - if (CPU_ISSET(i, &cpus)) - val |= arm_gic_map[i] << GICD_SGI_TARGET_SHIFT; - - gic_d_write_4(sc, GICD_SGIR, val | ipi); -} - -static int -arm_gic_ipi_read(device_t dev, int i) -{ - - if (i != -1) { - /* - * The intr code will automagically give the frame pointer - * if the interrupt argument is 0. - */ - if ((unsigned int)i > 16) - return (0); - return (i); - } - - return (0x3ff); -} - -static void -arm_gic_ipi_clear(device_t dev, int ipi) -{ - /* no-op */ -} -#endif - -static void -gic_post_filter(void *arg) -{ - struct arm_gic_softc *sc = gic_sc; - uintptr_t irq = (uintptr_t) arg; - - if (irq > GIC_LAST_SGI) - arm_irq_memory_barrier(irq); - gic_c_write_4(sc, GICC_EOIR, irq); -} - -static int -gic_config_irq(int irq, enum intr_trigger trig, enum intr_polarity pol) -{ - - return (arm_gic_config(gic_sc->gic_dev, irq, trig, pol)); -} - -void -arm_mask_irq(uintptr_t nb) -{ - - arm_gic_mask(gic_sc->gic_dev, nb); -} - -void -arm_unmask_irq(uintptr_t nb) -{ - - arm_gic_unmask(gic_sc->gic_dev, nb); -} - -int -arm_get_next_irq(int last_irq) -{ - - return (arm_gic_next_irq(gic_sc, last_irq)); -} - -#ifdef SMP -void -intr_pic_init_secondary(void) -{ - - arm_gic_init_secondary(gic_sc->gic_dev); -} - -void -pic_ipi_send(cpuset_t cpus, u_int ipi) -{ - - arm_gic_ipi_send(gic_sc->gic_dev, cpus, ipi); -} - -int -pic_ipi_read(int i) -{ - - return (arm_gic_ipi_read(gic_sc->gic_dev, i)); -} - -void -pic_ipi_clear(int ipi) -{ - - arm_gic_ipi_clear(gic_sc->gic_dev, ipi); -} -#endif -#endif /* INTRNG */ - static device_method_t arm_gic_methods[] = { -#ifdef INTRNG /* Bus interface */ DEVMETHOD(bus_print_child, arm_gic_print_child), DEVMETHOD(bus_add_child, bus_generic_add_child), @@ -1379,14 +1028,12 @@ static device_method_t arm_gic_methods[] = { DEVMETHOD(pic_ipi_send, arm_gic_ipi_send), DEVMETHOD(pic_ipi_setup, arm_gic_ipi_setup), #endif -#endif { 0, 0 } }; DEFINE_CLASS_0(gic, arm_gic_driver, arm_gic_methods, sizeof(struct arm_gic_softc)); -#ifdef INTRNG /* * GICv2m support -- the GICv2 MSI/MSI-X controller. */ @@ -1611,4 +1258,3 @@ static device_method_t arm_gicv2m_methods[] = { DEFINE_CLASS_0(gicv2m, arm_gicv2m_driver, arm_gicv2m_methods, sizeof(struct arm_gicv2m_softc)); -#endif Modified: head/sys/arm/arm/gic.h ============================================================================== --- head/sys/arm/arm/gic.h Mon Jul 30 10:23:29 2018 (r336893) +++ head/sys/arm/arm/gic.h Mon Jul 30 10:55:02 2018 (r336894) @@ -39,20 +39,16 @@ #ifndef _ARM_GIC_H_ #define _ARM_GIC_H_ -#ifdef INTRNG struct arm_gic_range { uint64_t bus; uint64_t host; uint64_t size; }; -#endif struct arm_gic_softc { device_t gic_dev; -#ifdef INTRNG void * gic_intrhand; struct gic_irqsrc * gic_irqs; -#endif struct resource * gic_res[3]; bus_space_tag_t gic_c_bst; bus_space_tag_t gic_d_bst; @@ -64,18 +60,15 @@ struct arm_gic_softc { uint32_t typer; uint32_t last_irq[MAXCPU]; -#ifdef INTRNG uint32_t gic_iidr; u_int gic_bus; int nranges; struct arm_gic_range * ranges; -#endif }; DECLARE_CLASS(arm_gic_driver); -#ifdef INTRNG struct arm_gicv2m_softc { struct resource *sc_mem; struct mtx sc_mutex; @@ -86,7 +79,6 @@ struct arm_gicv2m_softc { }; DECLARE_CLASS(arm_gicv2m_driver); -#endif int arm_gic_attach(device_t); int arm_gic_detach(device_t); Modified: head/sys/arm/arm/gic_fdt.c ============================================================================== --- head/sys/arm/arm/gic_fdt.c Mon Jul 30 10:23:29 2018 (r336893) +++ head/sys/arm/arm/gic_fdt.c Mon Jul 30 10:55:02 2018 (r336894) @@ -54,12 +54,10 @@ __FBSDID("$FreeBSD$"); #include #include -#ifdef INTRNG struct arm_gic_devinfo { struct ofw_bus_devinfo obdinfo; struct resource_list rl; }; -#endif struct arm_gic_fdt_softc { struct arm_gic_softc base; @@ -70,10 +68,8 @@ struct arm_gic_fdt_softc { static device_probe_t gic_fdt_probe; static device_attach_t gic_fdt_attach; static ofw_bus_get_devinfo_t gic_ofw_get_devinfo; -#ifdef INTRNG static bus_get_resource_list_t gic_fdt_get_resource_list; static bool arm_gic_add_children(device_t); -#endif static struct ofw_compat_data compat_data[] = { {"arm,gic", true}, /* Non-standard, used in FreeBSD dts. */ @@ -92,7 +88,6 @@ static device_method_t gic_fdt_methods[] = { DEVMETHOD(device_probe, gic_fdt_probe), DEVMETHOD(device_attach, gic_fdt_attach), -#ifdef INTRNG /* Bus interface */ DEVMETHOD(bus_get_resource_list,gic_fdt_get_resource_list), @@ -103,7 +98,6 @@ static device_method_t gic_fdt_methods[] = { DEVMETHOD(ofw_bus_get_name, ofw_bus_gen_get_name), DEVMETHOD(ofw_bus_get_node, ofw_bus_gen_get_node), DEVMETHOD(ofw_bus_get_type, ofw_bus_gen_get_type), -#endif DEVMETHOD_END, }; @@ -134,22 +128,17 @@ gic_fdt_probe(device_t dev) static int gic_fdt_attach(device_t dev) { -#ifdef INTRNG struct arm_gic_fdt_softc *sc = device_get_softc(dev); phandle_t pxref; intptr_t xref; -#endif int err; -#ifdef INTRNG sc->base.gic_bus = GIC_BUS_FDT; -#endif err = arm_gic_attach(dev); if (err != 0) return (err); -#ifdef INTRNG xref = OF_xref_from_node(ofw_bus_get_node(dev)); /* @@ -196,18 +185,14 @@ gic_fdt_attach(device_t dev) bus_generic_probe(dev); return (bus_generic_attach(dev)); } -#endif return (0); -#ifdef INTRNG cleanup: arm_gic_detach(dev); return(ENXIO); -#endif } -#ifdef INTRNG static struct resource_list * gic_fdt_get_resource_list(device_t bus, device_t child) { @@ -376,4 +361,3 @@ static devclass_t arm_gicv2m_fdt_devclass; EARLY_DRIVER_MODULE(gicv2m, gic, arm_gicv2m_fdt_driver, arm_gicv2m_fdt_devclass, 0, 0, BUS_PASS_INTERRUPT + BUS_PASS_ORDER_MIDDLE); -#endif From owner-svn-src-all@freebsd.org Mon Jul 30 11:15:21 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17F121052D1E; Mon, 30 Jul 2018 11:15:21 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2005944AA; Mon, 30 Jul 2018 11:15:20 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9F56823921; Mon, 30 Jul 2018 11:15:20 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UBFK0r080382; Mon, 30 Jul 2018 11:15:20 GMT (envelope-from royger@FreeBSD.org) Received: (from royger@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UBFKo7080381; Mon, 30 Jul 2018 11:15:20 GMT (envelope-from royger@FreeBSD.org) Message-Id: <201807301115.w6UBFKo7080381@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: royger set sender to royger@FreeBSD.org using -f From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= Date: Mon, 30 Jul 2018 11:15:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336895 - head/sys/dev/xen/blkfront X-SVN-Group: head X-SVN-Commit-Author: royger X-SVN-Commit-Paths: head/sys/dev/xen/blkfront X-SVN-Commit-Revision: 336895 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 11:15:21 -0000 Author: royger Date: Mon Jul 30 11:15:20 2018 New Revision: 336895 URL: https://svnweb.freebsd.org/changeset/base/336895 Log: xen-blkfront: fix length check Length is an unsigned integer, so checking against < 0 doesn't make sense. While there also make clear that a length of 0 always succeeds. Submitted by: Pratyush Yadav Differential Review: https://reviews.freebsd.org/D16045 Modified: head/sys/dev/xen/blkfront/blkfront.c Modified: head/sys/dev/xen/blkfront/blkfront.c ============================================================================== --- head/sys/dev/xen/blkfront/blkfront.c Mon Jul 30 10:55:02 2018 (r336894) +++ head/sys/dev/xen/blkfront/blkfront.c Mon Jul 30 11:15:20 2018 (r336895) @@ -602,8 +602,8 @@ xbd_dump(void *arg, void *virtual, vm_offset_t physica int sbp; int rc = 0; - if (length <= 0) - return (rc); + if (length == 0) + return (0); xbd_quiesce(sc); /* All quiet on the western front. */ From owner-svn-src-all@freebsd.org Mon Jul 30 11:27:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4F1D1053340; Mon, 30 Jul 2018 11:27:52 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 674E194D83; Mon, 30 Jul 2018 11:27:52 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 39A0023AB3; Mon, 30 Jul 2018 11:27:52 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UBRpUl085391; Mon, 30 Jul 2018 11:27:51 GMT (envelope-from royger@FreeBSD.org) Received: (from royger@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UBRpaV085390; Mon, 30 Jul 2018 11:27:51 GMT (envelope-from royger@FreeBSD.org) Message-Id: <201807301127.w6UBRpaV085390@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: royger set sender to royger@FreeBSD.org using -f From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= Date: Mon, 30 Jul 2018 11:27:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336896 - head/sys/dev/xen/blkfront X-SVN-Group: head X-SVN-Commit-Author: royger X-SVN-Commit-Paths: head/sys/dev/xen/blkfront X-SVN-Commit-Revision: 336896 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 11:27:52 -0000 Author: royger Date: Mon Jul 30 11:27:51 2018 New Revision: 336896 URL: https://svnweb.freebsd.org/changeset/base/336896 Log: xen-blkfront: fix memory leak in xbd_connect error path If gnttab_grant_foreign_access() fails for any of the indirection pages, the code breaks out of both the loops without freeing the local variable indirectpages, causing a memory leak. Submitted by: Pratyush Yadav Differential Review: https://reviews.freebsd.org/D16136 Modified: head/sys/dev/xen/blkfront/blkfront.c Modified: head/sys/dev/xen/blkfront/blkfront.c ============================================================================== --- head/sys/dev/xen/blkfront/blkfront.c Mon Jul 30 11:15:20 2018 (r336895) +++ head/sys/dev/xen/blkfront/blkfront.c Mon Jul 30 11:27:51 2018 (r336896) @@ -1333,7 +1333,10 @@ xbd_connect(struct xbd_softc *sc) if (sc->xbd_max_request_indirectpages > 0) { indirectpages = contigmalloc( PAGE_SIZE * sc->xbd_max_request_indirectpages, - M_XENBLOCKFRONT, M_ZERO, 0, ~0, PAGE_SIZE, 0); + M_XENBLOCKFRONT, M_ZERO | M_NOWAIT, 0, ~0, + PAGE_SIZE, 0); + if (indirectpages == NULL) + sc->xbd_max_request_indirectpages = 0; } else { indirectpages = NULL; } @@ -1345,8 +1348,12 @@ xbd_connect(struct xbd_softc *sc) &cm->cm_indirectionrefs[j])) break; } - if (j < sc->xbd_max_request_indirectpages) + if (j < sc->xbd_max_request_indirectpages) { + contigfree(indirectpages, + PAGE_SIZE * sc->xbd_max_request_indirectpages, + M_XENBLOCKFRONT); break; + } cm->cm_indirectionpages = indirectpages; xbd_free_command(cm); } From owner-svn-src-all@freebsd.org Mon Jul 30 11:41:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDA881053708; Mon, 30 Jul 2018 11:41:52 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F64B9548A; Mon, 30 Jul 2018 11:41:52 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 608DF23DE4; Mon, 30 Jul 2018 11:41:52 +0000 (UTC) (envelope-from royger@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UBfqv7093192; Mon, 30 Jul 2018 11:41:52 GMT (envelope-from royger@FreeBSD.org) Received: (from royger@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UBfq4j093191; Mon, 30 Jul 2018 11:41:52 GMT (envelope-from royger@FreeBSD.org) Message-Id: <201807301141.w6UBfq4j093191@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: royger set sender to royger@FreeBSD.org using -f From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= Date: Mon, 30 Jul 2018 11:41:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336897 - head/sys/dev/xen/grant_table X-SVN-Group: head X-SVN-Commit-Author: royger X-SVN-Commit-Paths: head/sys/dev/xen/grant_table X-SVN-Commit-Revision: 336897 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 11:41:53 -0000 Author: royger Date: Mon Jul 30 11:41:51 2018 New Revision: 336897 URL: https://svnweb.freebsd.org/changeset/base/336897 Log: xen/grants: fix deadlocks in the free callbacks This fixes the panic caused by deadlocking when grant-table free callbacks are used. The cause of the recursion is: check_free_callbacks() is always called with the lock gnttab_list_lock held. In turn the callback function is also called with the lock held. Then when the client uses any of the grant reference methods which also attempt the lock the gnttab_list_lock mutex from within the free callback a deadlock happens. Fix this by making the gnttab_list_lock recursive. Submitted by: Pratyush Yadav Differential Revision: https://reviews.freebsd.org/D16505 Modified: head/sys/dev/xen/grant_table/grant_table.c Modified: head/sys/dev/xen/grant_table/grant_table.c ============================================================================== --- head/sys/dev/xen/grant_table/grant_table.c Mon Jul 30 11:27:51 2018 (r336896) +++ head/sys/dev/xen/grant_table/grant_table.c Mon Jul 30 11:41:51 2018 (r336897) @@ -585,7 +585,7 @@ gnttab_expand(unsigned int req_entries) return (error); } -MTX_SYSINIT(gnttab, &gnttab_list_lock, "GNTTAB LOCK", MTX_DEF); +MTX_SYSINIT(gnttab, &gnttab_list_lock, "GNTTAB LOCK", MTX_DEF | MTX_RECURSE); /*------------------ Private Device Attachment Functions --------------------*/ /** From owner-svn-src-all@freebsd.org Mon Jul 30 12:10:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0606210551D2; Mon, 30 Jul 2018 12:10:25 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9FD2E96B0F; Mon, 30 Jul 2018 12:10:24 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6351824180; Mon, 30 Jul 2018 12:10:24 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UCAOK1006328; Mon, 30 Jul 2018 12:10:24 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UCAODb006327; Mon, 30 Jul 2018 12:10:24 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807301210.w6UCAODb006327@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 30 Jul 2018 12:10:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336898 - in stable/11/sys/fs: nfs nfsclient X-SVN-Group: stable-11 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in stable/11/sys/fs: nfs nfsclient X-SVN-Commit-Revision: 336898 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 12:10:25 -0000 Author: rmacklem Date: Mon Jul 30 12:10:23 2018 New Revision: 336898 URL: https://svnweb.freebsd.org/changeset/base/336898 Log: MFC: r334966 Add a couple of safety belt checks to the NFSv4.1 client related to sessions. There were a couple of cases in newnfs_request() that it assumed that it was an NFSv4.1 mount with a session. This should always be the case when a Sequence operation is in the reply or the server replies NFSERR_BADSESSION. However, if a server was broken and sent an erroneous reply, these safety belt checks should avoid trouble. The one check required a small tweak to nfsmnt_mdssession() so that it returns NULL when there is no session instead of the offset of the field in the structure (0x8 for i386). This patch should have no effect on normal operation of the client. Found by inspection during pNFS server development. Modified: stable/11/sys/fs/nfs/nfs_commonkrpc.c stable/11/sys/fs/nfsclient/nfsmount.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/fs/nfs/nfs_commonkrpc.c ============================================================================== --- stable/11/sys/fs/nfs/nfs_commonkrpc.c Mon Jul 30 11:41:51 2018 (r336897) +++ stable/11/sys/fs/nfs/nfs_commonkrpc.c Mon Jul 30 12:10:23 2018 (r336898) @@ -849,9 +849,9 @@ tryagain: if ((nmp != NULL && i == NFSV4OP_SEQUENCE && j != 0) || (clp != NULL && i == NFSV4OP_CBSEQUENCE && j != 0)) NFSCL_DEBUG(1, "failed seq=%d\n", j); - if ((nmp != NULL && i == NFSV4OP_SEQUENCE && j == 0) || - (clp != NULL && i == NFSV4OP_CBSEQUENCE && j == 0) - ) { + if (((nmp != NULL && i == NFSV4OP_SEQUENCE && j == 0) || + (clp != NULL && i == NFSV4OP_CBSEQUENCE && + j == 0)) && sep != NULL) { if (i == NFSV4OP_SEQUENCE) NFSM_DISSECT(tl, uint32_t *, NFSX_V4SESSIONID + @@ -893,7 +893,8 @@ tryagain: } if (nd->nd_repstat != 0) { if (nd->nd_repstat == NFSERR_BADSESSION && - nmp != NULL && dssep == NULL) { + nmp != NULL && dssep == NULL && + (nd->nd_flag & ND_NFSV41) != 0) { /* * If this is a client side MDS RPC, mark * the MDS session defunct and initiate Modified: stable/11/sys/fs/nfsclient/nfsmount.h ============================================================================== --- stable/11/sys/fs/nfsclient/nfsmount.h Mon Jul 30 11:41:51 2018 (r336897) +++ stable/11/sys/fs/nfsclient/nfsmount.h Mon Jul 30 12:10:23 2018 (r336898) @@ -127,8 +127,10 @@ nfsmnt_mdssession(struct nfsmount *nmp) { struct nfsclsession *tsep; + tsep = NULL; mtx_lock(&nmp->nm_mtx); - tsep = NFSMNT_MDSSESSION(nmp); + if (TAILQ_FIRST(&nmp->nm_sess) != NULL) + tsep = NFSMNT_MDSSESSION(nmp); mtx_unlock(&nmp->nm_mtx); return (tsep); } From owner-svn-src-all@freebsd.org Mon Jul 30 12:17:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 268C11055483; Mon, 30 Jul 2018 12:17:12 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A3D196FFA; Mon, 30 Jul 2018 12:17:11 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5B63724310; Mon, 30 Jul 2018 12:17:11 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UCHBer011763; Mon, 30 Jul 2018 12:17:11 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UCHBTP011762; Mon, 30 Jul 2018 12:17:11 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807301217.w6UCHBTP011762@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 30 Jul 2018 12:17:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r336899 - in stable/10/sys/fs: nfs nfsclient X-SVN-Group: stable-10 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in stable/10/sys/fs: nfs nfsclient X-SVN-Commit-Revision: 336899 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 12:17:12 -0000 Author: rmacklem Date: Mon Jul 30 12:17:10 2018 New Revision: 336899 URL: https://svnweb.freebsd.org/changeset/base/336899 Log: MFC: r334966 Add a couple of safety belt checks to the NFSv4.1 client related to sessions. There were a couple of cases in newnfs_request() that it assumed that it was an NFSv4.1 mount with a session. This should always be the case when a Sequence operation is in the reply or the server replies NFSERR_BADSESSION. However, if a server was broken and sent an erroneous reply, these safety belt checks should avoid trouble. The one check required a small tweak to nfsmnt_mdssession() so that it returns NULL when there is no session instead of the offset of the field in the structure (0x8 for i386). This patch should have no effect on normal operation of the client. Found by inspection during pNFS server development. Modified: stable/10/sys/fs/nfs/nfs_commonkrpc.c stable/10/sys/fs/nfsclient/nfsmount.h Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/fs/nfs/nfs_commonkrpc.c ============================================================================== --- stable/10/sys/fs/nfs/nfs_commonkrpc.c Mon Jul 30 12:10:23 2018 (r336898) +++ stable/10/sys/fs/nfs/nfs_commonkrpc.c Mon Jul 30 12:17:10 2018 (r336899) @@ -850,9 +850,9 @@ tryagain: if ((nmp != NULL && i == NFSV4OP_SEQUENCE && j != 0) || (clp != NULL && i == NFSV4OP_CBSEQUENCE && j != 0)) NFSCL_DEBUG(1, "failed seq=%d\n", j); - if ((nmp != NULL && i == NFSV4OP_SEQUENCE && j == 0) || - (clp != NULL && i == NFSV4OP_CBSEQUENCE && j == 0) - ) { + if (((nmp != NULL && i == NFSV4OP_SEQUENCE && j == 0) || + (clp != NULL && i == NFSV4OP_CBSEQUENCE && + j == 0)) && sep != NULL) { if (i == NFSV4OP_SEQUENCE) NFSM_DISSECT(tl, uint32_t *, NFSX_V4SESSIONID + @@ -894,7 +894,8 @@ tryagain: } if (nd->nd_repstat != 0) { if (nd->nd_repstat == NFSERR_BADSESSION && - nmp != NULL && dssep == NULL) { + nmp != NULL && dssep == NULL && + (nd->nd_flag & ND_NFSV41) != 0) { /* * If this is a client side MDS RPC, mark * the MDS session defunct and initiate Modified: stable/10/sys/fs/nfsclient/nfsmount.h ============================================================================== --- stable/10/sys/fs/nfsclient/nfsmount.h Mon Jul 30 12:10:23 2018 (r336898) +++ stable/10/sys/fs/nfsclient/nfsmount.h Mon Jul 30 12:17:10 2018 (r336899) @@ -122,8 +122,10 @@ nfsmnt_mdssession(struct nfsmount *nmp) { struct nfsclsession *tsep; + tsep = NULL; mtx_lock(&nmp->nm_mtx); - tsep = NFSMNT_MDSSESSION(nmp); + if (TAILQ_FIRST(&nmp->nm_sess) != NULL) + tsep = NFSMNT_MDSSESSION(nmp); mtx_unlock(&nmp->nm_mtx); return (tsep); } From owner-svn-src-all@freebsd.org Mon Jul 30 12:22:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D93E7105591D; Mon, 30 Jul 2018 12:22:50 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 76D1F97516; Mon, 30 Jul 2018 12:22:50 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 53E9A244A2; Mon, 30 Jul 2018 12:22:50 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UCMoV3016479; Mon, 30 Jul 2018 12:22:50 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UCMoVg016478; Mon, 30 Jul 2018 12:22:50 GMT (envelope-from br@FreeBSD.org) Message-Id: <201807301222.w6UCMoVg016478@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Mon, 30 Jul 2018 12:22:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336900 - head/sys/riscv/conf X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/sys/riscv/conf X-SVN-Commit-Revision: 336900 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 12:22:51 -0000 Author: br Date: Mon Jul 30 12:22:49 2018 New Revision: 336900 URL: https://svnweb.freebsd.org/changeset/base/336900 Log: Disable VIMAGE on RISC-V. Similar to r326179 ("Temporarily disable VIMAGE on arm64") creation of if_lagg or epair on RISC-V results a kernel panic. Sponsored by: DARPA, AFRL Modified: head/sys/riscv/conf/GENERIC Modified: head/sys/riscv/conf/GENERIC ============================================================================== --- head/sys/riscv/conf/GENERIC Mon Jul 30 12:17:10 2018 (r336899) +++ head/sys/riscv/conf/GENERIC Mon Jul 30 12:22:49 2018 (r336900) @@ -30,7 +30,7 @@ makeoptions WITHOUT_MODULES="usb otusfw mwlfw ispfw mw options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption -options VIMAGE # Subsystem virtualization, e.g. VNET +# options VIMAGE # Subsystem virtualization, e.g. VNET options INET # InterNETworking options INET6 # IPv6 communications protocols options TCP_HHOOK # hhook(9) framework for TCP From owner-svn-src-all@freebsd.org Mon Jul 30 12:38:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BA831055CCA; Mon, 30 Jul 2018 12:38:09 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 01E5E97D1E; Mon, 30 Jul 2018 12:38:09 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D727324634; Mon, 30 Jul 2018 12:38:08 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UCc8j4022074; Mon, 30 Jul 2018 12:38:08 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UCc8Nd022073; Mon, 30 Jul 2018 12:38:08 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807301238.w6UCc8Nd022073@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 12:38:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336901 - head/share/mk X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/share/mk X-SVN-Commit-Revision: 336901 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 12:38:09 -0000 Author: emaste Date: Mon Jul 30 12:38:08 2018 New Revision: 336901 URL: https://svnweb.freebsd.org/changeset/base/336901 Log: Enable ld.lld as bootstrap linker by default on i386 Akin to r327783 for amd64. lld has been usable for amd64 for quite some time, but a couple of issues remained that affected i386. These were recently addressed upstream in lld and merged into FreeBSD or addressed directly in FreeBSD (r326831, r326879, r326897, r326957, r333401, r334626, r336664). Similarly to the intial amd64 commit this change enables lld only as the bootstrap linker (used to link the kernel and userland libraries and executables), while GNU ld.bfd is still installed as /usr/bin/ld and used for ports builds. That will be changed shortly, after an exp-run. This is a recommit of r327823 after additional lld fixes. PR: 225128 (exp-run) Relnotes: Yes Sponsored by: The FreeBSD Foundation Modified: head/share/mk/src.opts.mk Modified: head/share/mk/src.opts.mk ============================================================================== --- head/share/mk/src.opts.mk Mon Jul 30 12:22:49 2018 (r336900) +++ head/share/mk/src.opts.mk Mon Jul 30 12:38:08 2018 (r336901) @@ -311,6 +311,9 @@ __DEFAULT_NO_OPTIONS+=LLVM_LIBUNWIND .endif .if ${__T} == "aarch64" || ${__T} == "amd64" __DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP LLD_IS_LD +.elif ${__T} == "i386" +__DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP +__DEFAULT_NO_OPTIONS+=LLD_IS_LD .else __DEFAULT_NO_OPTIONS+=LLD_BOOTSTRAP LLD_IS_LD .endif From owner-svn-src-all@freebsd.org Mon Jul 30 12:44:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 502C81056050; Mon, 30 Jul 2018 12:44:45 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04B82703CB; Mon, 30 Jul 2018 12:44:45 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D6135247DD; Mon, 30 Jul 2018 12:44:44 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UCiiEO027484; Mon, 30 Jul 2018 12:44:44 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UCiiSn027483; Mon, 30 Jul 2018 12:44:44 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807301244.w6UCiiSn027483@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Mon, 30 Jul 2018 12:44:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336902 - head/sys/net X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: head/sys/net X-SVN-Commit-Revision: 336902 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 12:44:45 -0000 Author: andrew Date: Mon Jul 30 12:44:44 2018 New Revision: 336902 URL: https://svnweb.freebsd.org/changeset/base/336902 Log: As with DPCPU_DEFINE make it a compile error to use static with VNET_DEFINE. There is the VNET_DEFINE_STATIC macro for that. Modified: head/sys/net/vnet.h Modified: head/sys/net/vnet.h ============================================================================== --- head/sys/net/vnet.h Mon Jul 30 12:38:08 2018 (r336901) +++ head/sys/net/vnet.h Mon Jul 30 12:44:44 2018 (r336902) @@ -270,8 +270,9 @@ extern struct sx vnet_sxlock; */ #define VNET_NAME(n) vnet_entry_##n #define VNET_DECLARE(t, n) extern t VNET_NAME(n) +/* struct _hack is to stop this from being used with static data */ #define VNET_DEFINE(t, n) \ - t VNET_NAME(n) __section(VNET_SETNAME) __used + struct _hack; t VNET_NAME(n) __section(VNET_SETNAME) __used #define VNET_DEFINE_STATIC(t, n) \ static t VNET_NAME(n) __section(VNET_SETNAME) __used #define _VNET_PTR(b, n) (__typeof(VNET_NAME(n))*) \ From owner-svn-src-all@freebsd.org Mon Jul 30 12:51:58 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5141D10562AC; Mon, 30 Jul 2018 12:51:58 +0000 (UTC) (envelope-from slavash@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0318970865; Mon, 30 Jul 2018 12:51:58 +0000 (UTC) (envelope-from slavash@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D8B7B24960; Mon, 30 Jul 2018 12:51:57 +0000 (UTC) (envelope-from slavash@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UCpvMH032337; Mon, 30 Jul 2018 12:51:57 GMT (envelope-from slavash@FreeBSD.org) Received: (from slavash@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UCpvcW032336; Mon, 30 Jul 2018 12:51:57 GMT (envelope-from slavash@FreeBSD.org) Message-Id: <201807301251.w6UCpvcW032336@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: slavash set sender to slavash@FreeBSD.org using -f From: Slava Shwartsman Date: Mon, 30 Jul 2018 12:51:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336903 - stable/11/contrib/tcpdump X-SVN-Group: stable-11 X-SVN-Commit-Author: slavash X-SVN-Commit-Paths: stable/11/contrib/tcpdump X-SVN-Commit-Revision: 336903 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 12:51:58 -0000 Author: slavash Date: Mon Jul 30 12:51:57 2018 New Revision: 336903 URL: https://svnweb.freebsd.org/changeset/base/336903 Log: MFC r334318: Vendor import two upstream commits: c1bb8784abd3ca978e376b0d10e324db0491237b 9c4af7213cc2543a1f5586d8f2c19f86aa0cbe72 Approved by: hselasky (mentor), kib (mentor) Sponsored by: Mellanox Technologies Modified: stable/11/contrib/tcpdump/tcpdump.c Directory Properties: stable/11/ (props changed) Modified: stable/11/contrib/tcpdump/tcpdump.c ============================================================================== --- stable/11/contrib/tcpdump/tcpdump.c Mon Jul 30 12:44:44 2018 (r336902) +++ stable/11/contrib/tcpdump/tcpdump.c Mon Jul 30 12:51:57 2018 (r336903) @@ -77,7 +77,6 @@ The Regents of the University of California. All righ */ #ifdef HAVE_CAPSICUM #include -#include #include #include #include @@ -114,6 +113,10 @@ The Regents of the University of California. All righ #endif /* HAVE_CAP_NG_H */ #endif /* HAVE_LIBCAP_NG */ +#ifdef __FreeBSD__ +#include +#endif /* __FreeBSD__ */ + #include "netdissect.h" #include "interface.h" #include "addrtoname.h" @@ -1095,6 +1098,10 @@ open_interface(const char *device, netdissect_options sysctlbyname(sysctl, parent, &s, NULL, 0); strlcpy(newdev, device, sizeof(newdev)); /* Suggest a new wlan device. */ + /* FIXME: incrementing the index this way is not going to work well + * when the index is 9 or greater but the only consequence in this + * specific case would be an error message that looks a bit odd. + */ newdev[strlen(newdev)-1]++; error("%s is not a monitor mode VAP\n" "To create a new monitor mode VAP use:\n" From owner-svn-src-all@freebsd.org Mon Jul 30 12:58:34 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E17D610563CD; Mon, 30 Jul 2018 12:58:34 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 55C9670BB6; Mon, 30 Jul 2018 12:58:34 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3735924986; Mon, 30 Jul 2018 12:58:34 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UCwYbL032672; Mon, 30 Jul 2018 12:58:34 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UCwYTO032671; Mon, 30 Jul 2018 12:58:34 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807301258.w6UCwYTO032671@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 12:58:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336904 - head/share/man/man5 X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/share/man/man5 X-SVN-Commit-Revision: 336904 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 12:58:35 -0000 Author: emaste Date: Mon Jul 30 12:58:33 2018 New Revision: 336904 URL: https://svnweb.freebsd.org/changeset/base/336904 Log: Regen src.conf.5 after r336901, WITH_LLD_BOOTSTRAP on i386 Sponsored by: The FreeBSD Foundation Modified: head/share/man/man5/src.conf.5 Modified: head/share/man/man5/src.conf.5 ============================================================================== --- head/share/man/man5/src.conf.5 Mon Jul 30 12:51:57 2018 (r336903) +++ head/share/man/man5/src.conf.5 Mon Jul 30 12:58:33 2018 (r336904) @@ -1,6 +1,6 @@ .\" DO NOT EDIT-- this file is @generated by tools/build/options/makeman. .\" $FreeBSD$ -.Dd July 25, 2018 +.Dd July 30, 2018 .Dt SRC.CONF 5 .Os .Sh NAME @@ -1001,12 +1001,12 @@ To be able to build the system, either Binutils or LLD enabled unless an alternate linker is provided via XLD. .Pp This is a default setting on -arm/arm, arm/armv6, arm/armv7, i386/i386, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. +arm/arm, arm/armv6, arm/armv7, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITH_LLD_BOOTSTRAP Set to build the LLD linker during the bootstrap phase of the build. .Pp This is a default setting on -amd64/amd64 and arm64/aarch64. +amd64/amd64, arm64/aarch64 and i386/i386. .It Va WITHOUT_LLD_IS_LD Set to use GNU binutils ld as the system linker, instead of LLVM's LLD. .Pp From owner-svn-src-all@freebsd.org Mon Jul 30 14:21:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BCB71057E84; Mon, 30 Jul 2018 14:21:50 +0000 (UTC) (envelope-from dab@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1297873A80; Mon, 30 Jul 2018 14:21:50 +0000 (UTC) (envelope-from dab@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E8FED256FC; Mon, 30 Jul 2018 14:21:49 +0000 (UTC) (envelope-from dab@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UELnYC076213; Mon, 30 Jul 2018 14:21:49 GMT (envelope-from dab@FreeBSD.org) Received: (from dab@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UELnWf076212; Mon, 30 Jul 2018 14:21:49 GMT (envelope-from dab@FreeBSD.org) Message-Id: <201807301421.w6UELnWf076212@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dab set sender to dab@FreeBSD.org using -f From: David Bright Date: Mon, 30 Jul 2018 14:21:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336905 - head/tests/sys/kqueue/libkqueue X-SVN-Group: head X-SVN-Commit-Author: dab X-SVN-Commit-Paths: head/tests/sys/kqueue/libkqueue X-SVN-Commit-Revision: 336905 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 14:21:50 -0000 Author: dab Date: Mon Jul 30 14:21:49 2018 New Revision: 336905 URL: https://svnweb.freebsd.org/changeset/base/336905 Log: Correct possible misleading error message in kqtest. ian@ pointed out that in the test_abstime() function time(NULL) is used twice; once in an "if" test and again in the enclosed error message. If the true branch was taken and the process got preempted before the second time(NULL) call, by the time the error message was generated enough time could have elapsed that the message could claim that the event came "too early" but print an event time that was after the expected timeout. Correct by making the time(NULL) call only once and using that returned time in both the "if" test and the error message. Reported by: ian@ MFC after: 4 days X-MFC-with: r336761, r336781, r336802 Sponsored by: Dell EMC Modified: head/tests/sys/kqueue/libkqueue/timer.c Modified: head/tests/sys/kqueue/libkqueue/timer.c ============================================================================== --- head/tests/sys/kqueue/libkqueue/timer.c Mon Jul 30 12:58:33 2018 (r336904) +++ head/tests/sys/kqueue/libkqueue/timer.c Mon Jul 30 14:21:49 2018 (r336905) @@ -220,16 +220,17 @@ test_abstime(void) { const char *test_id = "kevent(EVFILT_TIMER, EV_ONESHOT, NOTE_ABSTIME)"; struct kevent kev; - time_t when; + time_t start; + time_t stop; const int timeout = 3; test_begin(test_id); test_no_kevents(); - when = time(NULL); + start = time(NULL); EV_SET(&kev, vnode_fd, EVFILT_TIMER, EV_ADD | EV_ONESHOT, - NOTE_ABSTIME | NOTE_SECONDS, when + timeout, NULL); + NOTE_ABSTIME | NOTE_SECONDS, start + timeout, NULL); if (kevent(kqfd, &kev, 1, NULL, 0, NULL) < 0) err(1, "%s", test_id); @@ -238,8 +239,9 @@ test_abstime(void) kev.data = 1; kev.fflags = 0; kevent_cmp(&kev, kevent_get(kqfd)); - if (time(NULL) < when + timeout) - err(1, "too early %jd %jd", (intmax_t)time(NULL), (intmax_t)(when + timeout)); + stop = time(NULL); + if (stop < start + timeout) + err(1, "too early %jd %jd", (intmax_t)stop, (intmax_t)(start + timeout)); /* Check if the event occurs again */ sleep(3); From owner-svn-src-all@freebsd.org Mon Jul 30 14:25:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31BA41057F77; Mon, 30 Jul 2018 14:25:19 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD66D73CD7; Mon, 30 Jul 2018 14:25:18 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A19AF2583C; Mon, 30 Jul 2018 14:25:18 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UEPI4A078631; Mon, 30 Jul 2018 14:25:18 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UEPIVP078629; Mon, 30 Jul 2018 14:25:18 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807301425.w6UEPIVP078629@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Mon, 30 Jul 2018 14:25:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336906 - in head/sys: kern net X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: in head/sys: kern net X-SVN-Commit-Revision: 336906 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 14:25:19 -0000 Author: andrew Date: Mon Jul 30 14:25:17 2018 New Revision: 336906 URL: https://svnweb.freebsd.org/changeset/base/336906 Log: Ensure the DPCPU and VNET module spaces are aligned to hold a pointer. Previously they may have been aligned to a char, leading to misaligned DPCPU and VNET variables. Sponsored by: DARPA, AFRL Modified: head/sys/kern/subr_pcpu.c head/sys/net/vnet.c Modified: head/sys/kern/subr_pcpu.c ============================================================================== --- head/sys/kern/subr_pcpu.c Mon Jul 30 14:21:49 2018 (r336905) +++ head/sys/kern/subr_pcpu.c Mon Jul 30 14:25:17 2018 (r336906) @@ -72,7 +72,7 @@ struct dpcpu_free { TAILQ_ENTRY(dpcpu_free) df_link; }; -DPCPU_DEFINE_STATIC(char, modspace[DPCPU_MODMIN]); +DPCPU_DEFINE_STATIC(char, modspace[DPCPU_MODMIN] __aligned(__alignof(void *))); static TAILQ_HEAD(, dpcpu_free) dpcpu_head = TAILQ_HEAD_INITIALIZER(dpcpu_head); static struct sx dpcpu_lock; uintptr_t dpcpu_off[MAXCPU]; Modified: head/sys/net/vnet.c ============================================================================== --- head/sys/net/vnet.c Mon Jul 30 14:21:49 2018 (r336905) +++ head/sys/net/vnet.c Mon Jul 30 14:25:17 2018 (r336906) @@ -178,7 +178,7 @@ static MALLOC_DEFINE(M_VNET_DATA, "vnet_data", "VNET d * Space to store virtualized global variables from loadable kernel modules, * and the free list to manage it. */ -VNET_DEFINE_STATIC(char, modspace[VNET_MODMIN]); +VNET_DEFINE_STATIC(char, modspace[VNET_MODMIN] __aligned(__alignof(void *))); /* * Global lists of subsystem constructor and destructors for vnets. They are From owner-svn-src-all@freebsd.org Mon Jul 30 14:28:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72C411058026; Mon, 30 Jul 2018 14:28:01 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 29BF473E7C; Mon, 30 Jul 2018 14:28:01 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0636A2583D; Mon, 30 Jul 2018 14:28:01 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UES0Ba078784; Mon, 30 Jul 2018 14:28:00 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UES0gC078783; Mon, 30 Jul 2018 14:28:00 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807301428.w6UES0gC078783@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 14:28:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336907 - head X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head X-SVN-Commit-Revision: 336907 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 14:28:01 -0000 Author: emaste Date: Mon Jul 30 14:28:00 2018 New Revision: 336907 URL: https://svnweb.freebsd.org/changeset/base/336907 Log: Makefile.libcompat: use one variable assignment per line This makes it easier to carry or merge patches that remove gnu/ components. Modified: head/Makefile.libcompat Modified: head/Makefile.libcompat ============================================================================== --- head/Makefile.libcompat Mon Jul 30 14:25:17 2018 (r336906) +++ head/Makefile.libcompat Mon Jul 30 14:28:00 2018 (r336907) @@ -129,7 +129,8 @@ LIBCOMPATIMAKE+= ${LIBCOMPATWMAKE:NINSTALL=*:NDESTDIR= MK_TOOLCHAIN=no ${IMAKE_INSTALL} \ -DLIBRARIES_ONLY -_LC_LIBDIRS.yes= lib gnu/lib +_LC_LIBDIRS.yes= lib +_LC_LIBDIRS.yes= gnu/lib _LC_LIBDIRS.${MK_CDDL:tl}+= cddl/lib _LC_LIBDIRS.${MK_CRYPT:tl}+= secure/lib _LC_LIBDIRS.${MK_KERBEROS:tl}+= kerberos5/lib From owner-svn-src-all@freebsd.org Mon Jul 30 14:32:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BF8A1058337; Mon, 30 Jul 2018 14:32:57 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2B0BE74497; Mon, 30 Jul 2018 14:32:56 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.128.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 643E42600DB; Mon, 30 Jul 2018 16:32:54 +0200 (CEST) Subject: Re: svn commit: r336907 - head To: Ed Maste , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201807301428.w6UES0gC078783@repo.freebsd.org> From: Hans Petter Selasky Message-ID: <79db5503-ca9e-449e-9b9b-b48d1dfd1597@selasky.org> Date: Mon, 30 Jul 2018 16:32:33 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <201807301428.w6UES0gC078783@repo.freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 14:32:57 -0000 On 07/30/18 16:28, Ed Maste wrote: > +_LC_LIBDIRS.yes= lib > +_LC_LIBDIRS.yes= gnu/lib Should the latter line use += ? --HPS From owner-svn-src-all@freebsd.org Mon Jul 30 14:43:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E72D910587EF; Mon, 30 Jul 2018 14:43:00 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D7CE74AE9; Mon, 30 Jul 2018 14:43:00 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7E96225B6D; Mon, 30 Jul 2018 14:43:00 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UEh0pl089218; Mon, 30 Jul 2018 14:43:00 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UEh0RB089216; Mon, 30 Jul 2018 14:43:00 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807301443.w6UEh0RB089216@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 14:43:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336908 - in head: . share/mk X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: in head: . share/mk X-SVN-Commit-Revision: 336908 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 14:43:01 -0000 Author: emaste Date: Mon Jul 30 14:42:59 2018 New Revision: 336908 URL: https://svnweb.freebsd.org/changeset/base/336908 Log: Makefile.libcompat: correctly append to make variable Reported by: hselasky Pointy hat to: emaste Modified: head/Makefile.libcompat head/share/mk/src.opts.mk Modified: head/Makefile.libcompat ============================================================================== --- head/Makefile.libcompat Mon Jul 30 14:28:00 2018 (r336907) +++ head/Makefile.libcompat Mon Jul 30 14:42:59 2018 (r336908) @@ -130,7 +130,7 @@ LIBCOMPATIMAKE+= ${LIBCOMPATWMAKE:NINSTALL=*:NDESTDIR= -DLIBRARIES_ONLY _LC_LIBDIRS.yes= lib -_LC_LIBDIRS.yes= gnu/lib +_LC_LIBDIRS.yes+= gnu/lib _LC_LIBDIRS.${MK_CDDL:tl}+= cddl/lib _LC_LIBDIRS.${MK_CRYPT:tl}+= secure/lib _LC_LIBDIRS.${MK_KERBEROS:tl}+= kerberos5/lib Modified: head/share/mk/src.opts.mk ============================================================================== --- head/share/mk/src.opts.mk Mon Jul 30 14:28:00 2018 (r336907) +++ head/share/mk/src.opts.mk Mon Jul 30 14:42:59 2018 (r336908) @@ -309,11 +309,8 @@ __DEFAULT_YES_OPTIONS+=LLVM_LIBUNWIND .else __DEFAULT_NO_OPTIONS+=LLVM_LIBUNWIND .endif -.if ${__T} == "aarch64" || ${__T} == "amd64" +.if ${__T} == "aarch64" || ${__T} == "amd64" || ${__T} == "i386" __DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP LLD_IS_LD -.elif ${__T} == "i386" -__DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP -__DEFAULT_NO_OPTIONS+=LLD_IS_LD .else __DEFAULT_NO_OPTIONS+=LLD_BOOTSTRAP LLD_IS_LD .endif From owner-svn-src-all@freebsd.org Mon Jul 30 14:44:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BF9610588FD; Mon, 30 Jul 2018 14:44:32 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7DE874D26; Mon, 30 Jul 2018 14:44:31 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-it0-x235.google.com with SMTP id p7-v6so648835itf.5; Mon, 30 Jul 2018 07:44:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=I5OpnOj0+GQWDIFudSjS++q8uREMRPnriBHQhS9IY+0=; b=EbxZFzXXRxokAu9D0U3msHUuPvYS7os118dNePM6yZHToPeJ0lq6y/oc3Xjx+ukxLy 7vr6ULkDIuxY83NfWwU48KwljZvrlnsMmxG/LGOYRx0N0BquQgSB19mu0i2V+lxalC7T 3M+5+6mIkjeCnytQ9soxTrFw4dKmO0xtmu0lJD46CfLTF7y7ER1NsYTAKnjBbVfCFSqR uGMZS4dC1USHQcp0e1deP7Q1Hx7slCLVHB2KDiwDbPAS8T2i/9wzsutvaH6CU24/aeUK 9nSAhlP3F2Bakc1I6KwT8U5L2z2QDiZyaIFmCkoTYez4A2HTziMYALxqOm8+UnN8u7+s bxgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=I5OpnOj0+GQWDIFudSjS++q8uREMRPnriBHQhS9IY+0=; b=ex2yIG0EDTzq/vdw2CDE4y2EaTnZUzEix4KToZEFsKbgFEoNm4sSj9N9h1fhjP50ef h2i0Cz2ixuOB9Z1E9KTSYtjp3ItbIHahQtyOBicYuZ6AZ2olrpXT1GXykOXrSKdYZHBA 9kYgmGoXjoonGQm5AUcTIN/li1sKcXnfqW7gwQC8pY4OOLlRXtFzHDP0FKguQ5bQxzWy q6s7Dr2mUl2TH4Yh7oQlrD395AiyzlXdLucHXZ020YySPPhbjoCIryMF3N9HS0qWULvZ qfKrF6W4dzc+5Siw5ozhpAS0+fGoVnSVeYgm87Y3RYZN9KZUALcQTnioGpnuLfetCaes ANTw== X-Gm-Message-State: AOUpUlFVo0wJNS3vEBULegl7IqikGsuq44d0foxmEjhzH1OHvoGGkrtf 1jGEJEXtO3ceFvXew4WAqZ6PVqQzXIL9eU8QDitCsg== X-Google-Smtp-Source: AAOMgpeNgVEB7nuRqOUd9i7/GCRtZ6nmjJoZuDfyGB9oYNiGYGgw/eHaOSfLzhcxtaCDNlmGE4gLfpxY08EL5wQ2Fdk= X-Received: by 2002:a02:9a10:: with SMTP id b16-v6mr16794373jal.4.1532961871174; Mon, 30 Jul 2018 07:44:31 -0700 (PDT) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 2002:a6b:4a08:0:0:0:0:0 with HTTP; Mon, 30 Jul 2018 07:44:10 -0700 (PDT) In-Reply-To: <79db5503-ca9e-449e-9b9b-b48d1dfd1597@selasky.org> References: <201807301428.w6UES0gC078783@repo.freebsd.org> <79db5503-ca9e-449e-9b9b-b48d1dfd1597@selasky.org> From: Ed Maste Date: Mon, 30 Jul 2018 10:44:10 -0400 X-Google-Sender-Auth: N2h3HiTdmoNAyku7RYsOS9bY028 Message-ID: Subject: Re: svn commit: r336907 - head To: Hans Petter Selasky Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 14:44:32 -0000 On 30 July 2018 at 10:32, Hans Petter Selasky wrote: > On 07/30/18 16:28, Ed Maste wrote: >> >> +_LC_LIBDIRS.yes= lib >> +_LC_LIBDIRS.yes= gnu/lib > > Should the latter line use += ? It should indeed, I somehow broke this while manually applying the change from an old WIP branch. Fixed by r336908, thanks for pointing it out. From owner-svn-src-all@freebsd.org Mon Jul 30 15:05:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BDCF1058FC2; Mon, 30 Jul 2018 15:05:08 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E459F7583D; Mon, 30 Jul 2018 15:05:07 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C40B125ECE; Mon, 30 Jul 2018 15:05:07 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UF57FH099660; Mon, 30 Jul 2018 15:05:07 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UF57Pa099658; Mon, 30 Jul 2018 15:05:07 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807301505.w6UF57Pa099658@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Mon, 30 Jul 2018 15:05:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336909 - in head/sys: net sys X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: in head/sys: net sys X-SVN-Commit-Revision: 336909 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:05:08 -0000 Author: andrew Date: Mon Jul 30 15:05:07 2018 New Revision: 336909 URL: https://svnweb.freebsd.org/changeset/base/336909 Log: As with DPCPU_DEFINE_STATIC make VNET_DEFINE_STATIC non-static on arm64 in modules. It also fails in the same way, we are unable to relocate static variables as the compiler uses PC-relative loads with nothing for the kernel linker to relocate. Sponsored by: DARPA, AFRL Modified: head/sys/net/vnet.h head/sys/sys/pcpu.h Modified: head/sys/net/vnet.h ============================================================================== --- head/sys/net/vnet.h Mon Jul 30 14:42:59 2018 (r336908) +++ head/sys/net/vnet.h Mon Jul 30 15:05:07 2018 (r336909) @@ -273,8 +273,17 @@ extern struct sx vnet_sxlock; /* struct _hack is to stop this from being used with static data */ #define VNET_DEFINE(t, n) \ struct _hack; t VNET_NAME(n) __section(VNET_SETNAME) __used +#if defined(KLD_MODULE) && defined(__aarch64__) +/* + * As with DPCPU_DEFINE_STATIC we are unable to mark this data as static + * in modules on some architectures. + */ #define VNET_DEFINE_STATIC(t, n) \ + t VNET_NAME(n) __section(VNET_SETNAME) __used +#else +#define VNET_DEFINE_STATIC(t, n) \ static t VNET_NAME(n) __section(VNET_SETNAME) __used +#endif #define _VNET_PTR(b, n) (__typeof(VNET_NAME(n))*) \ ((b) + (uintptr_t)&VNET_NAME(n)) Modified: head/sys/sys/pcpu.h ============================================================================== --- head/sys/sys/pcpu.h Mon Jul 30 14:42:59 2018 (r336908) +++ head/sys/sys/pcpu.h Mon Jul 30 15:05:07 2018 (r336909) @@ -97,7 +97,9 @@ extern uintptr_t dpcpu_off[]; * wrong location. * * This is a workaround until a better solution can be found. -*/ + * + * VNET_DEFINE_STATIC also has the same workaround. + */ #define DPCPU_DEFINE_STATIC(t, n) \ t DPCPU_NAME(n) __section(DPCPU_SETNAME) __used #else From owner-svn-src-all@freebsd.org Mon Jul 30 15:10:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D85D81059243; Mon, 30 Jul 2018 15:10:06 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E3EC75BC3; Mon, 30 Jul 2018 15:10:06 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7088125EF0; Mon, 30 Jul 2018 15:10:06 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UFA69s099934; Mon, 30 Jul 2018 15:10:06 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UFA6bB099933; Mon, 30 Jul 2018 15:10:06 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807301510.w6UFA6bB099933@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 15:10:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336910 - head/share/mk X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/share/mk X-SVN-Commit-Revision: 336910 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:10:07 -0000 Author: emaste Date: Mon Jul 30 15:10:06 2018 New Revision: 336910 URL: https://svnweb.freebsd.org/changeset/base/336910 Log: Revert accidental change from r336908 By default ld.lld should be the bootstrap linker (only) on i386 right now. Once the i386 exp-run with LLD_IS_LD has a good result this will also be enabled by default. Reported by: andrew Pointy hat to: emaste Modified: head/share/mk/src.opts.mk Modified: head/share/mk/src.opts.mk ============================================================================== --- head/share/mk/src.opts.mk Mon Jul 30 15:05:07 2018 (r336909) +++ head/share/mk/src.opts.mk Mon Jul 30 15:10:06 2018 (r336910) @@ -309,8 +309,11 @@ __DEFAULT_YES_OPTIONS+=LLVM_LIBUNWIND .else __DEFAULT_NO_OPTIONS+=LLVM_LIBUNWIND .endif -.if ${__T} == "aarch64" || ${__T} == "amd64" || ${__T} == "i386" +.if ${__T} == "aarch64" || ${__T} == "amd64" __DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP LLD_IS_LD +.elif ${__T} == "i386" +__DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP +__DEFAULT_NO_OPTIONS+=LLD_IS_LD .else __DEFAULT_NO_OPTIONS+=LLD_BOOTSTRAP LLD_IS_LD .endif From owner-svn-src-all@freebsd.org Mon Jul 30 15:15:34 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE70B105956F; Mon, 30 Jul 2018 15:15:34 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 743C5760E5; Mon, 30 Jul 2018 15:15:34 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5236226080; Mon, 30 Jul 2018 15:15:34 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UFFYws005175; Mon, 30 Jul 2018 15:15:34 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UFFYKf005174; Mon, 30 Jul 2018 15:15:34 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201807301515.w6UFFYKf005174@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Mon, 30 Jul 2018 15:15:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336911 - head/sys/dev/sound/pci/hda X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/dev/sound/pci/hda X-SVN-Commit-Revision: 336911 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:15:34 -0000 Author: jhibbits Date: Mon Jul 30 15:15:33 2018 New Revision: 336911 URL: https://svnweb.freebsd.org/changeset/base/336911 Log: snd_hda: Only free streams DMA maps if the streams list has been created If hdac_attach fails prior to allocating sc->streams, cleanup in the hdac_attach_fail label will dereference a NULL pointer, panicking. Modified: head/sys/dev/sound/pci/hda/hdac.c Modified: head/sys/dev/sound/pci/hda/hdac.c ============================================================================== --- head/sys/dev/sound/pci/hda/hdac.c Mon Jul 30 15:10:06 2018 (r336910) +++ head/sys/dev/sound/pci/hda/hdac.c Mon Jul 30 15:15:33 2018 (r336911) @@ -1312,8 +1312,9 @@ hdac_attach(device_t dev) hdac_attach_fail: hdac_irq_free(sc); - for (i = 0; i < sc->num_ss; i++) - hdac_dma_free(sc, &sc->streams[i].bdl); + if (sc->streams != NULL) + for (i = 0; i < sc->num_ss; i++) + hdac_dma_free(sc, &sc->streams[i].bdl); free(sc->streams, M_HDAC); hdac_dma_free(sc, &sc->rirb_dma); hdac_dma_free(sc, &sc->corb_dma); From owner-svn-src-all@freebsd.org Mon Jul 30 15:20:00 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2960710596DB; Mon, 30 Jul 2018 15:20:00 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC71D76304; Mon, 30 Jul 2018 15:19:59 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A652626084; Mon, 30 Jul 2018 15:19:59 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UFJx0Z005376; Mon, 30 Jul 2018 15:19:59 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UFJxSK005375; Mon, 30 Jul 2018 15:19:59 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201807301519.w6UFJxSK005375@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Mon, 30 Jul 2018 15:19:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336912 - head/sys/dev/sound/pci/hda X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/dev/sound/pci/hda X-SVN-Commit-Revision: 336912 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:20:00 -0000 Author: jhibbits Date: Mon Jul 30 15:19:59 2018 New Revision: 336912 URL: https://svnweb.freebsd.org/changeset/base/336912 Log: snd_hda: Print error codes in decimal, rather than hex It's easy to confuse the error code as naked it looks decimal (EINVAL is reported as error 16, instead of error 22, so first reading looks like EBUSY). Modified: head/sys/dev/sound/pci/hda/hdac.c Modified: head/sys/dev/sound/pci/hda/hdac.c ============================================================================== --- head/sys/dev/sound/pci/hda/hdac.c Mon Jul 30 15:15:33 2018 (r336911) +++ head/sys/dev/sound/pci/hda/hdac.c Mon Jul 30 15:19:59 2018 (r336912) @@ -569,7 +569,7 @@ hdac_dma_alloc(struct hdac_softc *sc, struct hdac_dma NULL, /* lockfuncarg */ &dma->dma_tag); /* dmat */ if (result != 0) { - device_printf(sc->dev, "%s: bus_dma_tag_create failed (%x)\n", + device_printf(sc->dev, "%s: bus_dma_tag_create failed (%d)\n", __func__, result); goto hdac_dma_alloc_fail; } @@ -582,7 +582,7 @@ hdac_dma_alloc(struct hdac_softc *sc, struct hdac_dma ((sc->flags & HDAC_F_DMA_NOCACHE) ? BUS_DMA_NOCACHE : 0), &dma->dma_map); if (result != 0) { - device_printf(sc->dev, "%s: bus_dmamem_alloc failed (%x)\n", + device_printf(sc->dev, "%s: bus_dmamem_alloc failed (%d)\n", __func__, result); goto hdac_dma_alloc_fail; } @@ -597,7 +597,7 @@ hdac_dma_alloc(struct hdac_softc *sc, struct hdac_dma if (result != 0 || dma->dma_paddr == 0) { if (result == 0) result = ENOMEM; - device_printf(sc->dev, "%s: bus_dmamem_load failed (%x)\n", + device_printf(sc->dev, "%s: bus_dmamem_load failed (%d)\n", __func__, result); goto hdac_dma_alloc_fail; } @@ -718,7 +718,7 @@ hdac_irq_alloc(struct hdac_softc *sc) NULL, hdac_intr_handler, sc, &irq->irq_handle); if (result != 0) { device_printf(sc->dev, - "%s: Unable to setup interrupt handler (%x)\n", + "%s: Unable to setup interrupt handler (%d)\n", __func__, result); goto hdac_irq_alloc_fail; } @@ -1285,7 +1285,7 @@ hdac_attach(device_t dev) NULL, /* lockfuncarg */ &sc->chan_dmat); /* dmat */ if (result != 0) { - device_printf(dev, "%s: bus_dma_tag_create failed (%x)\n", + device_printf(dev, "%s: bus_dma_tag_create failed (%d)\n", __func__, result); goto hdac_attach_fail; } From owner-svn-src-all@freebsd.org Mon Jul 30 15:46:11 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E2F21059F13; Mon, 30 Jul 2018 15:46:11 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F107077093; Mon, 30 Jul 2018 15:46:10 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id CE4F35A9F17; Mon, 30 Jul 2018 15:46:03 +0000 (UTC) Date: Mon, 30 Jul 2018 15:46:03 +0000 From: Brooks Davis To: Konstantin Belousov Cc: Conrad Meyer , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336835 - head/lib/libc/gen Message-ID: <20180730154603.GA49837@spindle.one-eyed-alien.net> References: <201807281908.w6SJ80qo089859@repo.freebsd.org> <20180728200611.GA40119@kib.kiev.ua> <20180728203253.GB40119@kib.kiev.ua> <20180728214523.GC40119@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0F1p//8PRICkK4MW" Content-Disposition: inline In-Reply-To: <20180728214523.GC40119@kib.kiev.ua> User-Agent: Mutt/1.9.4 (2018-02-28) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:46:11 -0000 --0F1p//8PRICkK4MW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 29, 2018 at 12:45:23AM +0300, Konstantin Belousov wrote: > On Sat, Jul 28, 2018 at 01:59:16PM -0700, Conrad Meyer wrote: > > On Sat, Jul 28, 2018 at 1:32 PM, Konstantin Belousov > > wrote: > > > Our existing practice is to not provide an advise in the man pages > > > for the userspace code which would harm the portability or the standa= rd > > > conformance. > >=20 > > Perhaps, but you do not have to look beyond this manual page to find > > that such a practice, if it is even nominally applied, is not followed > > consistently. The remainder of the document spells the terminating > > value as plain "NULL." FreeBSD manual pages consistently refer to > > NULL as a pointer value. > >=20 > > The Linux manual page spells the terminating value (char *)NULL and > > explicitly documents why. That is perhaps both pedantically correct > > and clear. I am amenable to such a change. > I do not see much difference between (char *)0 and (char *)NULL. Both > are better there, because both are correct. >=20 > If you prefer (char *)NULL, please use it. >=20 > >=20 > > > I am not so sure about CheriBSD, for instance. > >=20 > > I observe simply that CheriBSD is not FreeBSD and that CHERI is an > > exotic architecture. > We should not give an advise which breaks other architectures, even > if not included into the mainline repo. CheriBSD would be fine because FreeBSD uses (void *)0 for NULL. That said, C++ requires that NULL be a bare 0 or nullptr so (char *)NULL is likely the most portable. -- Brooks --0F1p//8PRICkK4MW Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJbXzK7AAoJEKzQXbSebgfAOvQH/jXAUNrzJUb7QPKHK5JnfqvU RwwgQoo2PM4Gqis6tIgIgbYVHPIyqYfWmfrJTKcWPFzRq1IVGl3LHQ9xUO/YxI2R WyyTw/Nw7y1bfcmYdKqpOpMF78qcDg68EDlUvn5Oz1rbSh2aB2TgrRQmP7mSomAj 3V/cvk94+PDeGIqhApLRxgNKb66i1Y+FG+EDsBLhhE+HvEohw/9lmJSeP8Vm7Vf3 7gLh4rRINDdxBGODd509AVLlhYa9jwADh/v0BkkpUi6wU6LHnqD1CmB3AGuvSWZ3 2PwanPlwsZWTiFcQ5E+8UABi6eWSkiqGP9yjnDkIbSAxvyC0ctoqvfjYBAEjyZE= =oTja -----END PGP SIGNATURE----- --0F1p//8PRICkK4MW-- From owner-svn-src-all@freebsd.org Mon Jul 30 15:46:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DF101059F58; Mon, 30 Jul 2018 15:46:25 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E860677190; Mon, 30 Jul 2018 15:46:24 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CA28726563; Mon, 30 Jul 2018 15:46:24 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UFkOXm020282; Mon, 30 Jul 2018 15:46:24 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UFkOdr020280; Mon, 30 Jul 2018 15:46:24 GMT (envelope-from bapt@FreeBSD.org) Message-Id: <201807301546.w6UFkOdr020280@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f From: Baptiste Daroussin Date: Mon, 30 Jul 2018 15:46:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336913 - in head/usr.sbin/newsyslog: . tests X-SVN-Group: head X-SVN-Commit-Author: bapt X-SVN-Commit-Paths: in head/usr.sbin/newsyslog: . tests X-SVN-Commit-Revision: 336913 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:46:25 -0000 Author: bapt Date: Mon Jul 30 15:46:24 2018 New Revision: 336913 URL: https://svnweb.freebsd.org/changeset/base/336913 Log: newsyslog: fix compression arglist construction Reuse of the index variable in two nested loops resulted in only the first argument in the list being used (fine for gzip, not fine for zstd). Also add tests for xz and zstd, and fix the COMPRESS_SUFFIX_MAXLEN macro. Submitted by: dnelson_1901_yahoo.com Differential Revision: https://reviews.freebsd.org/D16509 Modified: head/usr.sbin/newsyslog/newsyslog.c head/usr.sbin/newsyslog/tests/legacy_test.sh Modified: head/usr.sbin/newsyslog/newsyslog.c ============================================================================== --- head/usr.sbin/newsyslog/newsyslog.c Mon Jul 30 15:19:59 2018 (r336912) +++ head/usr.sbin/newsyslog/newsyslog.c Mon Jul 30 15:46:24 2018 (r336913) @@ -105,7 +105,7 @@ __FBSDID("$FreeBSD$"); #define COMPRESS_SUFFIX_ZST ".zst" #endif -#define COMPRESS_SUFFIX_MAXLEN MAX(MAX(sizeof(COMPRESS_SUFFIX_GZ),sizeof(COMPRESS_SUFFIX_BZ2)),sizeof(COMPRESS_SUFFIX_XZ)) +#define COMPRESS_SUFFIX_MAXLEN MAX(MAX(MAX(sizeof(COMPRESS_SUFFIX_GZ),sizeof(COMPRESS_SUFFIX_BZ2)),sizeof(COMPRESS_SUFFIX_XZ)),sizeof(COMPRESS_SUFFIX_ZST)) /* * Compression types @@ -2027,7 +2027,7 @@ do_zipwork(struct zipwork_entry *zwork) char zresult[MAXPATHLEN]; char command[BUFSIZ]; char **args; - int c; + int c, i; assert(zwork != NULL); pgm_path = NULL; @@ -2043,10 +2043,10 @@ do_zipwork(struct zipwork_entry *zwork) (void) strlcat(zresult, compress_type[c].suffix, sizeof(zresult)); /* the first argument is always NULL, skip it */ - for (c = 1; c < ARGS_NUM; c++) { - if (compress_type[c].args[c] == NULL) + for (i = 1; i < ARGS_NUM; i++) { + if (compress_type[c].args[i] == NULL) break; - args[c] = compress_type[c].args[c]; + args[i] = compress_type[c].args[i]; } break; } Modified: head/usr.sbin/newsyslog/tests/legacy_test.sh ============================================================================== --- head/usr.sbin/newsyslog/tests/legacy_test.sh Mon Jul 30 15:19:59 2018 (r336912) +++ head/usr.sbin/newsyslog/tests/legacy_test.sh Mon Jul 30 15:46:24 2018 (r336913) @@ -498,6 +498,12 @@ tests_normal_rotate ".gz" echo "$LOGFPATH 640 3 * @T00 NCJ" > newsyslog.conf tests_normal_rotate ".bz2" +echo "$LOGFPATH 640 3 * @T00 NCX" > newsyslog.conf +tests_normal_rotate ".xz" + +echo "$LOGFPATH 640 3 * @T00 NCY" > newsyslog.conf +tests_normal_rotate ".zst" + # Normal, archive dir echo "$LOGFPATH 640 3 * @T00 NC" > newsyslog.conf tests_normal_rotate "" "${TMPDIR}/alog/" @@ -508,6 +514,12 @@ tests_normal_rotate ".gz" "${TMPDIR}/alog/" echo "$LOGFPATH 640 3 * @T00 NCJ" > newsyslog.conf tests_normal_rotate ".bz2" "${TMPDIR}/alog/" +echo "$LOGFPATH 640 3 * @T00 NCX" > newsyslog.conf +tests_normal_rotate ".xz" "${TMPDIR}/alog/" + +echo "$LOGFPATH 640 3 * @T00 NCY" > newsyslog.conf +tests_normal_rotate ".zst" "${TMPDIR}/alog/" + # Time based, no archive dir echo "$LOGFPATH 640 3 * @T00 NC" > newsyslog.conf tests_time_rotate @@ -518,6 +530,12 @@ tests_time_rotate "gz" "" echo "$LOGFPATH 640 3 * @T00 NCJ" > newsyslog.conf tests_time_rotate "bz2" "" +echo "$LOGFPATH 640 3 * @T00 NCX" > newsyslog.conf +tests_time_rotate "xz" "" + +echo "$LOGFPATH 640 3 * @T00 NCY" > newsyslog.conf +tests_time_rotate "zst" "" + # Time based, archive dir echo "$LOGFPATH 640 3 * @T00 NC" > newsyslog.conf tests_time_rotate "" "${TMPDIR}/alog/" @@ -527,6 +545,12 @@ tests_time_rotate "gz" "${TMPDIR}/alog/" echo "$LOGFPATH 640 3 * @T00 NCJ" > newsyslog.conf tests_time_rotate "bz2" "${TMPDIR}/alog/" + +echo "$LOGFPATH 640 3 * @T00 NCX" > newsyslog.conf +tests_time_rotate "xz" "${TMPDIR}/alog/" + +echo "$LOGFPATH 640 3 * @T00 NCY" > newsyslog.conf +tests_time_rotate "zst" "${TMPDIR}/alog/" # RFC-5424; Normal, no archive dir echo "$LOGFPATH5424 640 3 * @T00 NCT" > newsyslog.conf From owner-svn-src-all@freebsd.org Mon Jul 30 15:46:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDA651059FC0; Mon, 30 Jul 2018 15:46:48 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B6A4773AA; Mon, 30 Jul 2018 15:46:48 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5C35C26567; Mon, 30 Jul 2018 15:46:48 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UFkmZA020499; Mon, 30 Jul 2018 15:46:48 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UFkf5V020458; Mon, 30 Jul 2018 15:46:41 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201807301546.w6UFkf5V020458@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Mon, 30 Jul 2018 15:46:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336914 - in head: lib/libnetbsd/sys sbin/fsck_ffs share/man/man3 sys/compat/linux sys/compat/linuxkpi/common/include/linux sys/dev/acpica sys/dev/drm2/i915 sys/dev/efidev sys/dev/isp s... X-SVN-Group: head X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in head: lib/libnetbsd/sys sbin/fsck_ffs share/man/man3 sys/compat/linux sys/compat/linuxkpi/common/include/linux sys/dev/acpica sys/dev/drm2/i915 sys/dev/efidev sys/dev/isp sys/dev/joy sys/dev/xen/ti... X-SVN-Commit-Revision: 336914 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:46:49 -0000 Author: asomers Date: Mon Jul 30 15:46:40 2018 New Revision: 336914 URL: https://svnweb.freebsd.org/changeset/base/336914 Log: Make timespecadd(3) and friends public The timespecadd(3) family of macros were imported from NetBSD back in r35029. However, they were initially guarded by #ifdef _KERNEL. In the meantime, we have grown at least 28 syscalls that use timespecs in some way, leading many programs both inside and outside of the base system to redefine those macros. It's better just to make the definitions public. Our kernel currently defines two-argument versions of timespecadd and timespecsub. NetBSD, OpenBSD, and FreeDesktop.org's libbsd, however, define three-argument versions. Solaris also defines a three-argument version, but only in its kernel. This revision changes our definition to match the common three-argument version. Bump _FreeBSD_version due to the breaking KPI change. Discussed with: cem, jilles, ian, bde Differential Revision: https://reviews.freebsd.org/D14725 Deleted: head/lib/libnetbsd/sys/time.h Modified: head/sbin/fsck_ffs/fsutil.c head/share/man/man3/Makefile head/share/man/man3/timeradd.3 head/sys/compat/linux/linux_event.c head/sys/compat/linux/linux_futex.c head/sys/compat/linux/linux_misc.c head/sys/compat/linux/linux_socket.c head/sys/compat/linuxkpi/common/include/linux/time.h head/sys/dev/acpica/acpi_cmbat.c head/sys/dev/acpica/acpi_smbat.c head/sys/dev/acpica/acpi_thermal.c head/sys/dev/drm2/i915/i915_gem.c head/sys/dev/drm2/i915/intel_pm.c head/sys/dev/efidev/efirtc.c head/sys/dev/isp/isp_freebsd.c head/sys/dev/joy/joy.c head/sys/dev/xen/timer/timer.c head/sys/kern/kern_sig.c head/sys/kern/kern_tc.c head/sys/kern/kern_time.c head/sys/kern/kern_umtx.c head/sys/kern/subr_rtc.c head/sys/kern/uipc_mqueue.c head/sys/kern/uipc_sem.c head/sys/mips/ingenic/jz4780_smb.c head/sys/netinet/ip_input.c head/sys/netinet6/ip6_input.c head/sys/netsmb/smb_iod.c head/sys/netsmb/smb_trantcp.c head/sys/opencrypto/crypto.c head/sys/sys/param.h head/sys/sys/time.h head/sys/ufs/ffs/ffs_snapshot.c head/sys/x86/iommu/intel_dmar.h head/tools/regression/posixsem/posixsem.c head/tools/regression/sockets/udp_pingpong/udp_pingpong.c head/tools/regression/sockets/unix_cmsg/uc_check_time.c head/tools/tools/netrate/juggle/juggle.c head/tools/tools/netrate/tcpp/tcpp_client.c head/tools/tools/syscall_timing/syscall_timing.c head/usr.bin/truss/setup.c head/usr.bin/truss/syscalls.c head/usr.bin/truss/truss.h head/usr.sbin/camdd/camdd.c Modified: head/sbin/fsck_ffs/fsutil.c ============================================================================== --- head/sbin/fsck_ffs/fsutil.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sbin/fsck_ffs/fsutil.c Mon Jul 30 15:46:40 2018 (r336914) @@ -297,28 +297,6 @@ foundit: return (bp); } -/* - * Timespec operations (from ). - */ -#define timespecsub(vvp, uvp) \ - do { \ - (vvp)->tv_sec -= (uvp)->tv_sec; \ - (vvp)->tv_nsec -= (uvp)->tv_nsec; \ - if ((vvp)->tv_nsec < 0) { \ - (vvp)->tv_sec--; \ - (vvp)->tv_nsec += 1000000000; \ - } \ - } while (0) -#define timespecadd(vvp, uvp) \ - do { \ - (vvp)->tv_sec += (uvp)->tv_sec; \ - (vvp)->tv_nsec += (uvp)->tv_nsec; \ - if ((vvp)->tv_nsec >= 1000000000) { \ - (vvp)->tv_sec++; \ - (vvp)->tv_nsec -= 1000000000; \ - } \ - } while (0) - void getblk(struct bufarea *bp, ufs2_daddr_t blk, long size) { @@ -337,8 +315,9 @@ getblk(struct bufarea *bp, ufs2_daddr_t blk, long size bp->b_errs = blread(fsreadfd, bp->b_un.b_buf, dblk, size); if (debug) { clock_gettime(CLOCK_REALTIME_PRECISE, &finish); - timespecsub(&finish, &start); - timespecadd(&readtime[bp->b_type], &finish); + timespecsub(&finish, &start, &finish); + timespecadd(&readtime[bp->b_type], &finish, + &readtime[bp->b_type]); } bp->b_bno = dblk; bp->b_size = size; @@ -509,7 +488,7 @@ IOstats(char *what) totaldiskreads += diskreads; diskreads = 0; for (i = 0; i < BT_NUMBUFTYPES; i++) { - timespecadd(&totalreadtime[i], &readtime[i]); + timespecadd(&totalreadtime[i], &readtime[i], &totalreadtime[i]); totalreadcnt[i] += readcnt[i]; readtime[i].tv_sec = readtime[i].tv_nsec = 0; readcnt[i] = 0; @@ -529,7 +508,7 @@ finalIOstats(void) diskreads = totaldiskreads; startpass = startprog; for (i = 0; i < BT_NUMBUFTYPES; i++) { - timespecadd(&totalreadtime[i], &readtime[i]); + timespecadd(&totalreadtime[i], &readtime[i], &totalreadtime[i]); totalreadcnt[i] += readcnt[i]; readtime[i] = totalreadtime[i]; readcnt[i] = totalreadcnt[i]; @@ -543,7 +522,7 @@ static void printIOstats(void) int i; clock_gettime(CLOCK_REALTIME_PRECISE, &finishpass); - timespecsub(&finishpass, &startpass); + timespecsub(&finishpass, &startpass, &finishpass); printf("Running time: %jd.%03ld sec\n", (intmax_t)finishpass.tv_sec, finishpass.tv_nsec / 1000000); printf("buffer reads by type:\n"); Modified: head/share/man/man3/Makefile ============================================================================== --- head/share/man/man3/Makefile Mon Jul 30 15:46:24 2018 (r336913) +++ head/share/man/man3/Makefile Mon Jul 30 15:46:40 2018 (r336914) @@ -164,7 +164,12 @@ MLINKS+= stdarg.3 va_arg.3 \ MLINKS+= timeradd.3 timerclear.3 \ timeradd.3 timercmp.3 \ timeradd.3 timerisset.3 \ - timeradd.3 timersub.3 + timeradd.3 timersub.3 \ + timeradd.3 timespecadd.3 \ + timeradd.3 timespecsub.3 \ + timeradd.3 timespecclear.3 \ + timeradd.3 timespecisset.3 \ + timeradd.3 timespeccmp.3 MLINKS+= tree.3 RB_EMPTY.3 \ tree.3 RB_ENTRY.3 \ tree.3 RB_FIND.3 \ Modified: head/share/man/man3/timeradd.3 ============================================================================== --- head/share/man/man3/timeradd.3 Mon Jul 30 15:46:24 2018 (r336913) +++ head/share/man/man3/timeradd.3 Mon Jul 30 15:46:40 2018 (r336914) @@ -27,7 +27,7 @@ .\" .\" $FreeBSD$ .\" -.Dd August 11, 1999 +.Dd July 30, 2018 .Dt TIMERADD 3 .Os .Sh NAME @@ -35,8 +35,13 @@ .Nm timersub , .Nm timerclear , .Nm timerisset , -.Nm timercmp -.Nd operations on timevals +.Nm timercmp , +.Nm timespecadd , +.Nm timespecsub , +.Nm timespecclear , +.Nm timespecisset , +.Nm timespeccmp +.Nd operations on timevals and timespecs .Sh SYNOPSIS .In sys/time.h .Ft void @@ -49,15 +54,31 @@ .Fn timerisset "struct timeval *tvp" .Ft int .Fn timercmp "struct timeval *a" "struct timeval *b" CMP +.Ft void +.Fn timespecadd "struct timespec *a" "struct timespec *b" "struct timespec *res" +.Ft void +.Fn timespecsub "struct timespec *a" "struct timespec *b" "struct timespec *res" +.Ft void +.Fn timespecclear "struct timespec *ts" +.Ft int +.Fn timespecisset "struct timespec *ts" +.Ft int +.Fn timespeccmp "struct timespec *a" "struct timespec *b" CMP .Sh DESCRIPTION These macros are provided for manipulating .Fa timeval +and +.Fa timespec structures for use with the +.Xr clock_gettime 2 , +.Xr clock_settime 2 , .Xr gettimeofday 2 and .Xr settimeofday 2 calls. -The structure is defined in +The +.Fa timeval +structure is defined in .In sys/time.h as: .Bd -literal @@ -66,50 +87,67 @@ struct timeval { long tv_usec; /* and microseconds */ }; .Ed +And the +.Fa timespec +structure is defined in +.In time.h +as: +.Bd -literal +struct timespec { + time_t tv_nsec; /* seconds */ + long tv_nsec; /* and nanoseconds */ +}; +.Ed .Pp .Fn timeradd -adds the time information stored in +and +.Fn timespecadd +add the time information stored in .Fa a to .Fa b -and stores the resulting -.Vt timeval -in +and store the result in .Fa res . The results are simplified such that the value of .Fa res->tv_usec -is always less than 1,000,000 (1 second). +or +.Fa res->tv_nsec +is always less than 1 second. .Pp .Fn timersub -subtracts the time information stored in +and +.Fn timespecsub +subtract the time information stored in .Fa b from .Fa a -and stores the resulting -.Vt timeval +and store the result in .Fa res . .Pp .Fn timerclear -initializes -.Fa tvp -to midnight (0 hour) January 1st, 1970 (the Epoch). +and +.Fn timespecclear +initialize their argument to midnight (0 hour) January 1st, 1970 (the Epoch). .Pp .Fn timerisset -returns true if -.Fa tvp -is set to any time value other than the Epoch. +and +.Fn timespecisset +return true if their argument is set to any time value other than the Epoch. .Pp .Fn timercmp -compares +and +.Fn timespeccmp +compare .Fa a to .Fa b using the comparison operator given in .Fa CMP , -and returns the result of that comparison. +and return the result of that comparison. .Sh SEE ALSO -.Xr gettimeofday 2 +.Xr gettimeofday 2 , +.Xr clock_gettime 2 .Sh HISTORY The .Fn timeradd @@ -117,3 +155,11 @@ family of macros were imported from .Nx 1.1 , and appeared in .Fx 2.2.6 . +The +.Fn timespecadd +family of macros were imported from +.Nx 1.3 +into +.Fx 3.0 , +though they were not exposed to userland until +.Fx 12.0 . Modified: head/sys/compat/linux/linux_event.c ============================================================================== --- head/sys/compat/linux/linux_event.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/compat/linux/linux_event.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1183,7 +1183,7 @@ linux_timerfd_curval(struct timerfd *tfd, struct itime linux_timerfd_clocktime(tfd, &cts); *ots = tfd->tfd_time; if (ots->it_value.tv_sec != 0 || ots->it_value.tv_nsec != 0) { - timespecsub(&ots->it_value, &cts); + timespecsub(&ots->it_value, &cts, &ots->it_value); if (ots->it_value.tv_sec < 0 || (ots->it_value.tv_sec == 0 && ots->it_value.tv_nsec == 0)) { @@ -1265,9 +1265,10 @@ linux_timerfd_settime(struct thread *td, struct linux_ linux_timerfd_clocktime(tfd, &cts); ts = nts.it_value; if ((args->flags & LINUX_TFD_TIMER_ABSTIME) == 0) { - timespecadd(&tfd->tfd_time.it_value, &cts); + timespecadd(&tfd->tfd_time.it_value, &cts, + &tfd->tfd_time.it_value); } else { - timespecsub(&ts, &cts); + timespecsub(&ts, &cts, &ts); } TIMESPEC_TO_TIMEVAL(&tv, &ts); callout_reset(&tfd->tfd_callout, tvtohz(&tv), @@ -1303,13 +1304,13 @@ linux_timerfd_expire(void *arg) if (timespeccmp(&cts, &tfd->tfd_time.it_value, >=)) { if (timespecisset(&tfd->tfd_time.it_interval)) timespecadd(&tfd->tfd_time.it_value, - &tfd->tfd_time.it_interval); + &tfd->tfd_time.it_interval, + &tfd->tfd_time.it_value); else /* single shot timer */ timespecclear(&tfd->tfd_time.it_value); if (timespecisset(&tfd->tfd_time.it_value)) { - ts = tfd->tfd_time.it_value; - timespecsub(&ts, &cts); + timespecsub(&tfd->tfd_time.it_value, &cts, &ts); TIMESPEC_TO_TIMEVAL(&tv, &ts); callout_reset(&tfd->tfd_callout, tvtohz(&tv), linux_timerfd_expire, tfd); @@ -1319,8 +1320,7 @@ linux_timerfd_expire(void *arg) selwakeup(&tfd->tfd_sel); wakeup(&tfd->tfd_count); } else if (timespecisset(&tfd->tfd_time.it_value)) { - ts = tfd->tfd_time.it_value; - timespecsub(&ts, &cts); + timespecsub(&tfd->tfd_time.it_value, &cts, &ts); TIMESPEC_TO_TIMEVAL(&tv, &ts); callout_reset(&tfd->tfd_callout, tvtohz(&tv), linux_timerfd_expire, tfd); Modified: head/sys/compat/linux/linux_futex.c ============================================================================== --- head/sys/compat/linux/linux_futex.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/compat/linux/linux_futex.c Mon Jul 30 15:46:40 2018 (r336914) @@ -290,10 +290,10 @@ futex_copyin_timeout(int op, struct l_timespec *luts, return (error); if (clockrt) { nanotime(&kts); - timespecsub(ts, &kts); + timespecsub(ts, &kts, ts); } else if (op == LINUX_FUTEX_WAIT_BITSET) { nanouptime(&kts); - timespecsub(ts, &kts); + timespecsub(ts, &kts, ts); } return (error); } Modified: head/sys/compat/linux/linux_misc.c ============================================================================== --- head/sys/compat/linux/linux_misc.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/compat/linux/linux_misc.c Mon Jul 30 15:46:40 2018 (r336914) @@ -2382,8 +2382,8 @@ linux_ppoll(struct thread *td, struct linux_ppoll_args if (error == 0 && args->tsp != NULL) { if (td->td_retval[0]) { nanotime(&ts1); - timespecsub(&ts1, &ts0); - timespecsub(&uts, &ts1); + timespecsub(&ts1, &ts0, &ts1); + timespecsub(&uts, &ts1, &uts); if (uts.tv_sec < 0) timespecclear(&uts); } else Modified: head/sys/compat/linux/linux_socket.c ============================================================================== --- head/sys/compat/linux/linux_socket.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/compat/linux/linux_socket.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1463,7 +1463,7 @@ linux_recvmmsg(struct thread *td, struct linux_recvmms if (error != 0) return (error); getnanotime(&tts); - timespecadd(&tts, &ts); + timespecadd(&tts, &ts, &tts); } msg = PTRIN(args->msg); @@ -1492,7 +1492,7 @@ linux_recvmmsg(struct thread *td, struct linux_recvmms */ if (args->timeout) { getnanotime(&ts); - timespecsub(&ts, &tts); + timespecsub(&ts, &tts, &ts); if (!timespecisset(&ts) || ts.tv_sec > 0) break; } Modified: head/sys/compat/linuxkpi/common/include/linux/time.h ============================================================================== --- head/sys/compat/linuxkpi/common/include/linux/time.h Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/compat/linuxkpi/common/include/linux/time.h Mon Jul 30 15:46:40 2018 (r336914) @@ -76,9 +76,7 @@ timespec_sub(struct timespec lhs, struct timespec rhs) { struct timespec ts; - ts.tv_sec = lhs.tv_sec; - ts.tv_nsec = lhs.tv_nsec; - timespecsub(&ts, &rhs); + timespecsub(&lhs, &rhs, &ts); return ts; } Modified: head/sys/dev/acpica/acpi_cmbat.c ============================================================================== --- head/sys/dev/acpica/acpi_cmbat.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/acpica/acpi_cmbat.c Mon Jul 30 15:46:40 2018 (r336914) @@ -229,7 +229,7 @@ acpi_cmbat_info_expired(struct timespec *lastupdated) return (TRUE); getnanotime(&curtime); - timespecsub(&curtime, lastupdated); + timespecsub(&curtime, lastupdated, &curtime); return (curtime.tv_sec < 0 || curtime.tv_sec > acpi_battery_get_info_expire()); } Modified: head/sys/dev/acpica/acpi_smbat.c ============================================================================== --- head/sys/dev/acpica/acpi_smbat.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/acpica/acpi_smbat.c Mon Jul 30 15:46:40 2018 (r336914) @@ -170,7 +170,7 @@ acpi_smbat_info_expired(struct timespec *lastupdated) return (TRUE); getnanotime(&curtime); - timespecsub(&curtime, lastupdated); + timespecsub(&curtime, lastupdated, &curtime); return (curtime.tv_sec < 0 || curtime.tv_sec > acpi_battery_get_info_expire()); } Modified: head/sys/dev/acpica/acpi_thermal.c ============================================================================== --- head/sys/dev/acpica/acpi_thermal.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/acpica/acpi_thermal.c Mon Jul 30 15:46:40 2018 (r336914) @@ -535,7 +535,7 @@ acpi_tz_monitor(void *Context) (newactive == TZ_ACTIVE_NONE || newactive > sc->tz_active)) { getnanotime(&curtime); - timespecsub(&curtime, &sc->tz_cooling_started); + timespecsub(&curtime, &sc->tz_cooling_started, &curtime); if (curtime.tv_sec < acpi_tz_min_runtime) newactive = sc->tz_active; } Modified: head/sys/dev/drm2/i915/i915_gem.c ============================================================================== --- head/sys/dev/drm2/i915/i915_gem.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/drm2/i915/i915_gem.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1135,8 +1135,8 @@ static int __wait_seqno(struct intel_ring_buffer *ring #undef EXIT_COND if (timeout) { - timespecsub(&now, &before); - timespecsub(timeout, &now); + timespecsub(&now, &before, &now); + timespecsub(timeout, &now, timeout); } switch (end) { Modified: head/sys/dev/drm2/i915/intel_pm.c ============================================================================== --- head/sys/dev/drm2/i915/intel_pm.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/drm2/i915/intel_pm.c Mon Jul 30 15:46:40 2018 (r336914) @@ -3102,8 +3102,7 @@ static void __i915_update_gfx_val(struct drm_i915_priv mtx_assert(&mchdev_lock, MA_OWNED); nanotime(&now); - diff1 = now; - timespecsub(&diff1, &dev_priv->ips.last_time2); + timespecsub(&now, &dev_priv->ips.last_time2, &diff1); /* Don't divide by 0 */ diffms = diff1.tv_sec * 1000 + diff1.tv_nsec / 1000000; Modified: head/sys/dev/efidev/efirtc.c ============================================================================== --- head/sys/dev/efidev/efirtc.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/efidev/efirtc.c Mon Jul 30 15:46:40 2018 (r336914) @@ -163,7 +163,7 @@ efirtc_settime(device_t dev, struct timespec *ts) */ ts->tv_sec -= utc_offset(); if (!efirtc_zeroes_subseconds) - timespecadd(ts, &efirtc_resadj); + timespecadd(ts, &efirtc_resadj, ts); clock_ts_to_ct(ts, &ct); clock_dbgprint_ct(dev, CLOCK_DBG_WRITE, &ct); Modified: head/sys/dev/isp/isp_freebsd.c ============================================================================== --- head/sys/dev/isp/isp_freebsd.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/isp/isp_freebsd.c Mon Jul 30 15:46:40 2018 (r336914) @@ -4083,8 +4083,9 @@ uint64_t isp_nanotime_sub(struct timespec *b, struct timespec *a) { uint64_t elapsed; - struct timespec x = *b; - timespecsub(&x, a); + struct timespec x; + + timespecsub(b, a, &x); elapsed = GET_NANOSEC(&x); if (elapsed == 0) elapsed++; Modified: head/sys/dev/joy/joy.c ============================================================================== --- head/sys/dev/joy/joy.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/joy/joy.c Mon Jul 30 15:46:40 2018 (r336914) @@ -171,14 +171,14 @@ joyread(struct cdev *dev, struct uio *uio, int flag) nanotime(&t); end.tv_sec = 0; end.tv_nsec = joy->timeout[joypart(dev)] * 1000; - timespecadd(&end, &t); + timespecadd(&end, &t, &end); for (; timespeccmp(&t, &end, <) && (bus_space_read_1(bt, port, 0) & 0x0f); nanotime(&t)) ; /* nothing */ bus_space_write_1 (bt, port, 0, 0xff); nanotime(&start); end.tv_sec = 0; end.tv_nsec = joy->timeout[joypart(dev)] * 1000; - timespecadd(&end, &start); + timespecadd(&end, &start, &end); t = start; timespecclear(&x); timespecclear(&y); @@ -200,12 +200,12 @@ joyread(struct cdev *dev, struct uio *uio, int flag) enable_intr (); #endif if (timespecisset(&x)) { - timespecsub(&x, &start); + timespecsub(&x, &start, &x); c.x = joy->x_off[joypart(dev)] + x.tv_nsec / 1000; } else c.x = 0x80000000; if (timespecisset(&y)) { - timespecsub(&y, &start); + timespecsub(&y, &start, &y); c.y = joy->y_off[joypart(dev)] + y.tv_nsec / 1000; } else c.y = 0x80000000; Modified: head/sys/dev/xen/timer/timer.c ============================================================================== --- head/sys/dev/xen/timer/timer.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/dev/xen/timer/timer.c Mon Jul 30 15:46:40 2018 (r336914) @@ -262,7 +262,7 @@ xentimer_gettime(device_t dev, struct timespec *ts) timespecclear(ts); xen_fetch_wallclock(ts); xen_fetch_uptime(&u_ts); - timespecadd(ts, &u_ts); + timespecadd(ts, &u_ts, ts); return (0); } Modified: head/sys/kern/kern_sig.c ============================================================================== --- head/sys/kern/kern_sig.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/kern/kern_sig.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1261,8 +1261,7 @@ kern_sigtimedwait(struct thread *td, sigset_t waitset, if (timeout->tv_nsec >= 0 && timeout->tv_nsec < 1000000000) { timevalid = 1; getnanouptime(&rts); - ets = rts; - timespecadd(&ets, timeout); + timespecadd(&rts, timeout, &ets); } } ksiginfo_init(ksi); @@ -1302,8 +1301,7 @@ kern_sigtimedwait(struct thread *td, sigset_t waitset, error = EAGAIN; break; } - ts = ets; - timespecsub(&ts, &rts); + timespecsub(&ets, &rts, &ts); TIMESPEC_TO_TIMEVAL(&tv, &ts); timo = tvtohz(&tv); } else { Modified: head/sys/kern/kern_tc.c ============================================================================== --- head/sys/kern/kern_tc.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/kern/kern_tc.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1847,7 +1847,7 @@ pps_event(struct pps_state *pps, int event) *tsp = ts; if (foff) { - timespecadd(tsp, osp); + timespecadd(tsp, osp, tsp); if (tsp->tv_nsec < 0) { tsp->tv_nsec += 1000000000; tsp->tv_sec -= 1; Modified: head/sys/kern/kern_time.c ============================================================================== --- head/sys/kern/kern_time.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/kern/kern_time.c Mon Jul 30 15:46:40 2018 (r336914) @@ -543,7 +543,7 @@ kern_clock_nanosleep(struct thread *td, clockid_t cloc atomic_load_acq_int(&rtc_generation); error = kern_clock_gettime(td, clock_id, &now); KASSERT(error == 0, ("kern_clock_gettime: %d", error)); - timespecsub(&ts, &now); + timespecsub(&ts, &now, &ts); } if (ts.tv_sec < 0 || (ts.tv_sec == 0 && ts.tv_nsec == 0)) { error = EWOULDBLOCK; @@ -1520,7 +1520,7 @@ realtimer_gettime(struct itimer *it, struct itimerspec realtimer_clocktime(it->it_clockid, &cts); *ovalue = it->it_time; if (ovalue->it_value.tv_sec != 0 || ovalue->it_value.tv_nsec != 0) { - timespecsub(&ovalue->it_value, &cts); + timespecsub(&ovalue->it_value, &cts, &ovalue->it_value); if (ovalue->it_value.tv_sec < 0 || (ovalue->it_value.tv_sec == 0 && ovalue->it_value.tv_nsec == 0)) { @@ -1561,9 +1561,10 @@ realtimer_settime(struct itimer *it, int flags, ts = val.it_value; if ((flags & TIMER_ABSTIME) == 0) { /* Convert to absolute time. */ - timespecadd(&it->it_time.it_value, &cts); + timespecadd(&it->it_time.it_value, &cts, + &it->it_time.it_value); } else { - timespecsub(&ts, &cts); + timespecsub(&ts, &cts, &ts); /* * We don't care if ts is negative, tztohz will * fix it. @@ -1631,22 +1632,23 @@ realtimer_expire(void *arg) if (timespeccmp(&cts, &it->it_time.it_value, >=)) { if (timespecisset(&it->it_time.it_interval)) { timespecadd(&it->it_time.it_value, - &it->it_time.it_interval); + &it->it_time.it_interval, + &it->it_time.it_value); while (timespeccmp(&cts, &it->it_time.it_value, >=)) { if (it->it_overrun < INT_MAX) it->it_overrun++; else it->it_ksi.ksi_errno = ERANGE; timespecadd(&it->it_time.it_value, - &it->it_time.it_interval); + &it->it_time.it_interval, + &it->it_time.it_value); } } else { /* single shot timer ? */ timespecclear(&it->it_time.it_value); } if (timespecisset(&it->it_time.it_value)) { - ts = it->it_time.it_value; - timespecsub(&ts, &cts); + timespecsub(&it->it_time.it_value, &cts, &ts); TIMESPEC_TO_TIMEVAL(&tv, &ts); callout_reset(&it->it_callout, tvtohz(&tv), realtimer_expire, it); @@ -1658,7 +1660,7 @@ realtimer_expire(void *arg) itimer_leave(it); } else if (timespecisset(&it->it_time.it_value)) { ts = it->it_time.it_value; - timespecsub(&ts, &cts); + timespecsub(&ts, &cts, &ts); TIMESPEC_TO_TIMEVAL(&tv, &ts); callout_reset(&it->it_callout, tvtohz(&tv), realtimer_expire, it); Modified: head/sys/kern/kern_umtx.c ============================================================================== --- head/sys/kern/kern_umtx.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/kern/kern_umtx.c Mon Jul 30 15:46:40 2018 (r336914) @@ -772,8 +772,7 @@ abs_timeout_init(struct abs_timeout *timo, int clockid if (!absolute) { timo->is_abs_real = false; abs_timeout_update(timo); - timo->end = timo->cur; - timespecadd(&timo->end, timeout); + timespecadd(&timo->cur, timeout, &timo->end); } else { timo->end = *timeout; timo->is_abs_real = clockid == CLOCK_REALTIME || @@ -811,8 +810,7 @@ abs_timeout_gethz(struct abs_timeout *timo) if (timespeccmp(&timo->end, &timo->cur, <=)) return (-1); - tts = timo->end; - timespecsub(&tts, &timo->cur); + timespecsub(&timo->end, &timo->cur, &tts); return (tstohz(&tts)); } @@ -3247,8 +3245,8 @@ do_sem2_wait(struct thread *td, struct _usem2 *sem, st error = EINTR; if (error == EINTR) { abs_timeout_update(&timo); - timeout->_timeout = timo.end; - timespecsub(&timeout->_timeout, &timo.cur); + timespecsub(&timo.end, &timo.cur, + &timeout->_timeout); } } } Modified: head/sys/kern/subr_rtc.c ============================================================================== --- head/sys/kern/subr_rtc.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/kern/subr_rtc.c Mon Jul 30 15:46:40 2018 (r336914) @@ -144,7 +144,7 @@ settime_task_func(void *arg, int pending) getnanotime(&ts); if (!(rtc->flags & CLOCKF_SETTIME_NO_ADJ)) { ts.tv_sec -= utc_offset(); - timespecadd(&ts, &rtc->resadj); + timespecadd(&ts, &rtc->resadj, &ts); } } else { ts.tv_sec = 0; @@ -301,7 +301,7 @@ read_clocks(struct timespec *ts, bool debug_read) continue; } if (!(rtc->flags & CLOCKF_GETTIME_NO_ADJ)) { - timespecadd(ts, &rtc->resadj); + timespecadd(ts, &rtc->resadj, ts); ts->tv_sec += utc_offset(); } if (!debug_read) { Modified: head/sys/kern/uipc_mqueue.c ============================================================================== --- head/sys/kern/uipc_mqueue.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/kern/uipc_mqueue.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1735,9 +1735,8 @@ mqueue_send(struct mqueue *mq, const char *msg_ptr, goto bad; } for (;;) { - ts2 = *abs_timeout; getnanotime(&ts); - timespecsub(&ts2, &ts); + timespecsub(abs_timeout, &ts, &ts2); if (ts2.tv_sec < 0 || (ts2.tv_sec == 0 && ts2.tv_nsec <= 0)) { error = ETIMEDOUT; break; @@ -1887,9 +1886,8 @@ mqueue_receive(struct mqueue *mq, char *msg_ptr, } for (;;) { - ts2 = *abs_timeout; getnanotime(&ts); - timespecsub(&ts2, &ts); + timespecsub(abs_timeout, &ts, &ts2); if (ts2.tv_sec < 0 || (ts2.tv_sec == 0 && ts2.tv_nsec <= 0)) { error = ETIMEDOUT; return (error); Modified: head/sys/kern/uipc_sem.c ============================================================================== --- head/sys/kern/uipc_sem.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/kern/uipc_sem.c Mon Jul 30 15:46:40 2018 (r336914) @@ -843,7 +843,7 @@ kern_sem_wait(struct thread *td, semid_t id, int tryfl for (;;) { ts1 = *abstime; getnanotime(&ts2); - timespecsub(&ts1, &ts2); + timespecsub(&ts1, &ts2, &ts1); TIMESPEC_TO_TIMEVAL(&tv, &ts1); if (tv.tv_sec < 0) { error = ETIMEDOUT; Modified: head/sys/mips/ingenic/jz4780_smb.c ============================================================================== --- head/sys/mips/ingenic/jz4780_smb.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/mips/ingenic/jz4780_smb.c Mon Jul 30 15:46:40 2018 (r336914) @@ -248,7 +248,7 @@ jzsmb_transfer_read(device_t dev, struct iic_msg *msg) SMB_WRITE(sc, SMBDC, SMBDC_CMD); for (;;) { getnanouptime(&diff); - timespecsub(&diff, &start); + timespecsub(&diff, &start, &diff); if ((SMB_READ(sc, SMBST) & SMBST_RFNE) != 0) { msg->buf[msg->len - resid] = SMB_READ(sc, SMBDC) & SMBDC_DAT; @@ -293,7 +293,7 @@ jzsmb_transfer_write(device_t dev, struct iic_msg *msg for (resid = msg->len; resid > 0; resid--) { for (;;) { getnanouptime(&diff); - timespecsub(&diff, &start); + timespecsub(&diff, &start, &diff); if ((SMB_READ(sc, SMBST) & SMBST_TFNF) != 0) { SMB_WRITE(sc, SMBDC, msg->buf[msg->len - resid]); Modified: head/sys/netinet/ip_input.c ============================================================================== --- head/sys/netinet/ip_input.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/netinet/ip_input.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1193,7 +1193,7 @@ ip_savecontrol(struct inpcb *inp, struct mbuf **mp, st mbuf_tstmp2timespec(m, &ts); getboottimebin(&boottimebin); bintime2timespec(&boottimebin, &ts1); - timespecadd(&ts, &ts1); + timespecadd(&ts, &ts1, &ts); } else { nanotime(&ts); } Modified: head/sys/netinet6/ip6_input.c ============================================================================== --- head/sys/netinet6/ip6_input.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/netinet6/ip6_input.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1267,7 +1267,7 @@ ip6_savecontrol_v4(struct inpcb *inp, struct mbuf *m, mbuf_tstmp2timespec(m, &t.ts); getboottimebin(&boottimebin); bintime2timespec(&boottimebin, &ts1); - timespecadd(&t.ts, &ts1); + timespecadd(&t.ts, &ts1, &t.ts); } else { nanotime(&t.ts); } Modified: head/sys/netsmb/smb_iod.c ============================================================================== --- head/sys/netsmb/smb_iod.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/netsmb/smb_iod.c Mon Jul 30 15:46:40 2018 (r336914) @@ -557,9 +557,9 @@ smb_iod_sendall(struct smbiod *iod) break; case SMBRQ_SENT: SMB_TRAN_GETPARAM(vcp, SMBTP_TIMEOUT, &tstimeout); - timespecadd(&tstimeout, &tstimeout); + timespecadd(&tstimeout, &tstimeout, &tstimeout); getnanotime(&ts); - timespecsub(&ts, &tstimeout); + timespecsub(&ts, &tstimeout, &ts); if (timespeccmp(&ts, &rqp->sr_timesent, >)) { smb_iod_rqprocessed(rqp, ETIMEDOUT); } @@ -630,7 +630,7 @@ smb_iod_main(struct smbiod *iod) #if 0 if (iod->iod_state == SMBIOD_ST_VCACTIVE) { getnanotime(&tsnow); - timespecsub(&tsnow, &iod->iod_pingtimo); + timespecsub(&tsnow, &iod->iod_pingtimo, &tsnow); if (timespeccmp(&tsnow, &iod->iod_lastrqsent, >)) { smb_smb_echo(vcp, &iod->iod_scred); } Modified: head/sys/netsmb/smb_trantcp.c ============================================================================== --- head/sys/netsmb/smb_trantcp.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/netsmb/smb_trantcp.c Mon Jul 30 15:46:40 2018 (r336914) @@ -546,15 +546,14 @@ smb_nbst_connect(struct smb_vc *vcp, struct sockaddr * if (error) return error; getnanotime(&ts2); - timespecsub(&ts2, &ts1); + timespecsub(&ts2, &ts1, &ts2); if (ts2.tv_sec == 0) { ts2.tv_sec = 1; ts2.tv_nsec = 0; } - nbp->nbp_timo = ts2; - timespecadd(&nbp->nbp_timo, &ts2); - timespecadd(&nbp->nbp_timo, &ts2); - timespecadd(&nbp->nbp_timo, &ts2); /* * 4 */ + timespecadd(&ts2, &ts2, &nbp->nbp_timo); + timespecadd(&nbp->nbp_timo, &ts2, &nbp->nbp_timo); + timespecadd(&nbp->nbp_timo, &ts2, &nbp->nbp_timo); /* * 4 */ error = nbssn_rq_request(nbp, td); if (error) smb_nbst_disconnect(vcp, td); Modified: head/sys/opencrypto/crypto.c ============================================================================== --- head/sys/opencrypto/crypto.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/opencrypto/crypto.c Mon Jul 30 15:46:40 2018 (r336914) @@ -1188,7 +1188,7 @@ crypto_tstat(struct cryptotstat *ts, struct bintime *b if (u < delta.frac) delta.sec--; bintime2timespec(&delta, &t); - timespecadd(&ts->acc, &t); + timespecadd(&ts->acc, &t, &ts->acc); if (timespeccmp(&t, &ts->min, <)) ts->min = t; if (timespeccmp(&t, &ts->max, >)) Modified: head/sys/sys/param.h ============================================================================== --- head/sys/sys/param.h Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/sys/param.h Mon Jul 30 15:46:40 2018 (r336914) @@ -60,7 +60,7 @@ * in the range 5 to 9. */ #undef __FreeBSD_version -#define __FreeBSD_version 1200075 /* Master, propagated to newvers */ +#define __FreeBSD_version 1200076 /* Master, propagated to newvers */ /* * __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD, Modified: head/sys/sys/time.h ============================================================================== --- head/sys/sys/time.h Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/sys/time.h Mon Jul 30 15:46:40 2018 (r336914) @@ -306,6 +306,7 @@ tvtosbt(struct timeval _tv) #define USEC_2_TICKS(u) max(1, (uint32_t)((hz == 1000) ? \ ((u) / 1000) : ((uint64_t)(u) * (uint64_t)hz)/(uint64_t)1000000)) +#endif /* Operations on timespecs */ #define timespecclear(tvp) ((tvp)->tv_sec = (tvp)->tv_nsec = 0) #define timespecisset(tvp) ((tvp)->tv_sec || (tvp)->tv_nsec) @@ -313,24 +314,27 @@ tvtosbt(struct timeval _tv) (((tvp)->tv_sec == (uvp)->tv_sec) ? \ ((tvp)->tv_nsec cmp (uvp)->tv_nsec) : \ ((tvp)->tv_sec cmp (uvp)->tv_sec)) -#define timespecadd(vvp, uvp) \ + +#define timespecadd(tsp, usp, vsp) \ do { \ - (vvp)->tv_sec += (uvp)->tv_sec; \ - (vvp)->tv_nsec += (uvp)->tv_nsec; \ - if ((vvp)->tv_nsec >= 1000000000) { \ - (vvp)->tv_sec++; \ - (vvp)->tv_nsec -= 1000000000; \ + (vsp)->tv_sec = (tsp)->tv_sec + (usp)->tv_sec; \ + (vsp)->tv_nsec = (tsp)->tv_nsec + (usp)->tv_nsec; \ + if ((vsp)->tv_nsec >= 1000000000L) { \ + (vsp)->tv_sec++; \ + (vsp)->tv_nsec -= 1000000000L; \ } \ } while (0) -#define timespecsub(vvp, uvp) \ +#define timespecsub(tsp, usp, vsp) \ do { \ - (vvp)->tv_sec -= (uvp)->tv_sec; \ - (vvp)->tv_nsec -= (uvp)->tv_nsec; \ - if ((vvp)->tv_nsec < 0) { \ - (vvp)->tv_sec--; \ - (vvp)->tv_nsec += 1000000000; \ + (vsp)->tv_sec = (tsp)->tv_sec - (usp)->tv_sec; \ + (vsp)->tv_nsec = (tsp)->tv_nsec - (usp)->tv_nsec; \ + if ((vsp)->tv_nsec < 0) { \ + (vsp)->tv_sec--; \ + (vsp)->tv_nsec += 1000000000L; \ } \ } while (0) + +#ifdef _KERNEL /* Operations on timevals. */ Modified: head/sys/ufs/ffs/ffs_snapshot.c ============================================================================== --- head/sys/ufs/ffs/ffs_snapshot.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/ufs/ffs/ffs_snapshot.c Mon Jul 30 15:46:40 2018 (r336914) @@ -694,7 +694,7 @@ out1: vfs_write_resume(vp->v_mount, VR_START_WRITE | VR_NO_SUSPCLR); if (collectsnapstats && starttime.tv_sec > 0) { nanotime(&endtime); - timespecsub(&endtime, &starttime); + timespecsub(&endtime, &starttime, &endtime); printf("%s: suspended %ld.%03ld sec, redo %ld of %d\n", vp->v_mount->mnt_stat.f_mntonname, (long)endtime.tv_sec, endtime.tv_nsec / 1000000, redo, fs->fs_ncg); Modified: head/sys/x86/iommu/intel_dmar.h ============================================================================== --- head/sys/x86/iommu/intel_dmar.h Mon Jul 30 15:46:24 2018 (r336913) +++ head/sys/x86/iommu/intel_dmar.h Mon Jul 30 15:46:40 2018 (r336914) @@ -524,8 +524,7 @@ extern struct timespec dmar_hw_timeout; } else { \ forever = false; \ nanouptime(&curr); \ - last = curr; \ - timespecadd(&last, &dmar_hw_timeout); \ + timespecadd(&curr, &dmar_hw_timeout, &last); \ } \ for (;;) { \ if (cond) { \ Modified: head/tools/regression/posixsem/posixsem.c ============================================================================== --- head/tools/regression/posixsem/posixsem.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/tools/regression/posixsem/posixsem.c Mon Jul 30 15:46:40 2018 (r336914) @@ -55,35 +55,6 @@ __FBSDID("$FreeBSD$"); #include "test.h" -/* Cut and pasted from kernel header, bah! */ - -/* Operations on timespecs */ -#define timespecclear(tvp) ((tvp)->tv_sec = (tvp)->tv_nsec = 0) -#define timespecisset(tvp) ((tvp)->tv_sec || (tvp)->tv_nsec) -#define timespeccmp(tvp, uvp, cmp) \ - (((tvp)->tv_sec == (uvp)->tv_sec) ? \ - ((tvp)->tv_nsec cmp (uvp)->tv_nsec) : \ - ((tvp)->tv_sec cmp (uvp)->tv_sec)) -#define timespecadd(vvp, uvp) \ - do { \ - (vvp)->tv_sec += (uvp)->tv_sec; \ - (vvp)->tv_nsec += (uvp)->tv_nsec; \ - if ((vvp)->tv_nsec >= 1000000000) { \ - (vvp)->tv_sec++; \ - (vvp)->tv_nsec -= 1000000000; \ - } \ - } while (0) -#define timespecsub(vvp, uvp) \ - do { \ - (vvp)->tv_sec -= (uvp)->tv_sec; \ - (vvp)->tv_nsec -= (uvp)->tv_nsec; \ - if ((vvp)->tv_nsec < 0) { \ - (vvp)->tv_sec--; \ - (vvp)->tv_nsec += 1000000000; \ - } \ - } while (0) - - #define TEST_PATH "/tmp/posixsem_regression_test" #define ELAPSED(elapsed, limit) (abs((elapsed) - (limit)) < 100) @@ -791,7 +762,7 @@ timedwait(semid_t id, u_int msec, u_int *delta, int er } end.tv_sec = msec / 1000; end.tv_nsec = msec % 1000 * 1000000; - timespecadd(&end, &start); + timespecadd(&end, &start, &end); if (ksem_timedwait(id, &end) < 0) { if (errno != error) { fail_errno("ksem_timedwait"); @@ -805,7 +776,7 @@ timedwait(semid_t id, u_int msec, u_int *delta, int er fail_errno("clock_gettime(CLOCK_REALTIME)"); return (-1); } - timespecsub(&end, &start); + timespecsub(&end, &start, &end); *delta = end.tv_nsec / 1000000; *delta += end.tv_sec * 1000; return (0); @@ -944,7 +915,7 @@ testwait(semid_t id, u_int *delta) fail_errno("clock_gettime(CLOCK_REALTIME)"); return (-1); } - timespecsub(&end, &start); + timespecsub(&end, &start, &end); *delta = end.tv_nsec / 1000000; *delta += end.tv_sec * 1000; return (0); Modified: head/tools/regression/sockets/udp_pingpong/udp_pingpong.c ============================================================================== --- head/tools/regression/sockets/udp_pingpong/udp_pingpong.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/tools/regression/sockets/udp_pingpong/udp_pingpong.c Mon Jul 30 15:46:40 2018 (r336914) @@ -106,31 +106,6 @@ struct rtt { #define NSEC_MAX 1000000000L #define NSEC_IN_USEC 1000L -#define timespecsub2(r, v, u) \ - do { \ - SEC(r) = SEC(v) - SEC(u); \ - NSEC(r) = NSEC(v) - NSEC(u); \ - if (NSEC(r) < 0 && (SEC(r) > 0 || NSEC(r) <= -NSEC_MAX)) { \ - SEC(r)--; \ - NSEC(r) += NSEC_MAX; \ - } \ - } while (0); - -#define timespecadd2(r, v, u) \ - do { \ - SEC(r) = SEC(v) + SEC(u); \ - NSEC(r) = NSEC(v) + NSEC(u); \ - if (NSEC(r) >= NSEC_MAX) { \ - SEC(r)++; \ - NSEC(r) -= NSEC_MAX; \ - } \ - } while (0); - -#define timespeccmp(t, c, u) \ - ((SEC(t) == SEC(u)) ? \ - (NSEC(t) c NSEC(u)) : \ - (SEC(t) c SEC(u))) - #define timeval2timespec(tv, ts) \ do { \ SEC(ts) = (tv)->tv_sec; \ @@ -536,10 +511,10 @@ static void calc_rtt(struct test_pkt *tpp, struct rtt *rttp) { - timespecsub2(&rttp->a2b, &tpp->tss[1].recvd, &tpp->tss[0].sent); - timespecsub2(&rttp->b2a, &tpp->tss[0].recvd, &tpp->tss[1].sent); - timespecadd2(&rttp->a2b_b2a, &rttp->a2b, &rttp->b2a); - timespecsub2(&rttp->e2e, &tpp->tss[0].recvd, &tpp->tss[0].sent); + timespecsub(&tpp->tss[1].recvd, &tpp->tss[0].sent, &rttp->a2b); + timespecsub(&tpp->tss[0].recvd, &tpp->tss[1].sent, &rttp->b2a); + timespecadd(&rttp->a2b, &rttp->b2a, &rttp->a2b_b2a); + timespecsub(&tpp->tss[0].recvd, &tpp->tss[0].sent, &rttp->e2e); } static void @@ -604,13 +579,13 @@ test_run(int ts_type, int use_ipv6, int use_recvmsg, c continue; } calc_rtt(&test_ctx.test_pkts[i], &rtt); - if (!timespeccmp(&rtt.e2e, >, &rtt.a2b_b2a)) + if (!timespeccmp(&rtt.e2e, &rtt.a2b_b2a, >)) errx(1, "end-to-end trip time is too small"); - if (!timespeccmp(&rtt.e2e, <, &max_ts)) + if (!timespeccmp(&rtt.e2e, &max_ts, <)) errx(1, "end-to-end trip time is too large"); - if (!timespeccmp(&rtt.a2b, >, &zero_ts)) + if (!timespeccmp(&rtt.a2b, &zero_ts, >)) errx(1, "A2B trip time is not positive"); - if (!timespeccmp(&rtt.b2a, >, &zero_ts)) + if (!timespeccmp(&rtt.b2a, &zero_ts, >)) errx(1, "B2A trip time is not positive"); } teardown_udp(&test_ctx); Modified: head/tools/regression/sockets/unix_cmsg/uc_check_time.c ============================================================================== --- head/tools/regression/sockets/unix_cmsg/uc_check_time.c Mon Jul 30 15:46:24 2018 (r336913) +++ head/tools/regression/sockets/unix_cmsg/uc_check_time.c Mon Jul 30 15:46:40 2018 (r336914) @@ -35,20 +35,6 @@ __FBSDID("$FreeBSD$"); static const struct timeval max_diff_tv = {.tv_sec = 1, .tv_usec = 0}; static const struct timespec max_diff_ts = {.tv_sec = 1, .tv_nsec = 0}; -#define timespeccmp(tvp, uvp, cmp) \ - (((tvp)->tv_sec == (uvp)->tv_sec) ? \ - ((tvp)->tv_nsec cmp (uvp)->tv_nsec) : \ - ((tvp)->tv_sec cmp (uvp)->tv_sec)) -#define timespecsub(vvp, uvp) \ - do { \ *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Mon Jul 30 15:57:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4BD03105A562; Mon, 30 Jul 2018 15:57:59 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2F3677CA0; Mon, 30 Jul 2018 15:57:58 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D3F2226711; Mon, 30 Jul 2018 15:57:58 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UFvwD2025854; Mon, 30 Jul 2018 15:57:58 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UFvwOx025853; Mon, 30 Jul 2018 15:57:58 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807301557.w6UFvwOx025853@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Mon, 30 Jul 2018 15:57:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336915 - head/sys/arm64/conf X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: head/sys/arm64/conf X-SVN-Commit-Revision: 336915 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 15:57:59 -0000 Author: andrew Date: Mon Jul 30 15:57:58 2018 New Revision: 336915 URL: https://svnweb.freebsd.org/changeset/base/336915 Log: Enable VIMAGE on arm64 again. A workaround for modules with static VNET variables has been committed so these should work now. PR: 223670 Sponsored by: DARPA, AFRL Modified: head/sys/arm64/conf/GENERIC Modified: head/sys/arm64/conf/GENERIC ============================================================================== --- head/sys/arm64/conf/GENERIC Mon Jul 30 15:46:40 2018 (r336914) +++ head/sys/arm64/conf/GENERIC Mon Jul 30 15:57:58 2018 (r336915) @@ -26,7 +26,7 @@ makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption -#options VIMAGE # Subsystem virtualization, e.g. VNET +options VIMAGE # Subsystem virtualization, e.g. VNET options INET # InterNETworking options INET6 # IPv6 communications protocols options IPSEC # IP (v4/v6) security From owner-svn-src-all@freebsd.org Mon Jul 30 16:54:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90FF9105B761; Mon, 30 Jul 2018 16:54:52 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44C5D7A2B9; Mon, 30 Jul 2018 16:54:52 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2254327191; Mon, 30 Jul 2018 16:54:52 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UGsp91056074; Mon, 30 Jul 2018 16:54:51 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UGspk4056073; Mon, 30 Jul 2018 16:54:51 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807301654.w6UGspk4056073@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Mon, 30 Jul 2018 16:54:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336917 - head/sys/amd64/amd64 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/amd64/amd64 X-SVN-Commit-Revision: 336917 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 16:54:52 -0000 Author: kib Date: Mon Jul 30 16:54:51 2018 New Revision: 336917 URL: https://svnweb.freebsd.org/changeset/base/336917 Log: Remove unneeded CLDs instructions in the SMAP-ed version of several functions from support.S. I believe they re-appeared due to me mis-merging my r327820 into the topic branch. Sponsored by: The FreeBSD Foundation Modified: head/sys/amd64/amd64/support.S Modified: head/sys/amd64/amd64/support.S ============================================================================== --- head/sys/amd64/amd64/support.S Mon Jul 30 16:33:32 2018 (r336916) +++ head/sys/amd64/amd64/support.S Mon Jul 30 16:54:51 2018 (r336917) @@ -307,7 +307,6 @@ ENTRY(copyout_smap) movq %rdx,%rcx shrq $3,%rcx - cld stac rep movsq @@ -358,7 +357,6 @@ ENTRY(copyin_nosmap) movq %rdx,%rcx movb %cl,%al shrq $3,%rcx /* copy longword-wise */ - cld rep movsq movb %al,%cl @@ -887,7 +885,6 @@ ENTRY(copyinstr_nosmap) movq %rax,%r8 1: incq %rdx - cld 2: decq %rdx From owner-svn-src-all@freebsd.org Mon Jul 30 17:03:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3292A105BA76; Mon, 30 Jul 2018 17:03:16 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D7F067A87C; Mon, 30 Jul 2018 17:03:15 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B9B352734D; Mon, 30 Jul 2018 17:03:15 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UH3Fva061121; Mon, 30 Jul 2018 17:03:15 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UH3FJf061119; Mon, 30 Jul 2018 17:03:15 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201807301703.w6UH3FJf061119@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Mon, 30 Jul 2018 17:03:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336918 - head/sys/powerpc/powerpc X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/powerpc/powerpc X-SVN-Commit-Revision: 336918 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 17:03:16 -0000 Author: jhibbits Date: Mon Jul 30 17:03:15 2018 New Revision: 336918 URL: https://svnweb.freebsd.org/changeset/base/336918 Log: powerpcspe: Unconditionally save an restore SPEFSCR on task switch The SPEFSCR is not guarded by the SPV bit in MSR, it's just another SPR. Protect processes from other tasks setting the SPEFSCR for their own needs. Modified: head/sys/powerpc/powerpc/genassym.c head/sys/powerpc/powerpc/swtch32.S Modified: head/sys/powerpc/powerpc/genassym.c ============================================================================== --- head/sys/powerpc/powerpc/genassym.c Mon Jul 30 16:54:51 2018 (r336917) +++ head/sys/powerpc/powerpc/genassym.c Mon Jul 30 17:03:15 2018 (r336918) @@ -208,6 +208,8 @@ ASSYM(PCB_CDSCR, PCB_CDSCR); ASSYM(PCB_AIM_USR_VSID, offsetof(struct pcb, pcb_cpu.aim.usr_vsid)); ASSYM(PCB_BOOKE_DBCR0, offsetof(struct pcb, pcb_cpu.booke.dbcr0)); +ASSYM(PCB_VSCR, offsetof(struct pcb, pcb_vec.vscr)); + ASSYM(TD_LOCK, offsetof(struct thread, td_lock)); ASSYM(TD_PROC, offsetof(struct thread, td_proc)); ASSYM(TD_PCB, offsetof(struct thread, td_pcb)); Modified: head/sys/powerpc/powerpc/swtch32.S ============================================================================== --- head/sys/powerpc/powerpc/swtch32.S Mon Jul 30 16:54:51 2018 (r336917) +++ head/sys/powerpc/powerpc/swtch32.S Mon Jul 30 17:03:15 2018 (r336918) @@ -113,6 +113,10 @@ ENTRY(cpu_switch) bl save_vec .L2: +#if defined(__SPE__) + mfspr %r3,SPR_SPEFSCR + stw %r3,PCB_VSCR(%r17) +#endif mr %r3,%r14 /* restore old thread ptr */ bl pmap_deactivate /* Deactivate the current pmap */ @@ -163,6 +167,10 @@ blocked_loop: bl enable_vec .L4: +#if defined(__SPE__) + lwz %r3,PCB_VSCR(%r17) + mtspr SPR_SPEFSCR,%r3 +#endif /* thread to restore is in r3 */ mr %r3,%r17 /* Recover PCB ptr */ lmw %r12,PCB_CONTEXT(%r3) /* Load the non-volatile GP regs */ From owner-svn-src-all@freebsd.org Mon Jul 30 17:40:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 297F0105C5A6; Mon, 30 Jul 2018 17:40:28 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D26317BEEE; Mon, 30 Jul 2018 17:40:27 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B01B52797D; Mon, 30 Jul 2018 17:40:27 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UHeRXv077139; Mon, 30 Jul 2018 17:40:27 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UHeRM0077138; Mon, 30 Jul 2018 17:40:27 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807301740.w6UHeRM0077138@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Mon, 30 Jul 2018 17:40:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336919 - head/sys/dev/efidev X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: head/sys/dev/efidev X-SVN-Commit-Revision: 336919 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 17:40:28 -0000 Author: kevans Date: Mon Jul 30 17:40:27 2018 New Revision: 336919 URL: https://svnweb.freebsd.org/changeset/base/336919 Log: efirt: Add tunable to allow disabling EFI Runtime Services Leading up to enabling EFIRT in GENERIC, allow runtime services to be disabled with a new tunable: efi.rt_disabled. This makes it so that EFIRT can be disabled easily in case we run into some buggy UEFI implementation and fail to boot. Discussed with: imp, kib MFC after: 1 week Modified: head/sys/dev/efidev/efirt.c Modified: head/sys/dev/efidev/efirt.c ============================================================================== --- head/sys/dev/efidev/efirt.c Mon Jul 30 17:03:15 2018 (r336918) +++ head/sys/dev/efidev/efirt.c Mon Jul 30 17:40:27 2018 (r336919) @@ -133,7 +133,12 @@ efi_init(void) struct efi_md *map; caddr_t kmdp; size_t efisz; + int rt_disabled; + rt_disabled = 0; + TUNABLE_INT_FETCH("efi.rt_disabled", &rt_disabled); + if (rt_disabled == 1) + return (0); mtx_init(&efi_lock, "efi", NULL, MTX_DEF); if (efi_systbl_phys == 0) { @@ -222,6 +227,9 @@ static void efi_uninit(void) { + /* Most likely disabled by tunable */ + if (efi_runtime == NULL) + return; efi_destroy_1t1_map(); efi_systbl = NULL; From owner-svn-src-all@freebsd.org Mon Jul 30 17:49:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDE52105C90C; Mon, 30 Jul 2018 17:49:48 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69FA17C58D; Mon, 30 Jul 2018 17:49:48 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3F35E27B18; Mon, 30 Jul 2018 17:49:48 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UHnmiS082320; Mon, 30 Jul 2018 17:49:48 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UHnmqF082319; Mon, 30 Jul 2018 17:49:48 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807301749.w6UHnmqF082319@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 17:49:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336920 - stable/11/usr.sbin/acpi/acpidump X-SVN-Group: stable-11 X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: stable/11/usr.sbin/acpi/acpidump X-SVN-Commit-Revision: 336920 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 17:49:48 -0000 Author: emaste Date: Mon Jul 30 17:49:47 2018 New Revision: 336920 URL: https://svnweb.freebsd.org/changeset/base/336920 Log: MFC r335459: acpidump.8: include NFIT in the man page list of tables Was missed in r321298 [MFC'd in r322461]. Reported by: Ben Widawsky (in review D15931) Sponsored by: The FreeBSD Foundation Modified: stable/11/usr.sbin/acpi/acpidump/acpidump.8 Directory Properties: stable/11/ (props changed) Modified: stable/11/usr.sbin/acpi/acpidump/acpidump.8 ============================================================================== --- stable/11/usr.sbin/acpi/acpidump/acpidump.8 Mon Jul 30 17:40:27 2018 (r336919) +++ stable/11/usr.sbin/acpi/acpidump/acpidump.8 Mon Jul 30 17:49:47 2018 (r336920) @@ -29,7 +29,7 @@ .\" .\" $FreeBSD$ .\" -.Dd February 14, 2005 +.Dd June 20, 2018 .Dt ACPIDUMP 8 .Os .Sh NAME @@ -105,6 +105,7 @@ utility dumps contents of the following tables: .It HPET .It MADT .It MCFG +.It NFIT .It RSD PTR .It RSDT .El From owner-svn-src-all@freebsd.org Mon Jul 30 17:54:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9772105CB16; Mon, 30 Jul 2018 17:54:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E8137C9F8; Mon, 30 Jul 2018 17:54:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4A23727CAA; Mon, 30 Jul 2018 17:54:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UHsJpf087311; Mon, 30 Jul 2018 17:54:19 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UHsIU8087308; Mon, 30 Jul 2018 17:54:18 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807301754.w6UHsIU8087308@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Mon, 30 Jul 2018 17:54:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336921 - in head: . sys/amd64/conf X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: in head: . sys/amd64/conf X-SVN-Commit-Revision: 336921 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 17:54:20 -0000 Author: kevans Date: Mon Jul 30 17:54:18 2018 New Revision: 336921 URL: https://svnweb.freebsd.org/changeset/base/336921 Log: amd64/GENERIC: Enable EFIRT by default As noted in UDPATING, the new loader tunable efi.rt_disabled may be used to disable EFIRT at runtime. It should have no effect if you are not booted via UEFI boot. MFC after: 6 weeks Modified: head/UPDATING head/sys/amd64/conf/GENERIC head/sys/amd64/conf/NOTES Modified: head/UPDATING ============================================================================== --- head/UPDATING Mon Jul 30 17:49:47 2018 (r336920) +++ head/UPDATING Mon Jul 30 17:54:18 2018 (r336921) @@ -31,6 +31,13 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 12.x IS SLOW: disable the most expensive debugging functionality run "ln -s 'abort:false,junk:false' /etc/malloc.conf".) +20180730: + amd64/GENERIC now has EFI runtime services, EFIRT, enabled by default. + This should have no effect if the kernel is booted via UEFI/legacy boot. + EFIRT may be disabled via a loader tunable, efi.rt_disabled, if a system + has a buggy firmware that prevents a successful boot due to use of + runtime services. + 20180727: Atmel AT91RM9200 and AT91SAM9, Cavium CNS 11xx and XScale support has been removed from the tree. These ports were Modified: head/sys/amd64/conf/GENERIC ============================================================================== --- head/sys/amd64/conf/GENERIC Mon Jul 30 17:49:47 2018 (r336920) +++ head/sys/amd64/conf/GENERIC Mon Jul 30 17:54:18 2018 (r336921) @@ -54,6 +54,7 @@ options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_RAID # Soft RAID functionality. options GEOM_LABEL # Provides labelization +options EFIRT # EFI Runtime Services support options COMPAT_FREEBSD32 # Compatible with i386 binaries options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 Modified: head/sys/amd64/conf/NOTES ============================================================================== --- head/sys/amd64/conf/NOTES Mon Jul 30 17:49:47 2018 (r336920) +++ head/sys/amd64/conf/NOTES Mon Jul 30 17:54:18 2018 (r336921) @@ -613,7 +613,7 @@ device amdsmn # options NKPT=31 -# EFI Runtime Services support (not functional yet). +# EFI Runtime Services support options EFIRT From owner-svn-src-all@freebsd.org Mon Jul 30 17:58:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A31A105CC02; Mon, 30 Jul 2018 17:58:42 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 208D67CC30; Mon, 30 Jul 2018 17:58:42 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 01D1327CBB; Mon, 30 Jul 2018 17:58:42 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UHwfIK087556; Mon, 30 Jul 2018 17:58:41 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UHwfmu087555; Mon, 30 Jul 2018 17:58:41 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201807301758.w6UHwfmu087555@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Mon, 30 Jul 2018 17:58:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336922 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 336922 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 17:58:42 -0000 Author: markj Date: Mon Jul 30 17:58:41 2018 New Revision: 336922 URL: https://svnweb.freebsd.org/changeset/base/336922 Log: Remove a redundant check. MFC after: 3 days Sponsored by: The FreeBSD Foundation Modified: head/sys/kern/uipc_sockbuf.c Modified: head/sys/kern/uipc_sockbuf.c ============================================================================== --- head/sys/kern/uipc_sockbuf.c Mon Jul 30 17:54:18 2018 (r336921) +++ head/sys/kern/uipc_sockbuf.c Mon Jul 30 17:58:41 2018 (r336922) @@ -964,8 +964,6 @@ sbappendcontrol_locked(struct sockbuf *sb, struct mbuf SOCKBUF_LOCK_ASSERT(sb); - if (control == NULL) - panic("sbappendcontrol_locked"); space = m_length(control, &n) + m_length(m0, NULL); if (space > sbspace(sb)) From owner-svn-src-all@freebsd.org Mon Jul 30 17:59:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E201105CC4C; Mon, 30 Jul 2018 17:59:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E78357CD75; Mon, 30 Jul 2018 17:59:18 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C899927CBC; Mon, 30 Jul 2018 17:59:18 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UHxInM087631; Mon, 30 Jul 2018 17:59:18 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UHxIPH087630; Mon, 30 Jul 2018 17:59:18 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807301759.w6UHxIPH087630@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Mon, 30 Jul 2018 17:59:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336923 - head X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: head X-SVN-Commit-Revision: 336923 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 17:59:19 -0000 Author: kevans Date: Mon Jul 30 17:59:18 2018 New Revision: 336923 URL: https://svnweb.freebsd.org/changeset/base/336923 Log: Correct typo from r336921; BIOS/legacy boot... Modified: head/UPDATING Modified: head/UPDATING ============================================================================== --- head/UPDATING Mon Jul 30 17:58:41 2018 (r336922) +++ head/UPDATING Mon Jul 30 17:59:18 2018 (r336923) @@ -33,7 +33,7 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 12.x IS SLOW: 20180730: amd64/GENERIC now has EFI runtime services, EFIRT, enabled by default. - This should have no effect if the kernel is booted via UEFI/legacy boot. + This should have no effect if the kernel is booted via BIOS/legacy boot. EFIRT may be disabled via a loader tunable, efi.rt_disabled, if a system has a buggy firmware that prevents a successful boot due to use of runtime services. From owner-svn-src-all@freebsd.org Mon Jul 30 18:00:13 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E547C105CD0C for ; Mon, 30 Jul 2018 18:00:12 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B3297CF05 for ; Mon, 30 Jul 2018 18:00:12 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-ed1-x529.google.com with SMTP id b20-v6so4500569edt.10 for ; Mon, 30 Jul 2018 11:00:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=w+Eree0B3p2i/8oSvWrhuiQ330bcqkq9A8AZboFbZHg=; b=eHhD6QK5uBK0ugb6Bl7jtz8bjbD9EutG8QTXxJuZAi3BBUeb4p87V0ObWXJlnXtgb/ 1flZcgBFZxCBkbP0V9r8ztpgrCGYyc8oahm1xsVELPw2/VssS4+dZxeBDbukmU+aL8Xq q9QMlVha+ZmE9El1SgA0o+4qQoVrWJya6Cdr58vUMSEhY9CbYqXTSraIVP5WC1j730dY /lY97i53aXIM43segRJd5jaCFQbIQOh0jtnG9yUNWgy8GV9nWqZa6KRxuGMy5qhHbHw+ pFtk48IOF8u51OrWT/0QXC+Q0rI60ENkKsqc7kzaUo9/T/hctqywvfbRJpj6O4coVy6l +npw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=w+Eree0B3p2i/8oSvWrhuiQ330bcqkq9A8AZboFbZHg=; b=c1nZojSwJ+G4C/xq/sLxdbif/yoBYP9GzJsc1Ze3C/OBug3/BNDuP4TyX11k7fi5m4 Zj87V72OmhpfN4fLain37ntEtz8Twovxib76tZHNF6AE1Y8zjecvsNYg51YiJCLw3t81 YC3uUkeegSKoKmDDGAFX/+Oy6Gi706c8yv2Z7llAXH5rcuQxQPUalRSNjVFzF0Gi6QEq uHFWfzkAxcGk6gZj78XS8aRC8LCkMGHaCptzuEElePXItOdVaUrrqXDrZmzy02dmwHbv l4JgedIAN6iAUezXzzVxOfS6lrCz2V86OkkzFGI8bQYGmLSsi7qebQDVGMczDFYp+zY2 ogKg== X-Gm-Message-State: AOUpUlHcSmd371/Zsn4JCyUlUMYxtI5eQsz7VQpxbBXeEmmQRf4KLtq7 rFd69WvoYys5HDqg8fkynYIN1JaqZcO5VQ== X-Google-Smtp-Source: AAOMgpfjzWo+8nK21bTyuAKe+C6cePlu2KXLAAMhEcYAiWJsf8sTcBJSf+5YzEhT+4QrTqLjmuMT8Q== X-Received: by 2002:a50:87a8:: with SMTP id a37-v6mr9127729eda.31.1532973610854; Mon, 30 Jul 2018 11:00:10 -0700 (PDT) Received: from mutt-hbsd ([178.17.166.148]) by smtp.gmail.com with ESMTPSA id i15-v6sm2650744edq.56.2018.07.30.11.00.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Jul 2018 11:00:09 -0700 (PDT) Date: Mon, 30 Jul 2018 13:59:23 -0400 From: Shawn Webb To: Kyle Evans Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336919 - head/sys/dev/efidev Message-ID: <20180730175923.vtha72pcvzilsspu@mutt-hbsd> References: <201807301740.w6UHeRM0077138@repo.freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="bmsssuyu4hjjv3su" Content-Disposition: inline In-Reply-To: <201807301740.w6UHeRM0077138@repo.freebsd.org> X-Operating-System: FreeBSD mutt-hbsd 12.0-CURRENT FreeBSD 12.0-CURRENT X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: NeoMutt/20180622 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:00:13 -0000 --bmsssuyu4hjjv3su Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 30, 2018 at 05:40:27PM +0000, Kyle Evans wrote: > Author: kevans > Date: Mon Jul 30 17:40:27 2018 > New Revision: 336919 > URL: https://svnweb.freebsd.org/changeset/base/336919 >=20 > Log: > efirt: Add tunable to allow disabling EFI Runtime Services > =20 > Leading up to enabling EFIRT in GENERIC, allow runtime services to be > disabled with a new tunable: efi.rt_disabled. This makes it so that EFI= RT > can be disabled easily in case we run into some buggy UEFI implementati= on > and fail to boot. > =20 > Discussed with: imp, kib > MFC after: 1 week >=20 > Modified: > head/sys/dev/efidev/efirt.c >=20 > Modified: head/sys/dev/efidev/efirt.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- head/sys/dev/efidev/efirt.c Mon Jul 30 17:03:15 2018 (r336918) > +++ head/sys/dev/efidev/efirt.c Mon Jul 30 17:40:27 2018 (r336919) > @@ -133,7 +133,12 @@ efi_init(void) > struct efi_md *map; > caddr_t kmdp; > size_t efisz; > + int rt_disabled; > =20 > + rt_disabled =3D 0; > + TUNABLE_INT_FETCH("efi.rt_disabled", &rt_disabled); Would it be a good idea to document this tunable in loader(8)? Thanks, --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera@is.a.hacker.sx GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --bmsssuyu4hjjv3su Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEKrq2ve9q9Ia+iT2eaoRlj1JFbu4FAltfUfcACgkQaoRlj1JF bu77uRAAsvnFbf1X+VPWde3niomyp8HY2jF5a8LUuIbbcmjvTDOogVnZa8yFi1iE snTspDZFMqBuLrVhM943qFuG+rYQ6E60tNpngC9rvxvZHcQVarJVtYyi5Qnnbh2H bwklLkrmazLwGTe9sf3ncPdYiNnTtYOfRz9LcATZg88+d6Qmb+cz+wGD0S87SZxm dFVSA//2YnvNqvBVQBwAGU5vRIx0AfIGVxQwnBygL0dzGa/uUyXj8KFoX7Mdrq5o ZGPWb6MQIQdL1hvyIGuM9oDc01yevT1CoxIFzpzUDD95o8eOaGdkoDTFCevcPlGU cW0WTMmz1j/IWRKBxwygPJsMeashzQrlSwRnHDDqpXvwoW54uLZ+IuuHPOsGBVEP pJnSER8Sc/lJW+L2Tu/4Gs1LZFyyiFbhAKZ0AxbXb/iM7tkRiy3FqxAadVjpgbqP E8BmrFHsEsVVIUSx6eUKxNEPnTGNSTCruktL4kWkpppcc7rgM8lsIwWRJ+D7Dq5V zr1ANPxutobFZo4RAhISlrfkrZXJQo7wFwW1AQbky4F2pi51jTkq7bvDkO3bzIxY khx9ZiQCtKsbAfMHPwZVMrN2Jdt3UaTwTtl4S/kmIi8AEtNOg3Ch7y5YypzuzZN7 NITN5vm6NBlZGPdIiqWErQj9yFYrp9iFNVkB3yLGi2mPxiURbjw= =qAeG -----END PGP SIGNATURE----- --bmsssuyu4hjjv3su-- From owner-svn-src-all@freebsd.org Mon Jul 30 18:01:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0ACE7105CDDD for ; Mon, 30 Jul 2018 18:01:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22d.google.com (mail-it0-x22d.google.com [IPv6:2607:f8b0:4001:c0b::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F0C37D257 for ; Mon, 30 Jul 2018 18:01:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22d.google.com with SMTP id e14-v6so512491itf.1 for ; Mon, 30 Jul 2018 11:01:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=MGjKUZpx9pSI6M3/mM9kLWZUUKNSlGDXAW1770YkHd0=; b=a1+jcresgm6khk4PDSiXrSQ8BBeiTjtZbaEwQUsPDvmMz/qjuTn12Kbmpd/iPzbT3z eu6pVrzt7E6GWPIR2llYErLZgJkSBzLBS+nJlgv3MVaF89+h4piz2uRtbtOfRFpWjTVU 0Lnli3L1rADcTp0sRQCoKokXKeOa/npXRX0QQf5vJ/7AJwPQnAPt3T679AqokKg3Nnzg d3sMUYphMHlICDXQcIZguVpjd2k6dnoZ928eHNGT4LPZohKZfPA9Lb6QFt5T4JkgUYu5 eIis0TwtCEyRMtfsqwmCmZoSskUWvz07MVSDO4IGf+MhyVDqgW8568YPO0u3lWwJ1ndL HStw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=MGjKUZpx9pSI6M3/mM9kLWZUUKNSlGDXAW1770YkHd0=; b=N6Ijh47U9qaFBJqGlvrVlZRatFatEJpfRJrJtUdKWVzD2Ww/UeMfGSfOtWiMcThXs6 queGO46s3R6bi5RdUCJWsOPOCBdZLP9fpaCU1zd6Ek4xcciKl9Xan8bXMzpY8NWh+YxT rRIxuZQo+AX4vxg2IBLAToGAP4wpJuFe6GJjSZ5bLuh3LMIMdE/9VGz+am3aqvZqogTW cv3Jbz71ZILyu9NqIrdk0jMQ69M1ZQoCe9lcC3vHYUQDWEGL3G1lt0FjHonTQLnfFjHa AZ1RQWThfssvu+wTMld13xNRwdl0QCba9iBOtoXpJPjy9hGCRKP709WzGvTIh4RyKPZV snFQ== X-Gm-Message-State: AOUpUlEVPQBAr9GVCYnATMOYYJEOIBRarvwhj6kSnXDucBPZyq95ii2M VcYoO5pfvBBhAoMoDExYKS5++pEDTpTSU5rneSHGqw== X-Google-Smtp-Source: AAOMgpdXir/j8GfX2JIntjtzFDpLjLuZBhwH1nCm+cbSTMcCjUChldElns+CWMorP/AkgZMfFCkHE5RpN/kvSk+Z+Ks= X-Received: by 2002:a24:b211:: with SMTP id u17-v6mr301014ite.1.1532973696920; Mon, 30 Jul 2018 11:01:36 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:4485:0:0:0:0:0 with HTTP; Mon, 30 Jul 2018 11:01:36 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: <20180730175923.vtha72pcvzilsspu@mutt-hbsd> References: <201807301740.w6UHeRM0077138@repo.freebsd.org> <20180730175923.vtha72pcvzilsspu@mutt-hbsd> From: Warner Losh Date: Mon, 30 Jul 2018 12:01:36 -0600 X-Google-Sender-Auth: OXbJlzd9IlDRxQc0hr7pxjPPAmo Message-ID: Subject: Re: svn commit: r336919 - head/sys/dev/efidev To: Shawn Webb Cc: Kyle Evans , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:01:38 -0000 It would be a good idea to make it like the other tuneables. efi.rt.enabled=0 would be the pattern. Warner On Mon, Jul 30, 2018 at 11:59 AM, Shawn Webb wrote: > On Mon, Jul 30, 2018 at 05:40:27PM +0000, Kyle Evans wrote: > > Author: kevans > > Date: Mon Jul 30 17:40:27 2018 > > New Revision: 336919 > > URL: https://svnweb.freebsd.org/changeset/base/336919 > > > > Log: > > efirt: Add tunable to allow disabling EFI Runtime Services > > > > Leading up to enabling EFIRT in GENERIC, allow runtime services to be > > disabled with a new tunable: efi.rt_disabled. This makes it so that > EFIRT > > can be disabled easily in case we run into some buggy UEFI > implementation > > and fail to boot. > > > > Discussed with: imp, kib > > MFC after: 1 week > > > > Modified: > > head/sys/dev/efidev/efirt.c > > > > Modified: head/sys/dev/efidev/efirt.c > > ============================================================ > ================== > > --- head/sys/dev/efidev/efirt.c Mon Jul 30 17:03:15 2018 > (r336918) > > +++ head/sys/dev/efidev/efirt.c Mon Jul 30 17:40:27 2018 > (r336919) > > @@ -133,7 +133,12 @@ efi_init(void) > > struct efi_md *map; > > caddr_t kmdp; > > size_t efisz; > > + int rt_disabled; > > > > + rt_disabled = 0; > > + TUNABLE_INT_FETCH("efi.rt_disabled", &rt_disabled); > > Would it be a good idea to document this tunable in loader(8)? > > Thanks, > > -- > Shawn Webb > Cofounder and Security Engineer > HardenedBSD > > Tor-ified Signal: +1 443-546-8752 > Tor+XMPP+OTR: lattera@is.a.hacker.sx > GPG Key ID: 0x6A84658F52456EEE > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > From owner-svn-src-all@freebsd.org Mon Jul 30 18:05:36 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D978105CFC3; Mon, 30 Jul 2018 18:05:36 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED5017D4DC; Mon, 30 Jul 2018 18:05:35 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 8A24BC662; Mon, 30 Jul 2018 18:05:35 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-lj1-f178.google.com with SMTP id f8-v6so11332822ljk.1; Mon, 30 Jul 2018 11:05:35 -0700 (PDT) X-Gm-Message-State: AOUpUlEJg/g6Pi6Gv+5oFkKVuJBOhTJzUZcs7in6NxTevGp2AjoDX7Df BlEyNHphZPuA/vYjTO0wvptquUGU3AdCdyftWag= X-Google-Smtp-Source: AAOMgpcvHcpQc69IwpQiO/Oo/BNizSC52ppUMRDML98EyRNYcjn9wf7ZA1VMhOpGcsMD2J/HPwgvGCrvkTj4ejpbqmI= X-Received: by 2002:a2e:8617:: with SMTP id a23-v6mr64489lji.43.1532973934130; Mon, 30 Jul 2018 11:05:34 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:5742:0:0:0:0:0 with HTTP; Mon, 30 Jul 2018 11:05:13 -0700 (PDT) In-Reply-To: References: <201807301740.w6UHeRM0077138@repo.freebsd.org> <20180730175923.vtha72pcvzilsspu@mutt-hbsd> From: Kyle Evans Date: Mon, 30 Jul 2018 13:05:13 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r336919 - head/sys/dev/efidev To: Warner Losh Cc: Shawn Webb , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:05:36 -0000 On Mon, Jul 30, 2018 at 1:01 PM, Warner Losh wrote: > On Mon, Jul 30, 2018 at 11:59 AM, Shawn Webb > wrote: >> >> On Mon, Jul 30, 2018 at 05:40:27PM +0000, Kyle Evans wrote: >> > Author: kevans >> > Date: Mon Jul 30 17:40:27 2018 >> > New Revision: 336919 >> > URL: https://svnweb.freebsd.org/changeset/base/336919 >> > >> > Log: >> > efirt: Add tunable to allow disabling EFI Runtime Services >> > >> > Leading up to enabling EFIRT in GENERIC, allow runtime services to be >> > disabled with a new tunable: efi.rt_disabled. This makes it so that >> > EFIRT >> > can be disabled easily in case we run into some buggy UEFI >> > implementation >> > and fail to boot. >> > >> > Discussed with: imp, kib >> > MFC after: 1 week >> > >> > Modified: >> > head/sys/dev/efidev/efirt.c >> > >> > Modified: head/sys/dev/efidev/efirt.c >> > >> > ============================================================================== >> > --- head/sys/dev/efidev/efirt.c Mon Jul 30 17:03:15 2018 >> > (r336918) >> > +++ head/sys/dev/efidev/efirt.c Mon Jul 30 17:40:27 2018 >> > (r336919) >> > @@ -133,7 +133,12 @@ efi_init(void) >> > struct efi_md *map; >> > caddr_t kmdp; >> > size_t efisz; >> > + int rt_disabled; >> > >> > + rt_disabled = 0; >> > + TUNABLE_INT_FETCH("efi.rt_disabled", &rt_disabled); >> >> Would it be a good idea to document this tunable in loader(8)? >> >> Thanks, >> >> -- >> Shawn Webb >> Cofounder and Security Engineer >> HardenedBSD >> >> Tor-ified Signal: +1 443-546-8752 >> Tor+XMPP+OTR: lattera@is.a.hacker.sx >> GPG Key ID: 0x6A84658F52456EEE >> GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE > > It would be a good idea to make it like the other tuneables. > > efi.rt.enabled=0 would be the pattern. > In that case, I'll change this to efi.rt.disabled to more closely match newbus hinting, and I'll document it in loader(8) in the process. From owner-svn-src-all@freebsd.org Mon Jul 30 18:06:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4F533105CFFA for ; Mon, 30 Jul 2018 18:06:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6F457D607 for ; Mon, 30 Jul 2018 18:06:01 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x236.google.com with SMTP id p81-v6so520165itp.1 for ; Mon, 30 Jul 2018 11:06:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Yw1uxy5QurUHEZZ7GRvyFFp9GIWeok9xljozkPiZSDk=; b=JS++yqxfbMp4O630dNH9HC10IT3YceVZ8HId5UnlFuO3hYsirKUHzEz3oki+omVt3i w897Spyb1bUeMK6XhUvNFFOy424OJaJ3TIddMsEzTvyqnKYwlZG9D8xScTp6VSI3xQ9E fOXkGQgx98+l8Fxs09iQBeM6nPbGOyIi4bnymBlEijbGHQQ4w4l993nqeNU3zU/+5mE2 HXANpYr7DBOc+saGylwp07InUpeym0cQLyQfHnC4vqqFZjHtW2X3zU/cQsaPt3q+ptHv 3suqNLu4SMA4MnQHisZ14rcvLLlwROYt/1GmLwbFj8zDFbvgSh7emRfJS8e8JnBrCMOH Do7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Yw1uxy5QurUHEZZ7GRvyFFp9GIWeok9xljozkPiZSDk=; b=ShB9TcdyYBIViZPc0nAD1lWI9UBBotj/PHlrL03VRbdbSkVClqfJJjzFjJzDAJFMsl 88ILpkxzh+FR942+YmaFfXdTNYzFB2hhcYzbV7stTbPNVQiitjGhB5N8x4roec4wdtHm +TDoI6I/QEINPTbreYYlS69uumL+ThnBJ1wI/hsyeYAf31MMu1CGA2E4bBZWgn2KSLkL SA2hQCZO3FeEW6N6v8nezIOxGz+vlo6ttwuPjNoUZoDK3FYYvjoHOUksEtMROcvA+2jr TWsaY2VHO/9psa8cmSy7/ligNxkPOQ3Rcf49MNRxzVTKh5I3f33bUnAG+RRXPOLLkVxV 3E7Q== X-Gm-Message-State: AOUpUlGR4a0g5Wo0chRojKh7PZBc6kqmSNrwYiv/mwAk1m3APn7Ob+5s pPb7XpB3r7QpLgqUrJ/RFAxztxZIf/JhJBjE6M/KGg== X-Google-Smtp-Source: AAOMgpdTH6ttfWWvxplUSoU1CfQfYR+q9FYRpHw/EMmOWcZK8avcx6/KOpGOnhi7be0CaKD+U1x360lcwa3jv0leowM= X-Received: by 2002:a24:b211:: with SMTP id u17-v6mr318226ite.1.1532973961068; Mon, 30 Jul 2018 11:06:01 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:4485:0:0:0:0:0 with HTTP; Mon, 30 Jul 2018 11:06:00 -0700 (PDT) X-Originating-IP: [2603:300b:6:5100:1052:acc7:f9de:2b6d] In-Reply-To: References: <201807301740.w6UHeRM0077138@repo.freebsd.org> <20180730175923.vtha72pcvzilsspu@mutt-hbsd> From: Warner Losh Date: Mon, 30 Jul 2018 12:06:00 -0600 X-Google-Sender-Auth: ls9kCj280gVlB4QoxjjFvhBF9GQ Message-ID: Subject: Re: svn commit: r336919 - head/sys/dev/efidev To: Shawn Webb Cc: Kyle Evans , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:06:02 -0000 I take that back... efi.rt.disabled matches the newbus usage... Warner On Mon, Jul 30, 2018 at 12:01 PM, Warner Losh wrote: > It would be a good idea to make it like the other tuneables. > > efi.rt.enabled=0 would be the pattern. > > Warner > > On Mon, Jul 30, 2018 at 11:59 AM, Shawn Webb > wrote: > >> On Mon, Jul 30, 2018 at 05:40:27PM +0000, Kyle Evans wrote: >> > Author: kevans >> > Date: Mon Jul 30 17:40:27 2018 >> > New Revision: 336919 >> > URL: https://svnweb.freebsd.org/changeset/base/336919 >> > >> > Log: >> > efirt: Add tunable to allow disabling EFI Runtime Services >> > >> > Leading up to enabling EFIRT in GENERIC, allow runtime services to be >> > disabled with a new tunable: efi.rt_disabled. This makes it so that >> EFIRT >> > can be disabled easily in case we run into some buggy UEFI >> implementation >> > and fail to boot. >> > >> > Discussed with: imp, kib >> > MFC after: 1 week >> > >> > Modified: >> > head/sys/dev/efidev/efirt.c >> > >> > Modified: head/sys/dev/efidev/efirt.c >> > ============================================================ >> ================== >> > --- head/sys/dev/efidev/efirt.c Mon Jul 30 17:03:15 2018 >> (r336918) >> > +++ head/sys/dev/efidev/efirt.c Mon Jul 30 17:40:27 2018 >> (r336919) >> > @@ -133,7 +133,12 @@ efi_init(void) >> > struct efi_md *map; >> > caddr_t kmdp; >> > size_t efisz; >> > + int rt_disabled; >> > >> > + rt_disabled = 0; >> > + TUNABLE_INT_FETCH("efi.rt_disabled", &rt_disabled); >> >> Would it be a good idea to document this tunable in loader(8)? >> >> Thanks, >> >> -- >> Shawn Webb >> Cofounder and Security Engineer >> HardenedBSD >> >> Tor-ified Signal: +1 443-546-8752 >> Tor+XMPP+OTR: lattera@is.a.hacker.sx >> GPG Key ID: 0x6A84658F52456EEE >> GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE >> > > From owner-svn-src-all@freebsd.org Mon Jul 30 18:13:22 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9C45105D3F8; Mon, 30 Jul 2018 18:13:21 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D2397DC44; Mon, 30 Jul 2018 18:13:21 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6E49527FF0; Mon, 30 Jul 2018 18:13:21 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UIDLod097592; Mon, 30 Jul 2018 18:13:21 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UIDKjW097589; Mon, 30 Jul 2018 18:13:20 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807301813.w6UIDKjW097589@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Mon, 30 Jul 2018 18:13:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336924 - in head: . stand/man sys/dev/efidev X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: in head: . stand/man sys/dev/efidev X-SVN-Commit-Revision: 336924 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:13:22 -0000 Author: kevans Date: Mon Jul 30 18:13:20 2018 New Revision: 336924 URL: https://svnweb.freebsd.org/changeset/base/336924 Log: Follow up to r336919 and r336921: s/efi.rt_disabled/efi.rt.disabled/ The latter matches the rest of the tree better [0]. The UPDATING entry has been updated to reflect this, and the new tunable is now documented in loader(8) [1]. Reported by: imp [0], Shawn Webb [1] Modified: head/UPDATING head/stand/man/loader.8 head/sys/dev/efidev/efirt.c Modified: head/UPDATING ============================================================================== --- head/UPDATING Mon Jul 30 17:59:18 2018 (r336923) +++ head/UPDATING Mon Jul 30 18:13:20 2018 (r336924) @@ -34,7 +34,7 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 12.x IS SLOW: 20180730: amd64/GENERIC now has EFI runtime services, EFIRT, enabled by default. This should have no effect if the kernel is booted via BIOS/legacy boot. - EFIRT may be disabled via a loader tunable, efi.rt_disabled, if a system + EFIRT may be disabled via a loader tunable, efi.rt.disabled, if a system has a buggy firmware that prevents a successful boot due to use of runtime services. Modified: head/stand/man/loader.8 ============================================================================== --- head/stand/man/loader.8 Mon Jul 30 17:59:18 2018 (r336923) +++ head/stand/man/loader.8 Mon Jul 30 18:13:20 2018 (r336924) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd May 15, 2018 +.Dd July 30, 2018 .Dt LOADER 8 .Os .Sh NAME @@ -588,6 +588,10 @@ explicitly. Other variables are used to override kernel tunable parameters. The following tunables are available: .Bl -tag -width Va +.It Va efi.rt.disabled +Disable UEFI runtime services in the kernel, if applicable. +Runtime services are only available and used if the kernel is booted in a UEFI +environment. .It Va hw.physmem Limit the amount of physical memory the system will use. By default the size is in bytes, but the Modified: head/sys/dev/efidev/efirt.c ============================================================================== --- head/sys/dev/efidev/efirt.c Mon Jul 30 17:59:18 2018 (r336923) +++ head/sys/dev/efidev/efirt.c Mon Jul 30 18:13:20 2018 (r336924) @@ -136,7 +136,7 @@ efi_init(void) int rt_disabled; rt_disabled = 0; - TUNABLE_INT_FETCH("efi.rt_disabled", &rt_disabled); + TUNABLE_INT_FETCH("efi.rt.disabled", &rt_disabled); if (rt_disabled == 1) return (0); mtx_init(&efi_lock, "efi", NULL, MTX_DEF); From owner-svn-src-all@freebsd.org Mon Jul 30 18:29:21 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A6C5105D838; Mon, 30 Jul 2018 18:29:21 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6EA97E373; Mon, 30 Jul 2018 18:29:20 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B28BB22F; Mon, 30 Jul 2018 18:29:20 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UITK6Z002948; Mon, 30 Jul 2018 18:29:20 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UITKvp002947; Mon, 30 Jul 2018 18:29:20 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201807301829.w6UITKvp002947@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Mon, 30 Jul 2018 18:29:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336925 - head/sys/modules/ipmi X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/modules/ipmi X-SVN-Commit-Revision: 336925 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:29:21 -0000 Author: jhibbits Date: Mon Jul 30 18:29:20 2018 New Revision: 336925 URL: https://svnweb.freebsd.org/changeset/base/336925 Log: Add ofw_bus_if.h to the SRCS list for ipmi module on powerpc64 PR: 230194 Reported by: sbruno Modified: head/sys/modules/ipmi/Makefile Modified: head/sys/modules/ipmi/Makefile ============================================================================== --- head/sys/modules/ipmi/Makefile Mon Jul 30 18:13:20 2018 (r336924) +++ head/sys/modules/ipmi/Makefile Mon Jul 30 18:29:20 2018 (r336925) @@ -16,7 +16,7 @@ SRCS+= acpi_if.h isa_if.h pci_if.h smbus_if.h SUBDIR+= ipmi_linux .elif ${MACHINE_ARCH} == "powerpc64" -SRCS+= ipmi_opal.c +SRCS+= ipmi_opal.c ofw_bus_if.h .endif .include From owner-svn-src-all@freebsd.org Mon Jul 30 18:29:47 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC1BB105D88B; Mon, 30 Jul 2018 18:29:47 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8174E7E4AE; Mon, 30 Jul 2018 18:29:47 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 62963231; Mon, 30 Jul 2018 18:29:47 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UITlaB003021; Mon, 30 Jul 2018 18:29:47 GMT (envelope-from pfg@FreeBSD.org) Received: (from pfg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UITl9i003019; Mon, 30 Jul 2018 18:29:47 GMT (envelope-from pfg@FreeBSD.org) Message-Id: <201807301829.w6UITl9i003019@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: pfg set sender to pfg@FreeBSD.org using -f From: "Pedro F. Giffuni" Date: Mon, 30 Jul 2018 18:29:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336926 - head/usr.bin/sed X-SVN-Group: head X-SVN-Commit-Author: pfg X-SVN-Commit-Paths: head/usr.bin/sed X-SVN-Commit-Revision: 336926 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:29:48 -0000 Author: pfg Date: Mon Jul 30 18:29:46 2018 New Revision: 336926 URL: https://svnweb.freebsd.org/changeset/base/336926 Log: sed: unsign some indexes to fix sign-compare warnings. Hinted by: OpenBSD (CVS 1.32) MFC after: 1 week Modified: head/usr.bin/sed/extern.h head/usr.bin/sed/process.c Modified: head/usr.bin/sed/extern.h ============================================================================== --- head/usr.bin/sed/extern.h Mon Jul 30 18:29:20 2018 (r336925) +++ head/usr.bin/sed/extern.h Mon Jul 30 18:29:46 2018 (r336926) @@ -41,7 +41,7 @@ extern struct s_appends *appends; extern regmatch_t *match; extern size_t maxnsub; extern u_long linenum; -extern int appendnum; +extern unsigned int appendnum; extern int aflag, eflag, nflag; extern const char *fname, *outfname; extern FILE *infile, *outfile; Modified: head/usr.bin/sed/process.c ============================================================================== --- head/usr.bin/sed/process.c Mon Jul 30 18:29:20 2018 (r336925) +++ head/usr.bin/sed/process.c Mon Jul 30 18:29:46 2018 (r336926) @@ -79,8 +79,8 @@ static void regsub(SPACE *, char *, char *); static int substitute(struct s_command *); struct s_appends *appends; /* Array of pointers to strings to append. */ -static int appendx; /* Index into appends array. */ -int appendnum; /* Size of appends array. */ +static unsigned int appendx; /* Index into appends array. */ +unsigned int appendnum; /* Size of appends array. */ static int lastaddr; /* Set by applies if last address of a range. */ static int sdone; /* If any substitutes since last line input. */ @@ -385,7 +385,7 @@ substitute(struct s_command *cp) regex_t *re; regoff_t slen; int lastempty, n; - size_t le = 0; + regoff_t le = 0; char *s; s = ps; @@ -550,13 +550,13 @@ static void flush_appends(void) { FILE *f; - int count, i; + unsigned int count, idx; char buf[8 * 1024]; - for (i = 0; i < appendx; i++) - switch (appends[i].type) { + for (idx = 0; idx < appendx; idx++) + switch (appends[idx].type) { case AP_STRING: - fwrite(appends[i].s, sizeof(char), appends[i].len, + fwrite(appends[idx].s, sizeof(char), appends[idx].len, outfile); break; case AP_FILE: @@ -568,7 +568,7 @@ flush_appends(void) * would be truly bizarre, but possible. It's probably * not that big a performance win, anyhow. */ - if ((f = fopen(appends[i].s, "r")) == NULL) + if ((f = fopen(appends[idx].s, "r")) == NULL) break; while ((count = fread(buf, sizeof(char), sizeof(buf), f))) (void)fwrite(buf, sizeof(char), count, outfile); From owner-svn-src-all@freebsd.org Mon Jul 30 18:30:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58FC0105D8E8; Mon, 30 Jul 2018 18:30:18 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from mail.ignoranthack.me (ignoranthack.me [199.102.79.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EB1067E617; Mon, 30 Jul 2018 18:30:17 +0000 (UTC) (envelope-from sbruno@freebsd.org) Received: from [192.168.0.6] (unknown [75.161.254.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: sbruno@ignoranthack.me) by mail.ignoranthack.me (Postfix) with ESMTPSA id 7114B1AF510; Mon, 30 Jul 2018 10:36:45 +0000 (UTC) Subject: Re: svn commit: r336925 - head/sys/modules/ipmi To: Justin Hibbits , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201807301829.w6UITKvp002947@repo.freebsd.org> From: Sean Bruno Openpgp: preference=signencrypt Autocrypt: addr=sbruno@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFk+0UEBCADaf4bgxxKvMOhRV5NPoGWRCCGm49d6+1VFNlQ77WsY/+Zvf95TPULdRlnG w648KfxWt7+O3kdKhdRwnqlXWC7zA2Qt0dRE1yIqOGJ4jp4INvp/bcxWzgr0aoKOjrlnfxRV bh+s0rzdZt6TsNL3cVYxkC8oezjaUkHdW4mFJU249U1QJogkF8g0FeKNfEcjEkwJNX6lQJH+ EzCWT0NCk6J+Xyo+zOOljxPp1OUfdvZi3ulkU/qTZstGVWxFVsP8xQklV/y3AFcbIYx6iGJ4 5L7WuB0IWhO7Z4yHENr8wFaNYwpod9i4egX2BugbrM8pOfhN2/qqdeG1L5LMtXw3yyAhABEB AAHNN1NlYW4gQnJ1bm8gKEZyZWVCU0QgRGV2ZWxvcGVyIEtleSkgPHNicnVub0BmcmVlYnNk Lm9yZz7CwJQEEwEKAD4WIQToxOn4gDUE4eP0ujS95PX+ibX8tgUCWT7RQQIbAwUJBaOagAUL CQgHAwUVCgkICwUWAwIBAAIeAQIXgAAKCRC95PX+ibX8ttKTCACFKzRc56EBAlVotq02EjZP SfX+unlk6AuPBzShxqRxeK+bGYVCigrYd1M8nnskv0dEiZ5iYeND9HIxbpEyopqgpVTibA7w gBXaZ7SOEhNX1wXwg14JrralfSmPFMYni+sWegPMX/zwfAsn1z4mG1Nn44Xqo3o7CfpkMPy6 M5Bow2IDzIhEYISLR+urxs74/aHU35PLtBSDtu18914SEMDdva27MARN8mbeCDbuJVfGCPWy YHuy2t+9u2Zn5Dd+t3sBXLM9gpeaMm+4x6TNPpESygbVdh4tDdjVZ9DK/bWFg0kMgfZoaq6J l0jNsQXrZV3bzYNFbVw04pFcvA2GIJ7xzsBNBFk+0UEBCADIXBmQOaKMHGbc9vwjhV4Oj5aZ DdhNedn12FVeTdOXJvuTOusgxS29lla0RenHGDsgD08UiFpasBXWq/E+BhQ19d+iRbLLR17O KKc1ZGefoVbLARLXD68J5j4XAyK+6k2KqBLlqzAEpHTzsksM9naARkVXiEVcrt6ciw0FSm8n kuK3gDKKe93XfzfP+TQdbvvzJc7Fa+appLbXz61TM1aikaQlda8bWubDegwXbuoJdB34xU1m yjr/N4o+raL0x7QrzdH+wwgrTTo+H4S2c1972Skt5K5tbxLowfHicRl23V8itVQr3sBtlX4+ 66q+Apm7+R36bUS/k+G45Sp6iPpxABEBAAHCwHwEGAEKACYWIQToxOn4gDUE4eP0ujS95PX+ ibX8tgUCWT7RQQIbDAUJBaOagAAKCRC95PX+ibX8trrIB/9Pljqt/JGamD9tx4dOVmxSyFg9 z2xzgklTLuDgS73MM120mM7ao9AQUeWiSle/H0UCK7xPOzC/aeUC4oygDQKAfkkNbCNTo3+A qDjBRA8qx0e9a/QjDL+RFgD4L5kLT4tToY8T8HaBp8h03LBfk510IaI8oL/Jg7vpM3PDtJMW tUi2H+yNFmL3NfM2oBToWKLFsoP54f/eeeImrNnrlLjLHPzqS+/9apgYqX2Jwiv3tHBc4FTO GuY8VvF7BpixJs8Pc2RUuCfSyodrp1YG1kRGlXAH0cqwwr0Zmk4+7dZvtVQMCl6kS6q1+84q JwtItxS2eXSEA4NO0sQ3BXUywANh Message-ID: Date: Mon, 30 Jul 2018 12:30:16 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <201807301829.w6UITKvp002947@repo.freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="m7aeTI9afbovXDkgqdmnz6dCKqadFs1eq" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 18:30:18 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --m7aeTI9afbovXDkgqdmnz6dCKqadFs1eq Content-Type: multipart/mixed; boundary="rFt2gzNTpmgqGhCUHEi3RDDvKsenkGrxV"; protected-headers="v1" From: Sean Bruno To: Justin Hibbits , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Message-ID: Subject: Re: svn commit: r336925 - head/sys/modules/ipmi References: <201807301829.w6UITKvp002947@repo.freebsd.org> In-Reply-To: <201807301829.w6UITKvp002947@repo.freebsd.org> --rFt2gzNTpmgqGhCUHEi3RDDvKsenkGrxV Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 07/30/18 12:29, Justin Hibbits wrote: > Author: jhibbits > Date: Mon Jul 30 18:29:20 2018 > New Revision: 336925 > URL: https://svnweb.freebsd.org/changeset/base/336925 >=20 > Log: > Add ofw_bus_if.h to the SRCS list for ipmi module on powerpc64 > =20 > PR: 230194 > Reported by: sbruno >=20 > Modified: > head/sys/modules/ipmi/Makefile >=20 > Modified: head/sys/modules/ipmi/Makefile > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- head/sys/modules/ipmi/Makefile Mon Jul 30 18:13:20 2018 (r336924) > +++ head/sys/modules/ipmi/Makefile Mon Jul 30 18:29:20 2018 (r336925) > @@ -16,7 +16,7 @@ SRCS+=3D acpi_if.h isa_if.h pci_if.h smbus_if.h > SUBDIR+=3D ipmi_linux > =20 > .elif ${MACHINE_ARCH} =3D=3D "powerpc64" > -SRCS+=3D ipmi_opal.c > +SRCS+=3D ipmi_opal.c ofw_bus_if.h > .endif > =20 > .include >=20 >=20 Gracias! sean --rFt2gzNTpmgqGhCUHEi3RDDvKsenkGrxV-- --m7aeTI9afbovXDkgqdmnz6dCKqadFs1eq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAEBCgB9FiEE6MTp+IA1BOHj9Lo0veT1/om1/LYFAltfWThfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEU4 QzRFOUY4ODAzNTA0RTFFM0Y0QkEzNEJERTRGNUZFODlCNUZDQjYACgkQveT1/om1 /LbLQwgAnf2avzSYY6aVAjOkm1YEwk+xKmW8s/paIFgfASNdpsbX6QkOPiVzweP/ Cnd0yBKGJU/uvCRGXZbYkxM9SGPuNl8U6MjG0KOHe7gV9CkiHciGI9gg5B3EtJKG 3FQWW4CO2mXjgVg59uUWZeJl984oxCeoYZ44+CZiLnFrqYvyx4j0Ph5QKzOuHumB O2UXRxso7gD0IpoNWMh30sN4K5sJM6w1bLk79VOudJTg20kWJWNq7UvJDZdHx+he qDkfoRleEdd6+t5Lnzfwzhh/86FUiVq4m4pVG5rAAjGZhfmJq9kToAOMQq6tj3Fd z7vI+cc0+JMya5I0FSH9dSVpg667dg== =QfRK -----END PGP SIGNATURE----- --m7aeTI9afbovXDkgqdmnz6dCKqadFs1eq-- From owner-svn-src-all@freebsd.org Mon Jul 30 19:23:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D4E9105EE4E; Mon, 30 Jul 2018 19:23:53 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0FDD180A67; Mon, 30 Jul 2018 19:23:53 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DF700C22; Mon, 30 Jul 2018 19:23:52 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UJNqKT033454; Mon, 30 Jul 2018 19:23:52 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UJNqWd033453; Mon, 30 Jul 2018 19:23:52 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807301923.w6UJNqWd033453@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 30 Jul 2018 19:23:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336927 - stable/11/sys/rpc X-SVN-Group: stable-11 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: stable/11/sys/rpc X-SVN-Commit-Revision: 336927 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 19:23:53 -0000 Author: rmacklem Date: Mon Jul 30 19:23:52 2018 New Revision: 336927 URL: https://svnweb.freebsd.org/changeset/base/336927 Log: MFC: r335866 Fix the server side krpc so that the kernel nfsd threads terminate. Occationally the kernel nfsd threads would not terminate when a SIGKILL was posted for the kernel process (called nfsd (slave)). When this occurred, the thread associated with the process (called "ismaster") had returned from svc_run_internal() and was sleeping waiting for the other threads to terminate. The other threads (created by kthread_start()) were still in svc_run_internal() handling NFS RPCs. The only way this could occur is for the "ismaster" thread to return from svc_run_internal() without having called svc_exit(). There was only one place in the code where this could happen and this patch stops that from happening. Since the problem is intermittent, I cannot be sure if this has fixed the problem, but I have not seen an occurrence of the problem with this patch applied. Modified: stable/11/sys/rpc/svc.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/rpc/svc.c ============================================================================== --- stable/11/sys/rpc/svc.c Mon Jul 30 18:29:46 2018 (r336926) +++ stable/11/sys/rpc/svc.c Mon Jul 30 19:23:52 2018 (r336927) @@ -1183,7 +1183,8 @@ svc_run_internal(SVCGROUP *grp, bool_t ismaster) /* * Enforce maxthreads count. */ - if (grp->sg_threadcount > grp->sg_maxthreads) + if (!ismaster && grp->sg_threadcount > + grp->sg_maxthreads) break; /* From owner-svn-src-all@freebsd.org Mon Jul 30 19:29:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5732105F034; Mon, 30 Jul 2018 19:29:32 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 570CB80C72; Mon, 30 Jul 2018 19:29:32 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 32D55C24; Mon, 30 Jul 2018 19:29:32 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UJTWYs033741; Mon, 30 Jul 2018 19:29:32 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UJTW8M033740; Mon, 30 Jul 2018 19:29:32 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807301929.w6UJTW8M033740@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 30 Jul 2018 19:29:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r336928 - stable/10/sys/rpc X-SVN-Group: stable-10 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: stable/10/sys/rpc X-SVN-Commit-Revision: 336928 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 19:29:32 -0000 Author: rmacklem Date: Mon Jul 30 19:29:31 2018 New Revision: 336928 URL: https://svnweb.freebsd.org/changeset/base/336928 Log: MFC: r335866 Fix the server side krpc so that the kernel nfsd threads terminate. Occationally the kernel nfsd threads would not terminate when a SIGKILL was posted for the kernel process (called nfsd (slave)). When this occurred, the thread associated with the process (called "ismaster") had returned from svc_run_internal() and was sleeping waiting for the other threads to terminate. The other threads (created by kthread_start()) were still in svc_run_internal() handling NFS RPCs. The only way this could occur is for the "ismaster" thread to return from svc_run_internal() without having called svc_exit(). There was only one place in the code where this could happen and this patch stops that from happening. Since the problem is intermittent, I cannot be sure if this has fixed the problem, but I have not seen an occurrence of the problem with this patch applied. Modified: stable/10/sys/rpc/svc.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/rpc/svc.c ============================================================================== --- stable/10/sys/rpc/svc.c Mon Jul 30 19:23:52 2018 (r336927) +++ stable/10/sys/rpc/svc.c Mon Jul 30 19:29:31 2018 (r336928) @@ -1183,7 +1183,8 @@ svc_run_internal(SVCGROUP *grp, bool_t ismaster) /* * Enforce maxthreads count. */ - if (grp->sg_threadcount > grp->sg_maxthreads) + if (!ismaster && grp->sg_threadcount > + grp->sg_maxthreads) break; /* From owner-svn-src-all@freebsd.org Mon Jul 30 19:41:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 962B0105F40B; Mon, 30 Jul 2018 19:41:55 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 487EB81329; Mon, 30 Jul 2018 19:41:55 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 29B55F11; Mon, 30 Jul 2018 19:41:55 +0000 (UTC) (envelope-from pfg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UJfto1043240; Mon, 30 Jul 2018 19:41:55 GMT (envelope-from pfg@FreeBSD.org) Received: (from pfg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UJftmu043239; Mon, 30 Jul 2018 19:41:55 GMT (envelope-from pfg@FreeBSD.org) Message-Id: <201807301941.w6UJftmu043239@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: pfg set sender to pfg@FreeBSD.org using -f From: "Pedro F. Giffuni" Date: Mon, 30 Jul 2018 19:41:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336929 - head/usr.bin/sed X-SVN-Group: head X-SVN-Commit-Author: pfg X-SVN-Commit-Paths: head/usr.bin/sed X-SVN-Commit-Revision: 336929 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 19:41:55 -0000 Author: pfg Date: Mon Jul 30 19:41:54 2018 New Revision: 336929 URL: https://svnweb.freebsd.org/changeset/base/336929 Log: sed: compile with the default WARNS level. Modified: head/usr.bin/sed/Makefile Modified: head/usr.bin/sed/Makefile ============================================================================== --- head/usr.bin/sed/Makefile Mon Jul 30 19:29:31 2018 (r336928) +++ head/usr.bin/sed/Makefile Mon Jul 30 19:41:54 2018 (r336929) @@ -6,8 +6,6 @@ PROG= sed SRCS= compile.c main.c misc.c process.c -WARNS?= 2 - HAS_TESTS= SUBDIR.${MK_TESTS}+= tests From owner-svn-src-all@freebsd.org Mon Jul 30 19:44:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 42D31105F4EB; Mon, 30 Jul 2018 19:44:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DB79181519; Mon, 30 Jul 2018 19:44:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B70F7F3F; Mon, 30 Jul 2018 19:44:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UJiFAG043396; Mon, 30 Jul 2018 19:44:15 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UJiFdh043393; Mon, 30 Jul 2018 19:44:15 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807301944.w6UJiFdh043393@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 19:44:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336930 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 336930 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 19:44:16 -0000 Author: mav Date: Mon Jul 30 19:44:14 2018 New Revision: 336930 URL: https://svnweb.freebsd.org/changeset/base/336930 Log: 9284 arc_reclaim_thread has 2 jobs `arc_reclaim_thread()` calls `arc_adjust()` after calling `arc_kmem_reap_now()`; `arc_adjust()` signals `arc_get_data_buf()` to indicate that we may no longer be `arc_is_overflowing()`. The problem is, `arc_kmem_reap_now()` can take several seconds to complete, has no impact on `arc_is_overflowing()`, but due to how the code is structured, can impact how long the ARC will remain in the `arc_is_overflowing()` state. The fix is to use seperate threads to: 1. keep `arc_size` under `arc_c`, by calling `arc_adjust()`, which improves `arc_is_overflowing()` 2. keep enough free memory in the system, by calling `arc_kmem_reap_now()` plus `arc_shrink()`, which improves `arc_available_memory()`. illumos/illumos-gate@de753e34f9c399037936e8bc547d823bba9d4b0d Reviewed by: Matt Ahrens Reviewed by: Serapheim Dimitropoulos Reviewed by: Pavel Zakharov Reviewed by: Dan Kimmel Reviewed by: Paul Dagnelie Reviewed by: Dan McDonald Reviewed by: Tim Kordas Approved by: Garrett D'Amore Author: Brad Lewis Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zthr.h vendor-sys/illumos/dist/uts/common/fs/zfs/zthr.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Mon Jul 30 19:41:54 2018 (r336929) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Mon Jul 30 19:44:14 2018 (r336930) @@ -274,6 +274,7 @@ #endif #include #include +#include #include #include #include @@ -284,11 +285,23 @@ boolean_t arc_watch = B_FALSE; int arc_procfd; #endif -static kmutex_t arc_reclaim_lock; -static kcondvar_t arc_reclaim_thread_cv; -static boolean_t arc_reclaim_thread_exit; -static kcondvar_t arc_reclaim_waiters_cv; +/* + * This thread's job is to keep enough free memory in the system, by + * calling arc_kmem_reap_now() plus arc_shrink(), which improves + * arc_available_memory(). + */ +static zthr_t *arc_reap_zthr; +/* + * This thread's job is to keep arc_size under arc_c, by calling + * arc_adjust(), which improves arc_is_overflowing(). + */ +static zthr_t *arc_adjust_zthr; + +static kmutex_t arc_adjust_lock; +static kcondvar_t arc_adjust_waiters_cv; +static boolean_t arc_adjust_needed = B_FALSE; + uint_t arc_reduce_dnlc_percent = 3; /* @@ -301,19 +314,23 @@ uint_t arc_reduce_dnlc_percent = 3; int zfs_arc_evict_batch_limit = 10; /* number of seconds before growing cache again */ -static int arc_grow_retry = 60; +int arc_grow_retry = 60; -/* number of milliseconds before attempting a kmem-cache-reap */ -static int arc_kmem_cache_reap_retry_ms = 1000; +/* + * Minimum time between calls to arc_kmem_reap_soon(). Note that this will + * be converted to ticks, so with the default hz=100, a setting of 15 ms + * will actually wait 2 ticks, or 20ms. + */ +int arc_kmem_cache_reap_retry_ms = 1000; /* shift of arc_c for calculating overflow limit in arc_get_data_impl */ -int zfs_arc_overflow_shift = 8; +int zfs_arc_overflow_shift = 8; /* shift of arc_c for calculating both min and max arc_p */ -static int arc_p_min_shift = 4; +int arc_p_min_shift = 4; /* log2(fraction of arc to reclaim) */ -static int arc_shrink_shift = 7; +int arc_shrink_shift = 7; /* * log2(fraction of ARC which must be free to allow growing). @@ -338,7 +355,7 @@ static int arc_min_prefetch_lifespan; */ int arc_lotsfree_percent = 10; -static int arc_dead; +static boolean_t arc_initialized; /* * The arc has filled available memory and has now warmed up. @@ -840,6 +857,7 @@ aggsum_t astat_other_size; aggsum_t astat_l2_hdr_size; static int arc_no_grow; /* Don't try to grow cache size */ +static hrtime_t arc_growtime; static uint64_t arc_tempreserve; static uint64_t arc_loaned_bytes; @@ -1399,8 +1417,8 @@ hdr_recl(void *unused) * umem calls the reclaim func when we destroy the buf cache, * which is after we do arc_fini(). */ - if (!arc_dead) - cv_signal(&arc_reclaim_thread_cv); + if (arc_initialized) + zthr_wakeup(arc_reap_zthr); } static void @@ -3413,13 +3431,14 @@ arc_evict_state_impl(multilist_t *ml, int idx, arc_buf * function should proceed in this case). * * If threads are left sleeping, due to not - * using cv_broadcast, they will be woken up - * just before arc_reclaim_thread() sleeps. + * using cv_broadcast here, they will be woken + * up via cv_broadcast in arc_adjust_cb() just + * before arc_adjust_zthr sleeps. */ - mutex_enter(&arc_reclaim_lock); + mutex_enter(&arc_adjust_lock); if (!arc_is_overflowing()) - cv_signal(&arc_reclaim_waiters_cv); - mutex_exit(&arc_reclaim_lock); + cv_signal(&arc_adjust_waiters_cv); + mutex_exit(&arc_adjust_lock); } else { ARCSTAT_BUMP(arcstat_mutex_miss); } @@ -3892,8 +3911,8 @@ arc_flush(spa_t *spa, boolean_t retry) (void) arc_flush_state(arc_mfu_ghost, guid, ARC_BUFC_METADATA, retry); } -void -arc_shrink(int64_t to_free) +static void +arc_reduce_target_size(int64_t to_free) { uint64_t asize = aggsum_value(&arc_size); if (arc_c > arc_c_min) { @@ -3912,8 +3931,13 @@ arc_shrink(int64_t to_free) ASSERT((int64_t)arc_p >= 0); } - if (asize > arc_c) - (void) arc_adjust(); + if (asize > arc_c) { + /* See comment in arc_adjust_cb_check() on why lock+flag */ + mutex_enter(&arc_adjust_lock); + arc_adjust_needed = B_TRUE; + mutex_exit(&arc_adjust_lock); + zthr_wakeup(arc_adjust_zthr); + } } typedef enum free_memory_reason_t { @@ -4065,7 +4089,7 @@ arc_reclaim_needed(void) } static void -arc_kmem_reap_now(void) +arc_kmem_reap_soon(void) { size_t i; kmem_cache_t *prev_cache = NULL; @@ -4091,16 +4115,6 @@ arc_kmem_reap_now(void) #endif #endif - /* - * If a kmem reap is already active, don't schedule more. We must - * check for this because kmem_cache_reap_soon() won't actually - * block on the cache being reaped (this is to prevent callers from - * becoming implicitly blocked by a system-wide kmem reap -- which, - * on a system with many, many full magazines, can take minutes). - */ - if (kmem_cache_reap_active()) - return; - for (i = 0; i < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT; i++) { if (zio_buf_cache[i] != prev_cache) { prev_cache = zio_buf_cache[i]; @@ -4126,139 +4140,162 @@ arc_kmem_reap_now(void) } } +/* ARGSUSED */ +static boolean_t +arc_adjust_cb_check(void *arg, zthr_t *zthr) +{ + /* + * This is necessary in order for the mdb ::arc dcmd to + * show up to date information. Since the ::arc command + * does not call the kstat's update function, without + * this call, the command may show stale stats for the + * anon, mru, mru_ghost, mfu, and mfu_ghost lists. Even + * with this change, the data might be up to 1 second + * out of date(the arc_adjust_zthr has a maximum sleep + * time of 1 second); but that should suffice. The + * arc_state_t structures can be queried directly if more + * accurate information is needed. + */ + if (arc_ksp != NULL) + arc_ksp->ks_update(arc_ksp, KSTAT_READ); + + /* + * We have to rely on arc_get_data_impl() to tell us when to adjust, + * rather than checking if we are overflowing here, so that we are + * sure to not leave arc_get_data_impl() waiting on + * arc_adjust_waiters_cv. If we have become "not overflowing" since + * arc_get_data_impl() checked, we need to wake it up. We could + * broadcast the CV here, but arc_get_data_impl() may have not yet + * gone to sleep. We would need to use a mutex to ensure that this + * function doesn't broadcast until arc_get_data_impl() has gone to + * sleep (e.g. the arc_adjust_lock). However, the lock ordering of + * such a lock would necessarily be incorrect with respect to the + * zthr_lock, which is held before this function is called, and is + * held by arc_get_data_impl() when it calls zthr_wakeup(). + */ + return (arc_adjust_needed); +} + /* - * Threads can block in arc_get_data_impl() waiting for this thread to evict - * enough data and signal them to proceed. When this happens, the threads in - * arc_get_data_impl() are sleeping while holding the hash lock for their - * particular arc header. Thus, we must be careful to never sleep on a - * hash lock in this thread. This is to prevent the following deadlock: - * - * - Thread A sleeps on CV in arc_get_data_impl() holding hash lock "L", - * waiting for the reclaim thread to signal it. - * - * - arc_reclaim_thread() tries to acquire hash lock "L" using mutex_enter, - * fails, and goes to sleep forever. - * - * This possible deadlock is avoided by always acquiring a hash lock - * using mutex_tryenter() from arc_reclaim_thread(). + * Keep arc_size under arc_c by running arc_adjust which evicts data + * from the ARC. */ /* ARGSUSED */ -static void -arc_reclaim_thread(void *unused) +static int +arc_adjust_cb(void *arg, zthr_t *zthr) { - hrtime_t growtime = 0; - hrtime_t kmem_reap_time = 0; - callb_cpr_t cpr; + uint64_t evicted = 0; - CALLB_CPR_INIT(&cpr, &arc_reclaim_lock, callb_generic_cpr, FTAG); + /* Evict from cache */ + evicted = arc_adjust(); - mutex_enter(&arc_reclaim_lock); - while (!arc_reclaim_thread_exit) { - uint64_t evicted = 0; - + /* + * If evicted is zero, we couldn't evict anything + * via arc_adjust(). This could be due to hash lock + * collisions, but more likely due to the majority of + * arc buffers being unevictable. Therefore, even if + * arc_size is above arc_c, another pass is unlikely to + * be helpful and could potentially cause us to enter an + * infinite loop. Additionally, zthr_iscancelled() is + * checked here so that if the arc is shutting down, the + * broadcast will wake any remaining arc adjust waiters. + */ + mutex_enter(&arc_adjust_lock); + arc_adjust_needed = !zthr_iscancelled(arc_adjust_zthr) && + evicted > 0 && aggsum_compare(&arc_size, arc_c) > 0; + if (!arc_adjust_needed) { /* - * This is necessary in order for the mdb ::arc dcmd to - * show up to date information. Since the ::arc command - * does not call the kstat's update function, without - * this call, the command may show stale stats for the - * anon, mru, mru_ghost, mfu, and mfu_ghost lists. Even - * with this change, the data might be up to 1 second - * out of date; but that should suffice. The arc_state_t - * structures can be queried directly if more accurate - * information is needed. + * We're either no longer overflowing, or we + * can't evict anything more, so we should wake + * up any waiters. */ - if (arc_ksp != NULL) - arc_ksp->ks_update(arc_ksp, KSTAT_READ); + cv_broadcast(&arc_adjust_waiters_cv); + } + mutex_exit(&arc_adjust_lock); - mutex_exit(&arc_reclaim_lock); + return (0); +} +/* ARGSUSED */ +static boolean_t +arc_reap_cb_check(void *arg, zthr_t *zthr) +{ + int64_t free_memory = arc_available_memory(); + + /* + * If a kmem reap is already active, don't schedule more. We must + * check for this because kmem_cache_reap_soon() won't actually + * block on the cache being reaped (this is to prevent callers from + * becoming implicitly blocked by a system-wide kmem reap -- which, + * on a system with many, many full magazines, can take minutes). + */ + if (!kmem_cache_reap_active() && + free_memory < 0) { + arc_no_grow = B_TRUE; + arc_warm = B_TRUE; /* - * We call arc_adjust() before (possibly) calling - * arc_kmem_reap_now(), so that we can wake up - * arc_get_data_impl() sooner. + * Wait at least zfs_grow_retry (default 60) seconds + * before considering growing. */ - evicted = arc_adjust(); + arc_growtime = gethrtime() + SEC2NSEC(arc_grow_retry); + return (B_TRUE); + } else if (free_memory < arc_c >> arc_no_grow_shift) { + arc_no_grow = B_TRUE; + } else if (gethrtime() >= arc_growtime) { + arc_no_grow = B_FALSE; + } - int64_t free_memory = arc_available_memory(); - if (free_memory < 0) { - hrtime_t curtime = gethrtime(); - arc_no_grow = B_TRUE; - arc_warm = B_TRUE; + return (B_FALSE); +} - /* - * Wait at least zfs_grow_retry (default 60) seconds - * before considering growing. - */ - growtime = curtime + SEC2NSEC(arc_grow_retry); +/* + * Keep enough free memory in the system by reaping the ARC's kmem + * caches. To cause more slabs to be reapable, we may reduce the + * target size of the cache (arc_c), causing the arc_adjust_cb() + * to free more buffers. + */ +/* ARGSUSED */ +static int +arc_reap_cb(void *arg, zthr_t *zthr) +{ + int64_t free_memory; - /* - * Wait at least arc_kmem_cache_reap_retry_ms - * between arc_kmem_reap_now() calls. Without - * this check it is possible to end up in a - * situation where we spend lots of time - * reaping caches, while we're near arc_c_min. - */ - if (curtime >= kmem_reap_time) { - arc_kmem_reap_now(); - kmem_reap_time = gethrtime() + - MSEC2NSEC(arc_kmem_cache_reap_retry_ms); - } + /* + * Kick off asynchronous kmem_reap()'s of all our caches. + */ + arc_kmem_reap_soon(); - /* - * If we are still low on memory, shrink the ARC - * so that we have arc_shrink_min free space. - */ - free_memory = arc_available_memory(); + /* + * Wait at least arc_kmem_cache_reap_retry_ms between + * arc_kmem_reap_soon() calls. Without this check it is possible to + * end up in a situation where we spend lots of time reaping + * caches, while we're near arc_c_min. Waiting here also gives the + * subsequent free memory check a chance of finding that the + * asynchronous reap has already freed enough memory, and we don't + * need to call arc_reduce_target_size(). + */ + delay((hz * arc_kmem_cache_reap_retry_ms + 999) / 1000); - int64_t to_free = - (arc_c >> arc_shrink_shift) - free_memory; - if (to_free > 0) { + /* + * Reduce the target size as needed to maintain the amount of free + * memory in the system at a fraction of the arc_size (1/128th by + * default). If oversubscribed (free_memory < 0) then reduce the + * target arc_size by the deficit amount plus the fractional + * amount. If free memory is positive but less then the fractional + * amount, reduce by what is needed to hit the fractional amount. + */ + free_memory = arc_available_memory(); + + int64_t to_free = + (arc_c >> arc_shrink_shift) - free_memory; + if (to_free > 0) { #ifdef _KERNEL - to_free = MAX(to_free, ptob(needfree)); + to_free = MAX(to_free, ptob(needfree)); #endif - arc_shrink(to_free); - } - } else if (free_memory < arc_c >> arc_no_grow_shift) { - arc_no_grow = B_TRUE; - } else if (gethrtime() >= growtime) { - arc_no_grow = B_FALSE; - } - - mutex_enter(&arc_reclaim_lock); - - /* - * If evicted is zero, we couldn't evict anything via - * arc_adjust(). This could be due to hash lock - * collisions, but more likely due to the majority of - * arc buffers being unevictable. Therefore, even if - * arc_size is above arc_c, another pass is unlikely to - * be helpful and could potentially cause us to enter an - * infinite loop. - */ - if (aggsum_compare(&arc_size, arc_c) <= 0|| evicted == 0) { - /* - * We're either no longer overflowing, or we - * can't evict anything more, so we should wake - * up any threads before we go to sleep. - */ - cv_broadcast(&arc_reclaim_waiters_cv); - - /* - * Block until signaled, or after one second (we - * might need to perform arc_kmem_reap_now() - * even if we aren't being signalled) - */ - CALLB_CPR_SAFE_BEGIN(&cpr); - (void) cv_timedwait_hires(&arc_reclaim_thread_cv, - &arc_reclaim_lock, SEC2NSEC(1), MSEC2NSEC(1), 0); - CALLB_CPR_SAFE_END(&cpr, &arc_reclaim_lock); - } + arc_reduce_target_size(to_free); } - arc_reclaim_thread_exit = B_FALSE; - cv_broadcast(&arc_reclaim_thread_cv); - CALLB_CPR_EXIT(&cpr); /* drops arc_reclaim_lock */ - thread_exit(); + return (0); } /* @@ -4302,11 +4339,15 @@ arc_adapt(int bytes, arc_state_t *state) } ASSERT((int64_t)arc_p >= 0); + /* + * Wake reap thread if we do not have any available memory + */ if (arc_reclaim_needed()) { - cv_signal(&arc_reclaim_thread_cv); + zthr_wakeup(arc_reap_zthr); return; } + if (arc_no_grow) return; @@ -4410,7 +4451,7 @@ arc_get_data_impl(arc_buf_hdr_t *hdr, uint64_t size, v * overflowing; thus we don't use a while loop here. */ if (arc_is_overflowing()) { - mutex_enter(&arc_reclaim_lock); + mutex_enter(&arc_adjust_lock); /* * Now that we've acquired the lock, we may no longer be @@ -4424,11 +4465,12 @@ arc_get_data_impl(arc_buf_hdr_t *hdr, uint64_t size, v * shouldn't cause any harm. */ if (arc_is_overflowing()) { - cv_signal(&arc_reclaim_thread_cv); - cv_wait(&arc_reclaim_waiters_cv, &arc_reclaim_lock); + arc_adjust_needed = B_TRUE; + zthr_wakeup(arc_adjust_zthr); + (void) cv_wait(&arc_adjust_waiters_cv, + &arc_adjust_lock); } - - mutex_exit(&arc_reclaim_lock); + mutex_exit(&arc_adjust_lock); } VERIFY3U(hdr->b_type, ==, type); @@ -6081,11 +6123,9 @@ arc_init(void) #else uint64_t allmem = (physmem * PAGESIZE) / 2; #endif + mutex_init(&arc_adjust_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&arc_adjust_waiters_cv, NULL, CV_DEFAULT, NULL); - mutex_init(&arc_reclaim_lock, NULL, MUTEX_DEFAULT, NULL); - cv_init(&arc_reclaim_thread_cv, NULL, CV_DEFAULT, NULL); - cv_init(&arc_reclaim_waiters_cv, NULL, CV_DEFAULT, NULL); - /* Convert seconds to clock ticks */ arc_min_prefetch_lifespan = 1 * hz; @@ -6169,10 +6209,15 @@ arc_init(void) arc_c = arc_c_min; arc_state_init(); + + /* + * The arc must be "uninitialized", so that hdr_recl() (which is + * registered by buf_init()) will not access arc_reap_zthr before + * it is created. + */ + ASSERT(!arc_initialized); buf_init(); - arc_reclaim_thread_exit = B_FALSE; - arc_ksp = kstat_create("zfs", 0, "arcstats", "misc", KSTAT_TYPE_NAMED, sizeof (arc_stats) / sizeof (kstat_named_t), KSTAT_FLAG_VIRTUAL); @@ -6182,10 +6227,12 @@ arc_init(void) kstat_install(arc_ksp); } - (void) thread_create(NULL, 0, arc_reclaim_thread, NULL, 0, &p0, - TS_RUN, minclsyspri); + arc_adjust_zthr = zthr_create(arc_adjust_cb_check, + arc_adjust_cb, NULL); + arc_reap_zthr = zthr_create_timer(arc_reap_cb_check, + arc_reap_cb, NULL, SEC2NSEC(1)); - arc_dead = B_FALSE; + arc_initialized = B_TRUE; arc_warm = B_FALSE; /* @@ -6207,31 +6254,24 @@ arc_init(void) void arc_fini(void) { - mutex_enter(&arc_reclaim_lock); - arc_reclaim_thread_exit = B_TRUE; - /* - * The reclaim thread will set arc_reclaim_thread_exit back to - * B_FALSE when it is finished exiting; we're waiting for that. - */ - while (arc_reclaim_thread_exit) { - cv_signal(&arc_reclaim_thread_cv); - cv_wait(&arc_reclaim_thread_cv, &arc_reclaim_lock); - } - mutex_exit(&arc_reclaim_lock); - /* Use B_TRUE to ensure *all* buffers are evicted */ arc_flush(NULL, B_TRUE); - arc_dead = B_TRUE; + arc_initialized = B_FALSE; if (arc_ksp != NULL) { kstat_delete(arc_ksp); arc_ksp = NULL; } - mutex_destroy(&arc_reclaim_lock); - cv_destroy(&arc_reclaim_thread_cv); - cv_destroy(&arc_reclaim_waiters_cv); + (void) zthr_cancel(arc_adjust_zthr); + zthr_destroy(arc_adjust_zthr); + + (void) zthr_cancel(arc_reap_zthr); + zthr_destroy(arc_reap_zthr); + + mutex_destroy(&arc_adjust_lock); + cv_destroy(&arc_adjust_waiters_cv); arc_state_fini(); buf_fini(); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zthr.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zthr.h Mon Jul 30 19:41:54 2018 (r336929) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zthr.h Mon Jul 30 19:44:14 2018 (r336930) @@ -29,6 +29,7 @@ struct zthr { kmutex_t zthr_lock; kcondvar_t zthr_cv; boolean_t zthr_cancel; + hrtime_t zthr_wait_time; zthr_checkfunc_t *zthr_checkfunc; zthr_func_t *zthr_func; @@ -38,6 +39,9 @@ struct zthr { extern zthr_t *zthr_create(zthr_checkfunc_t checkfunc, zthr_func_t *func, void *arg); +extern zthr_t *zthr_create_timer(zthr_checkfunc_t *checkfunc, + zthr_func_t *func, void *arg, hrtime_t nano_wait); + extern void zthr_exit(zthr_t *t, int rc); extern void zthr_destroy(zthr_t *t); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zthr.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zthr.c Mon Jul 30 19:41:54 2018 (r336929) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zthr.c Mon Jul 30 19:44:14 2018 (r336930) @@ -47,6 +47,10 @@ * 3] When the zthr is done, it changes the indicator to stopped, allowing * a new cycle to start. * + * Besides being awakened by other threads, a zthr can be configured + * during creation to wakeup on it's own after a specified interval + * [see zthr_create_timer()]. + * * == ZTHR creation * * Every zthr needs three inputs to start running: @@ -74,6 +78,9 @@ * * To start a zthr: * zthr_t *zthr_pointer = zthr_create(checkfunc, func, args); + * or + * zthr_t *zthr_pointer = zthr_create_timer(checkfunc, func, + * args, max_sleep); * * After that you should be able to wakeup, cancel, and resume the * zthr from another thread using zthr_pointer. @@ -189,7 +196,13 @@ zthr_procedure(void *arg) mutex_enter(&t->zthr_lock); } else { /* go to sleep */ - cv_wait(&t->zthr_cv, &t->zthr_lock); + if (t->zthr_wait_time == 0) { + cv_wait(&t->zthr_cv, &t->zthr_lock); + } else { + (void) cv_timedwait_hires(&t->zthr_cv, + &t->zthr_lock, t->zthr_wait_time, + MSEC2NSEC(1), 0); + } } } mutex_exit(&t->zthr_lock); @@ -200,6 +213,18 @@ zthr_procedure(void *arg) zthr_t * zthr_create(zthr_checkfunc_t *checkfunc, zthr_func_t *func, void *arg) { + return (zthr_create_timer(checkfunc, func, arg, (hrtime_t)0)); +} + +/* + * Create a zthr with specified maximum sleep time. If the time + * in sleeping state exceeds max_sleep, a wakeup(do the check and + * start working if required) will be triggered. + */ +zthr_t * +zthr_create_timer(zthr_checkfunc_t *checkfunc, zthr_func_t *func, + void *arg, hrtime_t max_sleep) +{ zthr_t *t = kmem_zalloc(sizeof (*t), KM_SLEEP); mutex_init(&t->zthr_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&t->zthr_cv, NULL, CV_DEFAULT, NULL); @@ -208,6 +233,7 @@ zthr_create(zthr_checkfunc_t *checkfunc, zthr_func_t * t->zthr_checkfunc = checkfunc; t->zthr_func = func; t->zthr_arg = arg; + t->zthr_wait_time = max_sleep; t->zthr_thread = thread_create(NULL, 0, zthr_procedure, t, 0, &p0, TS_RUN, minclsyspri); From owner-svn-src-all@freebsd.org Mon Jul 30 20:00:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05BA5105F9D9; Mon, 30 Jul 2018 20:00:57 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ABF4E81FC3; Mon, 30 Jul 2018 20:00:56 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8CF7D1201; Mon, 30 Jul 2018 20:00:56 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UK0uIk049965; Mon, 30 Jul 2018 20:00:56 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UK0usQ049964; Mon, 30 Jul 2018 20:00:56 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201807302000.w6UK0usQ049964@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Mon, 30 Jul 2018 20:00:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336931 - head/sys/dev/sound/pci/hda X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/dev/sound/pci/hda X-SVN-Commit-Revision: 336931 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 20:00:57 -0000 Author: jhibbits Date: Mon Jul 30 20:00:56 2018 New Revision: 336931 URL: https://svnweb.freebsd.org/changeset/base/336931 Log: snd_hda: Make codec control path endian safe The CORB and RIRB buffers exist in DMA memory, but the device reads them as little-endian only. Read and write as LE into the DMA memory block, to work on BE platforms. Modified: head/sys/dev/sound/pci/hda/hdac.c Modified: head/sys/dev/sound/pci/hda/hdac.c ============================================================================== --- head/sys/dev/sound/pci/hda/hdac.c Mon Jul 30 19:44:14 2018 (r336930) +++ head/sys/dev/sound/pci/hda/hdac.c Mon Jul 30 20:00:56 2018 (r336931) @@ -41,6 +41,7 @@ #include #include +#include #include #include @@ -900,7 +901,7 @@ hdac_rirb_flush(struct hdac_softc *sc) { struct hdac_rirb *rirb_base, *rirb; nid_t cad; - uint32_t resp; + uint32_t resp, resp_ex; uint8_t rirbwp; int ret; @@ -916,9 +917,10 @@ hdac_rirb_flush(struct hdac_softc *sc) sc->rirb_rp++; sc->rirb_rp %= sc->rirb_size; rirb = &rirb_base[sc->rirb_rp]; - cad = HDAC_RIRB_RESPONSE_EX_SDATA_IN(rirb->response_ex); - resp = rirb->response; - if (rirb->response_ex & HDAC_RIRB_RESPONSE_EX_UNSOLICITED) { + resp = le32toh(rirb->response); + resp_ex = le32toh(rirb->response_ex); + cad = HDAC_RIRB_RESPONSE_EX_SDATA_IN(resp_ex); + if (resp_ex & HDAC_RIRB_RESPONSE_EX_UNSOLICITED) { sc->unsolq[sc->unsolq_wp++] = resp; sc->unsolq_wp %= HDAC_UNSOLQ_MAX; sc->unsolq[sc->unsolq_wp++] = cad; @@ -985,7 +987,7 @@ hdac_send_command(struct hdac_softc *sc, nid_t cad, ui bus_dmamap_sync(sc->corb_dma.dma_tag, sc->corb_dma.dma_map, BUS_DMASYNC_PREWRITE); #endif - corb[sc->corb_wp] = verb; + corb[sc->corb_wp] = htole32(verb); #if 0 bus_dmamap_sync(sc->corb_dma.dma_tag, sc->corb_dma.dma_map, BUS_DMASYNC_POSTWRITE); From owner-svn-src-all@freebsd.org Mon Jul 30 20:13:39 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4EBCF105FEF0; Mon, 30 Jul 2018 20:13:39 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04ABE829FB; Mon, 30 Jul 2018 20:13:39 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D9EF01421; Mon, 30 Jul 2018 20:13:38 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UKDcMw059710; Mon, 30 Jul 2018 20:13:38 GMT (envelope-from tuexen@FreeBSD.org) Received: (from tuexen@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UKDc1F059707; Mon, 30 Jul 2018 20:13:38 GMT (envelope-from tuexen@FreeBSD.org) Message-Id: <201807302013.w6UKDc1F059707@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tuexen set sender to tuexen@FreeBSD.org using -f From: Michael Tuexen Date: Mon, 30 Jul 2018 20:13:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336932 - head/sys/netinet X-SVN-Group: head X-SVN-Commit-Author: tuexen X-SVN-Commit-Paths: head/sys/netinet X-SVN-Commit-Revision: 336932 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 20:13:39 -0000 Author: tuexen Date: Mon Jul 30 20:13:38 2018 New Revision: 336932 URL: https://svnweb.freebsd.org/changeset/base/336932 Log: Add missing send/recv dtrace probes for TCP. These missing probe are mostly in the syncache and timewait code. Reviewed by: markj@, rrs@ MFC after: 1 month Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D16369 Modified: head/sys/netinet/tcp_input.c head/sys/netinet/tcp_syncache.c head/sys/netinet/tcp_timewait.c Modified: head/sys/netinet/tcp_input.c ============================================================================== --- head/sys/netinet/tcp_input.c Mon Jul 30 20:00:56 2018 (r336931) +++ head/sys/netinet/tcp_input.c Mon Jul 30 20:13:38 2018 (r336932) @@ -1145,6 +1145,7 @@ tfo_socket_result: * contains. tcp_do_segment() consumes * the mbuf chain and unlocks the inpcb. */ + TCP_PROBE5(receive, NULL, tp, m, tp, th); tp->t_fb->tfb_tcp_do_segment(m, th, so, tp, drop_hdrlen, tlen, iptos); if (ti_locked == TI_RLOCKED) Modified: head/sys/netinet/tcp_syncache.c ============================================================================== --- head/sys/netinet/tcp_syncache.c Mon Jul 30 20:00:56 2018 (r336931) +++ head/sys/netinet/tcp_syncache.c Mon Jul 30 20:13:38 2018 (r336932) @@ -69,6 +69,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -1400,6 +1401,7 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *t */ mac_syncache_destroy(&maclabel); #endif + TCP_PROBE5(receive, NULL, NULL, m, NULL, th); /* Retransmit SYN|ACK and reset retransmit count. */ if ((s = tcp_log_addrs(&sc->sc_inc, th, NULL, NULL))) { log(LOG_DEBUG, "%s; %s: Received duplicate SYN, " @@ -1414,7 +1416,7 @@ syncache_add(struct in_conninfo *inc, struct tcpopt *t TCPSTAT_INC(tcps_sndtotal); } SCH_UNLOCK(sch); - goto done; + goto donenoprobe; } if (tfo_cookie_valid) { @@ -1566,6 +1568,7 @@ skip_alloc: goto tfo_expanded; } + TCP_PROBE5(receive, NULL, NULL, m, NULL, th); /* * Do a standard 3-way handshake. */ @@ -1581,8 +1584,11 @@ skip_alloc: syncache_free(sc); TCPSTAT_INC(tcps_sc_dropped); } + goto donenoprobe; done: + TCP_PROBE5(receive, NULL, NULL, m, NULL, th); +donenoprobe: if (m) { *lsop = NULL; m_freem(m); @@ -1793,6 +1799,7 @@ syncache_respond(struct syncache *sc, struct syncache_ return (error); } #endif + TCP_PROBE5(send, NULL, NULL, ip6, NULL, th); error = ip6_output(m, NULL, NULL, 0, NULL, NULL, NULL); } #endif @@ -1813,6 +1820,7 @@ syncache_respond(struct syncache *sc, struct syncache_ return (error); } #endif + TCP_PROBE5(send, NULL, NULL, ip, NULL, th); error = ip_output(m, sc->sc_ipopts, NULL, 0, NULL, NULL); } #endif Modified: head/sys/netinet/tcp_timewait.c ============================================================================== --- head/sys/netinet/tcp_timewait.c Mon Jul 30 20:00:56 2018 (r336931) +++ head/sys/netinet/tcp_timewait.c Mon Jul 30 20:13:38 2018 (r336932) @@ -63,6 +63,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -450,9 +451,14 @@ tcp_twcheck(struct inpcb *inp, struct tcpopt *to __unu * Acknowledge the segment if it has data or is not a duplicate ACK. */ if (thflags != TH_ACK || tlen != 0 || - th->th_seq != tw->rcv_nxt || th->th_ack != tw->snd_nxt) + th->th_seq != tw->rcv_nxt || th->th_ack != tw->snd_nxt) { + TCP_PROBE5(receive, NULL, NULL, m, NULL, th); tcp_twrespond(tw, TH_ACK); + goto dropnoprobe; + } drop: + TCP_PROBE5(receive, NULL, NULL, m, NULL, th); +dropnoprobe: INP_WUNLOCK(inp); m_freem(m); return (0); @@ -598,6 +604,7 @@ tcp_twrespond(struct tcptw *tw, int flags) th->th_sum = in6_cksum_pseudo(ip6, sizeof(struct tcphdr) + optlen, IPPROTO_TCP, 0); ip6->ip6_hlim = in6_selecthlim(inp, NULL); + TCP_PROBE5(send, NULL, NULL, ip6, NULL, th); error = ip6_output(m, inp->in6p_outputopts, NULL, (tw->tw_so_options & SO_DONTROUTE), NULL, NULL, inp); } @@ -613,6 +620,7 @@ tcp_twrespond(struct tcptw *tw, int flags) ip->ip_len = htons(m->m_pkthdr.len); if (V_path_mtu_discovery) ip->ip_off |= htons(IP_DF); + TCP_PROBE5(send, NULL, NULL, ip, NULL, th); error = ip_output(m, inp->inp_options, NULL, ((tw->tw_so_options & SO_DONTROUTE) ? IP_ROUTETOIF : 0), NULL, inp); From owner-svn-src-all@freebsd.org Mon Jul 30 20:25:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B0B2106035F; Mon, 30 Jul 2018 20:25:33 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 115B083241; Mon, 30 Jul 2018 20:25:33 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E7DB215C9; Mon, 30 Jul 2018 20:25:32 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UKPWOt065546; Mon, 30 Jul 2018 20:25:32 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UKPWEp065545; Mon, 30 Jul 2018 20:25:32 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807302025.w6UKPWEp065545@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Mon, 30 Jul 2018 20:25:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336933 - head/sys/fs/nfsclient X-SVN-Group: head X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: head/sys/fs/nfsclient X-SVN-Commit-Revision: 336933 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 20:25:33 -0000 Author: rmacklem Date: Mon Jul 30 20:25:32 2018 New Revision: 336933 URL: https://svnweb.freebsd.org/changeset/base/336933 Log: Silence newer gcc warnings. Newer versions of gcc generate "set, but not used" warnings. Add __unused macros to silence these warnings. Although the variables are not being used, they are values parsed from arguments to callback RPCs that might be needed in the future. Requested by: mmacy Modified: head/sys/fs/nfsclient/nfs_clstate.c Modified: head/sys/fs/nfsclient/nfs_clstate.c ============================================================================== --- head/sys/fs/nfsclient/nfs_clstate.c Mon Jul 30 20:13:38 2018 (r336932) +++ head/sys/fs/nfsclient/nfs_clstate.c Mon Jul 30 20:25:32 2018 (r336933) @@ -3237,7 +3237,7 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) u_int32_t *tl; struct nfsclclient *clp; struct nfscldeleg *dp = NULL; - int numops, taglen = -1, error = 0, trunc; + int numops, taglen = -1, error = 0, trunc __unused; u_int32_t minorvers = 0, retops = 0, *retopsp = NULL, *repp, cbident; u_char tag[NFSV4_SMALLSTR + 1], *tagstr; vnode_t vp = NULL; @@ -3247,7 +3247,7 @@ nfscl_docb(struct nfsrv_descript *nd, NFSPROC_T *p) mount_t mp; nfsattrbit_t attrbits, rattrbits; nfsv4stateid_t stateid; - uint32_t seqid, slotid = 0, highslot, cachethis; + uint32_t seqid, slotid = 0, highslot, cachethis __unused; uint8_t sessionid[NFSX_V4SESSIONID]; struct mbuf *rep; struct nfscllayout *lyp; From owner-svn-src-all@freebsd.org Mon Jul 30 20:35:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA65B1060842; Mon, 30 Jul 2018 20:35:51 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F8DF83826; Mon, 30 Jul 2018 20:35:51 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4341A1771; Mon, 30 Jul 2018 20:35:51 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UKZpEY070690; Mon, 30 Jul 2018 20:35:51 GMT (envelope-from tuexen@FreeBSD.org) Received: (from tuexen@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UKZoFq070687; Mon, 30 Jul 2018 20:35:50 GMT (envelope-from tuexen@FreeBSD.org) Message-Id: <201807302035.w6UKZoFq070687@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tuexen set sender to tuexen@FreeBSD.org using -f From: Michael Tuexen Date: Mon, 30 Jul 2018 20:35:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336934 - head/sys/netinet X-SVN-Group: head X-SVN-Commit-Author: tuexen X-SVN-Commit-Paths: head/sys/netinet X-SVN-Commit-Revision: 336934 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 20:35:52 -0000 Author: tuexen Date: Mon Jul 30 20:35:50 2018 New Revision: 336934 URL: https://svnweb.freebsd.org/changeset/base/336934 Log: Fix some TCP fast open issues. The following issues are fixed: * Whenever a TCP server with TCP fast open enabled, calls accept(), recv(), send(), and close() before the TCP-ACK segment has been received, the TCP connection is just dropped and the reception of the TCP-ACK segment triggers the sending of a TCP-RST segment. * Whenever a TCP server with TCP fast open enabled, calls accept(), recv(), send(), send(), and close() before the TCP-ACK segment has been received, the first byte provided in the second send call is not transferred. * Whenever a TCP client with TCP fast open enabled calls sendto() followed by close() the TCP connection is just dropped. Reviewed by: jtl@, kbowling@, rrs@ Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D16485 Modified: head/sys/netinet/tcp_input.c head/sys/netinet/tcp_output.c head/sys/netinet/tcp_usrreq.c Modified: head/sys/netinet/tcp_input.c ============================================================================== --- head/sys/netinet/tcp_input.c Mon Jul 30 20:25:32 2018 (r336933) +++ head/sys/netinet/tcp_input.c Mon Jul 30 20:35:50 2018 (r336934) @@ -2407,6 +2407,16 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, stru * SYN-RECEIVED* -> FIN-WAIT-1 */ tp->t_starttime = ticks; + if (IS_FASTOPEN(tp->t_flags) && tp->t_tfo_pending) { + tcp_fastopen_decrement_counter(tp->t_tfo_pending); + tp->t_tfo_pending = NULL; + + /* + * Account for the ACK of our SYN prior to + * regular ACK processing below. + */ + tp->snd_una++; + } if (tp->t_flags & TF_NEEDFIN) { tcp_state_change(tp, TCPS_FIN_WAIT_1); tp->t_flags &= ~TF_NEEDFIN; @@ -2414,16 +2424,6 @@ tcp_do_segment(struct mbuf *m, struct tcphdr *th, stru tcp_state_change(tp, TCPS_ESTABLISHED); TCP_PROBE5(accept__established, NULL, tp, m, tp, th); - if (IS_FASTOPEN(tp->t_flags) && tp->t_tfo_pending) { - tcp_fastopen_decrement_counter(tp->t_tfo_pending); - tp->t_tfo_pending = NULL; - - /* - * Account for the ACK of our SYN prior to - * regular ACK processing below. - */ - tp->snd_una++; - } /* * TFO connections call cc_conn_init() during SYN * processing. Calling it again here for such Modified: head/sys/netinet/tcp_output.c ============================================================================== --- head/sys/netinet/tcp_output.c Mon Jul 30 20:25:32 2018 (r336933) +++ head/sys/netinet/tcp_output.c Mon Jul 30 20:35:50 2018 (r336934) @@ -228,13 +228,15 @@ tcp_output(struct tcpcb *tp) #endif /* - * For TFO connections in SYN_RECEIVED, only allow the initial - * SYN|ACK and those sent by the retransmit timer. + * For TFO connections in SYN_SENT or SYN_RECEIVED, + * only allow the initial SYN or SYN|ACK and those sent + * by the retransmit timer. */ if (IS_FASTOPEN(tp->t_flags) && - (tp->t_state == TCPS_SYN_RECEIVED) && - SEQ_GT(tp->snd_max, tp->snd_una) && /* initial SYN|ACK sent */ - (tp->snd_nxt != tp->snd_una)) /* not a retransmit */ + ((tp->t_state == TCPS_SYN_SENT) || + (tp->t_state == TCPS_SYN_RECEIVED)) && + SEQ_GT(tp->snd_max, tp->snd_una) && /* initial SYN or SYN|ACK sent */ + (tp->snd_nxt != tp->snd_una)) /* not a retransmit */ return (0); /* Modified: head/sys/netinet/tcp_usrreq.c ============================================================================== --- head/sys/netinet/tcp_usrreq.c Mon Jul 30 20:25:32 2018 (r336933) +++ head/sys/netinet/tcp_usrreq.c Mon Jul 30 20:35:50 2018 (r336934) @@ -2113,7 +2113,8 @@ tcp_disconnect(struct tcpcb *tp) * Neither tcp_close() nor tcp_drop() should return NULL, as the * socket is still open. */ - if (tp->t_state < TCPS_ESTABLISHED) { + if (tp->t_state < TCPS_ESTABLISHED && + !(tp->t_state > TCPS_LISTEN && IS_FASTOPEN(tp->t_flags))) { tp = tcp_close(tp); KASSERT(tp != NULL, ("tcp_disconnect: tcp_close() returned NULL")); From owner-svn-src-all@freebsd.org Mon Jul 30 20:36:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B65910608B7; Mon, 30 Jul 2018 20:36:52 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D2FB2839B4; Mon, 30 Jul 2018 20:36:51 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B5755178D; Mon, 30 Jul 2018 20:36:51 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UKapQ4070774; Mon, 30 Jul 2018 20:36:51 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UKapAF070772; Mon, 30 Jul 2018 20:36:51 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807302036.w6UKapAF070772@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Mon, 30 Jul 2018 20:36:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336935 - head/sys/fs/msdosfs X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/sys/fs/msdosfs X-SVN-Commit-Revision: 336935 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 20:36:52 -0000 Author: emaste Date: Mon Jul 30 20:36:51 2018 New Revision: 336935 URL: https://svnweb.freebsd.org/changeset/base/336935 Log: msdosfs: use same max filesize #define as NetBSD and move to header For use by makefs msdosfs support. Obtained from: NetBSD denode.h 1.6 Sponsored by: The FreeBSD Foundation Modified: head/sys/fs/msdosfs/denode.h head/sys/fs/msdosfs/msdosfs_vnops.c Modified: head/sys/fs/msdosfs/denode.h ============================================================================== --- head/sys/fs/msdosfs/denode.h Mon Jul 30 20:35:50 2018 (r336934) +++ head/sys/fs/msdosfs/denode.h Mon Jul 30 20:36:51 2018 (r336935) @@ -174,6 +174,8 @@ struct denode { #define DE_MODIFIED 0x0020 /* Denode has been modified */ #define DE_RENAME 0x0040 /* Denode is in the process of being renamed */ +/* Maximum size of a file on a FAT filesystem */ +#define MSDOSFS_FILESIZE_MAX 0xFFFFFFFFLL /* * Transfer directory entries between internal and external form. Modified: head/sys/fs/msdosfs/msdosfs_vnops.c ============================================================================== --- head/sys/fs/msdosfs/msdosfs_vnops.c Mon Jul 30 20:35:50 2018 (r336934) +++ head/sys/fs/msdosfs/msdosfs_vnops.c Mon Jul 30 20:36:51 2018 (r336935) @@ -79,8 +79,6 @@ #include #include -#define DOS_FILESIZE_MAX 0xffffffff - /* * Prototypes for MSDOSFS vnode operations */ @@ -548,7 +546,7 @@ msdosfs_read(struct vop_read_args *ap) * The caller is supposed to ensure that * uio->uio_offset >= 0 and uio->uio_resid >= 0. * We don't need to check for large offsets as in ffs because - * dep->de_FileSize <= DOS_FILESIZE_MAX < OFF_MAX, so large + * dep->de_FileSize <= MSDOSFS_FILESIZE_MAX < OFF_MAX, so large * offsets cannot cause overflow even in theory. */ @@ -663,7 +661,7 @@ msdosfs_write(struct vop_write_args *ap) * The caller is supposed to ensure that * uio->uio_offset >= 0 and uio->uio_resid >= 0. */ - if ((uoff_t)uio->uio_offset + uio->uio_resid > DOS_FILESIZE_MAX) + if ((uoff_t)uio->uio_offset + uio->uio_resid > MSDOSFS_FILESIZE_MAX) return (EFBIG); /* From owner-svn-src-all@freebsd.org Mon Jul 30 20:59:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F27B61060D71; Mon, 30 Jul 2018 20:59:58 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A6262841FE; Mon, 30 Jul 2018 20:59:58 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 71EEC1AC2; Mon, 30 Jul 2018 20:59:58 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UKxw7w080979; Mon, 30 Jul 2018 20:59:58 GMT (envelope-from gjb@FreeBSD.org) Received: (from gjb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UKxwk5080978; Mon, 30 Jul 2018 20:59:58 GMT (envelope-from gjb@FreeBSD.org) Message-Id: <201807302059.w6UKxwk5080978@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gjb set sender to gjb@FreeBSD.org using -f From: Glen Barber Date: Mon, 30 Jul 2018 20:59:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336936 - stable/11/release/tools X-SVN-Group: stable-11 X-SVN-Commit-Author: gjb X-SVN-Commit-Paths: stable/11/release/tools X-SVN-Commit-Revision: 336936 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 20:59:59 -0000 Author: gjb Date: Mon Jul 30 20:59:58 2018 New Revision: 336936 URL: https://svnweb.freebsd.org/changeset/base/336936 Log: As part of r336741, BOOTFILES needs special handling when cross building on stable/11, where the path should be: /usr/obj/TARGET.TARGET_ARCH/[...] The change in r336741, which was a direct commit to stable/11 was not complete enough to evaluate where BOOTFILES should be. This, as was r336741, is a direct commit to stable/11. Tested with: stable/11 BANANAPI build Sponsored by: The FreeBSD Foundation Modified: stable/11/release/tools/arm.subr Modified: stable/11/release/tools/arm.subr ============================================================================== --- stable/11/release/tools/arm.subr Mon Jul 30 20:36:51 2018 (r336935) +++ stable/11/release/tools/arm.subr Mon Jul 30 20:59:58 2018 (r336936) @@ -142,14 +142,13 @@ arm_install_boot() { ${FATMOUNT}/ubldr.bin fi - # Ensure the correct .OBJDIR exists. - chroot ${CHROOTDIR} env TARGET=${EMBEDDED_TARGET} \ - TARGET_ARCH=${EMBEDDED_TARGET_ARCH} WITH_UNIFIED_OBJDIR=yes \ - make -C ${WORLDDIR}/stand obj - BOOTFILES="$(chroot ${CHROOTDIR} \ - env TARGET=${EMBEDDED_TARGET} TARGET_ARCH=${EMBEDDED_TARGET_ARCH} \ - WITH_UNIFIED_OBJDIR=yes \ - make -C ${WORLDDIR}/stand -V .OBJDIR)" + # Ensure the correct .OBJDIR is used for BOOTFILES. + _OBJDIR="$(chroot ${CHROOTDIR} make -C /usr/src -V .OBJDIR)" + if [ -d "/${CHROOTDIR}/${_OBJDIR%%/usr/src}/${EMBEDDED_TARGET}.${EMBEDDED_TARGET_ARCH}/usr/src/stand" ]; then + BOOTFILES=/${_OBJDIR%%/usr/src}/${EMBEDDED_TARGET}.${EMBEDDED_TARGET_ARCH}/usr/src/stand + else + BOOTFILES=/${_OBJDIR}/stand + fi BOOTFILES="$(chroot ${CHROOTDIR} realpath ${BOOTFILES})" chroot ${CHROOTDIR} mkdir -p ${FATMOUNT}/EFI/BOOT From owner-svn-src-all@freebsd.org Mon Jul 30 21:08:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5B0F1060FA7; Mon, 30 Jul 2018 21:08:18 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x243.google.com (mail-lj1-x243.google.com [IPv6:2a00:1450:4864:20::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09F1484676; Mon, 30 Jul 2018 21:08:18 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x243.google.com with SMTP id r13-v6so11767705ljg.10; Mon, 30 Jul 2018 14:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=xvVqwBJruo1DvvT7BLsDWhr6evA4Gp9w25R7IBWEios=; b=nnYdOo1F35HQrHmzLKPvi2zoZM4EJ7Tr6vKcNBbPpk1UO+3Wg5+frKq8Cqp6/3nUGg 1Vahhhq+hlTPXYqrE3kNk30LvWKrKYxmaiWJjYQwtsZLD2w8opNzF8bvPiPQdqnu758O 1PFH/AhY6YRfeSjktXVLvYcHpRtnKLvVpFqg06AbkTVF+t2wfB0OIs4mclPDxQW0IQdu JETEvKGC3EbwkG8Qn2/WWVcHbV4aCwaBjN4Gki5I/2vK41MiTjcU2orR7T69D4RcXSmI lHQdESWJV5GdBMFV2f3tN3bpxPZgYB0Uei7Av7dD/6o7S0itpyHvriTv3jLK1Jgpw/6U m0BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=xvVqwBJruo1DvvT7BLsDWhr6evA4Gp9w25R7IBWEios=; b=ilM1zhIVs0MyDMdhiMCTVMcL0pnUjZ6st1jWeJK9KIDCufJzxosBDfNfxGYz5AtV78 1m+XSEM+60Do8NnBNyQfaphrXhooa4JsEW6+hYzyVv5/R/D6JPWKioxIQwRYzL592oro NuRmZ2r3Oo7iu38u/uOvKWmZntE+uvcDdLrbe6T6Qa/Gi3eta8aJT6kmHnYqCEAiWByh dlnKJhYmqmY7nwl0bAHhU8xihBzfxf6OO/MpSdjWLPTiDGUYYHYlaeITA2rENRH4V7aP AdIUMZDhcYqL34P9q68PrGaFPebMzObMAC0lnkvNlYlmc0XPFPpE9RZ6MpJ81ICJX432 6QhA== X-Gm-Message-State: AOUpUlHwIMHrOf8OD5FHf7dy5OJsqXC5kPNv7V8uV0W+oUzdGuhq6L6W zto3lyKVPhqfqG0sWXKd/I5vQ2+b97GqKLbc0KUIORAm X-Google-Smtp-Source: AAOMgpce+ZUjB58OIpxJBSebtZ8Nltuo/s5QDfJq8M9EHqnBuuwZUbFV6VFUNGPEu3Ic5jr9Ut08OjF9ISWIg8h3Ros= X-Received: by 2002:a2e:87da:: with SMTP id v26-v6mr13736450ljj.69.1532984896367; Mon, 30 Jul 2018 14:08:16 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:7851:0:0:0:0:0 with HTTP; Mon, 30 Jul 2018 14:08:15 -0700 (PDT) In-Reply-To: <201807231611.w6NGB3gh074167@repo.freebsd.org> References: <201807231611.w6NGB3gh074167@repo.freebsd.org> From: Alan Somers Date: Mon, 30 Jul 2018 15:08:15 -0600 X-Google-Sender-Auth: bqAhdT8maF6RznD78MniLyFDZLI Message-ID: Subject: Re: svn commit: r336640 - head/share/mk To: Brad Davis Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:08:18 -0000 On Mon, Jul 23, 2018 at 10:11 AM, Brad Davis wrote: > Author: brd > Date: Mon Jul 23 16:11:03 2018 > New Revision: 336640 > URL: https://svnweb.freebsd.org/changeset/base/336640 > > Log: > Add the initial DIRS infrastructure for creating directories with the > necessary owner, group, mode and flags. > > Approved by: bapt (mentor) > Differential Revision: https://reviews.freebsd.org/D16405 > > Added: > head/share/mk/bsd.dirs.mk (contents, props changed) > Modified: > head/share/mk/bsd.README > This change is causing loads of console spam during mergemaster. $ sudo mergemaster -m $PWD *** Creating the temporary root environment in /var/tmp/temproot *** /var/tmp/temproot ready for use *** Creating and populating directory structure in /var/tmp/temproot make[4]: "/usr/home/somers/freebsd/base/head/share/mk/bsd.dirs.mk" line 29: warning: duplicate script for target "/var/tmp/temproot/etc/periodic/daily" ignored make[4]: "/usr/home/somers/freebsd/base/head/share/mk/bsd.dirs.mk" line 31: warning: using previous script for "/var/tmp/temproot/etc/periodic/daily" defined here make[4]: "/usr/home/somers/freebsd/base/head/share/mk/bsd.dirs.mk" line 31: warning: duplicate script for target "/var/tmp/temproot/etc/periodic/daily" ignored make[4]: "/usr/home/somers/freebsd/base/head/share/mk/bsd.dirs.mk" line 31: warning: using previous script for "/var/tmp/temproot/etc/periodic/daily" defined here ... From owner-svn-src-all@freebsd.org Mon Jul 30 21:13:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B6F310613D1; Mon, 30 Jul 2018 21:13:43 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B44C584D35; Mon, 30 Jul 2018 21:13:42 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 958751E20; Mon, 30 Jul 2018 21:13:42 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6ULDgmn091059; Mon, 30 Jul 2018 21:13:42 GMT (envelope-from tuexen@FreeBSD.org) Received: (from tuexen@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6ULDgHQ091058; Mon, 30 Jul 2018 21:13:42 GMT (envelope-from tuexen@FreeBSD.org) Message-Id: <201807302113.w6ULDgHQ091058@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tuexen set sender to tuexen@FreeBSD.org using -f From: Michael Tuexen Date: Mon, 30 Jul 2018 21:13:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336937 - head/sys/netinet X-SVN-Group: head X-SVN-Commit-Author: tuexen X-SVN-Commit-Paths: head/sys/netinet X-SVN-Commit-Revision: 336937 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:13:43 -0000 Author: tuexen Date: Mon Jul 30 21:13:42 2018 New Revision: 336937 URL: https://svnweb.freebsd.org/changeset/base/336937 Log: Send consistent SEG.WIN when using timewait codepath for TCP. When sending TCP segments from the timewait code path, a stored value of the last sent window is used. Use the same code for computing this in the timewait code path as in the main code path used in tcp_output() to avoiv inconsistencies. Reviewed by: rrs@ MFC after: 1 month Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D16503 Modified: head/sys/netinet/tcp_timewait.c Modified: head/sys/netinet/tcp_timewait.c ============================================================================== --- head/sys/netinet/tcp_timewait.c Mon Jul 30 20:59:58 2018 (r336936) +++ head/sys/netinet/tcp_timewait.c Mon Jul 30 21:13:42 2018 (r336937) @@ -230,6 +230,7 @@ tcp_twstart(struct tcpcb *tp) struct tcptw twlocal, *tw; struct inpcb *inp = tp->t_inpcb; struct socket *so; + uint32_t recwin; bool acknow, local; #ifdef INET6 bool isipv6 = inp->inp_inc.inc_flags & INC_ISIPV6; @@ -292,10 +293,16 @@ tcp_twstart(struct tcpcb *tp) /* * Recover last window size sent. */ - if (SEQ_GT(tp->rcv_adv, tp->rcv_nxt)) - tw->last_win = (tp->rcv_adv - tp->rcv_nxt) >> tp->rcv_scale; - else - tw->last_win = 0; + so = inp->inp_socket; + recwin = lmin(lmax(sbspace(&so->so_rcv), 0), + (long)TCP_MAXWIN << tp->rcv_scale); + if (recwin < (so->so_rcv.sb_hiwat / 4) && + recwin < tp->t_maxseg) + recwin = 0; + if (SEQ_GT(tp->rcv_adv, tp->rcv_nxt) && + recwin < (tp->rcv_adv - tp->rcv_nxt)) + recwin = (tp->rcv_adv - tp->rcv_nxt); + tw->last_win = htons((u_short)(recwin >> tp->rcv_scale)); /* * Set t_recent if timestamps are used on the connection. @@ -332,7 +339,6 @@ tcp_twstart(struct tcpcb *tp) * and might not be needed here any longer. */ tcp_discardcb(tp); - so = inp->inp_socket; soisdisconnected(so); tw->tw_so_options = so->so_options; inp->inp_flags |= INP_TIMEWAIT; From owner-svn-src-all@freebsd.org Mon Jul 30 21:24:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A8B71061764; Mon, 30 Jul 2018 21:24:29 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E72385329; Mon, 30 Jul 2018 21:24:29 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 202751FE4; Mon, 30 Jul 2018 21:24:29 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6ULOS8n096079; Mon, 30 Jul 2018 21:24:28 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6ULOSsT096076; Mon, 30 Jul 2018 21:24:28 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807302124.w6ULOSsT096076@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Mon, 30 Jul 2018 21:24:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336938 - stable/11/usr.sbin/config X-SVN-Group: stable-11 X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: stable/11/usr.sbin/config X-SVN-Commit-Revision: 336938 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:24:29 -0000 Author: kevans Date: Mon Jul 30 21:24:27 2018 New Revision: 336938 URL: https://svnweb.freebsd.org/changeset/base/336938 Log: MFC r307967,324082,325955: config(8) fixes r307967: Allow config to be compiled from another source directory, such as one for building tools. This boils down to replacing ${.CURDIR} with ${SRCDIR}, where the latter is the directory in which this makefile lives. Also allow overriding where file2c comes from using ${FILE2C}. r324082: Typo in filename in comment. r325955: Fix 'local' to not look in the source tree for the file. Usually 'local' is used along with other rules such as 'no-implicit-rule' or 'dependency' which avoids this problem. It's possible to need to use 'local' while relying on the default rules though for a file which is not in the source tree nor generated in the kernel. Modified: stable/11/usr.sbin/config/Makefile stable/11/usr.sbin/config/config.h stable/11/usr.sbin/config/configvers.h stable/11/usr.sbin/config/mkmakefile.c Directory Properties: stable/11/ (props changed) Modified: stable/11/usr.sbin/config/Makefile ============================================================================== --- stable/11/usr.sbin/config/Makefile Mon Jul 30 21:13:42 2018 (r336937) +++ stable/11/usr.sbin/config/Makefile Mon Jul 30 21:24:27 2018 (r336938) @@ -1,15 +1,20 @@ # @(#)Makefile 8.1 (Berkeley) 6/6/93 # $FreeBSD$ +SRCDIR:=${.PARSEDIR:tA} + PROG= config MAN= config.5 config.8 SRCS= config.y main.c lang.l mkmakefile.c mkheaders.c \ mkoptions.c y.tab.h kernconf.c +FILE2C?=file2c + kernconf.c: kernconf.tmpl - file2c 'char kernconfstr[] = {' ',0};' < ${.CURDIR}/kernconf.tmpl > kernconf.c + ${FILE2C} 'char kernconfstr[] = {' ',0};' < \ + ${SRCDIR}/kernconf.tmpl > kernconf.c -CFLAGS+= -I. -I${.CURDIR} +CFLAGS+= -I. -I${SRCDIR} NO_WMISSING_VARIABLE_DECLARATIONS= Modified: stable/11/usr.sbin/config/config.h ============================================================================== --- stable/11/usr.sbin/config/config.h Mon Jul 30 21:13:42 2018 (r336937) +++ stable/11/usr.sbin/config/config.h Mon Jul 30 21:24:27 2018 (r336938) @@ -55,6 +55,7 @@ struct file_list { char *f_clean; /* File list to add to clean rule */ char *f_warn; /* warning message */ const char *f_objprefix; /* prefix string for object name */ + const char *f_srcprefix; /* source prefix such as $S/ */ }; struct files_name { @@ -161,7 +162,7 @@ struct includepath { SLIST_HEAD(, includepath) includepath; /* - * Tag present in the kernelconf.tmlp template file. It's mandatory for those + * Tag present in the kernconf.tmpl template file. It's mandatory for those * two strings to be the same. Otherwise you'll get into trouble. */ #define KERNCONFTAG "%%KERNCONFFILE%%" Modified: stable/11/usr.sbin/config/configvers.h ============================================================================== --- stable/11/usr.sbin/config/configvers.h Mon Jul 30 21:13:42 2018 (r336937) +++ stable/11/usr.sbin/config/configvers.h Mon Jul 30 21:24:27 2018 (r336938) @@ -49,5 +49,5 @@ * * $FreeBSD$ */ -#define CONFIGVERS 600014 +#define CONFIGVERS 600015 #define MAJOR_VERS(x) ((x) / 100000) Modified: stable/11/usr.sbin/config/mkmakefile.c ============================================================================== --- stable/11/usr.sbin/config/mkmakefile.c Mon Jul 30 21:13:42 2018 (r336937) +++ stable/11/usr.sbin/config/mkmakefile.c Mon Jul 30 21:24:27 2018 (r336938) @@ -535,6 +535,10 @@ nextparam:; tp = new_fent(); tp->f_fn = this; tp->f_type = filetype; + if (filetype == LOCAL) + tp->f_srcprefix = ""; + else + tp->f_srcprefix = "$S/"; if (imp_rule) tp->f_flags |= NO_IMPLCT_RULE; if (no_obj) @@ -610,7 +614,8 @@ do_before_depend(FILE *fp) if (tp->f_flags & NO_IMPLCT_RULE) fprintf(fp, "%s ", tp->f_fn); else - fprintf(fp, "$S/%s ", tp->f_fn); + fprintf(fp, "%s%s ", tp->f_srcprefix, + tp->f_fn); lpos += len + 1; } if (lpos != 8) @@ -675,10 +680,7 @@ do_xxfiles(char *tag, FILE *fp) lpos = 8; fputs("\\\n\t", fp); } - if (tp->f_type != LOCAL) - fprintf(fp, "$S/%s ", tp->f_fn); - else - fprintf(fp, "%s ", tp->f_fn); + fprintf(fp, "%s%s ", tp->f_srcprefix, tp->f_fn); lpos += len + 1; } free(suff); @@ -724,25 +726,30 @@ do_rules(FILE *f) else { *cp = '\0'; if (och == 'o') { - fprintf(f, "%s%so:\n\t-cp $S/%so .\n\n", - ftp->f_objprefix, tail(np), np); + fprintf(f, "%s%so:\n\t-cp %s%so .\n\n", + ftp->f_objprefix, tail(np), + ftp->f_srcprefix, np); continue; } if (ftp->f_depends) { - fprintf(f, "%s%sln: $S/%s%c %s\n", - ftp->f_objprefix, tail(np), np, och, + fprintf(f, "%s%sln: %s%s%c %s\n", + ftp->f_objprefix, tail(np), + ftp->f_srcprefix, np, och, ftp->f_depends); fprintf(f, "\t${NORMAL_LINT}\n\n"); - fprintf(f, "%s%so: $S/%s%c %s\n", - ftp->f_objprefix, tail(np), np, och, + fprintf(f, "%s%so: %s%s%c %s\n", + ftp->f_objprefix, tail(np), + ftp->f_srcprefix, np, och, ftp->f_depends); } else { - fprintf(f, "%s%sln: $S/%s%c\n", - ftp->f_objprefix, tail(np), np, och); + fprintf(f, "%s%sln: %s%s%c\n", + ftp->f_objprefix, tail(np), + ftp->f_srcprefix, np, och); fprintf(f, "\t${NORMAL_LINT}\n\n"); - fprintf(f, "%s%so: $S/%s%c\n", - ftp->f_objprefix, tail(np), np, och); + fprintf(f, "%s%so: %s%s%c\n", + ftp->f_objprefix, tail(np), + ftp->f_srcprefix, np, och); } } compilewith = ftp->f_compilewith; @@ -771,7 +778,8 @@ do_rules(FILE *f) } *cp = och; if (strlen(ftp->f_objprefix)) - fprintf(f, "\t%s $S/%s\n", compilewith, np); + fprintf(f, "\t%s %s%s\n", compilewith, + ftp->f_srcprefix, np); else fprintf(f, "\t%s\n", compilewith); From owner-svn-src-all@freebsd.org Mon Jul 30 21:25:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4C6410617D8; Mon, 30 Jul 2018 21:25:46 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 548F485495; Mon, 30 Jul 2018 21:25:46 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 358231FEE; Mon, 30 Jul 2018 21:25:46 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6ULPkQf096198; Mon, 30 Jul 2018 21:25:46 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6ULPk4o096197; Mon, 30 Jul 2018 21:25:46 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807302125.w6ULPk4o096197@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Mon, 30 Jul 2018 21:25:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336939 - stable/11 X-SVN-Group: stable-11 X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: stable/11 X-SVN-Commit-Revision: 336939 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:25:46 -0000 Author: kevans Date: Mon Jul 30 21:25:45 2018 New Revision: 336939 URL: https://svnweb.freebsd.org/changeset/base/336939 Log: MFC r336353, r336358-r336359, r336414: One big config(8) no-op r336353: config(8): Bump major version after r335998 config-generated hints.c/env.c from r335998 and later are incompatible with earlier kernels due to no longer setting envmode/hintmode. A minor bump for this is insufficient, as matching major version with a later minor version is still viewed as backwards-compatible. This was an MI kernel change, soo all VERSREQ's are bumped. r336358: Partially revert r336353: sys/conf/* %VERSREQ bumps The changes made in r335998 don't strictly require a newer config(8), though it is advised. The %VERSREQ bumps were premature. r336359: Revert 336358 and step away fron machine for the day... VERSREQ < 7.+ physically will not work with new config(8) due to major bump, which is why I bumped it in the first place... Back to the original version r336414: Revert r336353 completely based on protest; compatibility shims incoming Modified: Directory Properties: stable/11/ (props changed) From owner-svn-src-all@freebsd.org Mon Jul 30 21:27:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A18210618A6; Mon, 30 Jul 2018 21:27:27 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0AD7B857B4; Mon, 30 Jul 2018 21:27:27 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DEC751FF6; Mon, 30 Jul 2018 21:27:26 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6ULRQWp096330; Mon, 30 Jul 2018 21:27:26 GMT (envelope-from tuexen@FreeBSD.org) Received: (from tuexen@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6ULRQhP096328; Mon, 30 Jul 2018 21:27:26 GMT (envelope-from tuexen@FreeBSD.org) Message-Id: <201807302127.w6ULRQhP096328@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tuexen set sender to tuexen@FreeBSD.org using -f From: Michael Tuexen Date: Mon, 30 Jul 2018 21:27:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336940 - in head/sys: netinet netinet6 X-SVN-Group: head X-SVN-Commit-Author: tuexen X-SVN-Commit-Paths: in head/sys: netinet netinet6 X-SVN-Commit-Revision: 336940 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:27:27 -0000 Author: tuexen Date: Mon Jul 30 21:27:26 2018 New Revision: 336940 URL: https://svnweb.freebsd.org/changeset/base/336940 Log: Allow implicit TCP connection setup for TCP/IPv6. TCP/IPv4 allows an implicit connection setup using sendto(), which is used for TTCP and TCP fast open. This patch adds support for TCP/IPv6. While there, improve some tests for detecting multicast addresses, which are mapped. Reviewed by: bz@, kbowling@, rrs@ Sponsored by: Netflix, Inc. Differential Revision: https://reviews.freebsd.org/D16458 Modified: head/sys/netinet/tcp_usrreq.c head/sys/netinet6/in6_proto.c Modified: head/sys/netinet/tcp_usrreq.c ============================================================================== --- head/sys/netinet/tcp_usrreq.c Mon Jul 30 21:25:45 2018 (r336939) +++ head/sys/netinet/tcp_usrreq.c Mon Jul 30 21:27:26 2018 (r336940) @@ -378,6 +378,11 @@ tcp6_usr_bind(struct socket *so, struct sockaddr *nam, struct sockaddr_in sin; in6_sin6_2_sin(&sin, sin6p); + if (IN_MULTICAST(ntohl(sin.sin_addr.s_addr))) { + error = EAFNOSUPPORT; + INP_HASH_WUNLOCK(&V_tcbinfo); + goto out; + } inp->inp_vflag |= INP_IPV4; inp->inp_vflag &= ~INP_IPV6; error = in_pcbbind(inp, (struct sockaddr *)&sin, @@ -607,6 +612,10 @@ tcp6_usr_connect(struct socket *so, struct sockaddr *n } in6_sin6_2_sin(&sin, sin6p); + if (IN_MULTICAST(ntohl(sin.sin_addr.s_addr))) { + error = EAFNOSUPPORT; + goto out; + } inp->inp_vflag |= INP_IPV4; inp->inp_vflag &= ~INP_IPV6; if ((error = prison_remote_ip4(td->td_ucred, @@ -892,6 +901,9 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf struct inpcb *inp; struct tcpcb *tp = NULL; struct epoch_tracker net_et; +#ifdef INET + struct sockaddr_in sin, *sinp; +#endif #ifdef INET6 int isipv6; #endif @@ -918,11 +930,124 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf error = ECONNRESET; goto out; } -#ifdef INET6 - isipv6 = nam && nam->sa_family == AF_INET6; -#endif /* INET6 */ tp = intotcpcb(inp); TCPDEBUG1(); + if (nam != NULL && tp->t_state < TCPS_SYN_SENT) { + switch (nam->sa_family) { +#ifdef INET + case AF_INET: + sinp = (struct sockaddr_in *)nam; + if (sinp->sin_len != sizeof(struct sockaddr_in)) { + if (m) + m_freem(m); + error = EINVAL; + goto out; + } + if ((inp->inp_vflag & INP_IPV6) != 0) { + if (m) + m_freem(m); + error = EAFNOSUPPORT; + goto out; + } + if (IN_MULTICAST(ntohl(sinp->sin_addr.s_addr))) { + if (m) + m_freem(m); + error = EAFNOSUPPORT; + goto out; + } + if ((error = prison_remote_ip4(td->td_ucred, + &sinp->sin_addr))) { + if (m) + m_freem(m); + goto out; + } +#ifdef INET6 + isipv6 = 0; +#endif + break; +#endif /* INET */ +#ifdef INET6 + case AF_INET6: + { + struct sockaddr_in6 *sin6p; + + sin6p = (struct sockaddr_in6 *)nam; + if (sin6p->sin6_len != sizeof(struct sockaddr_in6)) { + if (m) + m_freem(m); + error = EINVAL; + goto out; + } + if (IN6_IS_ADDR_MULTICAST(&sin6p->sin6_addr)) { + if (m) + m_freem(m); + error = EAFNOSUPPORT; + goto out; + } + if (IN6_IS_ADDR_V4MAPPED(&sin6p->sin6_addr)) { +#ifdef INET + if ((inp->inp_flags & IN6P_IPV6_V6ONLY) != 0) { + error = EINVAL; + if (m) + m_freem(m); + goto out; + } + if ((inp->inp_vflag & INP_IPV4) == 0) { + error = EAFNOSUPPORT; + if (m) + m_freem(m); + goto out; + } + inp->inp_vflag &= ~INP_IPV6; + sinp = &sin; + in6_sin6_2_sin(sinp, sin6p); + if (IN_MULTICAST( + ntohl(sinp->sin_addr.s_addr))) { + error = EAFNOSUPPORT; + if (m) + m_freem(m); + goto out; + } + if ((error = prison_remote_ip4(td->td_ucred, + &sinp->sin_addr))) { + if (m) + m_freem(m); + goto out; + } + isipv6 = 0; +#else /* !INET */ + error = EAFNOSUPPORT; + if (m) + m_freem(m); + goto out; +#endif /* INET */ + } else { + if ((inp->inp_vflag & INP_IPV6) == 0) { + if (m) + m_freem(m); + error = EAFNOSUPPORT; + goto out; + } + inp->inp_vflag &= ~INP_IPV4; + inp->inp_inc.inc_flags |= INC_ISIPV6; + if ((error = prison_remote_ip6(td->td_ucred, + &sin6p->sin6_addr))) { + if (m) + m_freem(m); + goto out; + } + isipv6 = 1; + } + break; + } +#endif /* INET6 */ + default: + if (m) + m_freem(m); + error = EAFNOSUPPORT; + goto out; + } + } if (control) { /* TCP doesn't do control messages (rights, creds, etc) */ if (control->m_len) { @@ -950,7 +1075,8 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf else #endif #ifdef INET - error = tcp_connect(tp, nam, td); + error = tcp_connect(tp, + (struct sockaddr *)sinp, td); #endif if (error) goto out; @@ -1019,7 +1145,8 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf else #endif #ifdef INET - error = tcp_connect(tp, nam, td); + error = tcp_connect(tp, + (struct sockaddr *)sinp, td); #endif if (error) goto out; Modified: head/sys/netinet6/in6_proto.c ============================================================================== --- head/sys/netinet6/in6_proto.c Mon Jul 30 21:25:45 2018 (r336939) +++ head/sys/netinet6/in6_proto.c Mon Jul 30 21:27:26 2018 (r336940) @@ -171,7 +171,7 @@ struct protosw inet6sw[] = { .pr_type = SOCK_STREAM, .pr_domain = &inet6domain, .pr_protocol = IPPROTO_TCP, - .pr_flags = PR_CONNREQUIRED|PR_WANTRCVD|PR_LISTEN, + .pr_flags = PR_CONNREQUIRED|PR_IMPLOPCL|PR_WANTRCVD|PR_LISTEN, .pr_input = tcp6_input, .pr_ctlinput = tcp6_ctlinput, .pr_ctloutput = tcp_ctloutput, From owner-svn-src-all@freebsd.org Mon Jul 30 21:29:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9723D1061981; Mon, 30 Jul 2018 21:29:27 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B8D1859F1; Mon, 30 Jul 2018 21:29:27 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2DCB2201E; Mon, 30 Jul 2018 21:29:27 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6ULTRwo096488; Mon, 30 Jul 2018 21:29:27 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6ULTR56096487; Mon, 30 Jul 2018 21:29:27 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807302129.w6ULTR56096487@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Mon, 30 Jul 2018 21:29:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336941 - stable/11/usr.sbin/config X-SVN-Group: stable-11 X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: stable/11/usr.sbin/config X-SVN-Commit-Revision: 336941 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:29:27 -0000 Author: kevans Date: Mon Jul 30 21:29:26 2018 New Revision: 336941 URL: https://svnweb.freebsd.org/changeset/base/336941 Log: MFC r335526: Let -s actually work. Modified: stable/11/usr.sbin/config/main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/usr.sbin/config/main.c ============================================================================== --- stable/11/usr.sbin/config/main.c Mon Jul 30 21:27:26 2018 (r336940) +++ stable/11/usr.sbin/config/main.c Mon Jul 30 21:29:26 2018 (r336941) @@ -116,7 +116,7 @@ main(int argc, char **argv) printmachine = 0; kernfile = NULL; SLIST_INIT(&includepath); - while ((ch = getopt(argc, argv, "CI:d:gmpsVx:")) != -1) + while ((ch = getopt(argc, argv, "CI:d:gmps:Vx:")) != -1) switch (ch) { case 'C': filebased = 1; From owner-svn-src-all@freebsd.org Mon Jul 30 21:59:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5CA9410621FF; Mon, 30 Jul 2018 21:59:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 05CF8867C4; Mon, 30 Jul 2018 21:59:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D6D6F250E; Mon, 30 Jul 2018 21:59:44 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6ULxieu012088; Mon, 30 Jul 2018 21:59:44 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6ULxi1A012086; Mon, 30 Jul 2018 21:59:44 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302159.w6ULxi1A012086@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 21:59:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336942 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 336942 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 21:59:45 -0000 Author: mav Date: Mon Jul 30 21:59:44 2018 New Revision: 336942 URL: https://svnweb.freebsd.org/changeset/base/336942 Log: 9189 Add debug to vdev_label_read_config when txg check fails illumos/illumos-gate@b6bf6e1540f30bd97b8d6e2c21d95e17841e0f23 Reviewed by: George Wilson Reviewed by: Prashanth Sreenivasa Approved by: Matt Ahrens Author: Pavel Zakharov Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Mon Jul 30 21:29:26 2018 (r336941) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Mon Jul 30 21:59:44 2018 (r336942) @@ -1585,7 +1585,8 @@ vdev_validate(vdev_t *vd) if ((label = vdev_label_read_config(vd, txg)) == NULL) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_BAD_LABEL); - vdev_dbgmsg(vd, "vdev_validate: failed reading config"); + vdev_dbgmsg(vd, "vdev_validate: failed reading config for " + "txg %llu", (u_longlong_t)txg); return (0); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Mon Jul 30 21:29:26 2018 (r336941) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Mon Jul 30 21:59:44 2018 (r336942) @@ -540,6 +540,7 @@ vdev_label_read_config(vdev_t *vd, uint64_t txg) abd_t *vp_abd; zio_t *zio; uint64_t best_txg = 0; + uint64_t label_txg = 0; int error = 0; int flags = ZIO_FLAG_CONFIG_WRITER | ZIO_FLAG_CANFAIL | ZIO_FLAG_SPECULATIVE; @@ -565,8 +566,6 @@ retry: if (zio_wait(zio) == 0 && nvlist_unpack(vp->vp_nvlist, sizeof (vp->vp_nvlist), &label, 0) == 0) { - uint64_t label_txg = 0; - /* * Auxiliary vdevs won't have txg values in their * labels and newly added vdevs may not have been @@ -595,6 +594,15 @@ retry: if (config == NULL && !(flags & ZIO_FLAG_TRYHARD)) { flags |= ZIO_FLAG_TRYHARD; goto retry; + } + + /* + * We found a valid label but it didn't pass txg restrictions. + */ + if (config == NULL && label_txg != 0) { + vdev_dbgmsg(vd, "label discarded as txg is too large " + "(%llu > %llu)", (u_longlong_t)label_txg, + (u_longlong_t)txg); } abd_free(vp_abd); From owner-svn-src-all@freebsd.org Mon Jul 30 22:03:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1E4110624C2; Mon, 30 Jul 2018 22:03:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48ECA86BEB; Mon, 30 Jul 2018 22:03:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 26724269E; Mon, 30 Jul 2018 22:03:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UM3U3f016735; Mon, 30 Jul 2018 22:03:30 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UM3TmU016733; Mon, 30 Jul 2018 22:03:29 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302203.w6UM3TmU016733@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 22:03:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336943 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 336943 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 22:03:30 -0000 Author: mav Date: Mon Jul 30 22:03:29 2018 New Revision: 336943 URL: https://svnweb.freebsd.org/changeset/base/336943 Log: MFV r336942: 9189 Add debug to vdev_label_read_config when txg check fails illumos/illumos-gate@b6bf6e1540f30bd97b8d6e2c21d95e17841e0f23 Reviewed by: George Wilson Reviewed by: Prashanth Sreenivasa Approved by: Matt Ahrens Author: Pavel Zakharov Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Mon Jul 30 21:59:44 2018 (r336942) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Mon Jul 30 22:03:29 2018 (r336943) @@ -1725,7 +1725,8 @@ vdev_validate(vdev_t *vd) if ((label = vdev_label_read_config(vd, txg)) == NULL) { vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, VDEV_AUX_BAD_LABEL); - vdev_dbgmsg(vd, "vdev_validate: failed reading config"); + vdev_dbgmsg(vd, "vdev_validate: failed reading config for " + "txg %llu", (u_longlong_t)txg); return (0); } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Mon Jul 30 21:59:44 2018 (r336942) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Mon Jul 30 22:03:29 2018 (r336943) @@ -546,6 +546,7 @@ vdev_label_read_config(vdev_t *vd, uint64_t txg) abd_t *vp_abd; zio_t *zio; uint64_t best_txg = 0; + uint64_t label_txg = 0; int error = 0; int flags = ZIO_FLAG_CONFIG_WRITER | ZIO_FLAG_CANFAIL | ZIO_FLAG_SPECULATIVE; @@ -571,8 +572,6 @@ retry: if (zio_wait(zio) == 0 && nvlist_unpack(vp->vp_nvlist, sizeof (vp->vp_nvlist), &label, 0) == 0) { - uint64_t label_txg = 0; - /* * Auxiliary vdevs won't have txg values in their * labels and newly added vdevs may not have been @@ -601,6 +600,15 @@ retry: if (config == NULL && !(flags & ZIO_FLAG_TRYHARD)) { flags |= ZIO_FLAG_TRYHARD; goto retry; + } + + /* + * We found a valid label but it didn't pass txg restrictions. + */ + if (config == NULL && label_txg != 0) { + vdev_dbgmsg(vd, "label discarded as txg is too large " + "(%llu > %llu)", (u_longlong_t)label_txg, + (u_longlong_t)txg); } abd_free(vp_abd); From owner-svn-src-all@freebsd.org Mon Jul 30 22:10:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1D5210625D2; Mon, 30 Jul 2018 22:10:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5491D86DFD; Mon, 30 Jul 2018 22:10:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 30C5826A3; Mon, 30 Jul 2018 22:10:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UMAFrc017085; Mon, 30 Jul 2018 22:10:15 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UMAFkO017082; Mon, 30 Jul 2018 22:10:15 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302210.w6UMAFkO017082@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 22:10:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336944 - in vendor/illumos/dist: lib/libzfs/common man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor/illumos/dist: lib/libzfs/common man/man1m X-SVN-Commit-Revision: 336944 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 22:10:16 -0000 Author: mav Date: Mon Jul 30 22:10:15 2018 New Revision: 336944 URL: https://svnweb.freebsd.org/changeset/base/336944 Log: 9286 want refreservation=auto When a ZFS volume is created with zfs create -V (but without -s), the refreservation property is set to a value that is volsize plus the maximum size of metadata. If refreservation is ever set to another value, it is impossible to set it back to the automatically determined value. There are other cases where refreservation may be wrong. These include receiving a volume that was sent without properties and zfs clone. We need: zfs set refreservation=auto zfs clone -o refreservation=auto Each one would use the same function used by zfs create -V to determine the proper value for refreservation. illumos/illumos-gate@1c10ae76c0cb31326c320e7cef1d3f24a1f47125 Reviewed by: Allan Jude Reviewed by: Matthew Ahrens Reviewed by: John Kennedy Reviewed by: Andy Stormont Approved by: Richard Lowe Author: Mike Gerdts Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs/common/libzfs_util.c vendor/illumos/dist/man/man1m/zfs.1m Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Mon Jul 30 22:03:29 2018 (r336943) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Mon Jul 30 22:10:15 2018 (r336944) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, Joyent, Inc. All rights reserved. + * Copyright (c) 2018, Joyent, Inc. All rights reserved. * Copyright (c) 2011, 2016 by Delphix. All rights reserved. * Copyright (c) 2012 DEY Storage Systems, Inc. All rights reserved. * Copyright (c) 2011-2012 Pawel Jakub Dawidek. All rights reserved. @@ -1387,7 +1387,6 @@ badlabel: switch (prop) { case ZFS_PROP_RESERVATION: - case ZFS_PROP_REFRESERVATION: if (intval > volsize) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "'%s' is greater than current " @@ -1398,6 +1397,17 @@ badlabel: } break; + case ZFS_PROP_REFRESERVATION: + if (intval > volsize && intval != UINT64_MAX) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'%s' is greater than current " + "volume size"), propname); + (void) zfs_error(hdl, EZFS_BADPROP, + errbuf); + goto error; + } + break; + case ZFS_PROP_VOLSIZE: if (intval % blocksize != 0) { zfs_nicenum(blocksize, buf, @@ -1499,6 +1509,61 @@ zfs_add_synthetic_resv(zfs_handle_t *zhp, nvlist_t *nv return (1); } +/* + * Helper for 'zfs {set|clone} refreservation=auto'. Must be called after + * zfs_valid_proplist(), as it is what sets the UINT64_MAX sentinal value. + * Return codes must match zfs_add_synthetic_resv(). + */ +static int +zfs_fix_auto_resv(zfs_handle_t *zhp, nvlist_t *nvl) +{ + uint64_t volsize; + uint64_t resvsize; + zfs_prop_t prop; + nvlist_t *props; + + if (!ZFS_IS_VOLUME(zhp)) { + return (0); + } + + if (zfs_which_resv_prop(zhp, &prop) != 0) { + return (-1); + } + + if (prop != ZFS_PROP_REFRESERVATION) { + return (0); + } + + if (nvlist_lookup_uint64(nvl, zfs_prop_to_name(prop), &resvsize) != 0) { + /* No value being set, so it can't be "auto" */ + return (0); + } + if (resvsize != UINT64_MAX) { + /* Being set to a value other than "auto" */ + return (0); + } + + props = fnvlist_alloc(); + + fnvlist_add_uint64(props, zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), + zfs_prop_get_int(zhp, ZFS_PROP_VOLBLOCKSIZE)); + + if (nvlist_lookup_uint64(nvl, zfs_prop_to_name(ZFS_PROP_VOLSIZE), + &volsize) != 0) { + volsize = zfs_prop_get_int(zhp, ZFS_PROP_VOLSIZE); + } + + resvsize = zvol_volsize_to_reservation(volsize, props); + fnvlist_free(props); + + (void) nvlist_remove_all(nvl, zfs_prop_to_name(prop)); + if (nvlist_add_uint64(nvl, zfs_prop_to_name(prop), resvsize) != 0) { + (void) no_memory(zhp->zfs_hdl); + return (-1); + } + return (1); +} + void zfs_setprop_error(libzfs_handle_t *hdl, zfs_prop_t prop, int err, char *errbuf) @@ -1664,6 +1729,12 @@ zfs_prop_set_list(zfs_handle_t *zhp, nvlist_t *props) goto error; } } + + if (added_resv != 1 && + (added_resv = zfs_fix_auto_resv(zhp, nvl)) == -1) { + goto error; + } + /* * Check how many properties we're setting and allocate an array to * store changelist pointers for postfix(). @@ -3666,6 +3737,7 @@ zfs_clone(zfs_handle_t *zhp, const char *target, nvlis if (props) { zfs_type_t type; + if (ZFS_IS_VOLUME(zhp)) { type = ZFS_TYPE_VOLUME; } else { @@ -3674,6 +3746,10 @@ zfs_clone(zfs_handle_t *zhp, const char *target, nvlis if ((props = zfs_valid_proplist(hdl, type, props, zoned, zhp, zhp->zpool_hdl, errbuf)) == NULL) return (-1); + if (zfs_fix_auto_resv(zhp, props) == -1) { + nvlist_free(props); + return (-1); + } } ret = lzc_clone(target, zhp->zfs_name, props); Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_util.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Mon Jul 30 22:03:29 2018 (r336943) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Mon Jul 30 22:10:15 2018 (r336944) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, Joyent, Inc. All rights reserved. + * Copyright (c) 2018 Joyent, Inc. * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright 2016 Igor Kozhukhov * Copyright (c) 2017 Datto Inc. @@ -1184,6 +1184,7 @@ zprop_parse_value(libzfs_handle_t *hdl, nvpair_t *elem const char *propname; char *value; boolean_t isnone = B_FALSE; + boolean_t isauto = B_FALSE; if (type == ZFS_TYPE_POOL) { proptype = zpool_prop_get_type(prop); @@ -1219,8 +1220,9 @@ zprop_parse_value(libzfs_handle_t *hdl, nvpair_t *elem (void) nvpair_value_string(elem, &value); if (strcmp(value, "none") == 0) { isnone = B_TRUE; - } else if (zfs_nicestrtonum(hdl, value, ivalp) - != 0) { + } else if (strcmp(value, "auto") == 0) { + isauto = B_TRUE; + } else if (zfs_nicestrtonum(hdl, value, ivalp) != 0) { goto error; } } else if (datatype == DATA_TYPE_UINT64) { @@ -1250,6 +1252,31 @@ zprop_parse_value(libzfs_handle_t *hdl, nvpair_t *elem prop == ZFS_PROP_SNAPSHOT_LIMIT)) { *ivalp = UINT64_MAX; } + + /* + * Special handling for setting 'refreservation' to 'auto'. Use + * UINT64_MAX to tell the caller to use zfs_fix_auto_resv(). + * 'auto' is only allowed on volumes. + */ + if (isauto) { + switch (prop) { + case ZFS_PROP_REFRESERVATION: + if ((type & ZFS_TYPE_VOLUME) == 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'%s=auto' only allowed on " + "volumes"), nvpair_name(elem)); + goto error; + } + *ivalp = UINT64_MAX; + break; + default: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'auto' is invalid value for '%s'"), + nvpair_name(elem)); + goto error; + } + } + break; case PROP_TYPE_INDEX: Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Mon Jul 30 22:03:29 2018 (r336943) +++ vendor/illumos/dist/man/man1m/zfs.1m Mon Jul 30 22:10:15 2018 (r336944) @@ -27,6 +27,7 @@ .\" Copyright (c) 2014 by Adam Stevko. All rights reserved. .\" Copyright (c) 2014 Integros [integros.com] .\" Copyright 2017 Nexenta Systems, Inc. +.\" Copyright 2018 Joyent, Inc. .\" .Dd December 6, 2017 .Dt ZFS 1M @@ -1345,7 +1346,7 @@ Limits the amount of space a dataset can consume. This property enforces a hard limit on the amount of space used. This hard limit does not include space used by descendents, including file systems and snapshots. -.It Sy refreservation Ns = Ns Em size Ns | Ns Sy none +.It Sy refreservation Ns = Ns Em size Ns | Ns Sy none Ns | Ns Sy auto The minimum amount of space guaranteed to a dataset, not including its descendents. When the amount of space used is below this value, the dataset is treated as if @@ -1363,6 +1364,22 @@ this reservation to accommodate the current number of .Qq referenced bytes in the dataset. .Pp +If +.Sy refreservation +is set to +.Sy auto , +a volume is thick provisioned +.Po or +.Qq not sparse +.Pc . +.Sy refreservation Ns = Ns Sy auto +is only supported on volumes. +See +.Sy volsize +in the +.Sx Native Properties +section for more information about sparse volumes. +.Pp This property can also be referred to by its shortened column name, .Sy refreserv . .It Sy reservation Ns = Ns Em size Ns | Ns Sy none @@ -1577,22 +1594,39 @@ Extreme care should be used when adjusting the volume Though not recommended, a .Qq sparse volume .Po also known as -.Qq thin provisioning +.Qq thin provisioned .Pc can be created by specifying the .Fl s option to the .Nm zfs Cm create Fl V -command, or by changing the reservation after the volume has been created. +command, or by changing the value of the +.Sy refreservation +property +.Po or +.Sy reservation +property on pool version 8 or earlier +.Pc +after the volume has been created. A .Qq sparse volume -is a volume where the reservation is less then the volume size. +is a volume where the value of +.Sy refreservation +is less than the size of the volume plus the space required to store its +metadata. Consequently, writes to a sparse volume can fail with .Er ENOSPC when the pool is low on space. For a sparse volume, changes to .Sy volsize -are not reflected in the reservation. +are not reflected in the +.Sy refreservation. +A volume that is not sparse is said to be +.Qq thick provisioned . +A sparse volume can become thick provisioned by setting +.Sy refreservation +to +.Sy auto . .It Sy vscan Ns = Ns Sy on Ns | Ns Sy off Controls whether regular files should be scanned for viruses when a file is opened and closed. From owner-svn-src-all@freebsd.org Mon Jul 30 22:38:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F22E1062DCF; Mon, 30 Jul 2018 22:38:16 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3E54885F9; Mon, 30 Jul 2018 22:38:15 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 20ABF10AFCD; Mon, 30 Jul 2018 18:38:14 -0400 (EDT) Subject: Re: svn commit: r336805 - in stable/11: share/man/man4 sys/dev/vt/hw/vga To: Eitan Adler , Kostik Belousov References: <201807280737.w6S7b12F025452@repo.freebsd.org> <20180728233405.GD40119@kib.kiev.ua> Cc: src-committers , svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org From: John Baldwin Message-ID: Date: Mon, 30 Jul 2018 15:38:13 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Mon, 30 Jul 2018 18:38:14 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 22:38:16 -0000 On 7/28/18 10:12 PM, Eitan Adler wrote: > On Sat, 28 Jul 2018 at 16:34, Konstantin Belousov wrote: >> >> On Sat, Jul 28, 2018 at 07:37:01AM +0000, Eitan Adler wrote: >>> Author: eadler >>> Date: Sat Jul 28 07:37:01 2018 >>> New Revision: 336805 >>> URL: https://svnweb.freebsd.org/changeset/base/336805 >>> >>> Log: >>> MFC r335629: >>> >>> vt: add option to ignore NO_VGA flag in ACPI >>> >>> To workaround buggy firmware that sets this flag when there's actually >>> a VGA present. >>> >>> Ref D16003 >>> PR: 229235 >>> >>> Modified: >>> stable/11/share/man/man4/vt.4 >>> stable/11/sys/dev/vt/hw/vga/vt_vga.c >>> Directory Properties: >>> stable/11/ (props changed) >>> >>> Modified: stable/11/share/man/man4/vt.4 >>> ============================================================================== >>> --- stable/11/share/man/man4/vt.4 Sat Jul 28 06:46:10 2018 (r336804) >>> +++ stable/11/share/man/man4/vt.4 Sat Jul 28 07:37:01 2018 (r336805) >>> @@ -44,6 +44,7 @@ >>> In >>> .Xr loader.conf 5 : >>> .Cd hw.vga.textmode=1 >>> +.Cd hw.vga.acpi_ignore_no_vga=1 >>> .Cd kern.vty=vt >>> .Cd kern.vt.color..rgb="" >>> .Cd kern.vt.fb.default_mode="x" >>> @@ -196,6 +197,10 @@ prompt or in >>> Set to 1 to use virtual terminals in text mode instead of graphics mode. >>> Features that require graphics mode, like loadable fonts, will be >>> disabled. >>> +.It Va hw.vga.acpi_ignore_no_vga >>> +Set to 1 to force the usage of the VGA driver regardless of whether >>> +ACPI IAPC_BOOT_ARCH signals no VGA support. >>> +Can be used to workaround firmware bugs in the ACPI tables. >>> .It Va kern.vty >>> Set this value to >>> .Ql vt >>> >>> Modified: stable/11/sys/dev/vt/hw/vga/vt_vga.c >>> ============================================================================== >>> --- stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 06:46:10 2018 (r336804) >>> +++ stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 07:37:01 2018 (r336805) >>> @@ -1211,6 +1211,12 @@ vga_acpi_disabled(void) >>> ACPI_TABLE_FADT *fadt; >>> vm_paddr_t physaddr; >>> uint16_t flags; >>> + int ignore; >>> + >>> + TUNABLE_INT_FETCH("hw.vga.acpi_ignore_no_vga", &ignore); >>> + >>> + if (ignore) >>> + return (false); > >> If the tunable is not set, this code evaluates the stack garbage. >> Why did you merged this without asking the author and not looking >> at the followups ? > > Please assume some level of good faith. > > I MFCed this because I was asked to here > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229235 over a month > ago. During that time no one mentioned or linked any followups. I did > look in my email for mentions of r335629 as well - of which there are > none. > > Now that you mention it, I looked at the svn log and found r335631 > which I will MFC as well. If MFC'ing a change by another developer you should proactively look at the svn logs of the changed files to see if there are follow up fixes. I do this myself via svnweb for my own commits. I also have a script that takes a given svn revision ID to merge from head to the current working directory, grabs the list of files from that change, then outputs a list of all the not-yet-merged svn revisions that touch the effected files so I can review them to see which ones are followups: #!/bin/sh # # mfclist [rev ... ] # # Given 1 or more SVN revisions for commits to head, build a list of eligible # commits to those files that haven't been merged yet. Must be run with the # cwd being the root of a checkout to merge to from ^/head. WRKDIR=`mktemp -d -t mfclist` if [ $? -ne 0 ]; then echo "Can't create temporary directory." exit 1 fi cleanup() { [ -d ${WRKDIR} ] && rm -rf ${WRKDIR} } trap cleanup EXIT INT QUIT TERM for rev; do svn log -qvc ${rev} ^/head | \ awk '/^ [A-Z] / { sub(/\/head\//, "", $2); print $2 }' >> \ ${WRKDIR}/files done sort ${WRKDIR}/files | uniq > ${WRKDIR}/files.u xargs -I % -n 1 svn mergeinfo --show-revs eligible '^/head/%' % < \ ${WRKDIR}/files.u | sort | uniq exit 0 -- John Baldwin From owner-svn-src-all@freebsd.org Mon Jul 30 22:39:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D550D1062E5B; Mon, 30 Jul 2018 22:39:31 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 88C9E88762; Mon, 30 Jul 2018 22:39:31 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 658252B87; Mon, 30 Jul 2018 22:39:31 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UMdVKA032685; Mon, 30 Jul 2018 22:39:31 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UMdUxu032683; Mon, 30 Jul 2018 22:39:30 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302239.w6UMdUxu032683@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 22:39:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336945 - in head/cddl/contrib/opensolaris: cmd/zfs lib/libzfs/common X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/cddl/contrib/opensolaris: cmd/zfs lib/libzfs/common X-SVN-Commit-Revision: 336945 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 22:39:32 -0000 Author: mav Date: Mon Jul 30 22:39:30 2018 New Revision: 336945 URL: https://svnweb.freebsd.org/changeset/base/336945 Log: MFV r336944: 9286 want refreservation=auto When a ZFS volume is created with zfs create -V (but without -s), the refreservation property is set to a value that is volsize plus the maximum size of metadata. If refreservation is ever set to another value, it is impossible to set it back to the automatically determined value. There are other cases where refreservation may be wrong. These include receiving a volume that was sent without properties and zfs clone. We need: zfs set refreservation=auto zfs clone -o refreservation=auto Each one would use the same function used by zfs create -V to determine the proper value for refreservation. illumos/illumos-gate@1c10ae76c0cb31326c320e7cef1d3f24a1f47125 Reviewed by: Allan Jude Reviewed by: Matthew Ahrens Reviewed by: John Kennedy Reviewed by: Andy Stormont Approved by: Richard Lowe Author: Mike Gerdts Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 Mon Jul 30 22:10:15 2018 (r336944) +++ head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 Mon Jul 30 22:39:30 2018 (r336945) @@ -28,6 +28,7 @@ .\" Copyright (c) 2016 Nexenta Systems, Inc. All Rights Reserved. .\" Copyright (c) 2014, Xin LI .\" Copyright (c) 2014-2015, The FreeBSD Foundation, All Rights Reserved. +.\" Copyright 2018 Joyent, Inc. .\" .\" $FreeBSD$ .\" @@ -1311,7 +1312,7 @@ The default value is Limits the amount of space a dataset can consume. This property enforces a hard limit on the amount of space used. This hard limit does not include space used by descendents, including file systems and snapshots. -.It Sy refreservation Ns = Ns Ar size | Cm none +.It Sy refreservation Ns = Ns Ar size | Cm none | Cm auto The minimum amount of space guaranteed to a dataset, not including its descendents. When the amount of space used is below this value, the dataset is treated as if it were taking up the amount of space specified by @@ -1327,6 +1328,18 @@ is set, a snapshot is only allowed if there is enough of this reservation to accommodate the current number of "referenced" bytes in the dataset. .Pp +If +.Sy refreservation +is set to +.Sy auto , +a volume is thick provisioned or not sparse. +.Sy refreservation Ns = Cm auto +is only supported on volumes. +See +.Sy volsize +in the Native Properties +section for more information about sparse volumes. +.Pp This property can also be referred to by its shortened column name, .Sy refreserv . .It Sy reservation Ns = Ns Ar size | Cm none @@ -1459,18 +1472,33 @@ on how the volume is used. These effects can also occu changed while it is in use (particularly when shrinking the size). Extreme care should be used when adjusting the volume size. .Pp -Though not recommended, a "sparse volume" (also known as "thin provisioning") +Though not recommended, a "sparse volume" (also known as "thin provisioned") can be created by specifying the .Fl s option to the .Qq Nm Cm create Fl V -command, or by changing the reservation after the volume has been created. A -"sparse volume" is a volume where the reservation is less then the volume size. +command, or by changing the value of the +.Sy refreservation +property, or +.Sy reservation +property on pool version 8 or earlier +.Pc +after the volume has been created. +A "sparse volume" is a volume where the value of +.Sy refreservation +is less then the size of the volume plus the space required to store its +metadata. Consequently, writes to a sparse volume can fail with .Sy ENOSPC when the pool is low on space. For a sparse volume, changes to .Sy volsize -are not reflected in the reservation. +are not reflected in the +.Sy refreservation . +A volume that is not sparse is said to be "thick provisioned". +A sparse volume can become thick provisioned by setting +.Sy refreservation +to +.Sy auto . .It Sy volmode Ns = Ns Cm default | geom | dev | none This property specifies how volumes should be exposed to the OS. Setting it to Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Mon Jul 30 22:10:15 2018 (r336944) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Mon Jul 30 22:39:30 2018 (r336945) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, Joyent, Inc. All rights reserved. + * Copyright (c) 2018, Joyent, Inc. All rights reserved. * Copyright (c) 2011, 2016 by Delphix. All rights reserved. * Copyright (c) 2012 DEY Storage Systems, Inc. All rights reserved. * Copyright (c) 2011-2012 Pawel Jakub Dawidek. All rights reserved. @@ -1407,7 +1407,6 @@ badlabel: switch (prop) { case ZFS_PROP_RESERVATION: - case ZFS_PROP_REFRESERVATION: if (intval > volsize) { zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "'%s' is greater than current " @@ -1418,6 +1417,17 @@ badlabel: } break; + case ZFS_PROP_REFRESERVATION: + if (intval > volsize && intval != UINT64_MAX) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'%s' is greater than current " + "volume size"), propname); + (void) zfs_error(hdl, EZFS_BADPROP, + errbuf); + goto error; + } + break; + case ZFS_PROP_VOLSIZE: if (intval % blocksize != 0) { zfs_nicenum(blocksize, buf, @@ -1519,6 +1529,61 @@ zfs_add_synthetic_resv(zfs_handle_t *zhp, nvlist_t *nv return (1); } +/* + * Helper for 'zfs {set|clone} refreservation=auto'. Must be called after + * zfs_valid_proplist(), as it is what sets the UINT64_MAX sentinal value. + * Return codes must match zfs_add_synthetic_resv(). + */ +static int +zfs_fix_auto_resv(zfs_handle_t *zhp, nvlist_t *nvl) +{ + uint64_t volsize; + uint64_t resvsize; + zfs_prop_t prop; + nvlist_t *props; + + if (!ZFS_IS_VOLUME(zhp)) { + return (0); + } + + if (zfs_which_resv_prop(zhp, &prop) != 0) { + return (-1); + } + + if (prop != ZFS_PROP_REFRESERVATION) { + return (0); + } + + if (nvlist_lookup_uint64(nvl, zfs_prop_to_name(prop), &resvsize) != 0) { + /* No value being set, so it can't be "auto" */ + return (0); + } + if (resvsize != UINT64_MAX) { + /* Being set to a value other than "auto" */ + return (0); + } + + props = fnvlist_alloc(); + + fnvlist_add_uint64(props, zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), + zfs_prop_get_int(zhp, ZFS_PROP_VOLBLOCKSIZE)); + + if (nvlist_lookup_uint64(nvl, zfs_prop_to_name(ZFS_PROP_VOLSIZE), + &volsize) != 0) { + volsize = zfs_prop_get_int(zhp, ZFS_PROP_VOLSIZE); + } + + resvsize = zvol_volsize_to_reservation(volsize, props); + fnvlist_free(props); + + (void) nvlist_remove_all(nvl, zfs_prop_to_name(prop)); + if (nvlist_add_uint64(nvl, zfs_prop_to_name(prop), resvsize) != 0) { + (void) no_memory(zhp->zfs_hdl); + return (-1); + } + return (1); +} + void zfs_setprop_error(libzfs_handle_t *hdl, zfs_prop_t prop, int err, char *errbuf) @@ -1685,6 +1750,12 @@ zfs_prop_set_list(zfs_handle_t *zhp, nvlist_t *props) goto error; } } + + if (added_resv != 1 && + (added_resv = zfs_fix_auto_resv(zhp, nvl)) == -1) { + goto error; + } + /* * Check how many properties we're setting and allocate an array to * store changelist pointers for postfix(). @@ -3715,6 +3786,7 @@ zfs_clone(zfs_handle_t *zhp, const char *target, nvlis if (props) { zfs_type_t type; + if (ZFS_IS_VOLUME(zhp)) { type = ZFS_TYPE_VOLUME; } else { @@ -3723,6 +3795,10 @@ zfs_clone(zfs_handle_t *zhp, const char *target, nvlis if ((props = zfs_valid_proplist(hdl, type, props, zoned, zhp, zhp->zpool_hdl, errbuf)) == NULL) return (-1); + if (zfs_fix_auto_resv(zhp, props) == -1) { + nvlist_free(props); + return (-1); + } } ret = lzc_clone(target, zhp->zfs_name, props); Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c Mon Jul 30 22:10:15 2018 (r336944) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c Mon Jul 30 22:39:30 2018 (r336945) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, Joyent, Inc. All rights reserved. + * Copyright (c) 2018 Joyent, Inc. * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright 2016 Igor Kozhukhov * Copyright (c) 2017 Datto Inc. @@ -1227,6 +1227,7 @@ zprop_parse_value(libzfs_handle_t *hdl, nvpair_t *elem const char *propname; char *value; boolean_t isnone = B_FALSE; + boolean_t isauto = B_FALSE; if (type == ZFS_TYPE_POOL) { proptype = zpool_prop_get_type(prop); @@ -1262,8 +1263,9 @@ zprop_parse_value(libzfs_handle_t *hdl, nvpair_t *elem (void) nvpair_value_string(elem, &value); if (strcmp(value, "none") == 0) { isnone = B_TRUE; - } else if (zfs_nicestrtonum(hdl, value, ivalp) - != 0) { + } else if (strcmp(value, "auto") == 0) { + isauto = B_TRUE; + } else if (zfs_nicestrtonum(hdl, value, ivalp) != 0) { goto error; } } else if (datatype == DATA_TYPE_UINT64) { @@ -1293,6 +1295,31 @@ zprop_parse_value(libzfs_handle_t *hdl, nvpair_t *elem prop == ZFS_PROP_SNAPSHOT_LIMIT)) { *ivalp = UINT64_MAX; } + + /* + * Special handling for setting 'refreservation' to 'auto'. Use + * UINT64_MAX to tell the caller to use zfs_fix_auto_resv(). + * 'auto' is only allowed on volumes. + */ + if (isauto) { + switch (prop) { + case ZFS_PROP_REFRESERVATION: + if ((type & ZFS_TYPE_VOLUME) == 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'%s=auto' only allowed on " + "volumes"), nvpair_name(elem)); + goto error; + } + *ivalp = UINT64_MAX; + break; + default: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "'auto' is invalid value for '%s'"), + nvpair_name(elem)); + goto error; + } + } + break; case PROP_TYPE_INDEX: From owner-svn-src-all@freebsd.org Mon Jul 30 22:56:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79C01106365A; Mon, 30 Jul 2018 22:56:27 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CABE89729; Mon, 30 Jul 2018 22:56:27 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0DA282F0F; Mon, 30 Jul 2018 22:56:27 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UMuQDX043718; Mon, 30 Jul 2018 22:56:26 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UMuQMD043714; Mon, 30 Jul 2018 22:56:26 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302256.w6UMuQMD043714@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 22:56:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336946 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/... X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/ztest vendor/illumos/dist... X-SVN-Commit-Revision: 336946 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 22:56:27 -0000 Author: mav Date: Mon Jul 30 22:56:24 2018 New Revision: 336946 URL: https://svnweb.freebsd.org/changeset/base/336946 Log: 9238 ZFS Spacemap Encoding V2 The current space map encoding has the following disadvantages: [1] Assuming 512 sector size each entry can represent at most 16MB for a segment. This makes the encoding very inefficient for large regions of space. [2] As vdev-wide space maps have started to be used by new features (i.e. device removal, zpool checkpoint) we've started imposing limits in the vdevs that can be used with them based on the maximum addressable offset (currently 64PB for a top-level vdev). The new remains backwards compatible with the old one. The introduced two-word entry format, besides extending the limits imposed by the single-entry layout, also includes a vdev field and some extra padding after its prefix. The extra padding after the prefix should is reserved for future usage (e.g. new prefixes for future encodings or new fields for flags). The new vdev field not only makes the space maps more self-descriptive, but also opens the doors for pool-wide space maps. One final important note is that the number of bits used for vdevs is reduced to 24 bits for blkptrs. That was decided as we don't know of any setups that use more than 16M vdevs for the time being and we wanted to fit the vdev field in the space map. In addition that gives us some extra bits in dva_t. illumos/illumos-gate@17f11284b49b98353b5119463254074fd9bc0a28 Reviewed by: Matt Ahrens Reviewed by: George Wilson Approved by: Gordon Ross Author: Serapheim Dimitropoulos Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/man/man5/zpool-features.5 Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_checkpoint.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect_mapping.c Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Mon Jul 30 22:39:30 2018 (r336945) +++ vendor/illumos/dist/cmd/zdb/zdb.c Mon Jul 30 22:56:24 2018 (r336946) @@ -774,7 +774,6 @@ verify_spacemap_refcounts(spa_t *spa) static void dump_spacemap(objset_t *os, space_map_t *sm) { - uint64_t alloc, offset, entry; char *ddata[] = { "ALLOC", "FREE", "CONDENSE", "INVALID", "INVALID", "INVALID", "INVALID", "INVALID" }; @@ -791,41 +790,73 @@ dump_spacemap(objset_t *os, space_map_t *sm) /* * Print out the freelist entries in both encoded and decoded form. */ - alloc = 0; - for (offset = 0; offset < space_map_length(sm); - offset += sizeof (entry)) { - uint8_t mapshift = sm->sm_shift; + uint8_t mapshift = sm->sm_shift; + int64_t alloc = 0; + uint64_t word; + for (uint64_t offset = 0; offset < space_map_length(sm); + offset += sizeof (word)) { VERIFY0(dmu_read(os, space_map_object(sm), offset, - sizeof (entry), &entry, DMU_READ_PREFETCH)); - if (SM_DEBUG_DECODE(entry)) { + sizeof (word), &word, DMU_READ_PREFETCH)); + if (sm_entry_is_debug(word)) { (void) printf("\t [%6llu] %s: txg %llu, pass %llu\n", - (u_longlong_t)(offset / sizeof (entry)), - ddata[SM_DEBUG_ACTION_DECODE(entry)], - (u_longlong_t)SM_DEBUG_TXG_DECODE(entry), - (u_longlong_t)SM_DEBUG_SYNCPASS_DECODE(entry)); + (u_longlong_t)(offset / sizeof (word)), + ddata[SM_DEBUG_ACTION_DECODE(word)], + (u_longlong_t)SM_DEBUG_TXG_DECODE(word), + (u_longlong_t)SM_DEBUG_SYNCPASS_DECODE(word)); + continue; + } + + uint8_t words; + char entry_type; + uint64_t entry_off, entry_run, entry_vdev = SM_NO_VDEVID; + + if (sm_entry_is_single_word(word)) { + entry_type = (SM_TYPE_DECODE(word) == SM_ALLOC) ? + 'A' : 'F'; + entry_off = (SM_OFFSET_DECODE(word) << mapshift) + + sm->sm_start; + entry_run = SM_RUN_DECODE(word) << mapshift; + words = 1; } else { - (void) printf("\t [%6llu] %c range:" - " %010llx-%010llx size: %06llx\n", - (u_longlong_t)(offset / sizeof (entry)), - SM_TYPE_DECODE(entry) == SM_ALLOC ? 'A' : 'F', - (u_longlong_t)((SM_OFFSET_DECODE(entry) << - mapshift) + sm->sm_start), - (u_longlong_t)((SM_OFFSET_DECODE(entry) << - mapshift) + sm->sm_start + - (SM_RUN_DECODE(entry) << mapshift)), - (u_longlong_t)(SM_RUN_DECODE(entry) << mapshift)); - if (SM_TYPE_DECODE(entry) == SM_ALLOC) - alloc += SM_RUN_DECODE(entry) << mapshift; - else - alloc -= SM_RUN_DECODE(entry) << mapshift; + /* it is a two-word entry so we read another word */ + ASSERT(sm_entry_is_double_word(word)); + + uint64_t extra_word; + offset += sizeof (extra_word); + VERIFY0(dmu_read(os, space_map_object(sm), offset, + sizeof (extra_word), &extra_word, + DMU_READ_PREFETCH)); + + ASSERT3U(offset, <=, space_map_length(sm)); + + entry_run = SM2_RUN_DECODE(word) << mapshift; + entry_vdev = SM2_VDEV_DECODE(word); + entry_type = (SM2_TYPE_DECODE(extra_word) == SM_ALLOC) ? + 'A' : 'F'; + entry_off = (SM2_OFFSET_DECODE(extra_word) << + mapshift) + sm->sm_start; + words = 2; } + + (void) printf("\t [%6llu] %c range:" + " %010llx-%010llx size: %06llx vdev: %06llu words: %u\n", + (u_longlong_t)(offset / sizeof (word)), + entry_type, (u_longlong_t)entry_off, + (u_longlong_t)(entry_off + entry_run), + (u_longlong_t)entry_run, + (u_longlong_t)entry_vdev, words); + + if (entry_type == 'A') + alloc += entry_run; + else + alloc -= entry_run; } - if (alloc != space_map_allocated(sm)) { - (void) printf("space_map_object alloc (%llu) INCONSISTENT " - "with space map summary (%llu)\n", - (u_longlong_t)space_map_allocated(sm), (u_longlong_t)alloc); + if ((uint64_t)alloc != space_map_allocated(sm)) { + (void) printf("space_map_object alloc (%lld) INCONSISTENT " + "with space map summary (%lld)\n", + (longlong_t)space_map_allocated(sm), (longlong_t)alloc); } } @@ -1153,7 +1184,7 @@ dump_ddt(ddt_t *ddt, enum ddt_type type, enum ddt_clas while ((error = ddt_object_walk(ddt, type, class, &walk, &dde)) == 0) dump_dde(ddt, &dde, walk); - ASSERT(error == ENOENT); + ASSERT3U(error, ==, ENOENT); (void) printf("\n"); } @@ -3068,15 +3099,14 @@ typedef struct checkpoint_sm_exclude_entry_arg { } checkpoint_sm_exclude_entry_arg_t; static int -checkpoint_sm_exclude_entry_cb(maptype_t type, uint64_t offset, uint64_t size, - void *arg) +checkpoint_sm_exclude_entry_cb(space_map_entry_t *sme, void *arg) { checkpoint_sm_exclude_entry_arg_t *cseea = arg; vdev_t *vd = cseea->cseea_vd; - metaslab_t *ms = vd->vdev_ms[offset >> vd->vdev_ms_shift]; - uint64_t end = offset + size; + metaslab_t *ms = vd->vdev_ms[sme->sme_offset >> vd->vdev_ms_shift]; + uint64_t end = sme->sme_offset + sme->sme_run; - ASSERT(type == SM_FREE); + ASSERT(sme->sme_type == SM_FREE); /* * Since the vdev_checkpoint_sm exists in the vdev level @@ -3094,7 +3124,7 @@ checkpoint_sm_exclude_entry_cb(maptype_t type, uint64_ * metaslab boundaries. So if needed we could add code * that handles metaslab-crossing segments in the future. */ - VERIFY3U(offset, >=, ms->ms_start); + VERIFY3U(sme->sme_offset, >=, ms->ms_start); VERIFY3U(end, <=, ms->ms_start + ms->ms_size); /* @@ -3102,10 +3132,10 @@ checkpoint_sm_exclude_entry_cb(maptype_t type, uint64_ * also verify that the entry is there to begin with. */ mutex_enter(&ms->ms_lock); - range_tree_remove(ms->ms_allocatable, offset, size); + range_tree_remove(ms->ms_allocatable, sme->sme_offset, sme->sme_run); mutex_exit(&ms->ms_lock); - cseea->cseea_checkpoint_size += size; + cseea->cseea_checkpoint_size += sme->sme_run; return (0); } @@ -4080,15 +4110,14 @@ typedef struct verify_checkpoint_sm_entry_cb_arg { #define ENTRIES_PER_PROGRESS_UPDATE 10000 static int -verify_checkpoint_sm_entry_cb(maptype_t type, uint64_t offset, uint64_t size, - void *arg) +verify_checkpoint_sm_entry_cb(space_map_entry_t *sme, void *arg) { verify_checkpoint_sm_entry_cb_arg_t *vcsec = arg; vdev_t *vd = vcsec->vcsec_vd; - metaslab_t *ms = vd->vdev_ms[offset >> vd->vdev_ms_shift]; - uint64_t end = offset + size; + metaslab_t *ms = vd->vdev_ms[sme->sme_offset >> vd->vdev_ms_shift]; + uint64_t end = sme->sme_offset + sme->sme_run; - ASSERT(type == SM_FREE); + ASSERT(sme->sme_type == SM_FREE); if ((vcsec->vcsec_entryid % ENTRIES_PER_PROGRESS_UPDATE) == 0) { (void) fprintf(stderr, @@ -4102,7 +4131,7 @@ verify_checkpoint_sm_entry_cb(maptype_t type, uint64_t /* * See comment in checkpoint_sm_exclude_entry_cb() */ - VERIFY3U(offset, >=, ms->ms_start); + VERIFY3U(sme->sme_offset, >=, ms->ms_start); VERIFY3U(end, <=, ms->ms_start + ms->ms_size); /* @@ -4111,7 +4140,7 @@ verify_checkpoint_sm_entry_cb(maptype_t type, uint64_t * their respective ms_allocateable trees should not contain them. */ mutex_enter(&ms->ms_lock); - range_tree_verify(ms->ms_allocatable, offset, size); + range_tree_verify(ms->ms_allocatable, sme->sme_offset, sme->sme_run); mutex_exit(&ms->ms_lock); return (0); @@ -4357,7 +4386,7 @@ verify_checkpoint(spa_t *spa) DMU_POOL_ZPOOL_CHECKPOINT, sizeof (uint64_t), sizeof (uberblock_t) / sizeof (uint64_t), &checkpoint); - if (error == ENOENT) { + if (error == ENOENT && !dump_opt['L']) { /* * If the feature is active but the uberblock is missing * then we must be in the middle of discarding the @@ -4380,7 +4409,7 @@ verify_checkpoint(spa_t *spa) error = 3; } - if (error == 0) + if (error == 0 && !dump_opt['L']) verify_checkpoint_blocks(spa); return (error); @@ -4485,7 +4514,7 @@ dump_zpool(spa_t *spa) if (dump_opt['h']) dump_history(spa); - if (rc == 0 && !dump_opt['L']) + if (rc == 0) rc = verify_checkpoint(spa); if (rc != 0) { Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Mon Jul 30 22:39:30 2018 (r336945) +++ vendor/illumos/dist/cmd/ztest/ztest.c Mon Jul 30 22:56:24 2018 (r336946) @@ -193,6 +193,7 @@ extern uint64_t zfs_deadman_synctime_ms; extern int metaslab_preload_limit; extern boolean_t zfs_compressed_arc_enabled; extern boolean_t zfs_abd_scatter_enabled; +extern boolean_t zfs_force_some_double_word_sm_entries; static ztest_shared_opts_t *ztest_shared_opts; static ztest_shared_opts_t ztest_opts; @@ -6390,6 +6391,12 @@ main(int argc, char **argv) dprintf_setup(&argc, argv); zfs_deadman_synctime_ms = 300000; + /* + * As two-word space map entries may not come up often (especially + * if pool and vdev sizes are small) we want to force at least some + * of them so the feature get tested. + */ + zfs_force_some_double_word_sm_entries = B_TRUE; ztest_fd_rand = open("/dev/urandom", O_RDONLY); ASSERT3S(ztest_fd_rand, >=, 0); Modified: vendor/illumos/dist/man/man5/zpool-features.5 ============================================================================== --- vendor/illumos/dist/man/man5/zpool-features.5 Mon Jul 30 22:39:30 2018 (r336945) +++ vendor/illumos/dist/man/man5/zpool-features.5 Mon Jul 30 22:56:24 2018 (r336946) @@ -423,7 +423,6 @@ This feature becomes \fBactive\fR as soon as it is ena never return to being \fBenabled\fR. .RE - .sp .ne 2 .na @@ -488,6 +487,34 @@ This feature becomes \fBactive\fR when the "zpool chec is used to checkpoint the pool. The feature will only return back to being \fBenabled\fR when the pool is rewound or the checkpoint has been discarded. + +.RE +.sp +.ne 2 +.na +\fB\fBspacemap_v2\fR\fR +.ad +.RS 4n +.TS +l l . +GUID com.delphix:spacemap_v2 +READ\-ONLY COMPATIBLE yes +DEPENDENCIES none +.TE + +This feature enables the use of the new space map encoding which +consists of two words (instead of one) whenever it is advantageous. +The new encoding allows space maps to represent large regions of +space more efficiently on-disk while also increasing their maximum +addressable offset. + +This feature becomes \fBactive\fR once it is \fBenabled\fR, and never +returns back to being \fBenabled\fR. + +.RE +.sp +.ne 2 +.na \fB\fBlarge_blocks\fR\fR .ad .RS 4n From owner-svn-src-all@freebsd.org Mon Jul 30 22:56:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B93801063655; Mon, 30 Jul 2018 22:56:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A66C89728; Mon, 30 Jul 2018 22:56:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 470C02F0E; Mon, 30 Jul 2018 22:56:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UMuQId043708; Mon, 30 Jul 2018 22:56:26 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UMuPvv043700; Mon, 30 Jul 2018 22:56:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302256.w6UMuPvv043700@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 22:56:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336946 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/... X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/ztest vendor/illumos/dist... X-SVN-Commit-Revision: 336946 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 22:56:27 -0000 Author: mav Date: Mon Jul 30 22:56:24 2018 New Revision: 336946 URL: https://svnweb.freebsd.org/changeset/base/336946 Log: 9238 ZFS Spacemap Encoding V2 The current space map encoding has the following disadvantages: [1] Assuming 512 sector size each entry can represent at most 16MB for a segment. This makes the encoding very inefficient for large regions of space. [2] As vdev-wide space maps have started to be used by new features (i.e. device removal, zpool checkpoint) we've started imposing limits in the vdevs that can be used with them based on the maximum addressable offset (currently 64PB for a top-level vdev). The new remains backwards compatible with the old one. The introduced two-word entry format, besides extending the limits imposed by the single-entry layout, also includes a vdev field and some extra padding after its prefix. The extra padding after the prefix should is reserved for future usage (e.g. new prefixes for future encodings or new fields for flags). The new vdev field not only makes the space maps more self-descriptive, but also opens the doors for pool-wide space maps. One final important note is that the number of bits used for vdevs is reduced to 24 bits for blkptrs. That was decided as we don't know of any setups that use more than 16M vdevs for the time being and we wanted to fit the vdev field in the space map. In addition that gives us some extra bits in dva_t. illumos/illumos-gate@17f11284b49b98353b5119463254074fd9bc0a28 Reviewed by: Matt Ahrens Reviewed by: George Wilson Approved by: Gordon Ross Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c vendor-sys/illumos/dist/common/zfs/zfeature_common.h vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_checkpoint.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/space_map.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect_mapping.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/man/man5/zpool-features.5 Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.c Mon Jul 30 22:39:30 2018 (r336945) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.c Mon Jul 30 22:56:24 2018 (r336946) @@ -230,6 +230,12 @@ zpool_feature_init(void) "Pool state can be checkpointed, allowing rewind later.", ZFEATURE_FLAG_READONLY_COMPAT, NULL); + zfeature_register(SPA_FEATURE_SPACEMAP_V2, + "com.delphix:spacemap_v2", "spacemap_v2", + "Space maps representing large segments are more efficient.", + ZFEATURE_FLAG_READONLY_COMPAT | ZFEATURE_FLAG_ACTIVATE_ON_ENABLE, + NULL); + static const spa_feature_t large_blocks_deps[] = { SPA_FEATURE_EXTENSIBLE_DATASET, SPA_FEATURE_NONE Modified: vendor-sys/illumos/dist/common/zfs/zfeature_common.h ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfeature_common.h Mon Jul 30 22:39:30 2018 (r336945) +++ vendor-sys/illumos/dist/common/zfs/zfeature_common.h Mon Jul 30 22:56:24 2018 (r336946) @@ -59,6 +59,7 @@ typedef enum spa_feature { SPA_FEATURE_DEVICE_REMOVAL, SPA_FEATURE_OBSOLETE_COUNTS, SPA_FEATURE_POOL_CHECKPOINT, + SPA_FEATURE_SPACEMAP_V2, SPA_FEATURES } spa_feature_t; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Mon Jul 30 22:39:30 2018 (r336945) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Mon Jul 30 22:56:24 2018 (r336946) @@ -2091,17 +2091,6 @@ metaslab_group_preload(metaslab_group_t *mg) * * 3. The on-disk size of the space map should actually decrease. * - * Checking the first condition is tricky since we don't want to walk - * the entire AVL tree calculating the estimated on-disk size. Instead we - * use the size-ordered range tree in the metaslab and calculate the - * size required to write out the largest segment in our free tree. If the - * size required to represent that segment on disk is larger than the space - * map object then we avoid condensing this map. - * - * To determine the second criterion we use a best-case estimate and assume - * each segment can be represented on-disk as a single 64-bit entry. We refer - * to this best-case estimate as the space map's minimal form. - * * Unfortunately, we cannot compute the on-disk size of the space map in this * context because we cannot accurately compute the effects of compression, etc. * Instead, we apply the heuristic described in the block comment for @@ -2112,9 +2101,6 @@ static boolean_t metaslab_should_condense(metaslab_t *msp) { space_map_t *sm = msp->ms_sm; - range_seg_t *rs; - uint64_t size, entries, segsz, object_size, optimal_size, record_size; - dmu_object_info_t doi; vdev_t *vd = msp->ms_group->mg_vd; uint64_t vdev_blocksize = 1 << vd->vdev_ashift; uint64_t current_txg = spa_syncing_txg(vd->vdev_spa); @@ -2140,34 +2126,22 @@ metaslab_should_condense(metaslab_t *msp) msp->ms_condense_checked_txg = current_txg; /* - * Use the ms_allocatable_by_size range tree, which is ordered by - * size, to obtain the largest segment in the free tree. We always - * condense metaslabs that are empty and metaslabs for which a - * condense request has been made. + * We always condense metaslabs that are empty and metaslabs for + * which a condense request has been made. */ - rs = avl_last(&msp->ms_allocatable_by_size); - if (rs == NULL || msp->ms_condense_wanted) + if (avl_is_empty(&msp->ms_allocatable_by_size) || + msp->ms_condense_wanted) return (B_TRUE); - /* - * Calculate the number of 64-bit entries this segment would - * require when written to disk. If this single segment would be - * larger on-disk than the entire current on-disk structure, then - * clearly condensing will increase the on-disk structure size. - */ - size = (rs->rs_end - rs->rs_start) >> sm->sm_shift; - entries = size / (MIN(size, SM_RUN_MAX)); - segsz = entries * sizeof (uint64_t); + uint64_t object_size = space_map_length(msp->ms_sm); + uint64_t optimal_size = space_map_estimate_optimal_size(sm, + msp->ms_allocatable, SM_NO_VDEVID); - optimal_size = - sizeof (uint64_t) * avl_numnodes(&msp->ms_allocatable->rt_root); - object_size = space_map_length(msp->ms_sm); - + dmu_object_info_t doi; dmu_object_info_from_db(sm->sm_dbuf, &doi); - record_size = MAX(doi.doi_data_block_size, vdev_blocksize); + uint64_t record_size = MAX(doi.doi_data_block_size, vdev_blocksize); - return (segsz <= object_size && - object_size >= (optimal_size * zfs_condense_pct / 100) && + return (object_size >= (optimal_size * zfs_condense_pct / 100) && object_size > zfs_metaslab_condense_block_threshold * record_size); } @@ -2242,11 +2216,11 @@ metaslab_condense(metaslab_t *msp, uint64_t txg, dmu_t * optimal, this is typically close to optimal, and much cheaper to * compute. */ - space_map_write(sm, condense_tree, SM_ALLOC, tx); + space_map_write(sm, condense_tree, SM_ALLOC, SM_NO_VDEVID, tx); range_tree_vacate(condense_tree, NULL, NULL); range_tree_destroy(condense_tree); - space_map_write(sm, msp->ms_allocatable, SM_FREE, tx); + space_map_write(sm, msp->ms_allocatable, SM_FREE, SM_NO_VDEVID, tx); mutex_enter(&msp->ms_lock); msp->ms_condensing = B_FALSE; } @@ -2358,8 +2332,10 @@ metaslab_sync(metaslab_t *msp, uint64_t txg) metaslab_condense(msp, txg, tx); } else { mutex_exit(&msp->ms_lock); - space_map_write(msp->ms_sm, alloctree, SM_ALLOC, tx); - space_map_write(msp->ms_sm, msp->ms_freeing, SM_FREE, tx); + space_map_write(msp->ms_sm, alloctree, SM_ALLOC, + SM_NO_VDEVID, tx); + space_map_write(msp->ms_sm, msp->ms_freeing, SM_FREE, + SM_NO_VDEVID, tx); mutex_enter(&msp->ms_lock); } @@ -2374,7 +2350,7 @@ metaslab_sync(metaslab_t *msp, uint64_t txg) */ mutex_exit(&msp->ms_lock); space_map_write(vd->vdev_checkpoint_sm, - msp->ms_checkpointing, SM_FREE, tx); + msp->ms_checkpointing, SM_FREE, SM_NO_VDEVID, tx); mutex_enter(&msp->ms_lock); space_map_update(vd->vdev_checkpoint_sm); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c Mon Jul 30 22:39:30 2018 (r336945) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c Mon Jul 30 22:56:24 2018 (r336946) @@ -179,7 +179,7 @@ range_tree_add(void *arg, uint64_t start, uint64_t siz } /* Make sure we don't overlap with either of our neighbors */ - VERIFY(rs == NULL); + VERIFY3P(rs, ==, NULL); rs_before = avl_nearest(&rt->rt_root, where, AVL_BEFORE); rs_after = avl_nearest(&rt->rt_root, where, AVL_AFTER); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_checkpoint.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_checkpoint.c Mon Jul 30 22:39:30 2018 (r336945) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_checkpoint.c Mon Jul 30 22:56:24 2018 (r336946) @@ -203,13 +203,12 @@ typedef struct spa_checkpoint_discard_sync_callback_ar } spa_checkpoint_discard_sync_callback_arg_t; static int -spa_checkpoint_discard_sync_callback(maptype_t type, uint64_t offset, - uint64_t size, void *arg) +spa_checkpoint_discard_sync_callback(space_map_entry_t *sme, void *arg) { spa_checkpoint_discard_sync_callback_arg_t *sdc = arg; vdev_t *vd = sdc->sdc_vd; - metaslab_t *ms = vd->vdev_ms[offset >> vd->vdev_ms_shift]; - uint64_t end = offset + size; + metaslab_t *ms = vd->vdev_ms[sme->sme_offset >> vd->vdev_ms_shift]; + uint64_t end = sme->sme_offset + sme->sme_run; if (sdc->sdc_entry_limit == 0) return (EINTR); @@ -224,8 +223,8 @@ spa_checkpoint_discard_sync_callback(maptype_t type, u * metaslab boundaries. So if needed we could add code * that handles metaslab-crossing segments in the future. */ - VERIFY3U(type, ==, SM_FREE); - VERIFY3U(offset, >=, ms->ms_start); + VERIFY3U(sme->sme_type, ==, SM_FREE); + VERIFY3U(sme->sme_offset, >=, ms->ms_start); VERIFY3U(end, <=, ms->ms_start + ms->ms_size); /* @@ -237,14 +236,15 @@ spa_checkpoint_discard_sync_callback(maptype_t type, u mutex_enter(&ms->ms_lock); if (range_tree_is_empty(ms->ms_freeing)) vdev_dirty(vd, VDD_METASLAB, ms, sdc->sdc_txg); - range_tree_add(ms->ms_freeing, offset, size); + range_tree_add(ms->ms_freeing, sme->sme_offset, sme->sme_run); mutex_exit(&ms->ms_lock); - ASSERT3U(vd->vdev_spa->spa_checkpoint_info.sci_dspace, >=, size); - ASSERT3U(vd->vdev_stat.vs_checkpoint_space, >=, size); + ASSERT3U(vd->vdev_spa->spa_checkpoint_info.sci_dspace, >=, + sme->sme_run); + ASSERT3U(vd->vdev_stat.vs_checkpoint_space, >=, sme->sme_run); - vd->vdev_spa->spa_checkpoint_info.sci_dspace -= size; - vd->vdev_stat.vs_checkpoint_space -= size; + vd->vdev_spa->spa_checkpoint_info.sci_dspace -= sme->sme_run; + vd->vdev_stat.vs_checkpoint_space -= sme->sme_run; sdc->sdc_entry_limit--; return (0); @@ -289,13 +289,14 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t * Thus, we set the maximum entries that the space map callback * will be applied to be half the entries that could fit in the * imposed memory limit. + * + * Note that since this is a conservative estimate we also + * assume the worst case scenario in our computation where each + * entry is two-word. */ uint64_t max_entry_limit = - (zfs_spa_discard_memory_limit / sizeof (uint64_t)) >> 1; + (zfs_spa_discard_memory_limit / (2 * sizeof (uint64_t))) >> 1; - uint64_t entries_in_sm = - space_map_length(vd->vdev_checkpoint_sm) / sizeof (uint64_t); - /* * Iterate from the end of the space map towards the beginning, * placing its entries on ms_freeing and removing them from the @@ -318,14 +319,15 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t spa_checkpoint_discard_sync_callback_arg_t sdc; sdc.sdc_vd = vd; sdc.sdc_txg = tx->tx_txg; - sdc.sdc_entry_limit = MIN(entries_in_sm, max_entry_limit); + sdc.sdc_entry_limit = max_entry_limit; - uint64_t entries_before = entries_in_sm; + uint64_t words_before = + space_map_length(vd->vdev_checkpoint_sm) / sizeof (uint64_t); error = space_map_incremental_destroy(vd->vdev_checkpoint_sm, spa_checkpoint_discard_sync_callback, &sdc, tx); - uint64_t entries_after = + uint64_t words_after = space_map_length(vd->vdev_checkpoint_sm) / sizeof (uint64_t); #ifdef DEBUG @@ -333,9 +335,9 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t #endif zfs_dbgmsg("discarding checkpoint: txg %llu, vdev id %d, " - "deleted %llu entries - %llu entries are left", - tx->tx_txg, vd->vdev_id, (entries_before - entries_after), - entries_after); + "deleted %llu words - %llu words are left", + tx->tx_txg, vd->vdev_id, (words_before - words_after), + words_after); if (error != EINTR) { if (error != 0) { @@ -344,15 +346,15 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t "space map of vdev %llu\n", error, vd->vdev_id); } - ASSERT0(entries_after); + ASSERT0(words_after); ASSERT0(vd->vdev_checkpoint_sm->sm_alloc); - ASSERT0(vd->vdev_checkpoint_sm->sm_length); + ASSERT0(space_map_length(vd->vdev_checkpoint_sm)); space_map_free(vd->vdev_checkpoint_sm, tx); space_map_close(vd->vdev_checkpoint_sm); vd->vdev_checkpoint_sm = NULL; - VERIFY0(zap_remove(vd->vdev_spa->spa_meta_objset, + VERIFY0(zap_remove(spa_meta_objset(vd->vdev_spa), vd->vdev_top_zap, VDEV_TOP_ZAP_POOL_CHECKPOINT_SM, tx)); } } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Mon Jul 30 22:39:30 2018 (r336945) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Mon Jul 30 22:56:24 2018 (r336946) @@ -41,68 +41,194 @@ * Note on space map block size: * * The data for a given space map can be kept on blocks of any size. - * Larger blocks entail fewer i/o operations, but they also cause the - * DMU to keep more data in-core, and also to waste more i/o bandwidth + * Larger blocks entail fewer I/O operations, but they also cause the + * DMU to keep more data in-core, and also to waste more I/O bandwidth * when only a few blocks have changed since the last transaction group. */ /* + * Enabled whenever we want to stress test the use of double-word + * space map entries. + */ +boolean_t zfs_force_some_double_word_sm_entries = B_FALSE; + +boolean_t +sm_entry_is_debug(uint64_t e) +{ + return (SM_PREFIX_DECODE(e) == SM_DEBUG_PREFIX); +} + +boolean_t +sm_entry_is_single_word(uint64_t e) +{ + uint8_t prefix = SM_PREFIX_DECODE(e); + return (prefix != SM_DEBUG_PREFIX && prefix != SM2_PREFIX); +} + +boolean_t +sm_entry_is_double_word(uint64_t e) +{ + return (SM_PREFIX_DECODE(e) == SM2_PREFIX); +} + +/* * Iterate through the space map, invoking the callback on each (non-debug) * space map entry. */ int space_map_iterate(space_map_t *sm, sm_cb_t callback, void *arg) { - uint64_t *entry, *entry_map, *entry_map_end; - uint64_t bufsize, size, offset, end; + uint64_t sm_len = space_map_length(sm); + ASSERT3U(sm->sm_blksz, !=, 0); + + dmu_prefetch(sm->sm_os, space_map_object(sm), 0, 0, sm_len, + ZIO_PRIORITY_SYNC_READ); + + uint64_t blksz = sm->sm_blksz; int error = 0; + for (uint64_t block_base = 0; block_base < sm_len && error == 0; + block_base += blksz) { + dmu_buf_t *db; + error = dmu_buf_hold(sm->sm_os, space_map_object(sm), + block_base, FTAG, &db, DMU_READ_PREFETCH); + if (error != 0) + return (error); - end = space_map_length(sm); + uint64_t *block_start = db->db_data; + uint64_t block_length = MIN(sm_len - block_base, blksz); + uint64_t *block_end = block_start + + (block_length / sizeof (uint64_t)); - bufsize = MAX(sm->sm_blksz, SPA_MINBLOCKSIZE); - entry_map = zio_buf_alloc(bufsize); + VERIFY0(P2PHASE(block_length, sizeof (uint64_t))); + VERIFY3U(block_length, !=, 0); + ASSERT3U(blksz, ==, db->db_size); - if (end > bufsize) { - dmu_prefetch(sm->sm_os, space_map_object(sm), 0, bufsize, - end - bufsize, ZIO_PRIORITY_SYNC_READ); + for (uint64_t *block_cursor = block_start; + block_cursor < block_end && error == 0; block_cursor++) { + uint64_t e = *block_cursor; + + if (sm_entry_is_debug(e)) /* Skip debug entries */ + continue; + + uint64_t raw_offset, raw_run, vdev_id; + maptype_t type; + if (sm_entry_is_single_word(e)) { + type = SM_TYPE_DECODE(e); + vdev_id = SM_NO_VDEVID; + raw_offset = SM_OFFSET_DECODE(e); + raw_run = SM_RUN_DECODE(e); + } else { + /* it is a two-word entry */ + ASSERT(sm_entry_is_double_word(e)); + raw_run = SM2_RUN_DECODE(e); + vdev_id = SM2_VDEV_DECODE(e); + + /* move on to the second word */ + block_cursor++; + e = *block_cursor; + VERIFY3P(block_cursor, <=, block_end); + + type = SM2_TYPE_DECODE(e); + raw_offset = SM2_OFFSET_DECODE(e); + } + + uint64_t entry_offset = (raw_offset << sm->sm_shift) + + sm->sm_start; + uint64_t entry_run = raw_run << sm->sm_shift; + + VERIFY0(P2PHASE(entry_offset, 1ULL << sm->sm_shift)); + VERIFY0(P2PHASE(entry_run, 1ULL << sm->sm_shift)); + ASSERT3U(entry_offset, >=, sm->sm_start); + ASSERT3U(entry_offset, <, sm->sm_start + sm->sm_size); + ASSERT3U(entry_run, <=, sm->sm_size); + ASSERT3U(entry_offset + entry_run, <=, + sm->sm_start + sm->sm_size); + + space_map_entry_t sme = { + .sme_type = type, + .sme_vdev = vdev_id, + .sme_offset = entry_offset, + .sme_run = entry_run + }; + error = callback(&sme, arg); + } + dmu_buf_rele(db, FTAG); } + return (error); +} - for (offset = 0; offset < end && error == 0; offset += bufsize) { - size = MIN(end - offset, bufsize); - VERIFY(P2PHASE(size, sizeof (uint64_t)) == 0); - VERIFY(size != 0); - ASSERT3U(sm->sm_blksz, !=, 0); +/* + * Reads the entries from the last block of the space map into + * buf in reverse order. Populates nwords with number of words + * in the last block. + * + * Refer to block comment within space_map_incremental_destroy() + * to understand why this function is needed. + */ +static int +space_map_reversed_last_block_entries(space_map_t *sm, uint64_t *buf, + uint64_t bufsz, uint64_t *nwords) +{ + int error = 0; + dmu_buf_t *db; - dprintf("object=%llu offset=%llx size=%llx\n", - space_map_object(sm), offset, size); + /* + * Find the offset of the last word in the space map and use + * that to read the last block of the space map with + * dmu_buf_hold(). + */ + uint64_t last_word_offset = + sm->sm_phys->smp_objsize - sizeof (uint64_t); + error = dmu_buf_hold(sm->sm_os, space_map_object(sm), last_word_offset, + FTAG, &db, DMU_READ_NO_PREFETCH); + if (error != 0) + return (error); - error = dmu_read(sm->sm_os, space_map_object(sm), offset, size, - entry_map, DMU_READ_PREFETCH); - if (error != 0) - break; + ASSERT3U(sm->sm_object, ==, db->db_object); + ASSERT3U(sm->sm_blksz, ==, db->db_size); + ASSERT3U(bufsz, >=, db->db_size); + ASSERT(nwords != NULL); - entry_map_end = entry_map + (size / sizeof (uint64_t)); - for (entry = entry_map; entry < entry_map_end && error == 0; - entry++) { - uint64_t e = *entry; - uint64_t offset, size; + uint64_t *words = db->db_data; + *nwords = + (sm->sm_phys->smp_objsize - db->db_offset) / sizeof (uint64_t); - if (SM_DEBUG_DECODE(e)) /* Skip debug entries */ - continue; + ASSERT3U(*nwords, <=, bufsz / sizeof (uint64_t)); - offset = (SM_OFFSET_DECODE(e) << sm->sm_shift) + - sm->sm_start; - size = SM_RUN_DECODE(e) << sm->sm_shift; + uint64_t n = *nwords; + uint64_t j = n - 1; + for (uint64_t i = 0; i < n; i++) { + uint64_t entry = words[i]; + if (sm_entry_is_double_word(entry)) { + /* + * Since we are populating the buffer backwards + * we have to be extra careful and add the two + * words of the double-word entry in the right + * order. + */ + ASSERT3U(j, >, 0); + buf[j - 1] = entry; - VERIFY0(P2PHASE(offset, 1ULL << sm->sm_shift)); - VERIFY0(P2PHASE(size, 1ULL << sm->sm_shift)); - VERIFY3U(offset, >=, sm->sm_start); - VERIFY3U(offset + size, <=, sm->sm_start + sm->sm_size); - error = callback(SM_TYPE_DECODE(e), offset, size, arg); + i++; + ASSERT3U(i, <, n); + entry = words[i]; + buf[j] = entry; + j -= 2; + } else { + ASSERT(sm_entry_is_debug(entry) || + sm_entry_is_single_word(entry)); + buf[j] = entry; + j--; } } - zio_buf_free(entry_map, bufsize); + /* + * Assert that we wrote backwards all the + * way to the beginning of the buffer. + */ + ASSERT3S(j, ==, -1); + + dmu_buf_rele(db, FTAG); return (error); } @@ -116,124 +242,122 @@ int space_map_incremental_destroy(space_map_t *sm, sm_cb_t callback, void *arg, dmu_tx_t *tx) { - uint64_t bufsize, len; - uint64_t *entry_map; - int error = 0; + uint64_t bufsz = MAX(sm->sm_blksz, SPA_MINBLOCKSIZE); + uint64_t *buf = zio_buf_alloc(bufsz); - len = space_map_length(sm); - bufsize = MAX(sm->sm_blksz, SPA_MINBLOCKSIZE); - entry_map = zio_buf_alloc(bufsize); - dmu_buf_will_dirty(sm->sm_dbuf, tx); /* - * Since we can't move the starting offset of the space map - * (e.g there are reference on-disk pointing to it), we destroy - * its entries incrementally starting from the end. + * Ideally we would want to iterate from the beginning of the + * space map to the end in incremental steps. The issue with this + * approach is that we don't have any field on-disk that points + * us where to start between each step. We could try zeroing out + * entries that we've destroyed, but this doesn't work either as + * an entry that is 0 is a valid one (ALLOC for range [0x0:0x200]). * - * The logic that follows is basically the same as the one used - * in space_map_iterate() but it traverses the space map - * backwards: + * As a result, we destroy its entries incrementally starting from + * the end after applying the callback to each of them. * - * 1] We figure out the size of the buffer that we want to use - * to read the on-disk space map entries. - * 2] We figure out the offset at the end of the space map where - * we will start reading entries into our buffer. - * 3] We read the on-disk entries into the buffer. - * 4] We iterate over the entries from end to beginning calling - * the callback function on each one. As we move from entry - * to entry we decrease the size of the space map, deleting - * effectively each entry. - * 5] If there are no more entries in the space map or the - * callback returns a value other than 0, we stop iterating - * over the space map. If there are entries remaining and - * the callback returned zero we go back to step [1]. + * The problem with this approach is that we cannot literally + * iterate through the words in the space map backwards as we + * can't distinguish two-word space map entries from their second + * word. Thus we do the following: + * + * 1] We get all the entries from the last block of the space map + * and put them into a buffer in reverse order. This way the + * last entry comes first in the buffer, the second to last is + * second, etc. + * 2] We iterate through the entries in the buffer and we apply + * the callback to each one. As we move from entry to entry we + * we decrease the size of the space map, deleting effectively + * each entry. + * 3] If there are no more entries in the space map or the callback + * returns a value other than 0, we stop iterating over the + * space map. If there are entries remaining and the callback + * returned 0, we go back to step [1]. */ - uint64_t offset = 0, size = 0; - while (len > 0 && error == 0) { - size = MIN(bufsize, len); - - VERIFY(P2PHASE(size, sizeof (uint64_t)) == 0); - VERIFY3U(size, >, 0); - ASSERT3U(sm->sm_blksz, !=, 0); - - offset = len - size; - - IMPLY(bufsize > len, offset == 0); - IMPLY(bufsize == len, offset == 0); - IMPLY(bufsize < len, offset > 0); - - - EQUIV(size == len, offset == 0); - IMPLY(size < len, bufsize < len); - - dprintf("object=%llu offset=%llx size=%llx\n", - space_map_object(sm), offset, size); - - error = dmu_read(sm->sm_os, space_map_object(sm), - offset, size, entry_map, DMU_READ_PREFETCH); + int error = 0; + while (space_map_length(sm) > 0 && error == 0) { + uint64_t nwords = 0; + error = space_map_reversed_last_block_entries(sm, buf, bufsz, + &nwords); if (error != 0) break; - uint64_t num_entries = size / sizeof (uint64_t); + ASSERT3U(nwords, <=, bufsz / sizeof (uint64_t)); - ASSERT3U(num_entries, >, 0); + for (uint64_t i = 0; i < nwords; i++) { + uint64_t e = buf[i]; - while (num_entries > 0) { - uint64_t e, entry_offset, entry_size; + if (sm_entry_is_debug(e)) { + sm->sm_phys->smp_objsize -= sizeof (uint64_t); + space_map_update(sm); + continue; + } + + int words = 1; + uint64_t raw_offset, raw_run, vdev_id; maptype_t type; + if (sm_entry_is_single_word(e)) { + type = SM_TYPE_DECODE(e); + vdev_id = SM_NO_VDEVID; + raw_offset = SM_OFFSET_DECODE(e); + raw_run = SM_RUN_DECODE(e); + } else { + ASSERT(sm_entry_is_double_word(e)); + words = 2; - e = entry_map[num_entries - 1]; + raw_run = SM2_RUN_DECODE(e); + vdev_id = SM2_VDEV_DECODE(e); - ASSERT3U(num_entries, >, 0); - ASSERT0(error); + /* move to the second word */ + i++; + e = buf[i]; - if (SM_DEBUG_DECODE(e)) { - sm->sm_phys->smp_objsize -= sizeof (uint64_t); - space_map_update(sm); - len -= sizeof (uint64_t); - num_entries--; - continue; + ASSERT3P(i, <=, nwords); + + type = SM2_TYPE_DECODE(e); + raw_offset = SM2_OFFSET_DECODE(e); } - type = SM_TYPE_DECODE(e); - entry_offset = (SM_OFFSET_DECODE(e) << sm->sm_shift) + - sm->sm_start; - entry_size = SM_RUN_DECODE(e) << sm->sm_shift; + uint64_t entry_offset = + (raw_offset << sm->sm_shift) + sm->sm_start; + uint64_t entry_run = raw_run << sm->sm_shift; VERIFY0(P2PHASE(entry_offset, 1ULL << sm->sm_shift)); - VERIFY0(P2PHASE(entry_size, 1ULL << sm->sm_shift)); + VERIFY0(P2PHASE(entry_run, 1ULL << sm->sm_shift)); VERIFY3U(entry_offset, >=, sm->sm_start); - VERIFY3U(entry_offset + entry_size, <=, + VERIFY3U(entry_offset, <, sm->sm_start + sm->sm_size); + VERIFY3U(entry_run, <=, sm->sm_size); + VERIFY3U(entry_offset + entry_run, <=, sm->sm_start + sm->sm_size); - error = callback(type, entry_offset, entry_size, arg); + space_map_entry_t sme = { + .sme_type = type, + .sme_vdev = vdev_id, + .sme_offset = entry_offset, + .sme_run = entry_run + }; + error = callback(&sme, arg); if (error != 0) break; if (type == SM_ALLOC) - sm->sm_phys->smp_alloc -= entry_size; + sm->sm_phys->smp_alloc -= entry_run; else - sm->sm_phys->smp_alloc += entry_size; - - sm->sm_phys->smp_objsize -= sizeof (uint64_t); + sm->sm_phys->smp_alloc += entry_run; + sm->sm_phys->smp_objsize -= words * sizeof (uint64_t); space_map_update(sm); - len -= sizeof (uint64_t); - num_entries--; } - IMPLY(error == 0, num_entries == 0); - EQUIV(offset == 0 && error == 0, len == 0 && num_entries == 0); } - if (len == 0) { + if (space_map_length(sm) == 0) { ASSERT0(error); - ASSERT0(offset); - ASSERT0(sm->sm_length); ASSERT0(sm->sm_phys->smp_objsize); ASSERT0(sm->sm_alloc); } - zio_buf_free(entry_map, bufsize); + zio_buf_free(buf, bufsz); return (error); } @@ -244,16 +368,15 @@ typedef struct space_map_load_arg { } space_map_load_arg_t; static int -space_map_load_callback(maptype_t type, uint64_t offset, uint64_t size, - void *arg) +space_map_load_callback(space_map_entry_t *sme, void *arg) { space_map_load_arg_t *smla = arg; - if (type == smla->smla_type) { - VERIFY3U(range_tree_space(smla->smla_rt) + size, <=, + if (sme->sme_type == smla->smla_type) { + VERIFY3U(range_tree_space(smla->smla_rt) + sme->sme_run, <=, smla->smla_sm->sm_size); - range_tree_add(smla->smla_rt, offset, size); + range_tree_add(smla->smla_rt, sme->sme_offset, sme->sme_run); } else { - range_tree_remove(smla->smla_rt, offset, size); + range_tree_remove(smla->smla_rt, sme->sme_offset, sme->sme_run); } return (0); @@ -365,43 +488,239 @@ space_map_histogram_add(space_map_t *sm, range_tree_t } } -uint64_t -space_map_entries(space_map_t *sm, range_tree_t *rt) +static void +space_map_write_intro_debug(space_map_t *sm, maptype_t maptype, dmu_tx_t *tx) { - avl_tree_t *t = &rt->rt_root; - range_seg_t *rs; - uint64_t size, entries; + dmu_buf_will_dirty(sm->sm_dbuf, tx); + uint64_t dentry = SM_PREFIX_ENCODE(SM_DEBUG_PREFIX) | + SM_DEBUG_ACTION_ENCODE(maptype) | + SM_DEBUG_SYNCPASS_ENCODE(spa_sync_pass(tx->tx_pool->dp_spa)) | + SM_DEBUG_TXG_ENCODE(dmu_tx_get_txg(tx)); + + dmu_write(sm->sm_os, space_map_object(sm), sm->sm_phys->smp_objsize, + sizeof (dentry), &dentry, tx); + + sm->sm_phys->smp_objsize += sizeof (dentry); +} + +/* + * Writes one or more entries given a segment. + * + * Note: The function may release the dbuf from the pointer initially + * passed to it, and return a different dbuf. Also, the space map's + * dbuf must be dirty for the changes in sm_phys to take effect. + */ +static void +space_map_write_seg(space_map_t *sm, range_seg_t *rs, maptype_t maptype, + uint64_t vdev_id, uint8_t words, dmu_buf_t **dbp, void *tag, dmu_tx_t *tx) +{ + ASSERT3U(words, !=, 0); + ASSERT3U(words, <=, 2); + + /* ensure the vdev_id can be represented by the space map */ + ASSERT3U(vdev_id, <=, SM_NO_VDEVID); + /* - * All space_maps always have a debug entry so account for it here. + * if this is a single word entry, ensure that no vdev was + * specified. */ - entries = 1; + IMPLY(words == 1, vdev_id == SM_NO_VDEVID); + dmu_buf_t *db = *dbp; + ASSERT3U(db->db_size, ==, sm->sm_blksz); + + uint64_t *block_base = db->db_data; + uint64_t *block_end = block_base + (sm->sm_blksz / sizeof (uint64_t)); + uint64_t *block_cursor = block_base + + (sm->sm_phys->smp_objsize - db->db_offset) / sizeof (uint64_t); + + ASSERT3P(block_cursor, <=, block_end); + + uint64_t size = (rs->rs_end - rs->rs_start) >> sm->sm_shift; + uint64_t start = (rs->rs_start - sm->sm_start) >> sm->sm_shift; + uint64_t run_max = (words == 2) ? SM2_RUN_MAX : SM_RUN_MAX; + + ASSERT3U(rs->rs_start, >=, sm->sm_start); + ASSERT3U(rs->rs_start, <, sm->sm_start + sm->sm_size); + ASSERT3U(rs->rs_end - rs->rs_start, <=, sm->sm_size); + ASSERT3U(rs->rs_end, <=, sm->sm_start + sm->sm_size); + + while (size != 0) { + ASSERT3P(block_cursor, <=, block_end); + + /* + * If we are at the end of this block, flush it and start + * writing again from the beginning. + */ + if (block_cursor == block_end) { + dmu_buf_rele(db, tag); + + uint64_t next_word_offset = sm->sm_phys->smp_objsize; + VERIFY0(dmu_buf_hold(sm->sm_os, + space_map_object(sm), next_word_offset, + tag, &db, DMU_READ_PREFETCH)); + dmu_buf_will_dirty(db, tx); + + /* update caller's dbuf */ + *dbp = db; + + ASSERT3U(db->db_size, ==, sm->sm_blksz); + + block_base = db->db_data; + block_cursor = block_base; + block_end = block_base + + (db->db_size / sizeof (uint64_t)); + } + + /* + * If we are writing a two-word entry and we only have one + * word left on this block, just pad it with an empty debug + * entry and write the two-word entry in the next block. + */ + uint64_t *next_entry = block_cursor + 1; + if (next_entry == block_end && words > 1) { + ASSERT3U(words, ==, 2); + *block_cursor = SM_PREFIX_ENCODE(SM_DEBUG_PREFIX) | + SM_DEBUG_ACTION_ENCODE(0) | + SM_DEBUG_SYNCPASS_ENCODE(0) | + SM_DEBUG_TXG_ENCODE(0); + block_cursor++; + sm->sm_phys->smp_objsize += sizeof (uint64_t); + ASSERT3P(block_cursor, ==, block_end); + continue; + } + + uint64_t run_len = MIN(size, run_max); + switch (words) { + case 1: + *block_cursor = SM_OFFSET_ENCODE(start) | + SM_TYPE_ENCODE(maptype) | + SM_RUN_ENCODE(run_len); + block_cursor++; + break; + case 2: + /* write the first word of the entry */ + *block_cursor = SM_PREFIX_ENCODE(SM2_PREFIX) | + SM2_RUN_ENCODE(run_len) | + SM2_VDEV_ENCODE(vdev_id); + block_cursor++; + + /* move on to the second word of the entry */ + ASSERT3P(block_cursor, <, block_end); + *block_cursor = SM2_TYPE_ENCODE(maptype) | + SM2_OFFSET_ENCODE(start); + block_cursor++; + break; + default: + panic("%d-word space map entries are not supported", + words); + break; + } + sm->sm_phys->smp_objsize += words * sizeof (uint64_t); + + start += run_len; + size -= run_len; + } + ASSERT0(size); + +} + +/* + * Note: The space map's dbuf must be dirty for the changes in sm_phys to + * take effect. + */ +static void +space_map_write_impl(space_map_t *sm, range_tree_t *rt, maptype_t maptype, + uint64_t vdev_id, dmu_tx_t *tx) +{ + spa_t *spa = tx->tx_pool->dp_spa; + dmu_buf_t *db; + + space_map_write_intro_debug(sm, maptype, tx); + +#ifdef DEBUG /* - * Traverse the range tree and calculate the number of space map - * entries that would be required to write out the range tree. + * We do this right after we write the intro debug entry + * because the estimate does not take it into account. */ - for (rs = avl_first(t); rs != NULL; rs = AVL_NEXT(t, rs)) { - size = (rs->rs_end - rs->rs_start) >> sm->sm_shift; - entries += howmany(size, SM_RUN_MAX); + uint64_t initial_objsize = sm->sm_phys->smp_objsize; + uint64_t estimated_growth = + space_map_estimate_optimal_size(sm, rt, SM_NO_VDEVID); + uint64_t estimated_final_objsize = initial_objsize + estimated_growth; +#endif + + /* + * Find the offset right after the last word in the space map + * and use that to get a hold of the last block, so we can + * start appending to it. + */ + uint64_t next_word_offset = sm->sm_phys->smp_objsize; + VERIFY0(dmu_buf_hold(sm->sm_os, space_map_object(sm), + next_word_offset, FTAG, &db, DMU_READ_PREFETCH)); + ASSERT3U(db->db_size, ==, sm->sm_blksz); + + dmu_buf_will_dirty(db, tx); + + avl_tree_t *t = &rt->rt_root; + for (range_seg_t *rs = avl_first(t); rs != NULL; rs = AVL_NEXT(t, rs)) { + uint64_t offset = (rs->rs_start - sm->sm_start) >> sm->sm_shift; + uint64_t length = (rs->rs_end - rs->rs_start) >> sm->sm_shift; + uint8_t words = 1; + + /* + * We only write two-word entries when both of the following + * are true: + * + * [1] The feature is enabled. + * [2] The offset or run is too big for a single-word entry, + * or the vdev_id is set (meaning not equal to + * SM_NO_VDEVID). + * + * Note that for purposes of testing we've added the case that + * we write two-word entries occasionally when the feature is + * enabled and zfs_force_some_double_word_sm_entries has been + * set. + */ + if (spa_feature_is_active(spa, SPA_FEATURE_SPACEMAP_V2) && + (offset >= (1ULL << SM_OFFSET_BITS) || + length > SM_RUN_MAX || + vdev_id != SM_NO_VDEVID || + (zfs_force_some_double_word_sm_entries && + spa_get_random(100) == 0))) + words = 2; + + space_map_write_seg(sm, rs, maptype, vdev_id, words, + &db, FTAG, tx); } - return (entries); + + dmu_buf_rele(db, FTAG); + +#ifdef DEBUG + /* + * We expect our estimation to be based on the worst case + * scenario [see comment in space_map_estimate_optimal_size()]. + * Therefore we expect the actual objsize to be equal or less + * than whatever we estimated it to be. + */ + ASSERT3U(estimated_final_objsize, >=, sm->sm_phys->smp_objsize); +#endif } +/* + * Note: This function manipulates the state of the given space map but + * does not hold any locks implicitly. Thus the caller is responsible + * for synchronizing writes to the space map. + */ void space_map_write(space_map_t *sm, range_tree_t *rt, maptype_t maptype, - dmu_tx_t *tx) + uint64_t vdev_id, dmu_tx_t *tx) { objset_t *os = sm->sm_os; - spa_t *spa = dmu_objset_spa(os); - avl_tree_t *t = &rt->rt_root; - range_seg_t *rs; - uint64_t size, total, rt_space, nodes; - uint64_t *entry, *entry_map, *entry_map_end; - uint64_t expected_entries, actual_entries = 1; ASSERT(dsl_pool_sync_context(dmu_objset_pool(os))); VERIFY3U(space_map_object(sm), !=, 0); + dmu_buf_will_dirty(sm->sm_dbuf, tx); /* @@ -421,68 +740,17 @@ space_map_write(space_map_t *sm, range_tree_t *rt, map else sm->sm_phys->smp_alloc -= range_tree_space(rt); - expected_entries = space_map_entries(sm, rt); + uint64_t nodes = avl_numnodes(&rt->rt_root); + uint64_t rt_space = range_tree_space(rt); - entry_map = zio_buf_alloc(sm->sm_blksz); - entry_map_end = entry_map + (sm->sm_blksz / sizeof (uint64_t)); - entry = entry_map; + space_map_write_impl(sm, rt, maptype, vdev_id, tx); *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Mon Jul 30 23:47:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5CB7D10643D5; Mon, 30 Jul 2018 23:47:41 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F0B568B101; Mon, 30 Jul 2018 23:47:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CC9EA3AF7; Mon, 30 Jul 2018 23:47:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UNleMg069302; Mon, 30 Jul 2018 23:47:40 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UNld7M069293; Mon, 30 Jul 2018 23:47:39 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302347.w6UNld7M069293@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 23:47:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336947 - in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/cmd/zpool cddl/contrib/opensolaris/cmd/ztest sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/openso... X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/cmd/zpool cddl/contrib/opensolaris/cmd/ztest sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs s... X-SVN-Commit-Revision: 336947 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 23:47:41 -0000 Author: mav Date: Mon Jul 30 23:47:38 2018 New Revision: 336947 URL: https://svnweb.freebsd.org/changeset/base/336947 Log: MFV r336946: 9238 ZFS Spacemap Encoding V2 The current space map encoding has the following disadvantages: [1] Assuming 512 sector size each entry can represent at most 16MB for a segment. This makes the encoding very inefficient for large regions of space. [2] As vdev-wide space maps have started to be used by new features (i.e. device removal, zpool checkpoint) we've started imposing limits in the vdevs that can be used with them based on the maximum addressable offset (currently 64PB for a top-level vdev). The new remains backwards compatible with the old one. The introduced two-word entry format, besides extending the limits imposed by the single-entry layout, also includes a vdev field and some extra padding after its prefix. The extra padding after the prefix should is reserved for future usage (e.g. new prefixes for future encodings or new fields for flags). The new vdev field not only makes the space maps more self-descriptive, but also opens the doors for pool-wide space maps. One final important note is that the number of bits used for vdevs is reduced to 24 bits for blkptrs. That was decided as we don't know of any setups that use more than 16M vdevs for the time being and we wanted to fit the vdev field in the space map. In addition that gives us some extra bits in dva_t. illumos/illumos-gate@17f11284b49b98353b5119463254074fd9bc0a28 Reviewed by: Matt Ahrens Reviewed by: George Wilson Approved by: Gordon Ross Author: Serapheim Dimitropoulos Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c head/cddl/contrib/opensolaris/cmd/zpool/zpool-features.7 head/cddl/contrib/opensolaris/cmd/ztest/ztest.c head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_checkpoint.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/space_map.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_indirect.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_indirect_mapping.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Mon Jul 30 22:56:24 2018 (r336946) +++ head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Mon Jul 30 23:47:38 2018 (r336947) @@ -774,7 +774,6 @@ verify_spacemap_refcounts(spa_t *spa) static void dump_spacemap(objset_t *os, space_map_t *sm) { - uint64_t alloc, offset, entry; char *ddata[] = { "ALLOC", "FREE", "CONDENSE", "INVALID", "INVALID", "INVALID", "INVALID", "INVALID" }; @@ -791,41 +790,73 @@ dump_spacemap(objset_t *os, space_map_t *sm) /* * Print out the freelist entries in both encoded and decoded form. */ - alloc = 0; - for (offset = 0; offset < space_map_length(sm); - offset += sizeof (entry)) { - uint8_t mapshift = sm->sm_shift; + uint8_t mapshift = sm->sm_shift; + int64_t alloc = 0; + uint64_t word; + for (uint64_t offset = 0; offset < space_map_length(sm); + offset += sizeof (word)) { VERIFY0(dmu_read(os, space_map_object(sm), offset, - sizeof (entry), &entry, DMU_READ_PREFETCH)); - if (SM_DEBUG_DECODE(entry)) { + sizeof (word), &word, DMU_READ_PREFETCH)); + if (sm_entry_is_debug(word)) { (void) printf("\t [%6llu] %s: txg %llu, pass %llu\n", - (u_longlong_t)(offset / sizeof (entry)), - ddata[SM_DEBUG_ACTION_DECODE(entry)], - (u_longlong_t)SM_DEBUG_TXG_DECODE(entry), - (u_longlong_t)SM_DEBUG_SYNCPASS_DECODE(entry)); + (u_longlong_t)(offset / sizeof (word)), + ddata[SM_DEBUG_ACTION_DECODE(word)], + (u_longlong_t)SM_DEBUG_TXG_DECODE(word), + (u_longlong_t)SM_DEBUG_SYNCPASS_DECODE(word)); + continue; + } + + uint8_t words; + char entry_type; + uint64_t entry_off, entry_run, entry_vdev = SM_NO_VDEVID; + + if (sm_entry_is_single_word(word)) { + entry_type = (SM_TYPE_DECODE(word) == SM_ALLOC) ? + 'A' : 'F'; + entry_off = (SM_OFFSET_DECODE(word) << mapshift) + + sm->sm_start; + entry_run = SM_RUN_DECODE(word) << mapshift; + words = 1; } else { - (void) printf("\t [%6llu] %c range:" - " %010llx-%010llx size: %06llx\n", - (u_longlong_t)(offset / sizeof (entry)), - SM_TYPE_DECODE(entry) == SM_ALLOC ? 'A' : 'F', - (u_longlong_t)((SM_OFFSET_DECODE(entry) << - mapshift) + sm->sm_start), - (u_longlong_t)((SM_OFFSET_DECODE(entry) << - mapshift) + sm->sm_start + - (SM_RUN_DECODE(entry) << mapshift)), - (u_longlong_t)(SM_RUN_DECODE(entry) << mapshift)); - if (SM_TYPE_DECODE(entry) == SM_ALLOC) - alloc += SM_RUN_DECODE(entry) << mapshift; - else - alloc -= SM_RUN_DECODE(entry) << mapshift; + /* it is a two-word entry so we read another word */ + ASSERT(sm_entry_is_double_word(word)); + + uint64_t extra_word; + offset += sizeof (extra_word); + VERIFY0(dmu_read(os, space_map_object(sm), offset, + sizeof (extra_word), &extra_word, + DMU_READ_PREFETCH)); + + ASSERT3U(offset, <=, space_map_length(sm)); + + entry_run = SM2_RUN_DECODE(word) << mapshift; + entry_vdev = SM2_VDEV_DECODE(word); + entry_type = (SM2_TYPE_DECODE(extra_word) == SM_ALLOC) ? + 'A' : 'F'; + entry_off = (SM2_OFFSET_DECODE(extra_word) << + mapshift) + sm->sm_start; + words = 2; } + + (void) printf("\t [%6llu] %c range:" + " %010llx-%010llx size: %06llx vdev: %06llu words: %u\n", + (u_longlong_t)(offset / sizeof (word)), + entry_type, (u_longlong_t)entry_off, + (u_longlong_t)(entry_off + entry_run), + (u_longlong_t)entry_run, + (u_longlong_t)entry_vdev, words); + + if (entry_type == 'A') + alloc += entry_run; + else + alloc -= entry_run; } - if (alloc != space_map_allocated(sm)) { - (void) printf("space_map_object alloc (%llu) INCONSISTENT " - "with space map summary (%llu)\n", - (u_longlong_t)space_map_allocated(sm), (u_longlong_t)alloc); + if ((uint64_t)alloc != space_map_allocated(sm)) { + (void) printf("space_map_object alloc (%lld) INCONSISTENT " + "with space map summary (%lld)\n", + (longlong_t)space_map_allocated(sm), (longlong_t)alloc); } } @@ -1155,7 +1186,7 @@ dump_ddt(ddt_t *ddt, enum ddt_type type, enum ddt_clas while ((error = ddt_object_walk(ddt, type, class, &walk, &dde)) == 0) dump_dde(ddt, &dde, walk); - ASSERT(error == ENOENT); + ASSERT3U(error, ==, ENOENT); (void) printf("\n"); } @@ -3097,15 +3128,14 @@ typedef struct checkpoint_sm_exclude_entry_arg { } checkpoint_sm_exclude_entry_arg_t; static int -checkpoint_sm_exclude_entry_cb(maptype_t type, uint64_t offset, uint64_t size, - void *arg) +checkpoint_sm_exclude_entry_cb(space_map_entry_t *sme, void *arg) { checkpoint_sm_exclude_entry_arg_t *cseea = arg; vdev_t *vd = cseea->cseea_vd; - metaslab_t *ms = vd->vdev_ms[offset >> vd->vdev_ms_shift]; - uint64_t end = offset + size; + metaslab_t *ms = vd->vdev_ms[sme->sme_offset >> vd->vdev_ms_shift]; + uint64_t end = sme->sme_offset + sme->sme_run; - ASSERT(type == SM_FREE); + ASSERT(sme->sme_type == SM_FREE); /* * Since the vdev_checkpoint_sm exists in the vdev level @@ -3123,7 +3153,7 @@ checkpoint_sm_exclude_entry_cb(maptype_t type, uint64_ * metaslab boundaries. So if needed we could add code * that handles metaslab-crossing segments in the future. */ - VERIFY3U(offset, >=, ms->ms_start); + VERIFY3U(sme->sme_offset, >=, ms->ms_start); VERIFY3U(end, <=, ms->ms_start + ms->ms_size); /* @@ -3131,10 +3161,10 @@ checkpoint_sm_exclude_entry_cb(maptype_t type, uint64_ * also verify that the entry is there to begin with. */ mutex_enter(&ms->ms_lock); - range_tree_remove(ms->ms_allocatable, offset, size); + range_tree_remove(ms->ms_allocatable, sme->sme_offset, sme->sme_run); mutex_exit(&ms->ms_lock); - cseea->cseea_checkpoint_size += size; + cseea->cseea_checkpoint_size += sme->sme_run; return (0); } @@ -4109,15 +4139,14 @@ typedef struct verify_checkpoint_sm_entry_cb_arg { #define ENTRIES_PER_PROGRESS_UPDATE 10000 static int -verify_checkpoint_sm_entry_cb(maptype_t type, uint64_t offset, uint64_t size, - void *arg) +verify_checkpoint_sm_entry_cb(space_map_entry_t *sme, void *arg) { verify_checkpoint_sm_entry_cb_arg_t *vcsec = arg; vdev_t *vd = vcsec->vcsec_vd; - metaslab_t *ms = vd->vdev_ms[offset >> vd->vdev_ms_shift]; - uint64_t end = offset + size; + metaslab_t *ms = vd->vdev_ms[sme->sme_offset >> vd->vdev_ms_shift]; + uint64_t end = sme->sme_offset + sme->sme_run; - ASSERT(type == SM_FREE); + ASSERT(sme->sme_type == SM_FREE); if ((vcsec->vcsec_entryid % ENTRIES_PER_PROGRESS_UPDATE) == 0) { (void) fprintf(stderr, @@ -4131,7 +4160,7 @@ verify_checkpoint_sm_entry_cb(maptype_t type, uint64_t /* * See comment in checkpoint_sm_exclude_entry_cb() */ - VERIFY3U(offset, >=, ms->ms_start); + VERIFY3U(sme->sme_offset, >=, ms->ms_start); VERIFY3U(end, <=, ms->ms_start + ms->ms_size); /* @@ -4140,7 +4169,7 @@ verify_checkpoint_sm_entry_cb(maptype_t type, uint64_t * their respective ms_allocateable trees should not contain them. */ mutex_enter(&ms->ms_lock); - range_tree_verify(ms->ms_allocatable, offset, size); + range_tree_verify(ms->ms_allocatable, sme->sme_offset, sme->sme_run); mutex_exit(&ms->ms_lock); return (0); @@ -4386,7 +4415,7 @@ verify_checkpoint(spa_t *spa) DMU_POOL_ZPOOL_CHECKPOINT, sizeof (uint64_t), sizeof (uberblock_t) / sizeof (uint64_t), &checkpoint); - if (error == ENOENT) { + if (error == ENOENT && !dump_opt['L']) { /* * If the feature is active but the uberblock is missing * then we must be in the middle of discarding the @@ -4409,7 +4438,7 @@ verify_checkpoint(spa_t *spa) error = 3; } - if (error == 0) + if (error == 0 && !dump_opt['L']) verify_checkpoint_blocks(spa); return (error); @@ -4514,7 +4543,7 @@ dump_zpool(spa_t *spa) if (dump_opt['h']) dump_history(spa); - if (rc == 0 && !dump_opt['L']) + if (rc == 0) rc = verify_checkpoint(spa); if (rc != 0) { Modified: head/cddl/contrib/opensolaris/cmd/zpool/zpool-features.7 ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zpool/zpool-features.7 Mon Jul 30 22:56:24 2018 (r336946) +++ head/cddl/contrib/opensolaris/cmd/zpool/zpool-features.7 Mon Jul 30 23:47:38 2018 (r336947) @@ -482,6 +482,24 @@ This feature becomes when the "zpool remove" command is used on a top-level vdev, and will never return to being .Sy enabled . +.It Sy spacemap_v2 +.Bl -column "READ\-ONLY COMPATIBLE" "com.delphix:spacemap_v2" +.It GUID Ta com.delphix:spacemap_v2 +.It READ\-ONLY COMPATIBLE Ta yes +.It DEPENDENCIES Ta none +.El +.Pp +This feature enables the use of the new space map encoding which +consists of two words (instead of one) whenever it is advantageous. +The new encoding allows space maps to represent large regions of +space more efficiently on-disk while also increasing their maximum +addressable offset. +.Pp +This feature becomes +.Sy active +as soon as it is enabled and will +never return to being +.Sy enabled . .It Sy large_blocks .Bl -column "READ\-ONLY COMPATIBLE" "org.open-zfs:large_block" .It GUID Ta org.open-zfs:large_block Modified: head/cddl/contrib/opensolaris/cmd/ztest/ztest.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Mon Jul 30 22:56:24 2018 (r336946) +++ head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Mon Jul 30 23:47:38 2018 (r336947) @@ -195,6 +195,7 @@ extern uint64_t zfs_deadman_synctime_ms; extern int metaslab_preload_limit; extern boolean_t zfs_compressed_arc_enabled; extern boolean_t zfs_abd_scatter_enabled; +extern boolean_t zfs_force_some_double_word_sm_entries; static ztest_shared_opts_t *ztest_shared_opts; static ztest_shared_opts_t ztest_opts; @@ -6397,6 +6398,12 @@ main(int argc, char **argv) dprintf_setup(&argc, argv); zfs_deadman_synctime_ms = 300000; + /* + * As two-word space map entries may not come up often (especially + * if pool and vdev sizes are small) we want to force at least some + * of them so the feature get tested. + */ + zfs_force_some_double_word_sm_entries = B_TRUE; ztest_fd_rand = open("/dev/urandom", O_RDONLY); ASSERT3S(ztest_fd_rand, >=, 0); Modified: head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c Mon Jul 30 22:56:24 2018 (r336946) +++ head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.c Mon Jul 30 23:47:38 2018 (r336947) @@ -229,6 +229,12 @@ zpool_feature_init(void) "Pool state can be checkpointed, allowing rewind later.", ZFEATURE_FLAG_READONLY_COMPAT, NULL); + zfeature_register(SPA_FEATURE_SPACEMAP_V2, + "com.delphix:spacemap_v2", "spacemap_v2", + "Space maps representing large segments are more efficient.", + ZFEATURE_FLAG_READONLY_COMPAT | ZFEATURE_FLAG_ACTIVATE_ON_ENABLE, + NULL); + static const spa_feature_t large_blocks_deps[] = { SPA_FEATURE_EXTENSIBLE_DATASET, SPA_FEATURE_NONE Modified: head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.h Mon Jul 30 22:56:24 2018 (r336946) +++ head/sys/cddl/contrib/opensolaris/common/zfs/zfeature_common.h Mon Jul 30 23:47:38 2018 (r336947) @@ -60,6 +60,7 @@ typedef enum spa_feature { SPA_FEATURE_DEVICE_REMOVAL, SPA_FEATURE_OBSOLETE_COUNTS, SPA_FEATURE_POOL_CHECKPOINT, + SPA_FEATURE_SPACEMAP_V2, SPA_FEATURES } spa_feature_t; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Mon Jul 30 22:56:24 2018 (r336946) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Mon Jul 30 23:47:38 2018 (r336947) @@ -2105,17 +2105,6 @@ metaslab_group_preload(metaslab_group_t *mg) * * 3. The on-disk size of the space map should actually decrease. * - * Checking the first condition is tricky since we don't want to walk - * the entire AVL tree calculating the estimated on-disk size. Instead we - * use the size-ordered range tree in the metaslab and calculate the - * size required to write out the largest segment in our free tree. If the - * size required to represent that segment on disk is larger than the space - * map object then we avoid condensing this map. - * - * To determine the second criterion we use a best-case estimate and assume - * each segment can be represented on-disk as a single 64-bit entry. We refer - * to this best-case estimate as the space map's minimal form. - * * Unfortunately, we cannot compute the on-disk size of the space map in this * context because we cannot accurately compute the effects of compression, etc. * Instead, we apply the heuristic described in the block comment for @@ -2126,9 +2115,6 @@ static boolean_t metaslab_should_condense(metaslab_t *msp) { space_map_t *sm = msp->ms_sm; - range_seg_t *rs; - uint64_t size, entries, segsz, object_size, optimal_size, record_size; - dmu_object_info_t doi; vdev_t *vd = msp->ms_group->mg_vd; uint64_t vdev_blocksize = 1 << vd->vdev_ashift; uint64_t current_txg = spa_syncing_txg(vd->vdev_spa); @@ -2154,34 +2140,22 @@ metaslab_should_condense(metaslab_t *msp) msp->ms_condense_checked_txg = current_txg; /* - * Use the ms_allocatable_by_size range tree, which is ordered by - * size, to obtain the largest segment in the free tree. We always - * condense metaslabs that are empty and metaslabs for which a - * condense request has been made. + * We always condense metaslabs that are empty and metaslabs for + * which a condense request has been made. */ - rs = avl_last(&msp->ms_allocatable_by_size); - if (rs == NULL || msp->ms_condense_wanted) + if (avl_is_empty(&msp->ms_allocatable_by_size) || + msp->ms_condense_wanted) return (B_TRUE); - /* - * Calculate the number of 64-bit entries this segment would - * require when written to disk. If this single segment would be - * larger on-disk than the entire current on-disk structure, then - * clearly condensing will increase the on-disk structure size. - */ - size = (rs->rs_end - rs->rs_start) >> sm->sm_shift; - entries = size / (MIN(size, SM_RUN_MAX)); - segsz = entries * sizeof (uint64_t); + uint64_t object_size = space_map_length(msp->ms_sm); + uint64_t optimal_size = space_map_estimate_optimal_size(sm, + msp->ms_allocatable, SM_NO_VDEVID); - optimal_size = - sizeof (uint64_t) * avl_numnodes(&msp->ms_allocatable->rt_root); - object_size = space_map_length(msp->ms_sm); - + dmu_object_info_t doi; dmu_object_info_from_db(sm->sm_dbuf, &doi); - record_size = MAX(doi.doi_data_block_size, vdev_blocksize); + uint64_t record_size = MAX(doi.doi_data_block_size, vdev_blocksize); - return (segsz <= object_size && - object_size >= (optimal_size * zfs_condense_pct / 100) && + return (object_size >= (optimal_size * zfs_condense_pct / 100) && object_size > zfs_metaslab_condense_block_threshold * record_size); } @@ -2256,11 +2230,11 @@ metaslab_condense(metaslab_t *msp, uint64_t txg, dmu_t * optimal, this is typically close to optimal, and much cheaper to * compute. */ - space_map_write(sm, condense_tree, SM_ALLOC, tx); + space_map_write(sm, condense_tree, SM_ALLOC, SM_NO_VDEVID, tx); range_tree_vacate(condense_tree, NULL, NULL); range_tree_destroy(condense_tree); - space_map_write(sm, msp->ms_allocatable, SM_FREE, tx); + space_map_write(sm, msp->ms_allocatable, SM_FREE, SM_NO_VDEVID, tx); mutex_enter(&msp->ms_lock); msp->ms_condensing = B_FALSE; } @@ -2372,8 +2346,10 @@ metaslab_sync(metaslab_t *msp, uint64_t txg) metaslab_condense(msp, txg, tx); } else { mutex_exit(&msp->ms_lock); - space_map_write(msp->ms_sm, alloctree, SM_ALLOC, tx); - space_map_write(msp->ms_sm, msp->ms_freeing, SM_FREE, tx); + space_map_write(msp->ms_sm, alloctree, SM_ALLOC, + SM_NO_VDEVID, tx); + space_map_write(msp->ms_sm, msp->ms_freeing, SM_FREE, + SM_NO_VDEVID, tx); mutex_enter(&msp->ms_lock); } @@ -2388,7 +2364,7 @@ metaslab_sync(metaslab_t *msp, uint64_t txg) */ mutex_exit(&msp->ms_lock); space_map_write(vd->vdev_checkpoint_sm, - msp->ms_checkpointing, SM_FREE, tx); + msp->ms_checkpointing, SM_FREE, SM_NO_VDEVID, tx); mutex_enter(&msp->ms_lock); space_map_update(vd->vdev_checkpoint_sm); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_checkpoint.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_checkpoint.c Mon Jul 30 22:56:24 2018 (r336946) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_checkpoint.c Mon Jul 30 23:47:38 2018 (r336947) @@ -203,13 +203,12 @@ typedef struct spa_checkpoint_discard_sync_callback_ar } spa_checkpoint_discard_sync_callback_arg_t; static int -spa_checkpoint_discard_sync_callback(maptype_t type, uint64_t offset, - uint64_t size, void *arg) +spa_checkpoint_discard_sync_callback(space_map_entry_t *sme, void *arg) { spa_checkpoint_discard_sync_callback_arg_t *sdc = arg; vdev_t *vd = sdc->sdc_vd; - metaslab_t *ms = vd->vdev_ms[offset >> vd->vdev_ms_shift]; - uint64_t end = offset + size; + metaslab_t *ms = vd->vdev_ms[sme->sme_offset >> vd->vdev_ms_shift]; + uint64_t end = sme->sme_offset + sme->sme_run; if (sdc->sdc_entry_limit == 0) return (EINTR); @@ -224,8 +223,8 @@ spa_checkpoint_discard_sync_callback(maptype_t type, u * metaslab boundaries. So if needed we could add code * that handles metaslab-crossing segments in the future. */ - VERIFY3U(type, ==, SM_FREE); - VERIFY3U(offset, >=, ms->ms_start); + VERIFY3U(sme->sme_type, ==, SM_FREE); + VERIFY3U(sme->sme_offset, >=, ms->ms_start); VERIFY3U(end, <=, ms->ms_start + ms->ms_size); /* @@ -237,14 +236,15 @@ spa_checkpoint_discard_sync_callback(maptype_t type, u mutex_enter(&ms->ms_lock); if (range_tree_is_empty(ms->ms_freeing)) vdev_dirty(vd, VDD_METASLAB, ms, sdc->sdc_txg); - range_tree_add(ms->ms_freeing, offset, size); + range_tree_add(ms->ms_freeing, sme->sme_offset, sme->sme_run); mutex_exit(&ms->ms_lock); - ASSERT3U(vd->vdev_spa->spa_checkpoint_info.sci_dspace, >=, size); - ASSERT3U(vd->vdev_stat.vs_checkpoint_space, >=, size); + ASSERT3U(vd->vdev_spa->spa_checkpoint_info.sci_dspace, >=, + sme->sme_run); + ASSERT3U(vd->vdev_stat.vs_checkpoint_space, >=, sme->sme_run); - vd->vdev_spa->spa_checkpoint_info.sci_dspace -= size; - vd->vdev_stat.vs_checkpoint_space -= size; + vd->vdev_spa->spa_checkpoint_info.sci_dspace -= sme->sme_run; + vd->vdev_stat.vs_checkpoint_space -= sme->sme_run; sdc->sdc_entry_limit--; return (0); @@ -289,13 +289,14 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t * Thus, we set the maximum entries that the space map callback * will be applied to be half the entries that could fit in the * imposed memory limit. + * + * Note that since this is a conservative estimate we also + * assume the worst case scenario in our computation where each + * entry is two-word. */ uint64_t max_entry_limit = - (zfs_spa_discard_memory_limit / sizeof (uint64_t)) >> 1; + (zfs_spa_discard_memory_limit / (2 * sizeof (uint64_t))) >> 1; - uint64_t entries_in_sm = - space_map_length(vd->vdev_checkpoint_sm) / sizeof (uint64_t); - /* * Iterate from the end of the space map towards the beginning, * placing its entries on ms_freeing and removing them from the @@ -318,14 +319,15 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t spa_checkpoint_discard_sync_callback_arg_t sdc; sdc.sdc_vd = vd; sdc.sdc_txg = tx->tx_txg; - sdc.sdc_entry_limit = MIN(entries_in_sm, max_entry_limit); + sdc.sdc_entry_limit = max_entry_limit; - uint64_t entries_before = entries_in_sm; + uint64_t words_before = + space_map_length(vd->vdev_checkpoint_sm) / sizeof (uint64_t); error = space_map_incremental_destroy(vd->vdev_checkpoint_sm, spa_checkpoint_discard_sync_callback, &sdc, tx); - uint64_t entries_after = + uint64_t words_after = space_map_length(vd->vdev_checkpoint_sm) / sizeof (uint64_t); #ifdef DEBUG @@ -333,9 +335,9 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t #endif zfs_dbgmsg("discarding checkpoint: txg %llu, vdev id %d, " - "deleted %llu entries - %llu entries are left", - tx->tx_txg, vd->vdev_id, (entries_before - entries_after), - entries_after); + "deleted %llu words - %llu words are left", + tx->tx_txg, vd->vdev_id, (words_before - words_after), + words_after); if (error != EINTR) { if (error != 0) { @@ -344,15 +346,15 @@ spa_checkpoint_discard_thread_sync(void *arg, dmu_tx_t "space map of vdev %llu\n", error, vd->vdev_id); } - ASSERT0(entries_after); + ASSERT0(words_after); ASSERT0(vd->vdev_checkpoint_sm->sm_alloc); - ASSERT0(vd->vdev_checkpoint_sm->sm_length); + ASSERT0(space_map_length(vd->vdev_checkpoint_sm)); space_map_free(vd->vdev_checkpoint_sm, tx); space_map_close(vd->vdev_checkpoint_sm); vd->vdev_checkpoint_sm = NULL; - VERIFY0(zap_remove(vd->vdev_spa->spa_meta_objset, + VERIFY0(zap_remove(spa_meta_objset(vd->vdev_spa), vd->vdev_top_zap, VDEV_TOP_ZAP_POOL_CHECKPOINT_SM, tx)); } } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c Mon Jul 30 22:56:24 2018 (r336946) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c Mon Jul 30 23:47:38 2018 (r336947) @@ -43,68 +43,194 @@ SYSCTL_DECL(_vfs_zfs); * Note on space map block size: * * The data for a given space map can be kept on blocks of any size. - * Larger blocks entail fewer i/o operations, but they also cause the - * DMU to keep more data in-core, and also to waste more i/o bandwidth + * Larger blocks entail fewer I/O operations, but they also cause the + * DMU to keep more data in-core, and also to waste more I/O bandwidth * when only a few blocks have changed since the last transaction group. */ /* + * Enabled whenever we want to stress test the use of double-word + * space map entries. + */ +boolean_t zfs_force_some_double_word_sm_entries = B_FALSE; + +boolean_t +sm_entry_is_debug(uint64_t e) +{ + return (SM_PREFIX_DECODE(e) == SM_DEBUG_PREFIX); +} + +boolean_t +sm_entry_is_single_word(uint64_t e) +{ + uint8_t prefix = SM_PREFIX_DECODE(e); + return (prefix != SM_DEBUG_PREFIX && prefix != SM2_PREFIX); +} + +boolean_t +sm_entry_is_double_word(uint64_t e) +{ + return (SM_PREFIX_DECODE(e) == SM2_PREFIX); +} + +/* * Iterate through the space map, invoking the callback on each (non-debug) * space map entry. */ int space_map_iterate(space_map_t *sm, sm_cb_t callback, void *arg) { - uint64_t *entry, *entry_map, *entry_map_end; - uint64_t bufsize, size, offset, end; + uint64_t sm_len = space_map_length(sm); + ASSERT3U(sm->sm_blksz, !=, 0); + + dmu_prefetch(sm->sm_os, space_map_object(sm), 0, 0, sm_len, + ZIO_PRIORITY_SYNC_READ); + + uint64_t blksz = sm->sm_blksz; int error = 0; + for (uint64_t block_base = 0; block_base < sm_len && error == 0; + block_base += blksz) { + dmu_buf_t *db; + error = dmu_buf_hold(sm->sm_os, space_map_object(sm), + block_base, FTAG, &db, DMU_READ_PREFETCH); + if (error != 0) + return (error); - end = space_map_length(sm); + uint64_t *block_start = db->db_data; + uint64_t block_length = MIN(sm_len - block_base, blksz); + uint64_t *block_end = block_start + + (block_length / sizeof (uint64_t)); - bufsize = MAX(sm->sm_blksz, SPA_MINBLOCKSIZE); - entry_map = zio_buf_alloc(bufsize); + VERIFY0(P2PHASE(block_length, sizeof (uint64_t))); + VERIFY3U(block_length, !=, 0); + ASSERT3U(blksz, ==, db->db_size); - if (end > bufsize) { - dmu_prefetch(sm->sm_os, space_map_object(sm), 0, bufsize, - end - bufsize, ZIO_PRIORITY_SYNC_READ); + for (uint64_t *block_cursor = block_start; + block_cursor < block_end && error == 0; block_cursor++) { + uint64_t e = *block_cursor; + + if (sm_entry_is_debug(e)) /* Skip debug entries */ + continue; + + uint64_t raw_offset, raw_run, vdev_id; + maptype_t type; + if (sm_entry_is_single_word(e)) { + type = SM_TYPE_DECODE(e); + vdev_id = SM_NO_VDEVID; + raw_offset = SM_OFFSET_DECODE(e); + raw_run = SM_RUN_DECODE(e); + } else { + /* it is a two-word entry */ + ASSERT(sm_entry_is_double_word(e)); + raw_run = SM2_RUN_DECODE(e); + vdev_id = SM2_VDEV_DECODE(e); + + /* move on to the second word */ + block_cursor++; + e = *block_cursor; + VERIFY3P(block_cursor, <=, block_end); + + type = SM2_TYPE_DECODE(e); + raw_offset = SM2_OFFSET_DECODE(e); + } + + uint64_t entry_offset = (raw_offset << sm->sm_shift) + + sm->sm_start; + uint64_t entry_run = raw_run << sm->sm_shift; + + VERIFY0(P2PHASE(entry_offset, 1ULL << sm->sm_shift)); + VERIFY0(P2PHASE(entry_run, 1ULL << sm->sm_shift)); + ASSERT3U(entry_offset, >=, sm->sm_start); + ASSERT3U(entry_offset, <, sm->sm_start + sm->sm_size); + ASSERT3U(entry_run, <=, sm->sm_size); + ASSERT3U(entry_offset + entry_run, <=, + sm->sm_start + sm->sm_size); + + space_map_entry_t sme = { + .sme_type = type, + .sme_vdev = vdev_id, + .sme_offset = entry_offset, + .sme_run = entry_run + }; + error = callback(&sme, arg); + } + dmu_buf_rele(db, FTAG); } + return (error); +} - for (offset = 0; offset < end && error == 0; offset += bufsize) { - size = MIN(end - offset, bufsize); - VERIFY(P2PHASE(size, sizeof (uint64_t)) == 0); - VERIFY(size != 0); - ASSERT3U(sm->sm_blksz, !=, 0); +/* + * Reads the entries from the last block of the space map into + * buf in reverse order. Populates nwords with number of words + * in the last block. + * + * Refer to block comment within space_map_incremental_destroy() + * to understand why this function is needed. + */ +static int +space_map_reversed_last_block_entries(space_map_t *sm, uint64_t *buf, + uint64_t bufsz, uint64_t *nwords) +{ + int error = 0; + dmu_buf_t *db; - dprintf("object=%llu offset=%llx size=%llx\n", - space_map_object(sm), offset, size); + /* + * Find the offset of the last word in the space map and use + * that to read the last block of the space map with + * dmu_buf_hold(). + */ + uint64_t last_word_offset = + sm->sm_phys->smp_objsize - sizeof (uint64_t); + error = dmu_buf_hold(sm->sm_os, space_map_object(sm), last_word_offset, + FTAG, &db, DMU_READ_NO_PREFETCH); + if (error != 0) + return (error); - error = dmu_read(sm->sm_os, space_map_object(sm), offset, size, - entry_map, DMU_READ_PREFETCH); - if (error != 0) - break; + ASSERT3U(sm->sm_object, ==, db->db_object); + ASSERT3U(sm->sm_blksz, ==, db->db_size); + ASSERT3U(bufsz, >=, db->db_size); + ASSERT(nwords != NULL); - entry_map_end = entry_map + (size / sizeof (uint64_t)); - for (entry = entry_map; entry < entry_map_end && error == 0; - entry++) { - uint64_t e = *entry; - uint64_t offset, size; + uint64_t *words = db->db_data; + *nwords = + (sm->sm_phys->smp_objsize - db->db_offset) / sizeof (uint64_t); - if (SM_DEBUG_DECODE(e)) /* Skip debug entries */ - continue; + ASSERT3U(*nwords, <=, bufsz / sizeof (uint64_t)); - offset = (SM_OFFSET_DECODE(e) << sm->sm_shift) + - sm->sm_start; - size = SM_RUN_DECODE(e) << sm->sm_shift; + uint64_t n = *nwords; + uint64_t j = n - 1; + for (uint64_t i = 0; i < n; i++) { + uint64_t entry = words[i]; + if (sm_entry_is_double_word(entry)) { + /* + * Since we are populating the buffer backwards + * we have to be extra careful and add the two + * words of the double-word entry in the right + * order. + */ + ASSERT3U(j, >, 0); + buf[j - 1] = entry; - VERIFY0(P2PHASE(offset, 1ULL << sm->sm_shift)); - VERIFY0(P2PHASE(size, 1ULL << sm->sm_shift)); - VERIFY3U(offset, >=, sm->sm_start); - VERIFY3U(offset + size, <=, sm->sm_start + sm->sm_size); - error = callback(SM_TYPE_DECODE(e), offset, size, arg); + i++; + ASSERT3U(i, <, n); + entry = words[i]; + buf[j] = entry; + j -= 2; + } else { + ASSERT(sm_entry_is_debug(entry) || + sm_entry_is_single_word(entry)); + buf[j] = entry; + j--; } } - zio_buf_free(entry_map, bufsize); + /* + * Assert that we wrote backwards all the + * way to the beginning of the buffer. + */ + ASSERT3S(j, ==, -1); + + dmu_buf_rele(db, FTAG); return (error); } @@ -118,124 +244,122 @@ int space_map_incremental_destroy(space_map_t *sm, sm_cb_t callback, void *arg, dmu_tx_t *tx) { - uint64_t bufsize, len; - uint64_t *entry_map; - int error = 0; + uint64_t bufsz = MAX(sm->sm_blksz, SPA_MINBLOCKSIZE); + uint64_t *buf = zio_buf_alloc(bufsz); - len = space_map_length(sm); - bufsize = MAX(sm->sm_blksz, SPA_MINBLOCKSIZE); - entry_map = zio_buf_alloc(bufsize); - dmu_buf_will_dirty(sm->sm_dbuf, tx); /* - * Since we can't move the starting offset of the space map - * (e.g there are reference on-disk pointing to it), we destroy - * its entries incrementally starting from the end. + * Ideally we would want to iterate from the beginning of the + * space map to the end in incremental steps. The issue with this + * approach is that we don't have any field on-disk that points + * us where to start between each step. We could try zeroing out + * entries that we've destroyed, but this doesn't work either as + * an entry that is 0 is a valid one (ALLOC for range [0x0:0x200]). * - * The logic that follows is basically the same as the one used - * in space_map_iterate() but it traverses the space map - * backwards: + * As a result, we destroy its entries incrementally starting from + * the end after applying the callback to each of them. * - * 1] We figure out the size of the buffer that we want to use - * to read the on-disk space map entries. - * 2] We figure out the offset at the end of the space map where - * we will start reading entries into our buffer. - * 3] We read the on-disk entries into the buffer. - * 4] We iterate over the entries from end to beginning calling - * the callback function on each one. As we move from entry - * to entry we decrease the size of the space map, deleting - * effectively each entry. - * 5] If there are no more entries in the space map or the - * callback returns a value other than 0, we stop iterating - * over the space map. If there are entries remaining and - * the callback returned zero we go back to step [1]. + * The problem with this approach is that we cannot literally + * iterate through the words in the space map backwards as we + * can't distinguish two-word space map entries from their second + * word. Thus we do the following: + * + * 1] We get all the entries from the last block of the space map + * and put them into a buffer in reverse order. This way the + * last entry comes first in the buffer, the second to last is + * second, etc. + * 2] We iterate through the entries in the buffer and we apply + * the callback to each one. As we move from entry to entry we + * we decrease the size of the space map, deleting effectively + * each entry. + * 3] If there are no more entries in the space map or the callback + * returns a value other than 0, we stop iterating over the + * space map. If there are entries remaining and the callback + * returned 0, we go back to step [1]. */ - uint64_t offset = 0, size = 0; - while (len > 0 && error == 0) { - size = MIN(bufsize, len); - - VERIFY(P2PHASE(size, sizeof (uint64_t)) == 0); - VERIFY3U(size, >, 0); - ASSERT3U(sm->sm_blksz, !=, 0); - - offset = len - size; - - IMPLY(bufsize > len, offset == 0); - IMPLY(bufsize == len, offset == 0); - IMPLY(bufsize < len, offset > 0); - - - EQUIV(size == len, offset == 0); - IMPLY(size < len, bufsize < len); - - dprintf("object=%llu offset=%llx size=%llx\n", - space_map_object(sm), offset, size); - - error = dmu_read(sm->sm_os, space_map_object(sm), - offset, size, entry_map, DMU_READ_PREFETCH); + int error = 0; + while (space_map_length(sm) > 0 && error == 0) { + uint64_t nwords = 0; + error = space_map_reversed_last_block_entries(sm, buf, bufsz, + &nwords); if (error != 0) break; - uint64_t num_entries = size / sizeof (uint64_t); + ASSERT3U(nwords, <=, bufsz / sizeof (uint64_t)); - ASSERT3U(num_entries, >, 0); + for (uint64_t i = 0; i < nwords; i++) { + uint64_t e = buf[i]; - while (num_entries > 0) { - uint64_t e, entry_offset, entry_size; + if (sm_entry_is_debug(e)) { + sm->sm_phys->smp_objsize -= sizeof (uint64_t); + space_map_update(sm); + continue; + } + + int words = 1; + uint64_t raw_offset, raw_run, vdev_id; maptype_t type; + if (sm_entry_is_single_word(e)) { + type = SM_TYPE_DECODE(e); + vdev_id = SM_NO_VDEVID; + raw_offset = SM_OFFSET_DECODE(e); + raw_run = SM_RUN_DECODE(e); + } else { + ASSERT(sm_entry_is_double_word(e)); + words = 2; - e = entry_map[num_entries - 1]; + raw_run = SM2_RUN_DECODE(e); + vdev_id = SM2_VDEV_DECODE(e); - ASSERT3U(num_entries, >, 0); - ASSERT0(error); + /* move to the second word */ + i++; + e = buf[i]; - if (SM_DEBUG_DECODE(e)) { - sm->sm_phys->smp_objsize -= sizeof (uint64_t); - space_map_update(sm); - len -= sizeof (uint64_t); - num_entries--; - continue; + ASSERT3P(i, <=, nwords); + + type = SM2_TYPE_DECODE(e); + raw_offset = SM2_OFFSET_DECODE(e); } - type = SM_TYPE_DECODE(e); - entry_offset = (SM_OFFSET_DECODE(e) << sm->sm_shift) + - sm->sm_start; - entry_size = SM_RUN_DECODE(e) << sm->sm_shift; + uint64_t entry_offset = + (raw_offset << sm->sm_shift) + sm->sm_start; + uint64_t entry_run = raw_run << sm->sm_shift; VERIFY0(P2PHASE(entry_offset, 1ULL << sm->sm_shift)); - VERIFY0(P2PHASE(entry_size, 1ULL << sm->sm_shift)); + VERIFY0(P2PHASE(entry_run, 1ULL << sm->sm_shift)); VERIFY3U(entry_offset, >=, sm->sm_start); - VERIFY3U(entry_offset + entry_size, <=, + VERIFY3U(entry_offset, <, sm->sm_start + sm->sm_size); + VERIFY3U(entry_run, <=, sm->sm_size); + VERIFY3U(entry_offset + entry_run, <=, sm->sm_start + sm->sm_size); - error = callback(type, entry_offset, entry_size, arg); + space_map_entry_t sme = { + .sme_type = type, + .sme_vdev = vdev_id, + .sme_offset = entry_offset, + .sme_run = entry_run + }; + error = callback(&sme, arg); if (error != 0) break; if (type == SM_ALLOC) - sm->sm_phys->smp_alloc -= entry_size; + sm->sm_phys->smp_alloc -= entry_run; else - sm->sm_phys->smp_alloc += entry_size; - - sm->sm_phys->smp_objsize -= sizeof (uint64_t); + sm->sm_phys->smp_alloc += entry_run; + sm->sm_phys->smp_objsize -= words * sizeof (uint64_t); space_map_update(sm); - len -= sizeof (uint64_t); - num_entries--; } - IMPLY(error == 0, num_entries == 0); - EQUIV(offset == 0 && error == 0, len == 0 && num_entries == 0); } - if (len == 0) { + if (space_map_length(sm) == 0) { ASSERT0(error); - ASSERT0(offset); - ASSERT0(sm->sm_length); ASSERT0(sm->sm_phys->smp_objsize); ASSERT0(sm->sm_alloc); } - zio_buf_free(entry_map, bufsize); + zio_buf_free(buf, bufsz); return (error); } @@ -246,16 +370,15 @@ typedef struct space_map_load_arg { } space_map_load_arg_t; static int -space_map_load_callback(maptype_t type, uint64_t offset, uint64_t size, - void *arg) +space_map_load_callback(space_map_entry_t *sme, void *arg) { space_map_load_arg_t *smla = arg; - if (type == smla->smla_type) { - VERIFY3U(range_tree_space(smla->smla_rt) + size, <=, + if (sme->sme_type == smla->smla_type) { + VERIFY3U(range_tree_space(smla->smla_rt) + sme->sme_run, <=, smla->smla_sm->sm_size); - range_tree_add(smla->smla_rt, offset, size); + range_tree_add(smla->smla_rt, sme->sme_offset, sme->sme_run); } else { - range_tree_remove(smla->smla_rt, offset, size); + range_tree_remove(smla->smla_rt, sme->sme_offset, sme->sme_run); } return (0); @@ -367,43 +490,239 @@ space_map_histogram_add(space_map_t *sm, range_tree_t } } -uint64_t *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Mon Jul 30 23:53:26 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DD6B106471A; Mon, 30 Jul 2018 23:53:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32A4D8B688; Mon, 30 Jul 2018 23:53:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1394C3CB1; Mon, 30 Jul 2018 23:53:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6UNrPVT074421; Mon, 30 Jul 2018 23:53:25 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6UNrPFQ074418; Mon, 30 Jul 2018 23:53:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807302353.w6UNrPFQ074418@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Mon, 30 Jul 2018 23:53:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336948 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 336948 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2018 23:53:26 -0000 Author: mav Date: Mon Jul 30 23:53:25 2018 New Revision: 336948 URL: https://svnweb.freebsd.org/changeset/base/336948 Log: 9112 Improve allocation performance on high-end systems On high-end systems running async sequential write workloads, especially NUMA systems with flash or NVMe storage, one significant performance bottleneck is selecting a metaslab to do allocations from. This process can be parallelized, providing significant performance increases for these workloads. illumos/illumos-gate@f78cdc34af236a6199dd9e21376f4a46348c0d56 Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: Serapheim Dimitropoulos Reviewed by: Alexander Motin Approved by: Gordon Ross Author: Paul Dagnelie Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Mon Jul 30 23:47:38 2018 (r336947) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Mon Jul 30 23:53:25 2018 (r336948) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2015 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2014 Integros [integros.com] */ @@ -212,6 +212,8 @@ static uint64_t metaslab_weight(metaslab_t *); static void metaslab_set_fragmentation(metaslab_t *); static void metaslab_free_impl(vdev_t *, uint64_t, uint64_t, boolean_t); static void metaslab_check_free_impl(vdev_t *, uint64_t, uint64_t); +static void metaslab_passivate(metaslab_t *msp, uint64_t weight); +static uint64_t metaslab_weight_from_range_tree(metaslab_t *msp); kmem_cache_t *metaslab_alloc_trace_cache; @@ -231,7 +233,12 @@ metaslab_class_create(spa_t *spa, metaslab_ops_t *ops) mc->mc_rotor = NULL; mc->mc_ops = ops; mutex_init(&mc->mc_lock, NULL, MUTEX_DEFAULT, NULL); - refcount_create_tracked(&mc->mc_alloc_slots); + mc->mc_alloc_slots = kmem_zalloc(spa->spa_alloc_count * + sizeof (refcount_t), KM_SLEEP); + mc->mc_alloc_max_slots = kmem_zalloc(spa->spa_alloc_count * + sizeof (uint64_t), KM_SLEEP); + for (int i = 0; i < spa->spa_alloc_count; i++) + refcount_create_tracked(&mc->mc_alloc_slots[i]); return (mc); } @@ -245,7 +252,12 @@ metaslab_class_destroy(metaslab_class_t *mc) ASSERT(mc->mc_space == 0); ASSERT(mc->mc_dspace == 0); - refcount_destroy(&mc->mc_alloc_slots); + for (int i = 0; i < mc->mc_spa->spa_alloc_count; i++) + refcount_destroy(&mc->mc_alloc_slots[i]); + kmem_free(mc->mc_alloc_slots, mc->mc_spa->spa_alloc_count * + sizeof (refcount_t)); + kmem_free(mc->mc_alloc_max_slots, mc->mc_spa->spa_alloc_count * + sizeof (uint64_t)); mutex_destroy(&mc->mc_lock); kmem_free(mc, sizeof (metaslab_class_t)); } @@ -442,6 +454,30 @@ metaslab_compare(const void *x1, const void *x2) const metaslab_t *m1 = x1; const metaslab_t *m2 = x2; + int sort1 = 0; + int sort2 = 0; + if (m1->ms_allocator != -1 && m1->ms_primary) + sort1 = 1; + else if (m1->ms_allocator != -1 && !m1->ms_primary) + sort1 = 2; + if (m2->ms_allocator != -1 && m2->ms_primary) + sort2 = 1; + else if (m2->ms_allocator != -1 && !m2->ms_primary) + sort2 = 2; + + /* + * Sort inactive metaslabs first, then primaries, then secondaries. When + * selecting a metaslab to allocate from, an allocator first tries its + * primary, then secondary active metaslab. If it doesn't have active + * metaslabs, or can't allocate from them, it searches for an inactive + * metaslab to activate. If it can't find a suitable one, it will steal + * a primary or secondary metaslab from another allocator. + */ + if (sort1 < sort2) + return (-1); + if (sort1 > sort2) + return (1); + if (m1->ms_weight < m2->ms_weight) return (1); if (m1->ms_weight > m2->ms_weight) @@ -593,12 +629,16 @@ metaslab_group_alloc_update(metaslab_group_t *mg) } metaslab_group_t * -metaslab_group_create(metaslab_class_t *mc, vdev_t *vd) +metaslab_group_create(metaslab_class_t *mc, vdev_t *vd, int allocators) { metaslab_group_t *mg; mg = kmem_zalloc(sizeof (metaslab_group_t), KM_SLEEP); mutex_init(&mg->mg_lock, NULL, MUTEX_DEFAULT, NULL); + mg->mg_primaries = kmem_zalloc(allocators * sizeof (metaslab_t *), + KM_SLEEP); + mg->mg_secondaries = kmem_zalloc(allocators * sizeof (metaslab_t *), + KM_SLEEP); avl_create(&mg->mg_metaslab_tree, metaslab_compare, sizeof (metaslab_t), offsetof(struct metaslab, ms_group_node)); mg->mg_vd = vd; @@ -606,8 +646,17 @@ metaslab_group_create(metaslab_class_t *mc, vdev_t *vd mg->mg_activation_count = 0; mg->mg_initialized = B_FALSE; mg->mg_no_free_space = B_TRUE; - refcount_create_tracked(&mg->mg_alloc_queue_depth); + mg->mg_allocators = allocators; + mg->mg_alloc_queue_depth = kmem_zalloc(allocators * sizeof (refcount_t), + KM_SLEEP); + mg->mg_cur_max_alloc_queue_depth = kmem_zalloc(allocators * + sizeof (uint64_t), KM_SLEEP); + for (int i = 0; i < allocators; i++) { + refcount_create_tracked(&mg->mg_alloc_queue_depth[i]); + mg->mg_cur_max_alloc_queue_depth[i] = 0; + } + mg->mg_taskq = taskq_create("metaslab_group_taskq", metaslab_load_pct, minclsyspri, 10, INT_MAX, TASKQ_THREADS_CPU_PCT); @@ -628,8 +677,20 @@ metaslab_group_destroy(metaslab_group_t *mg) taskq_destroy(mg->mg_taskq); avl_destroy(&mg->mg_metaslab_tree); + kmem_free(mg->mg_primaries, mg->mg_allocators * sizeof (metaslab_t *)); + kmem_free(mg->mg_secondaries, mg->mg_allocators * + sizeof (metaslab_t *)); mutex_destroy(&mg->mg_lock); - refcount_destroy(&mg->mg_alloc_queue_depth); + + for (int i = 0; i < mg->mg_allocators; i++) { + refcount_destroy(&mg->mg_alloc_queue_depth[i]); + mg->mg_cur_max_alloc_queue_depth[i] = 0; + } + kmem_free(mg->mg_alloc_queue_depth, mg->mg_allocators * + sizeof (refcount_t)); + kmem_free(mg->mg_cur_max_alloc_queue_depth, mg->mg_allocators * + sizeof (uint64_t)); + kmem_free(mg, sizeof (metaslab_group_t)); } @@ -708,6 +769,22 @@ metaslab_group_passivate(metaslab_group_t *mg) taskq_wait(mg->mg_taskq); spa_config_enter(spa, locks & ~(SCL_ZIO - 1), spa, RW_WRITER); metaslab_group_alloc_update(mg); + for (int i = 0; i < mg->mg_allocators; i++) { + metaslab_t *msp = mg->mg_primaries[i]; + if (msp != NULL) { + mutex_enter(&msp->ms_lock); + metaslab_passivate(msp, + metaslab_weight_from_range_tree(msp)); + mutex_exit(&msp->ms_lock); + } + msp = mg->mg_secondaries[i]; + if (msp != NULL) { + mutex_enter(&msp->ms_lock); + metaslab_passivate(msp, + metaslab_weight_from_range_tree(msp)); + mutex_exit(&msp->ms_lock); + } + } mgprev = mg->mg_prev; mgnext = mg->mg_next; @@ -848,6 +925,17 @@ metaslab_group_remove(metaslab_group_t *mg, metaslab_t } static void +metaslab_group_sort_impl(metaslab_group_t *mg, metaslab_t *msp, uint64_t weight) +{ + ASSERT(MUTEX_HELD(&mg->mg_lock)); + ASSERT(msp->ms_group == mg); + avl_remove(&mg->mg_metaslab_tree, msp); + msp->ms_weight = weight; + avl_add(&mg->mg_metaslab_tree, msp); + +} + +static void metaslab_group_sort(metaslab_group_t *mg, metaslab_t *msp, uint64_t weight) { /* @@ -858,10 +946,7 @@ metaslab_group_sort(metaslab_group_t *mg, metaslab_t * ASSERT(MUTEX_HELD(&msp->ms_lock)); mutex_enter(&mg->mg_lock); - ASSERT(msp->ms_group == mg); - avl_remove(&mg->mg_metaslab_tree, msp); - msp->ms_weight = weight; - avl_add(&mg->mg_metaslab_tree, msp); + metaslab_group_sort_impl(mg, msp, weight); mutex_exit(&mg->mg_lock); } @@ -909,7 +994,7 @@ metaslab_group_fragmentation(metaslab_group_t *mg) */ static boolean_t metaslab_group_allocatable(metaslab_group_t *mg, metaslab_group_t *rotor, - uint64_t psize) + uint64_t psize, int allocator) { spa_t *spa = mg->mg_vd->vdev_spa; metaslab_class_t *mc = mg->mg_class; @@ -938,7 +1023,7 @@ metaslab_group_allocatable(metaslab_group_t *mg, metas if (mg->mg_allocatable) { metaslab_group_t *mgp; int64_t qdepth; - uint64_t qmax = mg->mg_max_alloc_queue_depth; + uint64_t qmax = mg->mg_cur_max_alloc_queue_depth[allocator]; if (!mc->mc_alloc_throttle_enabled) return (B_TRUE); @@ -950,7 +1035,7 @@ metaslab_group_allocatable(metaslab_group_t *mg, metas if (mg->mg_no_free_space) return (B_FALSE); - qdepth = refcount_count(&mg->mg_alloc_queue_depth); + qdepth = refcount_count(&mg->mg_alloc_queue_depth[allocator]); /* * If this metaslab group is below its qmax or it's @@ -969,9 +1054,10 @@ metaslab_group_allocatable(metaslab_group_t *mg, metas * groups at the same time when we make this check. */ for (mgp = mg->mg_next; mgp != rotor; mgp = mgp->mg_next) { - qmax = mgp->mg_max_alloc_queue_depth; + qmax = mgp->mg_cur_max_alloc_queue_depth[allocator]; - qdepth = refcount_count(&mgp->mg_alloc_queue_depth); + qdepth = refcount_count( + &mgp->mg_alloc_queue_depth[allocator]); /* * If there is another metaslab group that @@ -1458,6 +1544,8 @@ metaslab_init(metaslab_group_t *mg, uint64_t id, uint6 ms->ms_id = id; ms->ms_start = id << vd->vdev_ms_shift; ms->ms_size = 1ULL << vd->vdev_ms_shift; + ms->ms_allocator = -1; + ms->ms_new = B_TRUE; /* * We only open space map objects that already exist. All others @@ -1553,6 +1641,7 @@ metaslab_fini(metaslab_t *msp) cv_destroy(&msp->ms_load_cv); mutex_destroy(&msp->ms_lock); mutex_destroy(&msp->ms_sync_lock); + ASSERT3U(msp->ms_allocator, ==, -1); kmem_free(msp, sizeof (metaslab_t)); } @@ -1949,19 +2038,59 @@ metaslab_weight(metaslab_t *msp) } static int -metaslab_activate(metaslab_t *msp, uint64_t activation_weight) +metaslab_activate_allocator(metaslab_group_t *mg, metaslab_t *msp, + int allocator, uint64_t activation_weight) { + /* + * If we're activating for the claim code, we don't want to actually + * set the metaslab up for a specific allocator. + */ + if (activation_weight == METASLAB_WEIGHT_CLAIM) + return (0); + metaslab_t **arr = (activation_weight == METASLAB_WEIGHT_PRIMARY ? + mg->mg_primaries : mg->mg_secondaries); + ASSERT(MUTEX_HELD(&msp->ms_lock)); + mutex_enter(&mg->mg_lock); + if (arr[allocator] != NULL) { + mutex_exit(&mg->mg_lock); + return (EEXIST); + } + arr[allocator] = msp; + ASSERT3S(msp->ms_allocator, ==, -1); + msp->ms_allocator = allocator; + msp->ms_primary = (activation_weight == METASLAB_WEIGHT_PRIMARY); + mutex_exit(&mg->mg_lock); + + return (0); +} + +static int +metaslab_activate(metaslab_t *msp, int allocator, uint64_t activation_weight) +{ + ASSERT(MUTEX_HELD(&msp->ms_lock)); + if ((msp->ms_weight & METASLAB_ACTIVE_MASK) == 0) { + int error = 0; metaslab_load_wait(msp); if (!msp->ms_loaded) { - int error = metaslab_load(msp); - if (error) { + if ((error = metaslab_load(msp)) != 0) { metaslab_group_sort(msp->ms_group, msp, 0); return (error); } } + if ((msp->ms_weight & METASLAB_ACTIVE_MASK) != 0) { + /* + * The metaslab was activated for another allocator + * while we were waiting, we should reselect. + */ + return (EBUSY); + } + if ((error = metaslab_activate_allocator(msp->ms_group, msp, + allocator, activation_weight)) != 0) { + return (error); + } msp->ms_activation_weight = msp->ms_weight; metaslab_group_sort(msp->ms_group, msp, @@ -1974,6 +2103,34 @@ metaslab_activate(metaslab_t *msp, uint64_t activation } static void +metaslab_passivate_allocator(metaslab_group_t *mg, metaslab_t *msp, + uint64_t weight) +{ + ASSERT(MUTEX_HELD(&msp->ms_lock)); + if (msp->ms_weight & METASLAB_WEIGHT_CLAIM) { + metaslab_group_sort(mg, msp, weight); + return; + } + + mutex_enter(&mg->mg_lock); + ASSERT3P(msp->ms_group, ==, mg); + if (msp->ms_primary) { + ASSERT3U(0, <=, msp->ms_allocator); + ASSERT3U(msp->ms_allocator, <, mg->mg_allocators); + ASSERT3P(mg->mg_primaries[msp->ms_allocator], ==, msp); + ASSERT(msp->ms_weight & METASLAB_WEIGHT_PRIMARY); + mg->mg_primaries[msp->ms_allocator] = NULL; + } else { + ASSERT(msp->ms_weight & METASLAB_WEIGHT_SECONDARY); + ASSERT3P(mg->mg_secondaries[msp->ms_allocator], ==, msp); + mg->mg_secondaries[msp->ms_allocator] = NULL; + } + msp->ms_allocator = -1; + metaslab_group_sort_impl(mg, msp, weight); + mutex_exit(&mg->mg_lock); +} + +static void metaslab_passivate(metaslab_t *msp, uint64_t weight) { uint64_t size = weight & ~METASLAB_WEIGHT_TYPE; @@ -1988,7 +2145,7 @@ metaslab_passivate(metaslab_t *msp, uint64_t weight) ASSERT0(weight & METASLAB_ACTIVE_MASK); msp->ms_activation_weight = 0; - metaslab_group_sort(msp->ms_group, msp, weight); + metaslab_passivate_allocator(msp->ms_group, msp, weight); ASSERT((msp->ms_weight & METASLAB_ACTIVE_MASK) == 0); } @@ -2542,11 +2699,18 @@ metaslab_sync_done(metaslab_t *msp, uint64_t txg) vdev_dirty(vd, VDD_METASLAB, msp, txg + 1); } + if (msp->ms_new) { + msp->ms_new = B_FALSE; + mutex_enter(&mg->mg_lock); + mg->mg_ms_ready++; + mutex_exit(&mg->mg_lock); + } /* * Calculate the new weights before unloading any metaslabs. * This will give us the most accurate weighting. */ - metaslab_group_sort(mg, msp, metaslab_weight(msp)); + metaslab_group_sort(mg, msp, metaslab_weight(msp) | + (msp->ms_weight & METASLAB_ACTIVE_MASK)); /* * If the metaslab is loaded and we've not tried to load or allocate @@ -2558,6 +2722,10 @@ metaslab_sync_done(metaslab_t *msp, uint64_t txg) VERIFY0(range_tree_space( msp->ms_allocating[(txg + t) & TXG_MASK])); } + if (msp->ms_allocator != -1) { + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_ACTIVE_MASK); + } if (!metaslab_debug_unload) metaslab_unload(msp); @@ -2651,7 +2819,8 @@ metaslab_alloc_trace_fini(void) */ static void metaslab_trace_add(zio_alloc_list_t *zal, metaslab_group_t *mg, - metaslab_t *msp, uint64_t psize, uint32_t dva_id, uint64_t offset) + metaslab_t *msp, uint64_t psize, uint32_t dva_id, uint64_t offset, + int allocator) { if (!metaslab_trace_enabled) return; @@ -2684,6 +2853,7 @@ metaslab_trace_add(zio_alloc_list_t *zal, metaslab_gro mat->mat_dva_id = dva_id; mat->mat_offset = offset; mat->mat_weight = 0; + mat->mat_allocator = allocator; if (msp != NULL) mat->mat_weight = msp->ms_weight; @@ -2724,35 +2894,56 @@ metaslab_trace_fini(zio_alloc_list_t *zal) */ static void -metaslab_group_alloc_increment(spa_t *spa, uint64_t vdev, void *tag, int flags) +metaslab_group_alloc_increment(spa_t *spa, uint64_t vdev, void *tag, int flags, + int allocator) { if (!(flags & METASLAB_ASYNC_ALLOC) || - flags & METASLAB_DONT_THROTTLE) + (flags & METASLAB_DONT_THROTTLE)) return; metaslab_group_t *mg = vdev_lookup_top(spa, vdev)->vdev_mg; if (!mg->mg_class->mc_alloc_throttle_enabled) return; - (void) refcount_add(&mg->mg_alloc_queue_depth, tag); + (void) refcount_add(&mg->mg_alloc_queue_depth[allocator], tag); } +static void +metaslab_group_increment_qdepth(metaslab_group_t *mg, int allocator) +{ + uint64_t max = mg->mg_max_alloc_queue_depth; + uint64_t cur = mg->mg_cur_max_alloc_queue_depth[allocator]; + while (cur < max) { + if (atomic_cas_64(&mg->mg_cur_max_alloc_queue_depth[allocator], + cur, cur + 1) == cur) { + atomic_inc_64( + &mg->mg_class->mc_alloc_max_slots[allocator]); + return; + } + cur = mg->mg_cur_max_alloc_queue_depth[allocator]; + } +} + void -metaslab_group_alloc_decrement(spa_t *spa, uint64_t vdev, void *tag, int flags) +metaslab_group_alloc_decrement(spa_t *spa, uint64_t vdev, void *tag, int flags, + int allocator, boolean_t io_complete) { if (!(flags & METASLAB_ASYNC_ALLOC) || - flags & METASLAB_DONT_THROTTLE) + (flags & METASLAB_DONT_THROTTLE)) return; metaslab_group_t *mg = vdev_lookup_top(spa, vdev)->vdev_mg; if (!mg->mg_class->mc_alloc_throttle_enabled) return; - (void) refcount_remove(&mg->mg_alloc_queue_depth, tag); + (void) refcount_remove(&mg->mg_alloc_queue_depth[allocator], tag); + if (io_complete) + metaslab_group_increment_qdepth(mg, allocator); } void -metaslab_group_alloc_verify(spa_t *spa, const blkptr_t *bp, void *tag) +metaslab_group_alloc_verify(spa_t *spa, const blkptr_t *bp, void *tag, + int allocator) { #ifdef ZFS_DEBUG const dva_t *dva = bp->blk_dva; @@ -2761,7 +2952,8 @@ metaslab_group_alloc_verify(spa_t *spa, const blkptr_t for (int d = 0; d < ndvas; d++) { uint64_t vdev = DVA_GET_VDEV(&dva[d]); metaslab_group_t *mg = vdev_lookup_top(spa, vdev)->vdev_mg; - VERIFY(refcount_not_held(&mg->mg_alloc_queue_depth, tag)); + VERIFY(refcount_not_held(&mg->mg_alloc_queue_depth[allocator], + tag)); } #endif } @@ -2803,91 +2995,146 @@ metaslab_block_alloc(metaslab_t *msp, uint64_t size, u return (start); } +/* + * Find the metaslab with the highest weight that is less than what we've + * already tried. In the common case, this means that we will examine each + * metaslab at most once. Note that concurrent callers could reorder metaslabs + * by activation/passivation once we have dropped the mg_lock. If a metaslab is + * activated by another thread, and we fail to allocate from the metaslab we + * have selected, we may not try the newly-activated metaslab, and instead + * activate another metaslab. This is not optimal, but generally does not cause + * any problems (a possible exception being if every metaslab is completely full + * except for the the newly-activated metaslab which we fail to examine). + */ +static metaslab_t * +find_valid_metaslab(metaslab_group_t *mg, uint64_t activation_weight, + dva_t *dva, int d, uint64_t min_distance, uint64_t asize, int allocator, + zio_alloc_list_t *zal, metaslab_t *search, boolean_t *was_active) +{ + avl_index_t idx; + avl_tree_t *t = &mg->mg_metaslab_tree; + metaslab_t *msp = avl_find(t, search, &idx); + if (msp == NULL) + msp = avl_nearest(t, idx, AVL_AFTER); + + for (; msp != NULL; msp = AVL_NEXT(t, msp)) { + int i; + if (!metaslab_should_allocate(msp, asize)) { + metaslab_trace_add(zal, mg, msp, asize, d, + TRACE_TOO_SMALL, allocator); + continue; + } + + /* + * If the selected metaslab is condensing, skip it. + */ + if (msp->ms_condensing) + continue; + + *was_active = msp->ms_allocator != -1; + /* + * If we're activating as primary, this is our first allocation + * from this disk, so we don't need to check how close we are. + * If the metaslab under consideration was already active, + * we're getting desperate enough to steal another allocator's + * metaslab, so we still don't care about distances. + */ + if (activation_weight == METASLAB_WEIGHT_PRIMARY || *was_active) + break; + + uint64_t target_distance = min_distance + + (space_map_allocated(msp->ms_sm) != 0 ? 0 : + min_distance >> 1); + + for (i = 0; i < d; i++) { + if (metaslab_distance(msp, &dva[i]) < target_distance) + break; + } + if (i == d) + break; + } + + if (msp != NULL) { + search->ms_weight = msp->ms_weight; + search->ms_start = msp->ms_start + 1; + search->ms_allocator = msp->ms_allocator; + search->ms_primary = msp->ms_primary; + } + return (msp); +} + +/* ARGSUSED */ static uint64_t metaslab_group_alloc_normal(metaslab_group_t *mg, zio_alloc_list_t *zal, - uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d) + uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d, + int allocator) { metaslab_t *msp = NULL; uint64_t offset = -1ULL; uint64_t activation_weight; - uint64_t target_distance; - int i; + boolean_t tertiary = B_FALSE; activation_weight = METASLAB_WEIGHT_PRIMARY; - for (i = 0; i < d; i++) { - if (DVA_GET_VDEV(&dva[i]) == mg->mg_vd->vdev_id) { + for (int i = 0; i < d; i++) { + if (activation_weight == METASLAB_WEIGHT_PRIMARY && + DVA_GET_VDEV(&dva[i]) == mg->mg_vd->vdev_id) { activation_weight = METASLAB_WEIGHT_SECONDARY; + } else if (activation_weight == METASLAB_WEIGHT_SECONDARY && + DVA_GET_VDEV(&dva[i]) == mg->mg_vd->vdev_id) { + tertiary = B_TRUE; break; } } + /* + * If we don't have enough metaslabs active to fill the entire array, we + * just use the 0th slot. + */ + if (mg->mg_ms_ready < mg->mg_allocators * 2) { + tertiary = B_FALSE; + allocator = 0; + } + + ASSERT3U(mg->mg_vd->vdev_ms_count, >=, 2); + metaslab_t *search = kmem_alloc(sizeof (*search), KM_SLEEP); search->ms_weight = UINT64_MAX; search->ms_start = 0; + /* + * At the end of the metaslab tree are the already-active metaslabs, + * first the primaries, then the secondaries. When we resume searching + * through the tree, we need to consider ms_allocator and ms_primary so + * we start in the location right after where we left off, and don't + * accidentally loop forever considering the same metaslabs. + */ + search->ms_allocator = -1; + search->ms_primary = B_TRUE; for (;;) { - boolean_t was_active; - avl_tree_t *t = &mg->mg_metaslab_tree; - avl_index_t idx; + boolean_t was_active = B_FALSE; mutex_enter(&mg->mg_lock); - /* - * Find the metaslab with the highest weight that is less - * than what we've already tried. In the common case, this - * means that we will examine each metaslab at most once. - * Note that concurrent callers could reorder metaslabs - * by activation/passivation once we have dropped the mg_lock. - * If a metaslab is activated by another thread, and we fail - * to allocate from the metaslab we have selected, we may - * not try the newly-activated metaslab, and instead activate - * another metaslab. This is not optimal, but generally - * does not cause any problems (a possible exception being - * if every metaslab is completely full except for the - * the newly-activated metaslab which we fail to examine). - */ - msp = avl_find(t, search, &idx); - if (msp == NULL) - msp = avl_nearest(t, idx, AVL_AFTER); - for (; msp != NULL; msp = AVL_NEXT(t, msp)) { - - if (!metaslab_should_allocate(msp, asize)) { - metaslab_trace_add(zal, mg, msp, asize, d, - TRACE_TOO_SMALL); - continue; - } - - /* - * If the selected metaslab is condensing, skip it. - */ - if (msp->ms_condensing) - continue; - - was_active = msp->ms_weight & METASLAB_ACTIVE_MASK; - if (activation_weight == METASLAB_WEIGHT_PRIMARY) - break; - - target_distance = min_distance + - (space_map_allocated(msp->ms_sm) != 0 ? 0 : - min_distance >> 1); - - for (i = 0; i < d; i++) { - if (metaslab_distance(msp, &dva[i]) < - target_distance) - break; - } - if (i == d) - break; + if (activation_weight == METASLAB_WEIGHT_PRIMARY && + mg->mg_primaries[allocator] != NULL) { + msp = mg->mg_primaries[allocator]; + was_active = B_TRUE; + } else if (activation_weight == METASLAB_WEIGHT_SECONDARY && + mg->mg_secondaries[allocator] != NULL && !tertiary) { + msp = mg->mg_secondaries[allocator]; + was_active = B_TRUE; + } else { + msp = find_valid_metaslab(mg, activation_weight, dva, d, + min_distance, asize, allocator, zal, search, + &was_active); } + mutex_exit(&mg->mg_lock); if (msp == NULL) { kmem_free(search, sizeof (*search)); return (-1ULL); } - search->ms_weight = msp->ms_weight; - search->ms_start = msp->ms_start + 1; mutex_enter(&msp->ms_lock); - /* * Ensure that the metaslab we have selected is still * capable of handling our request. It's possible that @@ -2901,18 +3148,32 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ continue; } - if ((msp->ms_weight & METASLAB_WEIGHT_SECONDARY) && - activation_weight == METASLAB_WEIGHT_PRIMARY) { - metaslab_passivate(msp, - msp->ms_weight & ~METASLAB_ACTIVE_MASK); + /* + * If the metaslab is freshly activated for an allocator that + * isn't the one we're allocating from, or if it's a primary and + * we're seeking a secondary (or vice versa), we go back and + * select a new metaslab. + */ + if (!was_active && (msp->ms_weight & METASLAB_ACTIVE_MASK) && + (msp->ms_allocator != -1) && + (msp->ms_allocator != allocator || ((activation_weight == + METASLAB_WEIGHT_PRIMARY) != msp->ms_primary))) { mutex_exit(&msp->ms_lock); continue; } - if (metaslab_activate(msp, activation_weight) != 0) { + if (msp->ms_weight & METASLAB_WEIGHT_CLAIM) { + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_WEIGHT_CLAIM); mutex_exit(&msp->ms_lock); continue; } + + if (metaslab_activate(msp, allocator, activation_weight) != 0) { + mutex_exit(&msp->ms_lock); + continue; + } + msp->ms_selected_txg = txg; /* @@ -2925,7 +3186,7 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ if (!metaslab_should_allocate(msp, asize)) { /* Passivate this metaslab and select a new one. */ metaslab_trace_add(zal, mg, msp, asize, d, - TRACE_TOO_SMALL); + TRACE_TOO_SMALL, allocator); goto next; } @@ -2936,13 +3197,15 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ */ if (msp->ms_condensing) { metaslab_trace_add(zal, mg, msp, asize, d, - TRACE_CONDENSING); + TRACE_CONDENSING, allocator); + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_ACTIVE_MASK); mutex_exit(&msp->ms_lock); continue; } offset = metaslab_block_alloc(msp, asize, txg); - metaslab_trace_add(zal, mg, msp, asize, d, offset); + metaslab_trace_add(zal, mg, msp, asize, d, offset, allocator); if (offset != -1ULL) { /* Proactively passivate the metaslab, if needed */ @@ -2998,19 +3261,20 @@ next: static uint64_t metaslab_group_alloc(metaslab_group_t *mg, zio_alloc_list_t *zal, - uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d) + uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d, + int allocator) { uint64_t offset; ASSERT(mg->mg_initialized); offset = metaslab_group_alloc_normal(mg, zal, asize, txg, - min_distance, dva, d); + min_distance, dva, d, allocator); mutex_enter(&mg->mg_lock); if (offset == -1ULL) { mg->mg_failed_allocations++; metaslab_trace_add(zal, mg, NULL, asize, d, - TRACE_GROUP_FAILURE); + TRACE_GROUP_FAILURE, allocator); if (asize == SPA_GANGBLOCKSIZE) { /* * This metaslab group was unable to allocate @@ -3045,7 +3309,7 @@ int ditto_same_vdev_distance_shift = 3; int metaslab_alloc_dva(spa_t *spa, metaslab_class_t *mc, uint64_t psize, dva_t *dva, int d, dva_t *hintdva, uint64_t txg, int flags, - zio_alloc_list_t *zal) + zio_alloc_list_t *zal, int allocator) { metaslab_group_t *mg, *rotor; vdev_t *vd; @@ -3057,7 +3321,8 @@ metaslab_alloc_dva(spa_t *spa, metaslab_class_t *mc, u * For testing, make some blocks above a certain size be gang blocks. */ if (psize >= metaslab_force_ganging && (ddi_get_lbolt() & 3) == 0) { - metaslab_trace_add(zal, NULL, NULL, psize, d, TRACE_FORCE_GANG); + metaslab_trace_add(zal, NULL, NULL, psize, d, TRACE_FORCE_GANG, + allocator); return (SET_ERROR(ENOSPC)); } @@ -3143,12 +3408,12 @@ top: */ if (allocatable && !GANG_ALLOCATION(flags) && !try_hard) { allocatable = metaslab_group_allocatable(mg, rotor, - psize); + psize, allocator); } if (!allocatable) { metaslab_trace_add(zal, mg, NULL, psize, d, - TRACE_NOT_ALLOCATABLE); + TRACE_NOT_ALLOCATABLE, allocator); goto next; } @@ -3163,7 +3428,7 @@ top: vd->vdev_state < VDEV_STATE_HEALTHY) && d == 0 && !try_hard && vd->vdev_children == 0) { metaslab_trace_add(zal, mg, NULL, psize, d, - TRACE_VDEV_ERROR); + TRACE_VDEV_ERROR, allocator); goto next; } @@ -3187,7 +3452,7 @@ top: ASSERT(P2PHASE(asize, 1ULL << vd->vdev_ashift) == 0); uint64_t offset = metaslab_group_alloc(mg, zal, asize, txg, - distance, dva, d); + distance, dva, d, allocator); if (offset != -1ULL) { /* @@ -3250,7 +3515,7 @@ next: bzero(&dva[d], sizeof (dva_t)); - metaslab_trace_add(zal, rotor, NULL, psize, d, TRACE_ENOSPC); + metaslab_trace_add(zal, rotor, NULL, psize, d, TRACE_ENOSPC, allocator); return (SET_ERROR(ENOSPC)); } @@ -3551,18 +3816,20 @@ metaslab_free_dva(spa_t *spa, const dva_t *dva, boolea * the reservation. */ boolean_t -metaslab_class_throttle_reserve(metaslab_class_t *mc, int slots, zio_t *zio, - int flags) +metaslab_class_throttle_reserve(metaslab_class_t *mc, int slots, int allocator, + zio_t *zio, int flags) { uint64_t available_slots = 0; boolean_t slot_reserved = B_FALSE; + uint64_t max = mc->mc_alloc_max_slots[allocator]; ASSERT(mc->mc_alloc_throttle_enabled); mutex_enter(&mc->mc_lock); - uint64_t reserved_slots = refcount_count(&mc->mc_alloc_slots); - if (reserved_slots < mc->mc_alloc_max_slots) - available_slots = mc->mc_alloc_max_slots - reserved_slots; + uint64_t reserved_slots = + refcount_count(&mc->mc_alloc_slots[allocator]); + if (reserved_slots < max) + available_slots = max - reserved_slots; if (slots <= available_slots || GANG_ALLOCATION(flags)) { /* @@ -3570,7 +3837,9 @@ metaslab_class_throttle_reserve(metaslab_class_t *mc, * them individually when an I/O completes. */ for (int d = 0; d < slots; d++) { - reserved_slots = refcount_add(&mc->mc_alloc_slots, zio); + reserved_slots = + refcount_add(&mc->mc_alloc_slots[allocator], + zio); } zio->io_flags |= ZIO_FLAG_IO_ALLOCATING; slot_reserved = B_TRUE; @@ -3581,12 +3850,14 @@ metaslab_class_throttle_reserve(metaslab_class_t *mc, } void -metaslab_class_throttle_unreserve(metaslab_class_t *mc, int slots, zio_t *zio) +metaslab_class_throttle_unreserve(metaslab_class_t *mc, int slots, + int allocator, zio_t *zio) { ASSERT(mc->mc_alloc_throttle_enabled); mutex_enter(&mc->mc_lock); for (int d = 0; d < slots; d++) { - (void) refcount_remove(&mc->mc_alloc_slots, zio); + (void) refcount_remove(&mc->mc_alloc_slots[allocator], + zio); } mutex_exit(&mc->mc_lock); } @@ -3608,7 +3879,13 @@ metaslab_claim_concrete(vdev_t *vd, uint64_t offset, u mutex_enter(&msp->ms_lock); if ((txg != 0 && spa_writeable(spa)) || !msp->ms_loaded) - error = metaslab_activate(msp, METASLAB_WEIGHT_SECONDARY); + error = metaslab_activate(msp, 0, METASLAB_WEIGHT_CLAIM); + /* + * No need to fail in that case; someone else has activated the + * metaslab, but that doesn't preclude us from using it. + */ + if (error == EBUSY) + error = 0; if (error == 0 && !range_tree_contains(msp->ms_allocatable, offset, size)) @@ -3713,7 +3990,7 @@ metaslab_claim_dva(spa_t *spa, const dva_t *dva, uint6 int metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint64_t psize, blkptr_t *bp, int ndvas, uint64_t txg, blkptr_t *hintbp, int flags, - zio_alloc_list_t *zal, zio_t *zio) + zio_alloc_list_t *zal, zio_t *zio, int allocator) { dva_t *dva = bp->blk_dva; dva_t *hintdva = hintbp->blk_dva; @@ -3736,12 +4013,13 @@ metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint6 for (int d = 0; d < ndvas; d++) { error = metaslab_alloc_dva(spa, mc, psize, dva, d, hintdva, - txg, flags, zal); + txg, flags, zal, allocator); if (error != 0) { for (d--; d >= 0; d--) { metaslab_unalloc_dva(spa, &dva[d], txg); metaslab_group_alloc_decrement(spa, - DVA_GET_VDEV(&dva[d]), zio, flags); + DVA_GET_VDEV(&dva[d]), zio, flags, + allocator, B_FALSE); bzero(&dva[d], sizeof (dva_t)); } spa_config_exit(spa, SCL_ALLOC, FTAG); @@ -3752,7 +4030,7 @@ metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint6 * based on the newly allocated dva. */ metaslab_group_alloc_increment(spa, - DVA_GET_VDEV(&dva[d]), zio, flags); + DVA_GET_VDEV(&dva[d]), zio, flags, allocator); } } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Mon Jul 30 23:47:38 2018 (r336947) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Mon Jul 30 23:53:25 2018 (r336948) @@ -7398,9 +7398,11 @@ spa_sync(spa_t *spa, uint64_t txg) spa->spa_syncing_txg = txg; spa->spa_sync_pass = 0; - mutex_enter(&spa->spa_alloc_lock); - VERIFY0(avl_numnodes(&spa->spa_alloc_tree)); - mutex_exit(&spa->spa_alloc_lock); + for (int i = 0; i < spa->spa_alloc_count; i++) { + mutex_enter(&spa->spa_alloc_locks[i]); + VERIFY0(avl_numnodes(&spa->spa_alloc_trees[i])); + mutex_exit(&spa->spa_alloc_locks[i]); + } /* * If there are any pending vdev state changes, convert them @@ -7459,7 +7461,7 @@ spa_sync(spa_t *spa, uint64_t txg) * The max queue depth will not change in the middle of syncing * out this txg. */ - uint64_t queue_depth_total = 0; + uint64_t slots_per_allocator = 0; for (int c = 0; c < rvd->vdev_children; c++) { vdev_t *tvd = rvd->vdev_child[c]; metaslab_group_t *mg = tvd->vdev_mg; @@ -7473,18 +7475,23 @@ spa_sync(spa_t *spa, uint64_t txg) * allocations look at mg_max_alloc_queue_depth, and async * allocations all happen from spa_sync(). */ - ASSERT0(refcount_count(&mg->mg_alloc_queue_depth)); + for (int i = 0; i < spa->spa_alloc_count; i++) + ASSERT0(refcount_count(&(mg->mg_alloc_queue_depth[i]))); mg->mg_max_alloc_queue_depth = max_queue_depth; - queue_depth_total += mg->mg_max_alloc_queue_depth; + + for (int i = 0; i < spa->spa_alloc_count; i++) { + mg->mg_cur_max_alloc_queue_depth[i] = + zfs_vdev_def_queue_depth; + } + slots_per_allocator += zfs_vdev_def_queue_depth; } metaslab_class_t *mc = spa_normal_class(spa); - ASSERT0(refcount_count(&mc->mc_alloc_slots)); - mc->mc_alloc_max_slots = queue_depth_total; + for (int i = 0; i < spa->spa_alloc_count; i++) { + ASSERT0(refcount_count(&mc->mc_alloc_slots[i])); + mc->mc_alloc_max_slots[i] = slots_per_allocator; + } mc->mc_alloc_throttle_enabled = zio_dva_throttle_enabled; - ASSERT3U(mc->mc_alloc_max_slots, <=, - max_queue_depth * rvd->vdev_children); - for (int c = 0; c < rvd->vdev_children; c++) { vdev_t *vd = rvd->vdev_child[c]; vdev_indirect_state_sync_verify(vd); @@ -7661,9 +7668,11 @@ spa_sync(spa_t *spa, uint64_t txg) dsl_pool_sync_done(dp, txg); - mutex_enter(&spa->spa_alloc_lock); - VERIFY0(avl_numnodes(&spa->spa_alloc_tree)); - mutex_exit(&spa->spa_alloc_lock); + for (int i = 0; i < spa->spa_alloc_count; i++) { *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Tue Jul 31 00:02:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6EB491065089; Tue, 31 Jul 2018 00:02:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FA5A8BFD8; Tue, 31 Jul 2018 00:02:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 009BC3E4B; Tue, 31 Jul 2018 00:02:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V02gJZ079554; Tue, 31 Jul 2018 00:02:42 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V02g0a079551; Tue, 31 Jul 2018 00:02:42 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310002.w6V02g0a079551@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:02:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336949 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 336949 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:02:43 -0000 Author: mav Date: Tue Jul 31 00:02:42 2018 New Revision: 336949 URL: https://svnweb.freebsd.org/changeset/base/336949 Log: MFV r336948: 9112 Improve allocation performance on high-end systems On high-end systems running async sequential write workloads, especially NUMA systems with flash or NVMe storage, one significant performance bottleneck is selecting a metaslab to do allocations from. This process can be parallelized, providing significant performance increases for these workloads. illumos/illumos-gate@f78cdc34af236a6199dd9e21376f4a46348c0d56 Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: Serapheim Dimitropoulos Reviewed by: Alexander Motin Approved by: Gordon Ross Author: Paul Dagnelie Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Mon Jul 30 23:53:25 2018 (r336948) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:02:42 2018 (r336949) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2015 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2014 Integros [integros.com] */ @@ -275,6 +275,8 @@ static uint64_t metaslab_weight(metaslab_t *); static void metaslab_set_fragmentation(metaslab_t *); static void metaslab_free_impl(vdev_t *, uint64_t, uint64_t, boolean_t); static void metaslab_check_free_impl(vdev_t *, uint64_t, uint64_t); +static void metaslab_passivate(metaslab_t *msp, uint64_t weight); +static uint64_t metaslab_weight_from_range_tree(metaslab_t *msp); kmem_cache_t *metaslab_alloc_trace_cache; @@ -294,7 +296,12 @@ metaslab_class_create(spa_t *spa, metaslab_ops_t *ops) mc->mc_rotor = NULL; mc->mc_ops = ops; mutex_init(&mc->mc_lock, NULL, MUTEX_DEFAULT, NULL); - refcount_create_tracked(&mc->mc_alloc_slots); + mc->mc_alloc_slots = kmem_zalloc(spa->spa_alloc_count * + sizeof (refcount_t), KM_SLEEP); + mc->mc_alloc_max_slots = kmem_zalloc(spa->spa_alloc_count * + sizeof (uint64_t), KM_SLEEP); + for (int i = 0; i < spa->spa_alloc_count; i++) + refcount_create_tracked(&mc->mc_alloc_slots[i]); return (mc); } @@ -308,7 +315,12 @@ metaslab_class_destroy(metaslab_class_t *mc) ASSERT(mc->mc_space == 0); ASSERT(mc->mc_dspace == 0); - refcount_destroy(&mc->mc_alloc_slots); + for (int i = 0; i < mc->mc_spa->spa_alloc_count; i++) + refcount_destroy(&mc->mc_alloc_slots[i]); + kmem_free(mc->mc_alloc_slots, mc->mc_spa->spa_alloc_count * + sizeof (refcount_t)); + kmem_free(mc->mc_alloc_max_slots, mc->mc_spa->spa_alloc_count * + sizeof (uint64_t)); mutex_destroy(&mc->mc_lock); kmem_free(mc, sizeof (metaslab_class_t)); } @@ -532,6 +544,30 @@ metaslab_compare(const void *x1, const void *x2) const metaslab_t *m1 = x1; const metaslab_t *m2 = x2; + int sort1 = 0; + int sort2 = 0; + if (m1->ms_allocator != -1 && m1->ms_primary) + sort1 = 1; + else if (m1->ms_allocator != -1 && !m1->ms_primary) + sort1 = 2; + if (m2->ms_allocator != -1 && m2->ms_primary) + sort2 = 1; + else if (m2->ms_allocator != -1 && !m2->ms_primary) + sort2 = 2; + + /* + * Sort inactive metaslabs first, then primaries, then secondaries. When + * selecting a metaslab to allocate from, an allocator first tries its + * primary, then secondary active metaslab. If it doesn't have active + * metaslabs, or can't allocate from them, it searches for an inactive + * metaslab to activate. If it can't find a suitable one, it will steal + * a primary or secondary metaslab from another allocator. + */ + if (sort1 < sort2) + return (-1); + if (sort1 > sort2) + return (1); + if (m1->ms_weight < m2->ms_weight) return (1); if (m1->ms_weight > m2->ms_weight) @@ -683,12 +719,16 @@ metaslab_group_alloc_update(metaslab_group_t *mg) } metaslab_group_t * -metaslab_group_create(metaslab_class_t *mc, vdev_t *vd) +metaslab_group_create(metaslab_class_t *mc, vdev_t *vd, int allocators) { metaslab_group_t *mg; mg = kmem_zalloc(sizeof (metaslab_group_t), KM_SLEEP); mutex_init(&mg->mg_lock, NULL, MUTEX_DEFAULT, NULL); + mg->mg_primaries = kmem_zalloc(allocators * sizeof (metaslab_t *), + KM_SLEEP); + mg->mg_secondaries = kmem_zalloc(allocators * sizeof (metaslab_t *), + KM_SLEEP); avl_create(&mg->mg_metaslab_tree, metaslab_compare, sizeof (metaslab_t), offsetof(struct metaslab, ms_group_node)); mg->mg_vd = vd; @@ -696,8 +736,17 @@ metaslab_group_create(metaslab_class_t *mc, vdev_t *vd mg->mg_activation_count = 0; mg->mg_initialized = B_FALSE; mg->mg_no_free_space = B_TRUE; - refcount_create_tracked(&mg->mg_alloc_queue_depth); + mg->mg_allocators = allocators; + mg->mg_alloc_queue_depth = kmem_zalloc(allocators * sizeof (refcount_t), + KM_SLEEP); + mg->mg_cur_max_alloc_queue_depth = kmem_zalloc(allocators * + sizeof (uint64_t), KM_SLEEP); + for (int i = 0; i < allocators; i++) { + refcount_create_tracked(&mg->mg_alloc_queue_depth[i]); + mg->mg_cur_max_alloc_queue_depth[i] = 0; + } + mg->mg_taskq = taskq_create("metaslab_group_taskq", metaslab_load_pct, minclsyspri, 10, INT_MAX, TASKQ_THREADS_CPU_PCT); @@ -718,8 +767,20 @@ metaslab_group_destroy(metaslab_group_t *mg) taskq_destroy(mg->mg_taskq); avl_destroy(&mg->mg_metaslab_tree); + kmem_free(mg->mg_primaries, mg->mg_allocators * sizeof (metaslab_t *)); + kmem_free(mg->mg_secondaries, mg->mg_allocators * + sizeof (metaslab_t *)); mutex_destroy(&mg->mg_lock); - refcount_destroy(&mg->mg_alloc_queue_depth); + + for (int i = 0; i < mg->mg_allocators; i++) { + refcount_destroy(&mg->mg_alloc_queue_depth[i]); + mg->mg_cur_max_alloc_queue_depth[i] = 0; + } + kmem_free(mg->mg_alloc_queue_depth, mg->mg_allocators * + sizeof (refcount_t)); + kmem_free(mg->mg_cur_max_alloc_queue_depth, mg->mg_allocators * + sizeof (uint64_t)); + kmem_free(mg, sizeof (metaslab_group_t)); } @@ -799,6 +860,22 @@ metaslab_group_passivate(metaslab_group_t *mg) taskq_wait(mg->mg_taskq); spa_config_enter(spa, locks & ~(SCL_ZIO - 1), spa, RW_WRITER); metaslab_group_alloc_update(mg); + for (int i = 0; i < mg->mg_allocators; i++) { + metaslab_t *msp = mg->mg_primaries[i]; + if (msp != NULL) { + mutex_enter(&msp->ms_lock); + metaslab_passivate(msp, + metaslab_weight_from_range_tree(msp)); + mutex_exit(&msp->ms_lock); + } + msp = mg->mg_secondaries[i]; + if (msp != NULL) { + mutex_enter(&msp->ms_lock); + metaslab_passivate(msp, + metaslab_weight_from_range_tree(msp)); + mutex_exit(&msp->ms_lock); + } + } mgprev = mg->mg_prev; mgnext = mg->mg_next; @@ -940,6 +1017,17 @@ metaslab_group_remove(metaslab_group_t *mg, metaslab_t } static void +metaslab_group_sort_impl(metaslab_group_t *mg, metaslab_t *msp, uint64_t weight) +{ + ASSERT(MUTEX_HELD(&mg->mg_lock)); + ASSERT(msp->ms_group == mg); + avl_remove(&mg->mg_metaslab_tree, msp); + msp->ms_weight = weight; + avl_add(&mg->mg_metaslab_tree, msp); + +} + +static void metaslab_group_sort(metaslab_group_t *mg, metaslab_t *msp, uint64_t weight) { /* @@ -950,10 +1038,7 @@ metaslab_group_sort(metaslab_group_t *mg, metaslab_t * ASSERT(MUTEX_HELD(&msp->ms_lock)); mutex_enter(&mg->mg_lock); - ASSERT(msp->ms_group == mg); - avl_remove(&mg->mg_metaslab_tree, msp); - msp->ms_weight = weight; - avl_add(&mg->mg_metaslab_tree, msp); + metaslab_group_sort_impl(mg, msp, weight); mutex_exit(&mg->mg_lock); } @@ -1001,7 +1086,7 @@ metaslab_group_fragmentation(metaslab_group_t *mg) */ static boolean_t metaslab_group_allocatable(metaslab_group_t *mg, metaslab_group_t *rotor, - uint64_t psize) + uint64_t psize, int allocator) { spa_t *spa = mg->mg_vd->vdev_spa; metaslab_class_t *mc = mg->mg_class; @@ -1030,7 +1115,7 @@ metaslab_group_allocatable(metaslab_group_t *mg, metas if (mg->mg_allocatable) { metaslab_group_t *mgp; int64_t qdepth; - uint64_t qmax = mg->mg_max_alloc_queue_depth; + uint64_t qmax = mg->mg_cur_max_alloc_queue_depth[allocator]; if (!mc->mc_alloc_throttle_enabled) return (B_TRUE); @@ -1042,7 +1127,7 @@ metaslab_group_allocatable(metaslab_group_t *mg, metas if (mg->mg_no_free_space) return (B_FALSE); - qdepth = refcount_count(&mg->mg_alloc_queue_depth); + qdepth = refcount_count(&mg->mg_alloc_queue_depth[allocator]); /* * If this metaslab group is below its qmax or it's @@ -1061,9 +1146,10 @@ metaslab_group_allocatable(metaslab_group_t *mg, metas * groups at the same time when we make this check. */ for (mgp = mg->mg_next; mgp != rotor; mgp = mgp->mg_next) { - qmax = mgp->mg_max_alloc_queue_depth; + qmax = mgp->mg_cur_max_alloc_queue_depth[allocator]; - qdepth = refcount_count(&mgp->mg_alloc_queue_depth); + qdepth = refcount_count( + &mgp->mg_alloc_queue_depth[allocator]); /* * If there is another metaslab group that @@ -1471,6 +1557,8 @@ metaslab_init(metaslab_group_t *mg, uint64_t id, uint6 ms->ms_id = id; ms->ms_start = id << vd->vdev_ms_shift; ms->ms_size = 1ULL << vd->vdev_ms_shift; + ms->ms_allocator = -1; + ms->ms_new = B_TRUE; /* * We only open space map objects that already exist. All others @@ -1567,6 +1655,7 @@ metaslab_fini(metaslab_t *msp) cv_destroy(&msp->ms_load_cv); mutex_destroy(&msp->ms_lock); mutex_destroy(&msp->ms_sync_lock); + ASSERT3U(msp->ms_allocator, ==, -1); kmem_free(msp, sizeof (metaslab_t)); } @@ -1963,19 +2052,59 @@ metaslab_weight(metaslab_t *msp) } static int -metaslab_activate(metaslab_t *msp, uint64_t activation_weight) +metaslab_activate_allocator(metaslab_group_t *mg, metaslab_t *msp, + int allocator, uint64_t activation_weight) { + /* + * If we're activating for the claim code, we don't want to actually + * set the metaslab up for a specific allocator. + */ + if (activation_weight == METASLAB_WEIGHT_CLAIM) + return (0); + metaslab_t **arr = (activation_weight == METASLAB_WEIGHT_PRIMARY ? + mg->mg_primaries : mg->mg_secondaries); + ASSERT(MUTEX_HELD(&msp->ms_lock)); + mutex_enter(&mg->mg_lock); + if (arr[allocator] != NULL) { + mutex_exit(&mg->mg_lock); + return (EEXIST); + } + arr[allocator] = msp; + ASSERT3S(msp->ms_allocator, ==, -1); + msp->ms_allocator = allocator; + msp->ms_primary = (activation_weight == METASLAB_WEIGHT_PRIMARY); + mutex_exit(&mg->mg_lock); + + return (0); +} + +static int +metaslab_activate(metaslab_t *msp, int allocator, uint64_t activation_weight) +{ + ASSERT(MUTEX_HELD(&msp->ms_lock)); + if ((msp->ms_weight & METASLAB_ACTIVE_MASK) == 0) { + int error = 0; metaslab_load_wait(msp); if (!msp->ms_loaded) { - int error = metaslab_load(msp); - if (error) { + if ((error = metaslab_load(msp)) != 0) { metaslab_group_sort(msp->ms_group, msp, 0); return (error); } } + if ((msp->ms_weight & METASLAB_ACTIVE_MASK) != 0) { + /* + * The metaslab was activated for another allocator + * while we were waiting, we should reselect. + */ + return (EBUSY); + } + if ((error = metaslab_activate_allocator(msp->ms_group, msp, + allocator, activation_weight)) != 0) { + return (error); + } msp->ms_activation_weight = msp->ms_weight; metaslab_group_sort(msp->ms_group, msp, @@ -1988,6 +2117,34 @@ metaslab_activate(metaslab_t *msp, uint64_t activation } static void +metaslab_passivate_allocator(metaslab_group_t *mg, metaslab_t *msp, + uint64_t weight) +{ + ASSERT(MUTEX_HELD(&msp->ms_lock)); + if (msp->ms_weight & METASLAB_WEIGHT_CLAIM) { + metaslab_group_sort(mg, msp, weight); + return; + } + + mutex_enter(&mg->mg_lock); + ASSERT3P(msp->ms_group, ==, mg); + if (msp->ms_primary) { + ASSERT3U(0, <=, msp->ms_allocator); + ASSERT3U(msp->ms_allocator, <, mg->mg_allocators); + ASSERT3P(mg->mg_primaries[msp->ms_allocator], ==, msp); + ASSERT(msp->ms_weight & METASLAB_WEIGHT_PRIMARY); + mg->mg_primaries[msp->ms_allocator] = NULL; + } else { + ASSERT(msp->ms_weight & METASLAB_WEIGHT_SECONDARY); + ASSERT3P(mg->mg_secondaries[msp->ms_allocator], ==, msp); + mg->mg_secondaries[msp->ms_allocator] = NULL; + } + msp->ms_allocator = -1; + metaslab_group_sort_impl(mg, msp, weight); + mutex_exit(&mg->mg_lock); +} + +static void metaslab_passivate(metaslab_t *msp, uint64_t weight) { uint64_t size = weight & ~METASLAB_WEIGHT_TYPE; @@ -2002,7 +2159,7 @@ metaslab_passivate(metaslab_t *msp, uint64_t weight) ASSERT0(weight & METASLAB_ACTIVE_MASK); msp->ms_activation_weight = 0; - metaslab_group_sort(msp->ms_group, msp, weight); + metaslab_passivate_allocator(msp->ms_group, msp, weight); ASSERT((msp->ms_weight & METASLAB_ACTIVE_MASK) == 0); } @@ -2556,11 +2713,18 @@ metaslab_sync_done(metaslab_t *msp, uint64_t txg) vdev_dirty(vd, VDD_METASLAB, msp, txg + 1); } + if (msp->ms_new) { + msp->ms_new = B_FALSE; + mutex_enter(&mg->mg_lock); + mg->mg_ms_ready++; + mutex_exit(&mg->mg_lock); + } /* * Calculate the new weights before unloading any metaslabs. * This will give us the most accurate weighting. */ - metaslab_group_sort(mg, msp, metaslab_weight(msp)); + metaslab_group_sort(mg, msp, metaslab_weight(msp) | + (msp->ms_weight & METASLAB_ACTIVE_MASK)); /* * If the metaslab is loaded and we've not tried to load or allocate @@ -2572,6 +2736,10 @@ metaslab_sync_done(metaslab_t *msp, uint64_t txg) VERIFY0(range_tree_space( msp->ms_allocating[(txg + t) & TXG_MASK])); } + if (msp->ms_allocator != -1) { + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_ACTIVE_MASK); + } if (!metaslab_debug_unload) metaslab_unload(msp); @@ -2665,7 +2833,8 @@ metaslab_alloc_trace_fini(void) */ static void metaslab_trace_add(zio_alloc_list_t *zal, metaslab_group_t *mg, - metaslab_t *msp, uint64_t psize, uint32_t dva_id, uint64_t offset) + metaslab_t *msp, uint64_t psize, uint32_t dva_id, uint64_t offset, + int allocator) { if (!metaslab_trace_enabled) return; @@ -2698,6 +2867,7 @@ metaslab_trace_add(zio_alloc_list_t *zal, metaslab_gro mat->mat_dva_id = dva_id; mat->mat_offset = offset; mat->mat_weight = 0; + mat->mat_allocator = allocator; if (msp != NULL) mat->mat_weight = msp->ms_weight; @@ -2738,35 +2908,56 @@ metaslab_trace_fini(zio_alloc_list_t *zal) */ static void -metaslab_group_alloc_increment(spa_t *spa, uint64_t vdev, void *tag, int flags) +metaslab_group_alloc_increment(spa_t *spa, uint64_t vdev, void *tag, int flags, + int allocator) { if (!(flags & METASLAB_ASYNC_ALLOC) || - flags & METASLAB_DONT_THROTTLE) + (flags & METASLAB_DONT_THROTTLE)) return; metaslab_group_t *mg = vdev_lookup_top(spa, vdev)->vdev_mg; if (!mg->mg_class->mc_alloc_throttle_enabled) return; - (void) refcount_add(&mg->mg_alloc_queue_depth, tag); + (void) refcount_add(&mg->mg_alloc_queue_depth[allocator], tag); } +static void +metaslab_group_increment_qdepth(metaslab_group_t *mg, int allocator) +{ + uint64_t max = mg->mg_max_alloc_queue_depth; + uint64_t cur = mg->mg_cur_max_alloc_queue_depth[allocator]; + while (cur < max) { + if (atomic_cas_64(&mg->mg_cur_max_alloc_queue_depth[allocator], + cur, cur + 1) == cur) { + atomic_inc_64( + &mg->mg_class->mc_alloc_max_slots[allocator]); + return; + } + cur = mg->mg_cur_max_alloc_queue_depth[allocator]; + } +} + void -metaslab_group_alloc_decrement(spa_t *spa, uint64_t vdev, void *tag, int flags) +metaslab_group_alloc_decrement(spa_t *spa, uint64_t vdev, void *tag, int flags, + int allocator, boolean_t io_complete) { if (!(flags & METASLAB_ASYNC_ALLOC) || - flags & METASLAB_DONT_THROTTLE) + (flags & METASLAB_DONT_THROTTLE)) return; metaslab_group_t *mg = vdev_lookup_top(spa, vdev)->vdev_mg; if (!mg->mg_class->mc_alloc_throttle_enabled) return; - (void) refcount_remove(&mg->mg_alloc_queue_depth, tag); + (void) refcount_remove(&mg->mg_alloc_queue_depth[allocator], tag); + if (io_complete) + metaslab_group_increment_qdepth(mg, allocator); } void -metaslab_group_alloc_verify(spa_t *spa, const blkptr_t *bp, void *tag) +metaslab_group_alloc_verify(spa_t *spa, const blkptr_t *bp, void *tag, + int allocator) { #ifdef ZFS_DEBUG const dva_t *dva = bp->blk_dva; @@ -2775,7 +2966,8 @@ metaslab_group_alloc_verify(spa_t *spa, const blkptr_t for (int d = 0; d < ndvas; d++) { uint64_t vdev = DVA_GET_VDEV(&dva[d]); metaslab_group_t *mg = vdev_lookup_top(spa, vdev)->vdev_mg; - VERIFY(refcount_not_held(&mg->mg_alloc_queue_depth, tag)); + VERIFY(refcount_not_held(&mg->mg_alloc_queue_depth[allocator], + tag)); } #endif } @@ -2817,91 +3009,146 @@ metaslab_block_alloc(metaslab_t *msp, uint64_t size, u return (start); } +/* + * Find the metaslab with the highest weight that is less than what we've + * already tried. In the common case, this means that we will examine each + * metaslab at most once. Note that concurrent callers could reorder metaslabs + * by activation/passivation once we have dropped the mg_lock. If a metaslab is + * activated by another thread, and we fail to allocate from the metaslab we + * have selected, we may not try the newly-activated metaslab, and instead + * activate another metaslab. This is not optimal, but generally does not cause + * any problems (a possible exception being if every metaslab is completely full + * except for the the newly-activated metaslab which we fail to examine). + */ +static metaslab_t * +find_valid_metaslab(metaslab_group_t *mg, uint64_t activation_weight, + dva_t *dva, int d, uint64_t min_distance, uint64_t asize, int allocator, + zio_alloc_list_t *zal, metaslab_t *search, boolean_t *was_active) +{ + avl_index_t idx; + avl_tree_t *t = &mg->mg_metaslab_tree; + metaslab_t *msp = avl_find(t, search, &idx); + if (msp == NULL) + msp = avl_nearest(t, idx, AVL_AFTER); + + for (; msp != NULL; msp = AVL_NEXT(t, msp)) { + int i; + if (!metaslab_should_allocate(msp, asize)) { + metaslab_trace_add(zal, mg, msp, asize, d, + TRACE_TOO_SMALL, allocator); + continue; + } + + /* + * If the selected metaslab is condensing, skip it. + */ + if (msp->ms_condensing) + continue; + + *was_active = msp->ms_allocator != -1; + /* + * If we're activating as primary, this is our first allocation + * from this disk, so we don't need to check how close we are. + * If the metaslab under consideration was already active, + * we're getting desperate enough to steal another allocator's + * metaslab, so we still don't care about distances. + */ + if (activation_weight == METASLAB_WEIGHT_PRIMARY || *was_active) + break; + + uint64_t target_distance = min_distance + + (space_map_allocated(msp->ms_sm) != 0 ? 0 : + min_distance >> 1); + + for (i = 0; i < d; i++) { + if (metaslab_distance(msp, &dva[i]) < target_distance) + break; + } + if (i == d) + break; + } + + if (msp != NULL) { + search->ms_weight = msp->ms_weight; + search->ms_start = msp->ms_start + 1; + search->ms_allocator = msp->ms_allocator; + search->ms_primary = msp->ms_primary; + } + return (msp); +} + +/* ARGSUSED */ static uint64_t metaslab_group_alloc_normal(metaslab_group_t *mg, zio_alloc_list_t *zal, - uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d) + uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d, + int allocator) { metaslab_t *msp = NULL; uint64_t offset = -1ULL; uint64_t activation_weight; - uint64_t target_distance; - int i; + boolean_t tertiary = B_FALSE; activation_weight = METASLAB_WEIGHT_PRIMARY; - for (i = 0; i < d; i++) { - if (DVA_GET_VDEV(&dva[i]) == mg->mg_vd->vdev_id) { + for (int i = 0; i < d; i++) { + if (activation_weight == METASLAB_WEIGHT_PRIMARY && + DVA_GET_VDEV(&dva[i]) == mg->mg_vd->vdev_id) { activation_weight = METASLAB_WEIGHT_SECONDARY; + } else if (activation_weight == METASLAB_WEIGHT_SECONDARY && + DVA_GET_VDEV(&dva[i]) == mg->mg_vd->vdev_id) { + tertiary = B_TRUE; break; } } + /* + * If we don't have enough metaslabs active to fill the entire array, we + * just use the 0th slot. + */ + if (mg->mg_ms_ready < mg->mg_allocators * 2) { + tertiary = B_FALSE; + allocator = 0; + } + + ASSERT3U(mg->mg_vd->vdev_ms_count, >=, 2); + metaslab_t *search = kmem_alloc(sizeof (*search), KM_SLEEP); search->ms_weight = UINT64_MAX; search->ms_start = 0; + /* + * At the end of the metaslab tree are the already-active metaslabs, + * first the primaries, then the secondaries. When we resume searching + * through the tree, we need to consider ms_allocator and ms_primary so + * we start in the location right after where we left off, and don't + * accidentally loop forever considering the same metaslabs. + */ + search->ms_allocator = -1; + search->ms_primary = B_TRUE; for (;;) { - boolean_t was_active; - avl_tree_t *t = &mg->mg_metaslab_tree; - avl_index_t idx; + boolean_t was_active = B_FALSE; mutex_enter(&mg->mg_lock); - /* - * Find the metaslab with the highest weight that is less - * than what we've already tried. In the common case, this - * means that we will examine each metaslab at most once. - * Note that concurrent callers could reorder metaslabs - * by activation/passivation once we have dropped the mg_lock. - * If a metaslab is activated by another thread, and we fail - * to allocate from the metaslab we have selected, we may - * not try the newly-activated metaslab, and instead activate - * another metaslab. This is not optimal, but generally - * does not cause any problems (a possible exception being - * if every metaslab is completely full except for the - * the newly-activated metaslab which we fail to examine). - */ - msp = avl_find(t, search, &idx); - if (msp == NULL) - msp = avl_nearest(t, idx, AVL_AFTER); - for (; msp != NULL; msp = AVL_NEXT(t, msp)) { - - if (!metaslab_should_allocate(msp, asize)) { - metaslab_trace_add(zal, mg, msp, asize, d, - TRACE_TOO_SMALL); - continue; - } - - /* - * If the selected metaslab is condensing, skip it. - */ - if (msp->ms_condensing) - continue; - - was_active = msp->ms_weight & METASLAB_ACTIVE_MASK; - if (activation_weight == METASLAB_WEIGHT_PRIMARY) - break; - - target_distance = min_distance + - (space_map_allocated(msp->ms_sm) != 0 ? 0 : - min_distance >> 1); - - for (i = 0; i < d; i++) { - if (metaslab_distance(msp, &dva[i]) < - target_distance) - break; - } - if (i == d) - break; + if (activation_weight == METASLAB_WEIGHT_PRIMARY && + mg->mg_primaries[allocator] != NULL) { + msp = mg->mg_primaries[allocator]; + was_active = B_TRUE; + } else if (activation_weight == METASLAB_WEIGHT_SECONDARY && + mg->mg_secondaries[allocator] != NULL && !tertiary) { + msp = mg->mg_secondaries[allocator]; + was_active = B_TRUE; + } else { + msp = find_valid_metaslab(mg, activation_weight, dva, d, + min_distance, asize, allocator, zal, search, + &was_active); } + mutex_exit(&mg->mg_lock); if (msp == NULL) { kmem_free(search, sizeof (*search)); return (-1ULL); } - search->ms_weight = msp->ms_weight; - search->ms_start = msp->ms_start + 1; mutex_enter(&msp->ms_lock); - /* * Ensure that the metaslab we have selected is still * capable of handling our request. It's possible that @@ -2915,18 +3162,32 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ continue; } - if ((msp->ms_weight & METASLAB_WEIGHT_SECONDARY) && - activation_weight == METASLAB_WEIGHT_PRIMARY) { - metaslab_passivate(msp, - msp->ms_weight & ~METASLAB_ACTIVE_MASK); + /* + * If the metaslab is freshly activated for an allocator that + * isn't the one we're allocating from, or if it's a primary and + * we're seeking a secondary (or vice versa), we go back and + * select a new metaslab. + */ + if (!was_active && (msp->ms_weight & METASLAB_ACTIVE_MASK) && + (msp->ms_allocator != -1) && + (msp->ms_allocator != allocator || ((activation_weight == + METASLAB_WEIGHT_PRIMARY) != msp->ms_primary))) { mutex_exit(&msp->ms_lock); continue; } - if (metaslab_activate(msp, activation_weight) != 0) { + if (msp->ms_weight & METASLAB_WEIGHT_CLAIM) { + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_WEIGHT_CLAIM); mutex_exit(&msp->ms_lock); continue; } + + if (metaslab_activate(msp, allocator, activation_weight) != 0) { + mutex_exit(&msp->ms_lock); + continue; + } + msp->ms_selected_txg = txg; /* @@ -2939,7 +3200,7 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ if (!metaslab_should_allocate(msp, asize)) { /* Passivate this metaslab and select a new one. */ metaslab_trace_add(zal, mg, msp, asize, d, - TRACE_TOO_SMALL); + TRACE_TOO_SMALL, allocator); goto next; } @@ -2950,13 +3211,15 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ */ if (msp->ms_condensing) { metaslab_trace_add(zal, mg, msp, asize, d, - TRACE_CONDENSING); + TRACE_CONDENSING, allocator); + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_ACTIVE_MASK); mutex_exit(&msp->ms_lock); continue; } offset = metaslab_block_alloc(msp, asize, txg); - metaslab_trace_add(zal, mg, msp, asize, d, offset); + metaslab_trace_add(zal, mg, msp, asize, d, offset, allocator); if (offset != -1ULL) { /* Proactively passivate the metaslab, if needed */ @@ -3012,19 +3275,20 @@ next: static uint64_t metaslab_group_alloc(metaslab_group_t *mg, zio_alloc_list_t *zal, - uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d) + uint64_t asize, uint64_t txg, uint64_t min_distance, dva_t *dva, int d, + int allocator) { uint64_t offset; ASSERT(mg->mg_initialized); offset = metaslab_group_alloc_normal(mg, zal, asize, txg, - min_distance, dva, d); + min_distance, dva, d, allocator); mutex_enter(&mg->mg_lock); if (offset == -1ULL) { mg->mg_failed_allocations++; metaslab_trace_add(zal, mg, NULL, asize, d, - TRACE_GROUP_FAILURE); + TRACE_GROUP_FAILURE, allocator); if (asize == SPA_GANGBLOCKSIZE) { /* * This metaslab group was unable to allocate @@ -3059,7 +3323,7 @@ int ditto_same_vdev_distance_shift = 3; int metaslab_alloc_dva(spa_t *spa, metaslab_class_t *mc, uint64_t psize, dva_t *dva, int d, dva_t *hintdva, uint64_t txg, int flags, - zio_alloc_list_t *zal) + zio_alloc_list_t *zal, int allocator) { metaslab_group_t *mg, *rotor; vdev_t *vd; @@ -3071,7 +3335,8 @@ metaslab_alloc_dva(spa_t *spa, metaslab_class_t *mc, u * For testing, make some blocks above a certain size be gang blocks. */ if (psize >= metaslab_force_ganging && (ddi_get_lbolt() & 3) == 0) { - metaslab_trace_add(zal, NULL, NULL, psize, d, TRACE_FORCE_GANG); + metaslab_trace_add(zal, NULL, NULL, psize, d, TRACE_FORCE_GANG, + allocator); return (SET_ERROR(ENOSPC)); } @@ -3157,12 +3422,12 @@ top: */ if (allocatable && !GANG_ALLOCATION(flags) && !try_hard) { allocatable = metaslab_group_allocatable(mg, rotor, - psize); + psize, allocator); } if (!allocatable) { metaslab_trace_add(zal, mg, NULL, psize, d, - TRACE_NOT_ALLOCATABLE); + TRACE_NOT_ALLOCATABLE, allocator); goto next; } @@ -3177,7 +3442,7 @@ top: vd->vdev_state < VDEV_STATE_HEALTHY) && d == 0 && !try_hard && vd->vdev_children == 0) { metaslab_trace_add(zal, mg, NULL, psize, d, - TRACE_VDEV_ERROR); + TRACE_VDEV_ERROR, allocator); goto next; } @@ -3201,7 +3466,7 @@ top: ASSERT(P2PHASE(asize, 1ULL << vd->vdev_ashift) == 0); uint64_t offset = metaslab_group_alloc(mg, zal, asize, txg, - distance, dva, d); + distance, dva, d, allocator); if (offset != -1ULL) { /* @@ -3264,7 +3529,7 @@ next: bzero(&dva[d], sizeof (dva_t)); - metaslab_trace_add(zal, rotor, NULL, psize, d, TRACE_ENOSPC); + metaslab_trace_add(zal, rotor, NULL, psize, d, TRACE_ENOSPC, allocator); return (SET_ERROR(ENOSPC)); } @@ -3565,18 +3830,20 @@ metaslab_free_dva(spa_t *spa, const dva_t *dva, boolea * the reservation. */ boolean_t -metaslab_class_throttle_reserve(metaslab_class_t *mc, int slots, zio_t *zio, - int flags) +metaslab_class_throttle_reserve(metaslab_class_t *mc, int slots, int allocator, + zio_t *zio, int flags) { uint64_t available_slots = 0; boolean_t slot_reserved = B_FALSE; + uint64_t max = mc->mc_alloc_max_slots[allocator]; ASSERT(mc->mc_alloc_throttle_enabled); mutex_enter(&mc->mc_lock); - uint64_t reserved_slots = refcount_count(&mc->mc_alloc_slots); - if (reserved_slots < mc->mc_alloc_max_slots) - available_slots = mc->mc_alloc_max_slots - reserved_slots; + uint64_t reserved_slots = + refcount_count(&mc->mc_alloc_slots[allocator]); + if (reserved_slots < max) + available_slots = max - reserved_slots; if (slots <= available_slots || GANG_ALLOCATION(flags)) { /* @@ -3584,7 +3851,9 @@ metaslab_class_throttle_reserve(metaslab_class_t *mc, * them individually when an I/O completes. */ for (int d = 0; d < slots; d++) { - reserved_slots = refcount_add(&mc->mc_alloc_slots, zio); + reserved_slots = + refcount_add(&mc->mc_alloc_slots[allocator], + zio); } zio->io_flags |= ZIO_FLAG_IO_ALLOCATING; slot_reserved = B_TRUE; @@ -3595,12 +3864,14 @@ metaslab_class_throttle_reserve(metaslab_class_t *mc, } void -metaslab_class_throttle_unreserve(metaslab_class_t *mc, int slots, zio_t *zio) +metaslab_class_throttle_unreserve(metaslab_class_t *mc, int slots, + int allocator, zio_t *zio) { ASSERT(mc->mc_alloc_throttle_enabled); mutex_enter(&mc->mc_lock); for (int d = 0; d < slots; d++) { - (void) refcount_remove(&mc->mc_alloc_slots, zio); + (void) refcount_remove(&mc->mc_alloc_slots[allocator], + zio); } mutex_exit(&mc->mc_lock); } @@ -3622,7 +3893,13 @@ metaslab_claim_concrete(vdev_t *vd, uint64_t offset, u mutex_enter(&msp->ms_lock); if ((txg != 0 && spa_writeable(spa)) || !msp->ms_loaded) - error = metaslab_activate(msp, METASLAB_WEIGHT_SECONDARY); + error = metaslab_activate(msp, 0, METASLAB_WEIGHT_CLAIM); + /* + * No need to fail in that case; someone else has activated the + * metaslab, but that doesn't preclude us from using it. + */ + if (error == EBUSY) + error = 0; if (error == 0 && !range_tree_contains(msp->ms_allocatable, offset, size)) @@ -3727,7 +4004,7 @@ metaslab_claim_dva(spa_t *spa, const dva_t *dva, uint6 int metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint64_t psize, blkptr_t *bp, int ndvas, uint64_t txg, blkptr_t *hintbp, int flags, - zio_alloc_list_t *zal, zio_t *zio) + zio_alloc_list_t *zal, zio_t *zio, int allocator) { dva_t *dva = bp->blk_dva; dva_t *hintdva = hintbp->blk_dva; @@ -3750,12 +4027,13 @@ metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint6 for (int d = 0; d < ndvas; d++) { error = metaslab_alloc_dva(spa, mc, psize, dva, d, hintdva, - txg, flags, zal); + txg, flags, zal, allocator); if (error != 0) { for (d--; d >= 0; d--) { metaslab_unalloc_dva(spa, &dva[d], txg); metaslab_group_alloc_decrement(spa, - DVA_GET_VDEV(&dva[d]), zio, flags); + DVA_GET_VDEV(&dva[d]), zio, flags, + allocator, B_FALSE); bzero(&dva[d], sizeof (dva_t)); } spa_config_exit(spa, SCL_ALLOC, FTAG); @@ -3766,7 +4044,7 @@ metaslab_alloc(spa_t *spa, metaslab_class_t *mc, uint6 * based on the newly allocated dva. */ metaslab_group_alloc_increment(spa, - DVA_GET_VDEV(&dva[d]), zio, flags); + DVA_GET_VDEV(&dva[d]), zio, flags, allocator); } } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Mon Jul 30 23:53:25 2018 (r336948) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Tue Jul 31 00:02:42 2018 (r336949) @@ -7776,9 +7776,11 @@ spa_sync(spa_t *spa, uint64_t txg) spa->spa_syncing_txg = txg; spa->spa_sync_pass = 0; - mutex_enter(&spa->spa_alloc_lock); - VERIFY0(avl_numnodes(&spa->spa_alloc_tree)); - mutex_exit(&spa->spa_alloc_lock); + for (int i = 0; i < spa->spa_alloc_count; i++) { + mutex_enter(&spa->spa_alloc_locks[i]); + VERIFY0(avl_numnodes(&spa->spa_alloc_trees[i])); + mutex_exit(&spa->spa_alloc_locks[i]); + } /* * If there are any pending vdev state changes, convert them @@ -7844,7 +7846,7 @@ spa_sync(spa_t *spa, uint64_t txg) * The max queue depth will not change in the middle of syncing * out this txg. */ - uint64_t queue_depth_total = 0; + uint64_t slots_per_allocator = 0; for (int c = 0; c < rvd->vdev_children; c++) { vdev_t *tvd = rvd->vdev_child[c]; metaslab_group_t *mg = tvd->vdev_mg; @@ -7858,18 +7860,23 @@ spa_sync(spa_t *spa, uint64_t txg) * allocations look at mg_max_alloc_queue_depth, and async * allocations all happen from spa_sync(). */ - ASSERT0(refcount_count(&mg->mg_alloc_queue_depth)); + for (int i = 0; i < spa->spa_alloc_count; i++) + ASSERT0(refcount_count(&(mg->mg_alloc_queue_depth[i]))); mg->mg_max_alloc_queue_depth = max_queue_depth; - queue_depth_total += mg->mg_max_alloc_queue_depth; + + for (int i = 0; i < spa->spa_alloc_count; i++) { + mg->mg_cur_max_alloc_queue_depth[i] = + zfs_vdev_def_queue_depth; + } + slots_per_allocator += zfs_vdev_def_queue_depth; } metaslab_class_t *mc = spa_normal_class(spa); - ASSERT0(refcount_count(&mc->mc_alloc_slots)); - mc->mc_alloc_max_slots = queue_depth_total; + for (int i = 0; i < spa->spa_alloc_count; i++) { + ASSERT0(refcount_count(&mc->mc_alloc_slots[i])); + mc->mc_alloc_max_slots[i] = slots_per_allocator; + } mc->mc_alloc_throttle_enabled = zio_dva_throttle_enabled; - ASSERT3U(mc->mc_alloc_max_slots, <=, - max_queue_depth * rvd->vdev_children); - for (int c = 0; c < rvd->vdev_children; c++) { vdev_t *vd = rvd->vdev_child[c]; vdev_indirect_state_sync_verify(vd); @@ -8052,9 +8059,11 @@ spa_sync(spa_t *spa, uint64_t txg) dsl_pool_sync_done(dp, txg); - mutex_enter(&spa->spa_alloc_lock); - VERIFY0(avl_numnodes(&spa->spa_alloc_tree)); - mutex_exit(&spa->spa_alloc_lock); + for (int i = 0; i < spa->spa_alloc_count; i++) { *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Tue Jul 31 00:13:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D9471065B79; Tue, 31 Jul 2018 00:13:07 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 201E88E4AA; Tue, 31 Jul 2018 00:13:07 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F0B0A4058; Tue, 31 Jul 2018 00:13:06 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0D6fB088739; Tue, 31 Jul 2018 00:13:06 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0D48W088729; Tue, 31 Jul 2018 00:13:04 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310013.w6V0D48W088729@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:13:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336950 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/ztest vendor/illumos/dist/lib/libzf... X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/ztest vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 336950 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:13:07 -0000 Author: mav Date: Tue Jul 31 00:13:04 2018 New Revision: 336950 URL: https://svnweb.freebsd.org/changeset/base/336950 Log: 9290 device removal reduces redundancy of mirrors Mirrors are supposed to provide redundancy in the face of whole-disk failure and silent damage (e.g. some data on disk is not right, but ZFS hasn't detected the whole device as being broken). However, the current device removal implementation bypasses some of the mirror's redundancy. illumos/illumos-gate@3a4b1be953ee5601bab748afa07c26ed4996cde6 Reviewed by: George Wilson Reviewed by: Prashanth Sreenivasa Reviewed by: Sara Hartse Reviewed by: Serapheim Dimitropoulos Reviewed by: Brian Behlendorf Reviewed by: Tim Chase Approved by: Richard Lowe Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 00:13:04 2018 (r336950) @@ -1988,7 +1988,16 @@ dsl_scan_scrub_cb(dsl_pool_t *dp, /* if it's a resilver, this may not be in the target range */ if (!needs_io) { - if (DVA_GET_GANG(&bp->blk_dva[d])) { + if (vd->vdev_ops == &vdev_indirect_ops) { + /* + * The indirect vdev can point to multiple + * vdevs. For simplicity, always create + * the resilver zio_t. zio_vdev_io_start() + * will bypass the child resilver i/o's if + * they are on vdevs that don't have DTL's. + */ + needs_io = B_TRUE; + } else if (DVA_GET_GANG(&bp->blk_dva[d])) { /* * Gang members may be spread across multiple * vdevs, so the best estimate we have is the Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:13:04 2018 (r336950) @@ -3582,7 +3582,7 @@ metaslab_free_impl(vdev_t *vd, uint64_t offset, uint64 return; if (spa->spa_vdev_removal != NULL && - spa->spa_vdev_removal->svr_vdev == vd && + spa->spa_vdev_removal->svr_vdev_id == vd->vdev_id && vdev_is_concrete(vd)) { /* * Note: we check if the vdev is concrete because when Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Tue Jul 31 00:13:04 2018 (r336950) @@ -5509,8 +5509,7 @@ spa_vdev_add(spa_t *spa, nvlist_t *nvroot) for (int c = 0; c < vd->vdev_children; c++) { tvd = vd->vdev_child[c]; if (spa->spa_vdev_removal != NULL && - tvd->vdev_ashift != - spa->spa_vdev_removal->svr_vdev->vdev_ashift) { + tvd->vdev_ashift != spa->spa_max_ashift) { return (spa_vdev_exit(spa, vd, txg, EINVAL)); } /* Fail if top level vdev is raidz */ @@ -5626,10 +5625,8 @@ spa_vdev_attach(spa_t *spa, uint64_t guid, nvlist_t *n return (spa_vdev_exit(spa, NULL, txg, error)); } - if (spa->spa_vdev_removal != NULL || - spa->spa_removing_phys.sr_prev_indirect_vdev != -1) { + if (spa->spa_vdev_removal != NULL) return (spa_vdev_exit(spa, NULL, txg, EBUSY)); - } if (oldvd == NULL) return (spa_vdev_exit(spa, NULL, txg, ENODEV)); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:13:04 2018 (r336950) @@ -1761,9 +1761,12 @@ spa_update_dspace(spa_t *spa) * allocated twice (on the old device and the new * device). */ - vdev_t *vd = spa->spa_vdev_removal->svr_vdev; + spa_config_enter(spa, SCL_VDEV, FTAG, RW_READER); + vdev_t *vd = + vdev_lookup_top(spa, spa->spa_vdev_removal->svr_vdev_id); spa->spa_dspace -= spa_deflate(spa) ? vd->vdev_stat.vs_dspace : vd->vdev_stat.vs_space; + spa_config_exit(spa, SCL_VDEV, FTAG); } } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h Tue Jul 31 00:13:04 2018 (r336950) @@ -30,7 +30,7 @@ extern "C" { #endif typedef struct spa_vdev_removal { - vdev_t *svr_vdev; + uint64_t svr_vdev_id; uint64_t svr_max_offset_to_sync[TXG_SIZE]; /* Thread performing a vdev removal. */ kthread_t *svr_thread; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h Tue Jul 31 00:13:04 2018 (r336950) @@ -550,7 +550,7 @@ extern zio_t *zio_vdev_child_io(zio_t *zio, blkptr_t * zio_done_func_t *done, void *private); extern zio_t *zio_vdev_delegated_io(vdev_t *vd, uint64_t offset, - struct abd *data, uint64_t size, int type, zio_priority_t priority, + struct abd *data, uint64_t size, zio_type_t type, zio_priority_t priority, enum zio_flag flags, zio_done_func_t *done, void *private); extern void zio_vdev_io_bypass(zio_t *zio); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Tue Jul 31 00:13:04 2018 (r336950) @@ -867,6 +867,32 @@ vdev_top_transfer(vdev_t *svd, vdev_t *tvd) svd->vdev_stat.vs_space = 0; svd->vdev_stat.vs_dspace = 0; + /* + * State which may be set on a top-level vdev that's in the + * process of being removed. + */ + ASSERT0(tvd->vdev_indirect_config.vic_births_object); + ASSERT0(tvd->vdev_indirect_config.vic_mapping_object); + ASSERT3U(tvd->vdev_indirect_config.vic_prev_indirect_vdev, ==, -1ULL); + ASSERT3P(tvd->vdev_indirect_mapping, ==, NULL); + ASSERT3P(tvd->vdev_indirect_births, ==, NULL); + ASSERT3P(tvd->vdev_obsolete_sm, ==, NULL); + ASSERT0(tvd->vdev_removing); + tvd->vdev_removing = svd->vdev_removing; + tvd->vdev_indirect_config = svd->vdev_indirect_config; + tvd->vdev_indirect_mapping = svd->vdev_indirect_mapping; + tvd->vdev_indirect_births = svd->vdev_indirect_births; + range_tree_swap(&svd->vdev_obsolete_segments, + &tvd->vdev_obsolete_segments); + tvd->vdev_obsolete_sm = svd->vdev_obsolete_sm; + svd->vdev_indirect_config.vic_mapping_object = 0; + svd->vdev_indirect_config.vic_births_object = 0; + svd->vdev_indirect_config.vic_prev_indirect_vdev = -1ULL; + svd->vdev_indirect_mapping = NULL; + svd->vdev_indirect_births = NULL; + svd->vdev_obsolete_sm = NULL; + svd->vdev_removing = 0; + for (t = 0; t < TXG_SIZE; t++) { while ((msp = txg_list_remove(&svd->vdev_ms_list, t)) != NULL) (void) txg_list_add(&tvd->vdev_ms_list, msp, t); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c Tue Jul 31 00:13:04 2018 (r336950) @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -46,10 +47,11 @@ * "vdev_remap" operation that executes a callback on each contiguous * segment of the new location. This function is used in multiple ways: * - * - reads and repair writes to this device use the callback to create - * a child io for each mapped segment. + * - i/os to this vdev use the callback to determine where the + * data is now located, and issue child i/os for each segment's new + * location. * - * - frees and claims to this device use the callback to free or claim + * - frees and claims to this vdev use the callback to free or claim * each mapped segment. (Note that we don't actually need to claim * log blocks on indirect vdevs, because we don't allocate to * removing vdevs. However, zdb uses zio_claim() for its leak @@ -204,6 +206,94 @@ uint64_t zfs_condense_min_mapping_bytes = 128 * 1024; int zfs_condense_indirect_commit_entry_delay_ticks = 0; /* + * If a split block contains more than this many segments, consider it too + * computationally expensive to check all (2^num_segments) possible + * combinations. Instead, try at most 2^_segments_max randomly-selected + * combinations. + * + * This is reasonable if only a few segment copies are damaged and the + * majority of segment copies are good. This allows all the segment copies to + * participate fairly in the reconstruction and prevents the repeated use of + * one bad copy. + */ +int zfs_reconstruct_indirect_segments_max = 10; + +/* + * The indirect_child_t represents the vdev that we will read from, when we + * need to read all copies of the data (e.g. for scrub or reconstruction). + * For plain (non-mirror) top-level vdevs (i.e. is_vdev is not a mirror), + * ic_vdev is the same as is_vdev. However, for mirror top-level vdevs, + * ic_vdev is a child of the mirror. + */ +typedef struct indirect_child { + abd_t *ic_data; + vdev_t *ic_vdev; +} indirect_child_t; + +/* + * The indirect_split_t represents one mapped segment of an i/o to the + * indirect vdev. For non-split (contiguously-mapped) blocks, there will be + * only one indirect_split_t, with is_split_offset==0 and is_size==io_size. + * For split blocks, there will be several of these. + */ +typedef struct indirect_split { + list_node_t is_node; /* link on iv_splits */ + + /* + * is_split_offset is the offset into the i/o. + * This is the sum of the previous splits' is_size's. + */ + uint64_t is_split_offset; + + vdev_t *is_vdev; /* top-level vdev */ + uint64_t is_target_offset; /* offset on is_vdev */ + uint64_t is_size; + int is_children; /* number of entries in is_child[] */ + + /* + * is_good_child is the child that we are currently using to + * attempt reconstruction. + */ + int is_good_child; + + indirect_child_t is_child[1]; /* variable-length */ +} indirect_split_t; + +/* + * The indirect_vsd_t is associated with each i/o to the indirect vdev. + * It is the "Vdev-Specific Data" in the zio_t's io_vsd. + */ +typedef struct indirect_vsd { + boolean_t iv_split_block; + boolean_t iv_reconstruct; + + list_t iv_splits; /* list of indirect_split_t's */ +} indirect_vsd_t; + +static void +vdev_indirect_map_free(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + indirect_split_t *is; + while ((is = list_head(&iv->iv_splits)) != NULL) { + for (int c = 0; c < is->is_children; c++) { + indirect_child_t *ic = &is->is_child[c]; + if (ic->ic_data != NULL) + abd_free(ic->ic_data); + } + list_remove(&iv->iv_splits, is); + kmem_free(is, + offsetof(indirect_split_t, is_child[is->is_children])); + } + kmem_free(iv, sizeof (*iv)); +} + +static const zio_vsd_ops_t vdev_indirect_vsd_ops = { + vdev_indirect_map_free, + zio_vsd_default_cksum_report +}; +/* * Mark the given offset and size as being obsolete. */ void @@ -817,12 +907,6 @@ vdev_indirect_close(vdev_t *vd) } /* ARGSUSED */ -static void -vdev_indirect_io_done(zio_t *zio) -{ -} - -/* ARGSUSED */ static int vdev_indirect_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, uint64_t *ashift) @@ -1065,39 +1149,473 @@ vdev_indirect_child_io_done(zio_t *zio) abd_put(zio->io_abd); } +/* + * This is a callback for vdev_indirect_remap() which allocates an + * indirect_split_t for each split segment and adds it to iv_splits. + */ static void -vdev_indirect_io_start_cb(uint64_t split_offset, vdev_t *vd, uint64_t offset, +vdev_indirect_gather_splits(uint64_t split_offset, vdev_t *vd, uint64_t offset, uint64_t size, void *arg) { zio_t *zio = arg; + indirect_vsd_t *iv = zio->io_vsd; ASSERT3P(vd, !=, NULL); if (vd->vdev_ops == &vdev_indirect_ops) return; - zio_nowait(zio_vdev_child_io(zio, NULL, vd, offset, - abd_get_offset(zio->io_abd, split_offset), - size, zio->io_type, zio->io_priority, - 0, vdev_indirect_child_io_done, zio)); + int n = 1; + if (vd->vdev_ops == &vdev_mirror_ops) + n = vd->vdev_children; + + indirect_split_t *is = + kmem_zalloc(offsetof(indirect_split_t, is_child[n]), KM_SLEEP); + + is->is_children = n; + is->is_size = size; + is->is_split_offset = split_offset; + is->is_target_offset = offset; + is->is_vdev = vd; + + /* + * Note that we only consider multiple copies of the data for + * *mirror* vdevs. We don't for "replacing" or "spare" vdevs, even + * though they use the same ops as mirror, because there's only one + * "good" copy under the replacing/spare. + */ + if (vd->vdev_ops == &vdev_mirror_ops) { + for (int i = 0; i < n; i++) { + is->is_child[i].ic_vdev = vd->vdev_child[i]; + } + } else { + is->is_child[0].ic_vdev = vd; + } + + list_insert_tail(&iv->iv_splits, is); } static void +vdev_indirect_read_split_done(zio_t *zio) +{ + indirect_child_t *ic = zio->io_private; + + if (zio->io_error != 0) { + /* + * Clear ic_data to indicate that we do not have data for this + * child. + */ + abd_free(ic->ic_data); + ic->ic_data = NULL; + } +} + +/* + * Issue reads for all copies (mirror children) of all splits. + */ +static void +vdev_indirect_read_all(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + for (int i = 0; i < is->is_children; i++) { + indirect_child_t *ic = &is->is_child[i]; + + if (!vdev_readable(ic->ic_vdev)) + continue; + + /* + * Note, we may read from a child whose DTL + * indicates that the data may not be present here. + * While this might result in a few i/os that will + * likely return incorrect data, it simplifies the + * code since we can treat scrub and resilver + * identically. (The incorrect data will be + * detected and ignored when we verify the + * checksum.) + */ + + ic->ic_data = abd_alloc_sametype(zio->io_abd, + is->is_size); + + zio_nowait(zio_vdev_child_io(zio, NULL, + ic->ic_vdev, is->is_target_offset, ic->ic_data, + is->is_size, zio->io_type, zio->io_priority, 0, + vdev_indirect_read_split_done, ic)); + } + } + iv->iv_reconstruct = B_TRUE; +} + +static void vdev_indirect_io_start(zio_t *zio) { spa_t *spa = zio->io_spa; + indirect_vsd_t *iv = kmem_zalloc(sizeof (*iv), KM_SLEEP); + list_create(&iv->iv_splits, + sizeof (indirect_split_t), offsetof(indirect_split_t, is_node)); + zio->io_vsd = iv; + zio->io_vsd_ops = &vdev_indirect_vsd_ops; + ASSERT(spa_config_held(spa, SCL_ALL, RW_READER) != 0); if (zio->io_type != ZIO_TYPE_READ) { ASSERT3U(zio->io_type, ==, ZIO_TYPE_WRITE); - ASSERT((zio->io_flags & - (ZIO_FLAG_SELF_HEAL | ZIO_FLAG_INDUCE_DAMAGE)) != 0); + /* + * Note: this code can handle other kinds of writes, + * but we don't expect them. + */ + ASSERT((zio->io_flags & (ZIO_FLAG_SELF_HEAL | + ZIO_FLAG_RESILVER | ZIO_FLAG_INDUCE_DAMAGE)) != 0); } vdev_indirect_remap(zio->io_vd, zio->io_offset, zio->io_size, - vdev_indirect_io_start_cb, zio); + vdev_indirect_gather_splits, zio); + indirect_split_t *first = list_head(&iv->iv_splits); + if (first->is_size == zio->io_size) { + /* + * This is not a split block; we are pointing to the entire + * data, which will checksum the same as the original data. + * Pass the BP down so that the child i/o can verify the + * checksum, and try a different location if available + * (e.g. on a mirror). + * + * While this special case could be handled the same as the + * general (split block) case, doing it this way ensures + * that the vast majority of blocks on indirect vdevs + * (which are not split) are handled identically to blocks + * on non-indirect vdevs. This allows us to be less strict + * about performance in the general (but rare) case. + */ + ASSERT0(first->is_split_offset); + ASSERT3P(list_next(&iv->iv_splits, first), ==, NULL); + zio_nowait(zio_vdev_child_io(zio, zio->io_bp, + first->is_vdev, first->is_target_offset, + abd_get_offset(zio->io_abd, 0), + zio->io_size, zio->io_type, zio->io_priority, 0, + vdev_indirect_child_io_done, zio)); + } else { + iv->iv_split_block = B_TRUE; + if (zio->io_flags & (ZIO_FLAG_SCRUB | ZIO_FLAG_RESILVER)) { + /* + * Read all copies. Note that for simplicity, + * we don't bother consulting the DTL in the + * resilver case. + */ + vdev_indirect_read_all(zio); + } else { + /* + * Read one copy of each split segment, from the + * top-level vdev. Since we don't know the + * checksum of each split individually, the child + * zio can't ensure that we get the right data. + * E.g. if it's a mirror, it will just read from a + * random (healthy) leaf vdev. We have to verify + * the checksum in vdev_indirect_io_done(). + */ + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + zio_nowait(zio_vdev_child_io(zio, NULL, + is->is_vdev, is->is_target_offset, + abd_get_offset(zio->io_abd, + is->is_split_offset), + is->is_size, zio->io_type, + zio->io_priority, 0, + vdev_indirect_child_io_done, zio)); + } + } + } + zio_execute(zio); +} + +/* + * Report a checksum error for a child. + */ +static void +vdev_indirect_checksum_error(zio_t *zio, + indirect_split_t *is, indirect_child_t *ic) +{ + vdev_t *vd = ic->ic_vdev; + + if (zio->io_flags & ZIO_FLAG_SPECULATIVE) + return; + + mutex_enter(&vd->vdev_stat_lock); + vd->vdev_stat.vs_checksum_errors++; + mutex_exit(&vd->vdev_stat_lock); + + zio_bad_cksum_t zbc = { 0 }; + void *bad_buf = abd_borrow_buf_copy(ic->ic_data, is->is_size); + abd_t *good_abd = is->is_child[is->is_good_child].ic_data; + void *good_buf = abd_borrow_buf_copy(good_abd, is->is_size); + zfs_ereport_post_checksum(zio->io_spa, vd, zio, + is->is_target_offset, is->is_size, good_buf, bad_buf, &zbc); + abd_return_buf(ic->ic_data, bad_buf, is->is_size); + abd_return_buf(good_abd, good_buf, is->is_size); +} + +/* + * Issue repair i/os for any incorrect copies. We do this by comparing + * each split segment's correct data (is_good_child's ic_data) with each + * other copy of the data. If they differ, then we overwrite the bad data + * with the good copy. Note that we do this without regard for the DTL's, + * which simplifies this code and also issues the optimal number of writes + * (based on which copies actually read bad data, as opposed to which we + * think might be wrong). For the same reason, we always use + * ZIO_FLAG_SELF_HEAL, to bypass the DTL check in zio_vdev_io_start(). + */ +static void +vdev_indirect_repair(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + enum zio_flag flags = ZIO_FLAG_IO_REPAIR; + + if (!(zio->io_flags & (ZIO_FLAG_SCRUB | ZIO_FLAG_RESILVER))) + flags |= ZIO_FLAG_SELF_HEAL; + + if (!spa_writeable(zio->io_spa)) + return; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + indirect_child_t *good_child = &is->is_child[is->is_good_child]; + + for (int c = 0; c < is->is_children; c++) { + indirect_child_t *ic = &is->is_child[c]; + if (ic == good_child) + continue; + if (ic->ic_data == NULL) + continue; + if (abd_cmp(good_child->ic_data, ic->ic_data, + is->is_size) == 0) + continue; + + zio_nowait(zio_vdev_child_io(zio, NULL, + ic->ic_vdev, is->is_target_offset, + good_child->ic_data, is->is_size, + ZIO_TYPE_WRITE, ZIO_PRIORITY_ASYNC_WRITE, + ZIO_FLAG_IO_REPAIR | ZIO_FLAG_SELF_HEAL, + NULL, NULL)); + + vdev_indirect_checksum_error(zio, is, ic); + } + } +} + +/* + * Report checksum errors on all children that we read from. + */ +static void +vdev_indirect_all_checksum_errors(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + if (zio->io_flags & ZIO_FLAG_SPECULATIVE) + return; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + for (int c = 0; c < is->is_children; c++) { + indirect_child_t *ic = &is->is_child[c]; + + if (ic->ic_data == NULL) + continue; + + vdev_t *vd = ic->ic_vdev; + + mutex_enter(&vd->vdev_stat_lock); + vd->vdev_stat.vs_checksum_errors++; + mutex_exit(&vd->vdev_stat_lock); + + zfs_ereport_post_checksum(zio->io_spa, vd, zio, + is->is_target_offset, is->is_size, + NULL, NULL, NULL); + } + } +} + +/* + * This function is called when we have read all copies of the data and need + * to try to find a combination of copies that gives us the right checksum. + * + * If we pointed to any mirror vdevs, this effectively does the job of the + * mirror. The mirror vdev code can't do its own job because we don't know + * the checksum of each split segment individually. We have to try every + * combination of copies of split segments, until we find one that checksums + * correctly. (Or until we have tried all combinations, or have tried + * 2^zfs_reconstruct_indirect_segments_max combinations. In these cases we + * set io_error to ECKSUM to propagate the error up to the user.) + * + * For example, if we have 3 segments in the split, + * and each points to a 2-way mirror, we will have the following pieces of + * data: + * + * | mirror child + * split | [0] [1] + * ======|===================== + * A | data_A_0 data_A_1 + * B | data_B_0 data_B_1 + * C | data_C_0 data_C_1 + * + * We will try the following (mirror children)^(number of splits) (2^3=8) + * combinations, which is similar to bitwise-little-endian counting in + * binary. In general each "digit" corresponds to a split segment, and the + * base of each digit is is_children, which can be different for each + * digit. + * + * "low bit" "high bit" + * v v + * data_A_0 data_B_0 data_C_0 + * data_A_1 data_B_0 data_C_0 + * data_A_0 data_B_1 data_C_0 + * data_A_1 data_B_1 data_C_0 + * data_A_0 data_B_0 data_C_1 + * data_A_1 data_B_0 data_C_1 + * data_A_0 data_B_1 data_C_1 + * data_A_1 data_B_1 data_C_1 + * + * Note that the split segments may be on the same or different top-level + * vdevs. In either case, we try lots of combinations (see + * zfs_reconstruct_indirect_segments_max). This ensures that if a mirror has + * small silent errors on all of its children, we can still reconstruct the + * correct data, as long as those errors are at sufficiently-separated + * offsets (specifically, separated by the largest block size - default of + * 128KB, but up to 16MB). + */ +static void +vdev_indirect_reconstruct_io_done(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + uint64_t attempts = 0; + uint64_t attempts_max = 1ULL << zfs_reconstruct_indirect_segments_max; + int segments = 0; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) + segments++; + + for (;;) { + /* copy data from splits to main zio */ + int ret; + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + + /* + * If this child failed, its ic_data will be NULL. + * Skip this combination. + */ + if (is->is_child[is->is_good_child].ic_data == NULL) { + ret = EIO; + goto next; + } + + abd_copy_off(zio->io_abd, + is->is_child[is->is_good_child].ic_data, + is->is_split_offset, 0, is->is_size); + } + + /* See if this checksum matches. */ + zio_bad_cksum_t zbc; + ret = zio_checksum_error(zio, &zbc); + if (ret == 0) { + /* Found a matching checksum. Issue repair i/os. */ + vdev_indirect_repair(zio); + zio_checksum_verified(zio); + return; + } + + /* + * Checksum failed; try a different combination of split + * children. + */ + boolean_t more; +next: + more = B_FALSE; + if (segments <= zfs_reconstruct_indirect_segments_max) { + /* + * There are relatively few segments, so + * deterministically check all combinations. We do + * this by by adding one to the first split's + * good_child. If it overflows, then "carry over" to + * the next split (like counting in base is_children, + * but each digit can have a different base). + */ + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + is->is_good_child++; + if (is->is_good_child < is->is_children) { + more = B_TRUE; + break; + } + is->is_good_child = 0; + } + } else if (++attempts < attempts_max) { + /* + * There are too many combinations to try all of them + * in a reasonable amount of time, so try a fixed + * number of random combinations, after which we'll + * consider the block unrecoverable. + */ + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + is->is_good_child = + spa_get_random(is->is_children); + } + more = B_TRUE; + } + if (!more) { + /* All combinations failed. */ + zio->io_error = ret; + vdev_indirect_all_checksum_errors(zio); + zio_checksum_verified(zio); + return; + } + } +} + +static void +vdev_indirect_io_done(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + if (iv->iv_reconstruct) { + /* + * We have read all copies of the data (e.g. from mirrors), + * either because this was a scrub/resilver, or because the + * one-copy read didn't checksum correctly. + */ + vdev_indirect_reconstruct_io_done(zio); + return; + } + + if (!iv->iv_split_block) { + /* + * This was not a split block, so we passed the BP down, + * and the checksum was handled by the (one) child zio. + */ + return; + } + + zio_bad_cksum_t zbc; + int ret = zio_checksum_error(zio, &zbc); + if (ret == 0) { + zio_checksum_verified(zio); + return; + } + + /* + * The checksum didn't match. Read all copies of all splits, and + * then we will try to reconstruct. The next time + * vdev_indirect_io_done() is called, iv_reconstruct will be set. + */ + vdev_indirect_read_all(zio); + + zio_vdev_io_redone(zio); } vdev_ops_t vdev_indirect_ops = { Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Tue Jul 31 00:13:04 2018 (r336950) @@ -24,7 +24,7 @@ */ /* - * Copyright (c) 2012, 2015 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. */ #include @@ -345,12 +345,15 @@ vdev_mirror_io_start(zio_t *zio) } if (zio->io_type == ZIO_TYPE_READ) { - if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering) { + if (zio->io_bp != NULL && + (zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering) { /* - * For scrubbing reads we need to allocate a read - * buffer for each child and issue reads to all - * children. If any child succeeds, it will copy its - * data into zio->io_data in vdev_mirror_scrub_done. + * For scrubbing reads (if we can verify the + * checksum here, as indicated by io_bp being + * non-NULL) we need to allocate a read buffer for + * each child and issue reads to all children. If + * any child succeeds, it will copy its data into + * zio->io_data in vdev_mirror_scrub_done. */ for (c = 0; c < mm->mm_children; c++) { mc = &mm->mm_child[c]; @@ -502,7 +505,21 @@ vdev_mirror_io_done(zio_t *zio) if (mc->mc_error == 0) { if (mc->mc_tried) continue; + /* + * We didn't try this child. We need to + * repair it if: + * 1. it's a scrub (in which case we have + * tried everything that was healthy) + * - or - + * 2. it's an indirect vdev (in which case + * it could point to any other vdev, which + * might have a bad DTL) + * - or - + * 3. the DTL indicates that this data is + * missing from this vdev + */ if (!(zio->io_flags & ZIO_FLAG_SCRUB) && + mc->mc_vd->vdev_ops != &vdev_indirect_ops && !vdev_dtl_contains(mc->mc_vd, DTL_PARTIAL, zio->io_txg, 1)) continue; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 00:13:04 2018 (r336950) @@ -83,18 +83,12 @@ typedef struct vdev_copy_arg { kmutex_t vca_lock; } vdev_copy_arg_t; -typedef struct vdev_copy_seg_arg { - vdev_copy_arg_t *vcsa_copy_arg; - uint64_t vcsa_txg; - dva_t *vcsa_dest_dva; - blkptr_t *vcsa_dest_bp; -} vdev_copy_seg_arg_t; - /* - * The maximum amount of allowed data we're allowed to copy from a device - * at a time when removing it. + * The maximum amount of memory we can use for outstanding i/o while + * doing a device removal. This determines how much i/o we can have + * in flight concurrently. */ -int zfs_remove_max_copy_bytes = 8 * 1024 * 1024; +int zfs_remove_max_copy_bytes = 64 * 1024 * 1024; /* * The largest contiguous segment that we will attempt to allocate when @@ -176,7 +170,7 @@ spa_vdev_removal_create(vdev_t *vd) mutex_init(&svr->svr_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&svr->svr_cv, NULL, CV_DEFAULT, NULL); svr->svr_allocd_segs = range_tree_create(NULL, NULL); - svr->svr_vdev = vd; + svr->svr_vdev_id = vd->vdev_id; for (int i = 0; i < TXG_SIZE; i++) { svr->svr_frees[i] = range_tree_create(NULL, NULL); @@ -218,9 +212,10 @@ spa_vdev_removal_destroy(spa_vdev_removal_t *svr) static void vdev_remove_initiate_sync(void *arg, dmu_tx_t *tx) { - vdev_t *vd = arg; + int vdev_id = (uintptr_t)arg; + spa_t *spa = dmu_tx_pool(tx)->dp_spa; + vdev_t *vd = vdev_lookup_top(spa, vdev_id); vdev_indirect_config_t *vic = &vd->vdev_indirect_config; - spa_t *spa = vd->vdev_spa; objset_t *mos = spa->spa_dsl_pool->dp_meta_objset; spa_vdev_removal_t *svr = NULL; uint64_t txg = dmu_tx_get_txg(tx); @@ -342,7 +337,7 @@ vdev_remove_initiate_sync(void *arg, dmu_tx_t *tx) ASSERT3P(spa->spa_vdev_removal, ==, NULL); spa->spa_vdev_removal = svr; svr->svr_thread = thread_create(NULL, 0, - spa_vdev_remove_thread, vd, 0, &p0, TS_RUN, minclsyspri); + spa_vdev_remove_thread, spa, 0, &p0, TS_RUN, minclsyspri); } /* @@ -383,21 +378,24 @@ spa_remove_init(spa_t *spa) spa_config_enter(spa, SCL_STATE, FTAG, RW_READER); vdev_t *vd = vdev_lookup_top(spa, spa->spa_removing_phys.sr_removing_vdev); - spa_config_exit(spa, SCL_STATE, FTAG); - if (vd == NULL) + if (vd == NULL) { + spa_config_exit(spa, SCL_STATE, FTAG); return (EINVAL); + } vdev_indirect_config_t *vic = &vd->vdev_indirect_config; ASSERT(vdev_is_concrete(vd)); spa_vdev_removal_t *svr = spa_vdev_removal_create(vd); - ASSERT(svr->svr_vdev->vdev_removing); + ASSERT3U(svr->svr_vdev_id, ==, vd->vdev_id); + ASSERT(vd->vdev_removing); vd->vdev_indirect_mapping = vdev_indirect_mapping_open( spa->spa_meta_objset, vic->vic_mapping_object); vd->vdev_indirect_births = vdev_indirect_births_open( spa->spa_meta_objset, vic->vic_births_object); + spa_config_exit(spa, SCL_STATE, FTAG); spa->spa_vdev_removal = svr; } @@ -450,15 +448,8 @@ spa_restart_removal(spa_t *spa) if (!spa_writeable(spa)) return; - vdev_t *vd = svr->svr_vdev; - vdev_indirect_mapping_t *vim = vd->vdev_indirect_mapping; - - ASSERT3P(vd, !=, NULL); - ASSERT(vd->vdev_removing); - - zfs_dbgmsg("restarting removal of %llu at count=%llu", - vd->vdev_id, vdev_indirect_mapping_num_entries(vim)); - svr->svr_thread = thread_create(NULL, 0, spa_vdev_remove_thread, vd, + zfs_dbgmsg("restarting removal of %llu", svr->svr_vdev_id); + svr->svr_thread = thread_create(NULL, 0, spa_vdev_remove_thread, spa, 0, &p0, TS_RUN, minclsyspri); } @@ -479,7 +470,7 @@ free_from_removing_vdev(vdev_t *vd, uint64_t offset, u ASSERT(vd->vdev_indirect_config.vic_mapping_object != 0); ASSERT3U(vd->vdev_indirect_config.vic_mapping_object, ==, vdev_indirect_mapping_object(vim)); - ASSERT3P(vd, ==, svr->svr_vdev); + ASSERT3U(vd->vdev_id, ==, svr->svr_vdev_id); mutex_enter(&svr->svr_lock); @@ -662,7 +653,7 @@ spa_finish_removal(spa_t *spa, dsl_scan_state_t state, if (state == DSS_FINISHED) { spa_removing_phys_t *srp = &spa->spa_removing_phys; - vdev_t *vd = svr->svr_vdev; + vdev_t *vd = vdev_lookup_top(spa, svr->svr_vdev_id); vdev_indirect_config_t *vic = &vd->vdev_indirect_config; if (srp->sr_prev_indirect_vdev != UINT64_MAX) { @@ -705,7 +696,7 @@ vdev_mapping_sync(void *arg, dmu_tx_t *tx) { spa_vdev_removal_t *svr = arg; spa_t *spa = dmu_tx_pool(tx)->dp_spa; - vdev_t *vd = svr->svr_vdev; + vdev_t *vd = vdev_lookup_top(spa, svr->svr_vdev_id); vdev_indirect_config_t *vic = &vd->vdev_indirect_config; uint64_t txg = dmu_tx_get_txg(tx); vdev_indirect_mapping_t *vim = vd->vdev_indirect_mapping; @@ -733,64 +724,128 @@ vdev_mapping_sync(void *arg, dmu_tx_t *tx) spa_sync_removing_state(spa, tx); } +/* + * All reads and writes associated with a call to spa_vdev_copy_segment() + * are done. + */ static void +spa_vdev_copy_nullzio_done(zio_t *zio) +{ + spa_config_exit(zio->io_spa, SCL_STATE, zio->io_spa); +} + +/* + * The write of the new location is done. + */ +static void spa_vdev_copy_segment_write_done(zio_t *zio) { - vdev_copy_seg_arg_t *vcsa = zio->io_private; - vdev_copy_arg_t *vca = vcsa->vcsa_copy_arg; - spa_config_exit(zio->io_spa, SCL_STATE, FTAG); + vdev_copy_arg_t *vca = zio->io_private; + abd_free(zio->io_abd); mutex_enter(&vca->vca_lock); vca->vca_outstanding_bytes -= zio->io_size; cv_signal(&vca->vca_cv); mutex_exit(&vca->vca_lock); - - ASSERT0(zio->io_error); - kmem_free(vcsa->vcsa_dest_bp, sizeof (blkptr_t)); - kmem_free(vcsa, sizeof (vdev_copy_seg_arg_t)); } +/* + * The read of the old location is done. The parent zio is the write to + * the new location. Allow it to start. + */ static void spa_vdev_copy_segment_read_done(zio_t *zio) { - vdev_copy_seg_arg_t *vcsa = zio->io_private; - dva_t *dest_dva = vcsa->vcsa_dest_dva; - uint64_t txg = vcsa->vcsa_txg; - spa_t *spa = zio->io_spa; - vdev_t *dest_vd = vdev_lookup_top(spa, DVA_GET_VDEV(dest_dva)); - blkptr_t *bp = NULL; - dva_t *dva = NULL; - uint64_t size = zio->io_size; + zio_nowait(zio_unique_parent(zio)); +} - ASSERT3P(dest_vd, !=, NULL); - ASSERT0(zio->io_error); +/* + * If the old and new vdevs are mirrors, we will read both sides of the old + * mirror, and write each copy to the corresponding side of the new mirror. + * If the old and new vdevs have a different number of children, we will do + * this as best as possible. Since we aren't verifying checksums, this + * ensures that as long as there's a good copy of the data, we'll have a + * good copy after the removal, even if there's silent damage to one side + * of the mirror. If we're removing a mirror that has some silent damage, + * we'll have exactly the same damage in the new location (assuming that + * the new location is also a mirror). + * + * We accomplish this by creating a tree of zio_t's, with as many writes as + * there are "children" of the new vdev (a non-redundant vdev counts as one + * child, a 2-way mirror has 2 children, etc). Each write has an associated + * read from a child of the old vdev. Typically there will be the same + * number of children of the old and new vdevs. However, if there are more + * children of the new vdev, some child(ren) of the old vdev will be issued *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Tue Jul 31 00:13:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D5511065B7D; Tue, 31 Jul 2018 00:13:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D53878E4AB; Tue, 31 Jul 2018 00:13:07 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B72AE4059; Tue, 31 Jul 2018 00:13:07 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0D7Uk088747; Tue, 31 Jul 2018 00:13:07 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0D7dV088745; Tue, 31 Jul 2018 00:13:07 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310013.w6V0D7dV088745@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:13:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336950 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/ztest vendor/illumos/dist/lib/libzf... X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zdb vendor/illumos/dist/cmd/ztest vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 336950 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:13:08 -0000 Author: mav Date: Tue Jul 31 00:13:04 2018 New Revision: 336950 URL: https://svnweb.freebsd.org/changeset/base/336950 Log: 9290 device removal reduces redundancy of mirrors Mirrors are supposed to provide redundancy in the face of whole-disk failure and silent damage (e.g. some data on disk is not right, but ZFS hasn't detected the whole device as being broken). However, the current device removal implementation bypasses some of the mirror's redundancy. illumos/illumos-gate@3a4b1be953ee5601bab748afa07c26ed4996cde6 Reviewed by: George Wilson Reviewed by: Prashanth Sreenivasa Reviewed by: Sara Hartse Reviewed by: Serapheim Dimitropoulos Reviewed by: Brian Behlendorf Reviewed by: Tim Chase Approved by: Richard Lowe Author: Matthew Ahrens Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor/illumos/dist/cmd/zdb/zdb.c Tue Jul 31 00:13:04 2018 (r336950) @@ -3004,7 +3004,7 @@ zdb_claim_removing(spa_t *spa, zdb_cb_t *zcb) spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); spa_vdev_removal_t *svr = spa->spa_vdev_removal; - vdev_t *vd = svr->svr_vdev; + vdev_t *vd = vdev_lookup_top(spa, svr->svr_vdev_id); vdev_indirect_mapping_t *vim = vd->vdev_indirect_mapping; for (uint64_t msi = 0; msi < vd->vdev_ms_count; msi++) { @@ -3020,13 +3020,17 @@ zdb_claim_removing(spa_t *spa, zdb_cb_t *zcb) svr->svr_allocd_segs, SM_ALLOC)); /* - * Clear everything past what has been synced, - * because we have not allocated mappings for it yet. + * Clear everything past what has been synced unless + * it's past the spacemap, because we have not allocated + * mappings for it yet. */ - range_tree_clear(svr->svr_allocd_segs, - vdev_indirect_mapping_max_offset(vim), - msp->ms_sm->sm_start + msp->ms_sm->sm_size - - vdev_indirect_mapping_max_offset(vim)); + uint64_t vim_max_offset = + vdev_indirect_mapping_max_offset(vim); + uint64_t sm_end = msp->ms_sm->sm_start + + msp->ms_sm->sm_size; + if (sm_end > vim_max_offset) + range_tree_clear(svr->svr_allocd_segs, + vim_max_offset, sm_end - vim_max_offset); } zcb->zcb_removing_size += Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor/illumos/dist/cmd/ztest/ztest.c Tue Jul 31 00:13:04 2018 (r336950) @@ -436,6 +436,7 @@ static ztest_ds_t *ztest_ds; static kmutex_t ztest_vdev_lock; static kmutex_t ztest_checkpoint_lock; +static boolean_t ztest_device_removal_active = B_FALSE; /* * The ztest_name_lock protects the pool and dataset namespace used by @@ -2880,7 +2881,7 @@ ztest_vdev_attach_detach(ztest_ds_t *zd, uint64_t id) * value. Don't bother trying to attach while we are in the middle * of removal. */ - if (spa->spa_vdev_removal != NULL) { + if (ztest_device_removal_active) { spa_config_exit(spa, SCL_ALL, FTAG); mutex_exit(&ztest_vdev_lock); return; @@ -3055,16 +3056,49 @@ ztest_device_removal(ztest_ds_t *zd, uint64_t id) spa_t *spa = ztest_spa; vdev_t *vd; uint64_t guid; + int error; mutex_enter(&ztest_vdev_lock); + if (ztest_device_removal_active) { + mutex_exit(&ztest_vdev_lock); + return; + } + + /* + * Remove a random top-level vdev and wait for removal to finish. + */ spa_config_enter(spa, SCL_VDEV, FTAG, RW_READER); vd = vdev_lookup_top(spa, ztest_random_vdev_top(spa, B_FALSE)); guid = vd->vdev_guid; spa_config_exit(spa, SCL_VDEV, FTAG); - (void) spa_vdev_remove(spa, guid, B_FALSE); + error = spa_vdev_remove(spa, guid, B_FALSE); + if (error == 0) { + ztest_device_removal_active = B_TRUE; + mutex_exit(&ztest_vdev_lock); + while (spa->spa_vdev_removal != NULL) + txg_wait_synced(spa_get_dsl(spa), 0); + } else { + mutex_exit(&ztest_vdev_lock); + return; + } + + /* + * The pool needs to be scrubbed after completing device removal. + * Failure to do so may result in checksum errors due to the + * strategy employed by ztest_fault_inject() when selecting which + * offset are redundant and can be damaged. + */ + error = spa_scan(spa, POOL_SCAN_SCRUB); + if (error == 0) { + while (dsl_scan_scrubbing(spa_get_dsl(spa))) + txg_wait_synced(spa_get_dsl(spa), 0); + } + + mutex_enter(&ztest_vdev_lock); + ztest_device_removal_active = B_FALSE; mutex_exit(&ztest_vdev_lock); } @@ -3203,7 +3237,7 @@ ztest_vdev_LUN_growth(ztest_ds_t *zd, uint64_t id) * that the metaslab_class space increased (because it decreases * when the device removal completes). */ - if (spa->spa_vdev_removal != NULL) { + if (ztest_device_removal_active) { spa_config_exit(spa, SCL_STATE, spa); mutex_exit(&ztest_vdev_lock); mutex_exit(&ztest_checkpoint_lock); @@ -4985,6 +5019,18 @@ ztest_fault_inject(ztest_ds_t *zd, uint64_t id) boolean_t islog = B_FALSE; mutex_enter(&ztest_vdev_lock); + + /* + * Device removal is in progress, fault injection must be disabled + * until it completes and the pool is scrubbed. The fault injection + * strategy for damaging blocks does not take in to account evacuated + * blocks which may have already been damaged. + */ + if (ztest_device_removal_active) { + mutex_exit(&ztest_vdev_lock); + return; + } + maxfaults = MAXFAULTS(); leaves = MAX(zs->zs_mirrors, 1) * ztest_opts.zo_raidz; mirror_save = zs->zs_mirrors; @@ -5329,6 +5375,12 @@ void ztest_scrub(ztest_ds_t *zd, uint64_t id) { spa_t *spa = ztest_spa; + + /* + * Scrub in progress by device removal. + */ + if (ztest_device_removal_active) + return; (void) spa_scan(spa, POOL_SCAN_SCRUB); (void) poll(NULL, 0, 100); /* wait a moment, then force a restart */ Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Tue Jul 31 00:02:42 2018 (r336949) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Tue Jul 31 00:13:04 2018 (r336950) @@ -2807,7 +2807,7 @@ zpool_vdev_attach(zpool_handle_t *zhp, case EBUSY: zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "%s is busy, " - "or pool has removing/removed vdevs"), + "or device removal is in progress"), new_disk); (void) zfs_error(hdl, EZFS_BADDEV, msg); break; From owner-svn-src-all@freebsd.org Tue Jul 31 00:25:44 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B38B51065E7E; Tue, 31 Jul 2018 00:25:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BFFB8EACB; Tue, 31 Jul 2018 00:25:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3255D4231; Tue, 31 Jul 2018 00:25:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0Ph8Q094202; Tue, 31 Jul 2018 00:25:43 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0Pej2094185; Tue, 31 Jul 2018 00:25:40 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310025.w6V0Pej2094185@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:25:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336951 - in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/cmd/ztest cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl... X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/cmd/ztest cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/... X-SVN-Commit-Revision: 336951 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:25:44 -0000 Author: mav Date: Tue Jul 31 00:25:39 2018 New Revision: 336951 URL: https://svnweb.freebsd.org/changeset/base/336951 Log: MFV r336950: 9290 device removal reduces redundancy of mirrors Mirrors are supposed to provide redundancy in the face of whole-disk failure and silent damage (e.g. some data on disk is not right, but ZFS hasn't detected the whole device as being broken). However, the current device removal implementation bypasses some of the mirror's redundancy. illumos/illumos-gate@3a4b1be953ee5601bab748afa07c26ed4996cde6 Reviewed by: George Wilson Reviewed by: Prashanth Sreenivasa Reviewed by: Sara Hartse Reviewed by: Serapheim Dimitropoulos Reviewed by: Brian Behlendorf Reviewed by: Tim Chase Approved by: Richard Lowe Author: Matthew Ahrens Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c head/cddl/contrib/opensolaris/cmd/ztest/ztest.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_indirect.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Tue Jul 31 00:25:39 2018 (r336951) @@ -3033,7 +3033,7 @@ zdb_claim_removing(spa_t *spa, zdb_cb_t *zcb) spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); spa_vdev_removal_t *svr = spa->spa_vdev_removal; - vdev_t *vd = svr->svr_vdev; + vdev_t *vd = vdev_lookup_top(spa, svr->svr_vdev_id); vdev_indirect_mapping_t *vim = vd->vdev_indirect_mapping; for (uint64_t msi = 0; msi < vd->vdev_ms_count; msi++) { @@ -3049,13 +3049,17 @@ zdb_claim_removing(spa_t *spa, zdb_cb_t *zcb) svr->svr_allocd_segs, SM_ALLOC)); /* - * Clear everything past what has been synced, - * because we have not allocated mappings for it yet. + * Clear everything past what has been synced unless + * it's past the spacemap, because we have not allocated + * mappings for it yet. */ - range_tree_clear(svr->svr_allocd_segs, - vdev_indirect_mapping_max_offset(vim), - msp->ms_sm->sm_start + msp->ms_sm->sm_size - - vdev_indirect_mapping_max_offset(vim)); + uint64_t vim_max_offset = + vdev_indirect_mapping_max_offset(vim); + uint64_t sm_end = msp->ms_sm->sm_start + + msp->ms_sm->sm_size; + if (sm_end > vim_max_offset) + range_tree_clear(svr->svr_allocd_segs, + vim_max_offset, sm_end - vim_max_offset); } zcb->zcb_removing_size += Modified: head/cddl/contrib/opensolaris/cmd/ztest/ztest.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Jul 31 00:25:39 2018 (r336951) @@ -438,6 +438,7 @@ static ztest_ds_t *ztest_ds; static kmutex_t ztest_vdev_lock; static kmutex_t ztest_checkpoint_lock; +static boolean_t ztest_device_removal_active = B_FALSE; /* * The ztest_name_lock protects the pool and dataset namespace used by @@ -2882,7 +2883,7 @@ ztest_vdev_attach_detach(ztest_ds_t *zd, uint64_t id) * value. Don't bother trying to attach while we are in the middle * of removal. */ - if (spa->spa_vdev_removal != NULL) { + if (ztest_device_removal_active) { spa_config_exit(spa, SCL_ALL, FTAG); mutex_exit(&ztest_vdev_lock); return; @@ -3057,16 +3058,49 @@ ztest_device_removal(ztest_ds_t *zd, uint64_t id) spa_t *spa = ztest_spa; vdev_t *vd; uint64_t guid; + int error; mutex_enter(&ztest_vdev_lock); + if (ztest_device_removal_active) { + mutex_exit(&ztest_vdev_lock); + return; + } + + /* + * Remove a random top-level vdev and wait for removal to finish. + */ spa_config_enter(spa, SCL_VDEV, FTAG, RW_READER); vd = vdev_lookup_top(spa, ztest_random_vdev_top(spa, B_FALSE)); guid = vd->vdev_guid; spa_config_exit(spa, SCL_VDEV, FTAG); - (void) spa_vdev_remove(spa, guid, B_FALSE); + error = spa_vdev_remove(spa, guid, B_FALSE); + if (error == 0) { + ztest_device_removal_active = B_TRUE; + mutex_exit(&ztest_vdev_lock); + while (spa->spa_vdev_removal != NULL) + txg_wait_synced(spa_get_dsl(spa), 0); + } else { + mutex_exit(&ztest_vdev_lock); + return; + } + + /* + * The pool needs to be scrubbed after completing device removal. + * Failure to do so may result in checksum errors due to the + * strategy employed by ztest_fault_inject() when selecting which + * offset are redundant and can be damaged. + */ + error = spa_scan(spa, POOL_SCAN_SCRUB); + if (error == 0) { + while (dsl_scan_scrubbing(spa_get_dsl(spa))) + txg_wait_synced(spa_get_dsl(spa), 0); + } + + mutex_enter(&ztest_vdev_lock); + ztest_device_removal_active = B_FALSE; mutex_exit(&ztest_vdev_lock); } @@ -3205,7 +3239,7 @@ ztest_vdev_LUN_growth(ztest_ds_t *zd, uint64_t id) * that the metaslab_class space increased (because it decreases * when the device removal completes). */ - if (spa->spa_vdev_removal != NULL) { + if (ztest_device_removal_active) { spa_config_exit(spa, SCL_STATE, spa); mutex_exit(&ztest_vdev_lock); mutex_exit(&ztest_checkpoint_lock); @@ -4986,6 +5020,18 @@ ztest_fault_inject(ztest_ds_t *zd, uint64_t id) boolean_t islog = B_FALSE; mutex_enter(&ztest_vdev_lock); + + /* + * Device removal is in progress, fault injection must be disabled + * until it completes and the pool is scrubbed. The fault injection + * strategy for damaging blocks does not take in to account evacuated + * blocks which may have already been damaged. + */ + if (ztest_device_removal_active) { + mutex_exit(&ztest_vdev_lock); + return; + } + maxfaults = MAXFAULTS(); leaves = MAX(zs->zs_mirrors, 1) * ztest_opts.zo_raidz; mirror_save = zs->zs_mirrors; @@ -5330,6 +5376,12 @@ void ztest_scrub(ztest_ds_t *zd, uint64_t id) { spa_t *spa = ztest_spa; + + /* + * Scrub in progress by device removal. + */ + if (ztest_device_removal_active) + return; (void) spa_scan(spa, POOL_SCAN_SCRUB); (void) poll(NULL, 0, 100); /* wait a moment, then force a restart */ Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c Tue Jul 31 00:25:39 2018 (r336951) @@ -2836,7 +2836,7 @@ zpool_vdev_attach(zpool_handle_t *zhp, case EBUSY: zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, "%s is busy, " - "or pool has removing/removed vdevs"), + "or device removal is in progress"), new_disk); (void) zfs_error(hdl, EZFS_BADDEV, msg); break; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 00:25:39 2018 (r336951) @@ -2959,6 +2959,16 @@ dsl_scan_need_resilver(spa_t *spa, const dva_t *dva, s { vdev_t *vd; + if (vd->vdev_ops == &vdev_indirect_ops) { + /* + * The indirect vdev can point to multiple + * vdevs. For simplicity, always create + * the resilver zio_t. zio_vdev_io_start() + * will bypass the child resilver i/o's if + * they are on vdevs that don't have DTL's. + */ + return (B_TRUE); + } if (DVA_GET_GANG(dva)) { /* * Gang members may be spread across multiple Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:25:39 2018 (r336951) @@ -3596,7 +3596,7 @@ metaslab_free_impl(vdev_t *vd, uint64_t offset, uint64 return; if (spa->spa_vdev_removal != NULL && - spa->spa_vdev_removal->svr_vdev == vd && + spa->spa_vdev_removal->svr_vdev_id == vd->vdev_id && vdev_is_concrete(vd)) { /* * Note: we check if the vdev is concrete because when Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Tue Jul 31 00:25:39 2018 (r336951) @@ -5837,8 +5837,7 @@ spa_vdev_add(spa_t *spa, nvlist_t *nvroot) for (int c = 0; c < vd->vdev_children; c++) { tvd = vd->vdev_child[c]; if (spa->spa_vdev_removal != NULL && - tvd->vdev_ashift != - spa->spa_vdev_removal->svr_vdev->vdev_ashift) { + tvd->vdev_ashift != spa->spa_max_ashift) { return (spa_vdev_exit(spa, vd, txg, EINVAL)); } /* Fail if top level vdev is raidz */ @@ -5954,10 +5953,8 @@ spa_vdev_attach(spa_t *spa, uint64_t guid, nvlist_t *n return (spa_vdev_exit(spa, NULL, txg, error)); } - if (spa->spa_vdev_removal != NULL || - spa->spa_removing_phys.sr_prev_indirect_vdev != -1) { + if (spa->spa_vdev_removal != NULL) return (spa_vdev_exit(spa, NULL, txg, EBUSY)); - } if (oldvd == NULL) return (spa_vdev_exit(spa, NULL, txg, ENODEV)); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:25:39 2018 (r336951) @@ -1878,9 +1878,12 @@ spa_update_dspace(spa_t *spa) * allocated twice (on the old device and the new * device). */ - vdev_t *vd = spa->spa_vdev_removal->svr_vdev; + spa_config_enter(spa, SCL_VDEV, FTAG, RW_READER); + vdev_t *vd = + vdev_lookup_top(spa, spa->spa_vdev_removal->svr_vdev_id); spa->spa_dspace -= spa_deflate(spa) ? vd->vdev_stat.vs_dspace : vd->vdev_stat.vs_space; + spa_config_exit(spa, SCL_VDEV, FTAG); } } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h Tue Jul 31 00:25:39 2018 (r336951) @@ -30,7 +30,7 @@ extern "C" { #endif typedef struct spa_vdev_removal { - vdev_t *svr_vdev; + uint64_t svr_vdev_id; uint64_t svr_max_offset_to_sync[TXG_SIZE]; /* Thread performing a vdev removal. */ kthread_t *svr_thread; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h Tue Jul 31 00:25:39 2018 (r336951) @@ -587,7 +587,7 @@ extern zio_t *zio_vdev_child_io(zio_t *zio, blkptr_t * zio_done_func_t *done, void *priv); extern zio_t *zio_vdev_delegated_io(vdev_t *vd, uint64_t offset, - struct abd *data, uint64_t size, int type, zio_priority_t priority, + struct abd *data, uint64_t size, zio_type_t type, zio_priority_t priority, enum zio_flag flags, zio_done_func_t *done, void *priv); extern void zio_vdev_io_bypass(zio_t *zio); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Tue Jul 31 00:25:39 2018 (r336951) @@ -988,6 +988,32 @@ vdev_top_transfer(vdev_t *svd, vdev_t *tvd) svd->vdev_stat.vs_space = 0; svd->vdev_stat.vs_dspace = 0; + /* + * State which may be set on a top-level vdev that's in the + * process of being removed. + */ + ASSERT0(tvd->vdev_indirect_config.vic_births_object); + ASSERT0(tvd->vdev_indirect_config.vic_mapping_object); + ASSERT3U(tvd->vdev_indirect_config.vic_prev_indirect_vdev, ==, -1ULL); + ASSERT3P(tvd->vdev_indirect_mapping, ==, NULL); + ASSERT3P(tvd->vdev_indirect_births, ==, NULL); + ASSERT3P(tvd->vdev_obsolete_sm, ==, NULL); + ASSERT0(tvd->vdev_removing); + tvd->vdev_removing = svd->vdev_removing; + tvd->vdev_indirect_config = svd->vdev_indirect_config; + tvd->vdev_indirect_mapping = svd->vdev_indirect_mapping; + tvd->vdev_indirect_births = svd->vdev_indirect_births; + range_tree_swap(&svd->vdev_obsolete_segments, + &tvd->vdev_obsolete_segments); + tvd->vdev_obsolete_sm = svd->vdev_obsolete_sm; + svd->vdev_indirect_config.vic_mapping_object = 0; + svd->vdev_indirect_config.vic_births_object = 0; + svd->vdev_indirect_config.vic_prev_indirect_vdev = -1ULL; + svd->vdev_indirect_mapping = NULL; + svd->vdev_indirect_births = NULL; + svd->vdev_obsolete_sm = NULL; + svd->vdev_removing = 0; + for (t = 0; t < TXG_SIZE; t++) { while ((msp = txg_list_remove(&svd->vdev_ms_list, t)) != NULL) (void) txg_list_add(&tvd->vdev_ms_list, msp, t); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_indirect.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_indirect.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_indirect.c Tue Jul 31 00:25:39 2018 (r336951) @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include @@ -46,10 +47,11 @@ * "vdev_remap" operation that executes a callback on each contiguous * segment of the new location. This function is used in multiple ways: * - * - reads and repair writes to this device use the callback to create - * a child io for each mapped segment. + * - i/os to this vdev use the callback to determine where the + * data is now located, and issue child i/os for each segment's new + * location. * - * - frees and claims to this device use the callback to free or claim + * - frees and claims to this vdev use the callback to free or claim * each mapped segment. (Note that we don't actually need to claim * log blocks on indirect vdevs, because we don't allocate to * removing vdevs. However, zdb uses zio_claim() for its leak @@ -204,6 +206,94 @@ uint64_t zfs_condense_min_mapping_bytes = 128 * 1024; int zfs_condense_indirect_commit_entry_delay_ticks = 0; /* + * If a split block contains more than this many segments, consider it too + * computationally expensive to check all (2^num_segments) possible + * combinations. Instead, try at most 2^_segments_max randomly-selected + * combinations. + * + * This is reasonable if only a few segment copies are damaged and the + * majority of segment copies are good. This allows all the segment copies to + * participate fairly in the reconstruction and prevents the repeated use of + * one bad copy. + */ +int zfs_reconstruct_indirect_segments_max = 10; + +/* + * The indirect_child_t represents the vdev that we will read from, when we + * need to read all copies of the data (e.g. for scrub or reconstruction). + * For plain (non-mirror) top-level vdevs (i.e. is_vdev is not a mirror), + * ic_vdev is the same as is_vdev. However, for mirror top-level vdevs, + * ic_vdev is a child of the mirror. + */ +typedef struct indirect_child { + abd_t *ic_data; + vdev_t *ic_vdev; +} indirect_child_t; + +/* + * The indirect_split_t represents one mapped segment of an i/o to the + * indirect vdev. For non-split (contiguously-mapped) blocks, there will be + * only one indirect_split_t, with is_split_offset==0 and is_size==io_size. + * For split blocks, there will be several of these. + */ +typedef struct indirect_split { + list_node_t is_node; /* link on iv_splits */ + + /* + * is_split_offset is the offset into the i/o. + * This is the sum of the previous splits' is_size's. + */ + uint64_t is_split_offset; + + vdev_t *is_vdev; /* top-level vdev */ + uint64_t is_target_offset; /* offset on is_vdev */ + uint64_t is_size; + int is_children; /* number of entries in is_child[] */ + + /* + * is_good_child is the child that we are currently using to + * attempt reconstruction. + */ + int is_good_child; + + indirect_child_t is_child[1]; /* variable-length */ +} indirect_split_t; + +/* + * The indirect_vsd_t is associated with each i/o to the indirect vdev. + * It is the "Vdev-Specific Data" in the zio_t's io_vsd. + */ +typedef struct indirect_vsd { + boolean_t iv_split_block; + boolean_t iv_reconstruct; + + list_t iv_splits; /* list of indirect_split_t's */ +} indirect_vsd_t; + +static void +vdev_indirect_map_free(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + indirect_split_t *is; + while ((is = list_head(&iv->iv_splits)) != NULL) { + for (int c = 0; c < is->is_children; c++) { + indirect_child_t *ic = &is->is_child[c]; + if (ic->ic_data != NULL) + abd_free(ic->ic_data); + } + list_remove(&iv->iv_splits, is); + kmem_free(is, + offsetof(indirect_split_t, is_child[is->is_children])); + } + kmem_free(iv, sizeof (*iv)); +} + +static const zio_vsd_ops_t vdev_indirect_vsd_ops = { + vdev_indirect_map_free, + zio_vsd_default_cksum_report +}; +/* * Mark the given offset and size as being obsolete. */ void @@ -818,12 +908,6 @@ vdev_indirect_close(vdev_t *vd) } /* ARGSUSED */ -static void -vdev_indirect_io_done(zio_t *zio) -{ -} - -/* ARGSUSED */ static int vdev_indirect_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, uint64_t *logical_ashift, uint64_t *physical_ashift) @@ -1067,39 +1151,473 @@ vdev_indirect_child_io_done(zio_t *zio) abd_put(zio->io_abd); } +/* + * This is a callback for vdev_indirect_remap() which allocates an + * indirect_split_t for each split segment and adds it to iv_splits. + */ static void -vdev_indirect_io_start_cb(uint64_t split_offset, vdev_t *vd, uint64_t offset, +vdev_indirect_gather_splits(uint64_t split_offset, vdev_t *vd, uint64_t offset, uint64_t size, void *arg) { zio_t *zio = arg; + indirect_vsd_t *iv = zio->io_vsd; ASSERT3P(vd, !=, NULL); if (vd->vdev_ops == &vdev_indirect_ops) return; - zio_nowait(zio_vdev_child_io(zio, NULL, vd, offset, - abd_get_offset(zio->io_abd, split_offset), - size, zio->io_type, zio->io_priority, - 0, vdev_indirect_child_io_done, zio)); + int n = 1; + if (vd->vdev_ops == &vdev_mirror_ops) + n = vd->vdev_children; + + indirect_split_t *is = + kmem_zalloc(offsetof(indirect_split_t, is_child[n]), KM_SLEEP); + + is->is_children = n; + is->is_size = size; + is->is_split_offset = split_offset; + is->is_target_offset = offset; + is->is_vdev = vd; + + /* + * Note that we only consider multiple copies of the data for + * *mirror* vdevs. We don't for "replacing" or "spare" vdevs, even + * though they use the same ops as mirror, because there's only one + * "good" copy under the replacing/spare. + */ + if (vd->vdev_ops == &vdev_mirror_ops) { + for (int i = 0; i < n; i++) { + is->is_child[i].ic_vdev = vd->vdev_child[i]; + } + } else { + is->is_child[0].ic_vdev = vd; + } + + list_insert_tail(&iv->iv_splits, is); } static void +vdev_indirect_read_split_done(zio_t *zio) +{ + indirect_child_t *ic = zio->io_private; + + if (zio->io_error != 0) { + /* + * Clear ic_data to indicate that we do not have data for this + * child. + */ + abd_free(ic->ic_data); + ic->ic_data = NULL; + } +} + +/* + * Issue reads for all copies (mirror children) of all splits. + */ +static void +vdev_indirect_read_all(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + for (int i = 0; i < is->is_children; i++) { + indirect_child_t *ic = &is->is_child[i]; + + if (!vdev_readable(ic->ic_vdev)) + continue; + + /* + * Note, we may read from a child whose DTL + * indicates that the data may not be present here. + * While this might result in a few i/os that will + * likely return incorrect data, it simplifies the + * code since we can treat scrub and resilver + * identically. (The incorrect data will be + * detected and ignored when we verify the + * checksum.) + */ + + ic->ic_data = abd_alloc_sametype(zio->io_abd, + is->is_size); + + zio_nowait(zio_vdev_child_io(zio, NULL, + ic->ic_vdev, is->is_target_offset, ic->ic_data, + is->is_size, zio->io_type, zio->io_priority, 0, + vdev_indirect_read_split_done, ic)); + } + } + iv->iv_reconstruct = B_TRUE; +} + +static void vdev_indirect_io_start(zio_t *zio) { spa_t *spa = zio->io_spa; + indirect_vsd_t *iv = kmem_zalloc(sizeof (*iv), KM_SLEEP); + list_create(&iv->iv_splits, + sizeof (indirect_split_t), offsetof(indirect_split_t, is_node)); + zio->io_vsd = iv; + zio->io_vsd_ops = &vdev_indirect_vsd_ops; + ASSERT(spa_config_held(spa, SCL_ALL, RW_READER) != 0); if (zio->io_type != ZIO_TYPE_READ) { ASSERT3U(zio->io_type, ==, ZIO_TYPE_WRITE); - ASSERT((zio->io_flags & - (ZIO_FLAG_SELF_HEAL | ZIO_FLAG_INDUCE_DAMAGE)) != 0); + /* + * Note: this code can handle other kinds of writes, + * but we don't expect them. + */ + ASSERT((zio->io_flags & (ZIO_FLAG_SELF_HEAL | + ZIO_FLAG_RESILVER | ZIO_FLAG_INDUCE_DAMAGE)) != 0); } vdev_indirect_remap(zio->io_vd, zio->io_offset, zio->io_size, - vdev_indirect_io_start_cb, zio); + vdev_indirect_gather_splits, zio); + indirect_split_t *first = list_head(&iv->iv_splits); + if (first->is_size == zio->io_size) { + /* + * This is not a split block; we are pointing to the entire + * data, which will checksum the same as the original data. + * Pass the BP down so that the child i/o can verify the + * checksum, and try a different location if available + * (e.g. on a mirror). + * + * While this special case could be handled the same as the + * general (split block) case, doing it this way ensures + * that the vast majority of blocks on indirect vdevs + * (which are not split) are handled identically to blocks + * on non-indirect vdevs. This allows us to be less strict + * about performance in the general (but rare) case. + */ + ASSERT0(first->is_split_offset); + ASSERT3P(list_next(&iv->iv_splits, first), ==, NULL); + zio_nowait(zio_vdev_child_io(zio, zio->io_bp, + first->is_vdev, first->is_target_offset, + abd_get_offset(zio->io_abd, 0), + zio->io_size, zio->io_type, zio->io_priority, 0, + vdev_indirect_child_io_done, zio)); + } else { + iv->iv_split_block = B_TRUE; + if (zio->io_flags & (ZIO_FLAG_SCRUB | ZIO_FLAG_RESILVER)) { + /* + * Read all copies. Note that for simplicity, + * we don't bother consulting the DTL in the + * resilver case. + */ + vdev_indirect_read_all(zio); + } else { + /* + * Read one copy of each split segment, from the + * top-level vdev. Since we don't know the + * checksum of each split individually, the child + * zio can't ensure that we get the right data. + * E.g. if it's a mirror, it will just read from a + * random (healthy) leaf vdev. We have to verify + * the checksum in vdev_indirect_io_done(). + */ + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + zio_nowait(zio_vdev_child_io(zio, NULL, + is->is_vdev, is->is_target_offset, + abd_get_offset(zio->io_abd, + is->is_split_offset), + is->is_size, zio->io_type, + zio->io_priority, 0, + vdev_indirect_child_io_done, zio)); + } + } + } + zio_execute(zio); +} + +/* + * Report a checksum error for a child. + */ +static void +vdev_indirect_checksum_error(zio_t *zio, + indirect_split_t *is, indirect_child_t *ic) +{ + vdev_t *vd = ic->ic_vdev; + + if (zio->io_flags & ZIO_FLAG_SPECULATIVE) + return; + + mutex_enter(&vd->vdev_stat_lock); + vd->vdev_stat.vs_checksum_errors++; + mutex_exit(&vd->vdev_stat_lock); + + zio_bad_cksum_t zbc = { 0 }; + void *bad_buf = abd_borrow_buf_copy(ic->ic_data, is->is_size); + abd_t *good_abd = is->is_child[is->is_good_child].ic_data; + void *good_buf = abd_borrow_buf_copy(good_abd, is->is_size); + zfs_ereport_post_checksum(zio->io_spa, vd, zio, + is->is_target_offset, is->is_size, good_buf, bad_buf, &zbc); + abd_return_buf(ic->ic_data, bad_buf, is->is_size); + abd_return_buf(good_abd, good_buf, is->is_size); +} + +/* + * Issue repair i/os for any incorrect copies. We do this by comparing + * each split segment's correct data (is_good_child's ic_data) with each + * other copy of the data. If they differ, then we overwrite the bad data + * with the good copy. Note that we do this without regard for the DTL's, + * which simplifies this code and also issues the optimal number of writes + * (based on which copies actually read bad data, as opposed to which we + * think might be wrong). For the same reason, we always use + * ZIO_FLAG_SELF_HEAL, to bypass the DTL check in zio_vdev_io_start(). + */ +static void +vdev_indirect_repair(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + enum zio_flag flags = ZIO_FLAG_IO_REPAIR; + + if (!(zio->io_flags & (ZIO_FLAG_SCRUB | ZIO_FLAG_RESILVER))) + flags |= ZIO_FLAG_SELF_HEAL; + + if (!spa_writeable(zio->io_spa)) + return; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + indirect_child_t *good_child = &is->is_child[is->is_good_child]; + + for (int c = 0; c < is->is_children; c++) { + indirect_child_t *ic = &is->is_child[c]; + if (ic == good_child) + continue; + if (ic->ic_data == NULL) + continue; + if (abd_cmp(good_child->ic_data, ic->ic_data, + is->is_size) == 0) + continue; + + zio_nowait(zio_vdev_child_io(zio, NULL, + ic->ic_vdev, is->is_target_offset, + good_child->ic_data, is->is_size, + ZIO_TYPE_WRITE, ZIO_PRIORITY_ASYNC_WRITE, + ZIO_FLAG_IO_REPAIR | ZIO_FLAG_SELF_HEAL, + NULL, NULL)); + + vdev_indirect_checksum_error(zio, is, ic); + } + } +} + +/* + * Report checksum errors on all children that we read from. + */ +static void +vdev_indirect_all_checksum_errors(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + if (zio->io_flags & ZIO_FLAG_SPECULATIVE) + return; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + for (int c = 0; c < is->is_children; c++) { + indirect_child_t *ic = &is->is_child[c]; + + if (ic->ic_data == NULL) + continue; + + vdev_t *vd = ic->ic_vdev; + + mutex_enter(&vd->vdev_stat_lock); + vd->vdev_stat.vs_checksum_errors++; + mutex_exit(&vd->vdev_stat_lock); + + zfs_ereport_post_checksum(zio->io_spa, vd, zio, + is->is_target_offset, is->is_size, + NULL, NULL, NULL); + } + } +} + +/* + * This function is called when we have read all copies of the data and need + * to try to find a combination of copies that gives us the right checksum. + * + * If we pointed to any mirror vdevs, this effectively does the job of the + * mirror. The mirror vdev code can't do its own job because we don't know + * the checksum of each split segment individually. We have to try every + * combination of copies of split segments, until we find one that checksums + * correctly. (Or until we have tried all combinations, or have tried + * 2^zfs_reconstruct_indirect_segments_max combinations. In these cases we + * set io_error to ECKSUM to propagate the error up to the user.) + * + * For example, if we have 3 segments in the split, + * and each points to a 2-way mirror, we will have the following pieces of + * data: + * + * | mirror child + * split | [0] [1] + * ======|===================== + * A | data_A_0 data_A_1 + * B | data_B_0 data_B_1 + * C | data_C_0 data_C_1 + * + * We will try the following (mirror children)^(number of splits) (2^3=8) + * combinations, which is similar to bitwise-little-endian counting in + * binary. In general each "digit" corresponds to a split segment, and the + * base of each digit is is_children, which can be different for each + * digit. + * + * "low bit" "high bit" + * v v + * data_A_0 data_B_0 data_C_0 + * data_A_1 data_B_0 data_C_0 + * data_A_0 data_B_1 data_C_0 + * data_A_1 data_B_1 data_C_0 + * data_A_0 data_B_0 data_C_1 + * data_A_1 data_B_0 data_C_1 + * data_A_0 data_B_1 data_C_1 + * data_A_1 data_B_1 data_C_1 + * + * Note that the split segments may be on the same or different top-level + * vdevs. In either case, we try lots of combinations (see + * zfs_reconstruct_indirect_segments_max). This ensures that if a mirror has + * small silent errors on all of its children, we can still reconstruct the + * correct data, as long as those errors are at sufficiently-separated + * offsets (specifically, separated by the largest block size - default of + * 128KB, but up to 16MB). + */ +static void +vdev_indirect_reconstruct_io_done(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + uint64_t attempts = 0; + uint64_t attempts_max = 1ULL << zfs_reconstruct_indirect_segments_max; + int segments = 0; + + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) + segments++; + + for (;;) { + /* copy data from splits to main zio */ + int ret; + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + + /* + * If this child failed, its ic_data will be NULL. + * Skip this combination. + */ + if (is->is_child[is->is_good_child].ic_data == NULL) { + ret = EIO; + goto next; + } + + abd_copy_off(zio->io_abd, + is->is_child[is->is_good_child].ic_data, + is->is_split_offset, 0, is->is_size); + } + + /* See if this checksum matches. */ + zio_bad_cksum_t zbc; + ret = zio_checksum_error(zio, &zbc); + if (ret == 0) { + /* Found a matching checksum. Issue repair i/os. */ + vdev_indirect_repair(zio); + zio_checksum_verified(zio); + return; + } + + /* + * Checksum failed; try a different combination of split + * children. + */ + boolean_t more; +next: + more = B_FALSE; + if (segments <= zfs_reconstruct_indirect_segments_max) { + /* + * There are relatively few segments, so + * deterministically check all combinations. We do + * this by by adding one to the first split's + * good_child. If it overflows, then "carry over" to + * the next split (like counting in base is_children, + * but each digit can have a different base). + */ + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + is->is_good_child++; + if (is->is_good_child < is->is_children) { + more = B_TRUE; + break; + } + is->is_good_child = 0; + } + } else if (++attempts < attempts_max) { + /* + * There are too many combinations to try all of them + * in a reasonable amount of time, so try a fixed + * number of random combinations, after which we'll + * consider the block unrecoverable. + */ + for (indirect_split_t *is = list_head(&iv->iv_splits); + is != NULL; is = list_next(&iv->iv_splits, is)) { + is->is_good_child = + spa_get_random(is->is_children); + } + more = B_TRUE; + } + if (!more) { + /* All combinations failed. */ + zio->io_error = ret; + vdev_indirect_all_checksum_errors(zio); + zio_checksum_verified(zio); + return; + } + } +} + +static void +vdev_indirect_io_done(zio_t *zio) +{ + indirect_vsd_t *iv = zio->io_vsd; + + if (iv->iv_reconstruct) { + /* + * We have read all copies of the data (e.g. from mirrors), + * either because this was a scrub/resilver, or because the + * one-copy read didn't checksum correctly. + */ + vdev_indirect_reconstruct_io_done(zio); + return; + } + + if (!iv->iv_split_block) { + /* + * This was not a split block, so we passed the BP down, + * and the checksum was handled by the (one) child zio. + */ + return; + } + + zio_bad_cksum_t zbc; + int ret = zio_checksum_error(zio, &zbc); + if (ret == 0) { + zio_checksum_verified(zio); + return; + } + + /* + * The checksum didn't match. Read all copies of all splits, and + * then we will try to reconstruct. The next time + * vdev_indirect_io_done() is called, iv_reconstruct will be set. + */ + vdev_indirect_read_all(zio); + + zio_vdev_io_redone(zio); } vdev_ops_t vdev_indirect_ops = { Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c Tue Jul 31 00:25:39 2018 (r336951) @@ -24,7 +24,7 @@ */ /* - * Copyright (c) 2012, 2015 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. */ #include @@ -516,13 +516,16 @@ vdev_mirror_io_start(zio_t *zio) } if (zio->io_type == ZIO_TYPE_READ) { - if ((zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering && + if (zio->io_bp != NULL && + (zio->io_flags & ZIO_FLAG_SCRUB) && !mm->mm_resilvering && mm->mm_children > 1) { /* - * For scrubbing reads we need to allocate a read - * buffer for each child and issue reads to all - * children. If any child succeeds, it will copy its - * data into zio->io_data in vdev_mirror_scrub_done. + * For scrubbing reads (if we can verify the + * checksum here, as indicated by io_bp being + * non-NULL) we need to allocate a read buffer for + * each child and issue reads to all children. If + * any child succeeds, it will copy its data into + * zio->io_data in vdev_mirror_scrub_done. */ for (c = 0; c < mm->mm_children; c++) { mc = &mm->mm_child[c]; @@ -677,7 +680,21 @@ vdev_mirror_io_done(zio_t *zio) if (mc->mc_error == 0) { if (mc->mc_tried) continue; + /* + * We didn't try this child. We need to + * repair it if: + * 1. it's a scrub (in which case we have + * tried everything that was healthy) + * - or - + * 2. it's an indirect vdev (in which case + * it could point to any other vdev, which + * might have a bad DTL) + * - or - + * 3. the DTL indicates that this data is + * missing from this vdev + */ if (!(zio->io_flags & ZIO_FLAG_SCRUB) && + mc->mc_vd->vdev_ops != &vdev_indirect_ops && !vdev_dtl_contains(mc->mc_vd, DTL_PARTIAL, zio->io_txg, 1)) continue; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 00:13:04 2018 (r336950) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 00:25:39 2018 (r336951) @@ -83,18 +83,12 @@ typedef struct vdev_copy_arg { kmutex_t vca_lock; } vdev_copy_arg_t; -typedef struct vdev_copy_seg_arg { - vdev_copy_arg_t *vcsa_copy_arg; - uint64_t vcsa_txg; - dva_t *vcsa_dest_dva; - blkptr_t *vcsa_dest_bp; -} vdev_copy_seg_arg_t; - /* - * The maximum amount of allowed data we're allowed to copy from a device - * at a time when removing it. + * The maximum amount of memory we can use for outstanding i/o while + * doing a device removal. This determines how much i/o we can have + * in flight concurrently. */ -int zfs_remove_max_copy_bytes = 8 * 1024 * 1024; +int zfs_remove_max_copy_bytes = 64 * 1024 * 1024; /* * The largest contiguous segment that we will attempt to allocate when @@ -176,7 +170,7 @@ spa_vdev_removal_create(vdev_t *vd) mutex_init(&svr->svr_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&svr->svr_cv, NULL, CV_DEFAULT, NULL); svr->svr_allocd_segs = range_tree_create(NULL, NULL); - svr->svr_vdev = vd; + svr->svr_vdev_id = vd->vdev_id; for (int i = 0; i < TXG_SIZE; i++) { svr->svr_frees[i] = range_tree_create(NULL, NULL); @@ -218,9 +212,10 @@ spa_vdev_removal_destroy(spa_vdev_removal_t *svr) static void vdev_remove_initiate_sync(void *arg, dmu_tx_t *tx) { - vdev_t *vd = arg; + int vdev_id = (uintptr_t)arg; + spa_t *spa = dmu_tx_pool(tx)->dp_spa; + vdev_t *vd = vdev_lookup_top(spa, vdev_id); vdev_indirect_config_t *vic = &vd->vdev_indirect_config; - spa_t *spa = vd->vdev_spa; objset_t *mos = spa->spa_dsl_pool->dp_meta_objset; spa_vdev_removal_t *svr = NULL; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Tue Jul 31 00:34:40 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E82D310662BB; Tue, 31 Jul 2018 00:34:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B3218F08C; Tue, 31 Jul 2018 00:34:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 77C3043D9; Tue, 31 Jul 2018 00:34:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0YdTE099743; Tue, 31 Jul 2018 00:34:39 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0YdAp099742; Tue, 31 Jul 2018 00:34:39 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310034.w6V0YdAp099742@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:34:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336952 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 336952 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:34:40 -0000 Author: mav Date: Tue Jul 31 00:34:39 2018 New Revision: 336952 URL: https://svnweb.freebsd.org/changeset/base/336952 Log: 9192 explicitly pass good_writes to vdev_uberblock/label_sync Currently vdev_label_sync and vdev_uberblock_sync take a zio_t and assume that its io_private is a pointer to the good_writes count. They should instead accept this argument explicitly. illumos/illumos-gate@a3b5583021b7b45676bf1f0cc68adf7a97900b56 Reviewed by: Pavel Zakharov Reviewed by: George Wilson Approved by: Richard Lowe Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Tue Jul 31 00:25:39 2018 (r336951) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Tue Jul 31 00:34:39 2018 (r336952) @@ -1117,10 +1117,13 @@ vdev_uberblock_sync_done(zio_t *zio) * Write the uberblock to all labels of all leaves of the specified vdev. */ static void -vdev_uberblock_sync(zio_t *zio, uberblock_t *ub, vdev_t *vd, int flags) +vdev_uberblock_sync(zio_t *zio, uint64_t *good_writes, + uberblock_t *ub, vdev_t *vd, int flags) { - for (uint64_t c = 0; c < vd->vdev_children; c++) - vdev_uberblock_sync(zio, ub, vd->vdev_child[c], flags); + for (uint64_t c = 0; c < vd->vdev_children; c++) { + vdev_uberblock_sync(zio, good_writes, + ub, vd->vdev_child[c], flags); + } if (!vd->vdev_ops->vdev_op_leaf) return; @@ -1138,7 +1141,7 @@ vdev_uberblock_sync(zio_t *zio, uberblock_t *ub, vdev_ for (int l = 0; l < VDEV_LABELS; l++) vdev_label_write(zio, vd, l, ub_abd, VDEV_UBERBLOCK_OFFSET(vd, n), VDEV_UBERBLOCK_SIZE(vd), - vdev_uberblock_sync_done, zio->io_private, + vdev_uberblock_sync_done, good_writes, flags | ZIO_FLAG_DONT_PROPAGATE); abd_free(ub_abd); @@ -1152,10 +1155,10 @@ vdev_uberblock_sync_list(vdev_t **svd, int svdcount, u zio_t *zio; uint64_t good_writes = 0; - zio = zio_root(spa, NULL, &good_writes, flags); + zio = zio_root(spa, NULL, NULL, flags); for (int v = 0; v < svdcount; v++) - vdev_uberblock_sync(zio, ub, svd[v], flags); + vdev_uberblock_sync(zio, &good_writes, ub, svd[v], flags); (void) zio_wait(zio); @@ -1216,7 +1219,8 @@ vdev_label_sync_ignore_done(zio_t *zio) * Write all even or odd labels to all leaves of the specified vdev. */ static void -vdev_label_sync(zio_t *zio, vdev_t *vd, int l, uint64_t txg, int flags) +vdev_label_sync(zio_t *zio, uint64_t *good_writes, + vdev_t *vd, int l, uint64_t txg, int flags) { nvlist_t *label; vdev_phys_t *vp; @@ -1224,8 +1228,10 @@ vdev_label_sync(zio_t *zio, vdev_t *vd, int l, uint64_ char *buf; size_t buflen; - for (int c = 0; c < vd->vdev_children; c++) - vdev_label_sync(zio, vd->vdev_child[c], l, txg, flags); + for (int c = 0; c < vd->vdev_children; c++) { + vdev_label_sync(zio, good_writes, + vd->vdev_child[c], l, txg, flags); + } if (!vd->vdev_ops->vdev_op_leaf) return; @@ -1250,7 +1256,7 @@ vdev_label_sync(zio_t *zio, vdev_t *vd, int l, uint64_ vdev_label_write(zio, vd, l, vp_abd, offsetof(vdev_label_t, vl_vdev_phys), sizeof (vdev_phys_t), - vdev_label_sync_done, zio->io_private, + vdev_label_sync_done, good_writes, flags | ZIO_FLAG_DONT_PROPAGATE); } } @@ -1282,7 +1288,7 @@ vdev_label_sync_list(spa_t *spa, int l, uint64_t txg, (vd->vdev_islog || vd->vdev_aux != NULL) ? vdev_label_sync_ignore_done : vdev_label_sync_top_done, good_writes, flags); - vdev_label_sync(vio, vd, l, txg, flags); + vdev_label_sync(vio, good_writes, vd, l, txg, flags); zio_nowait(vio); } From owner-svn-src-all@freebsd.org Tue Jul 31 00:37:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D0BC10663F5; Tue, 31 Jul 2018 00:37:27 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 40F268F2A3; Tue, 31 Jul 2018 00:37:27 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0350243DD; Tue, 31 Jul 2018 00:37:27 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0bQsK099925; Tue, 31 Jul 2018 00:37:26 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0bPlm099916; Tue, 31 Jul 2018 00:37:25 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201807310037.w6V0bPlm099916@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Tue, 31 Jul 2018 00:37:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r336953 - in stable: 10/usr.bin/mail 11/usr.bin/mail X-SVN-Group: stable-10 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in stable: 10/usr.bin/mail 11/usr.bin/mail X-SVN-Commit-Revision: 336953 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:37:28 -0000 Author: markj Date: Tue Jul 31 00:37:25 2018 New Revision: 336953 URL: https://svnweb.freebsd.org/changeset/base/336953 Log: Revert r335693, r335694, r335695 by eadler. They cause mail(1) to crash in certain scenarios. PR: 230196 Reported by: Pete French Modified: stable/10/usr.bin/mail/cmd1.c stable/10/usr.bin/mail/cmd2.c stable/10/usr.bin/mail/cmd3.c stable/10/usr.bin/mail/edit.c stable/10/usr.bin/mail/extern.h stable/10/usr.bin/mail/fio.c stable/10/usr.bin/mail/getname.c stable/10/usr.bin/mail/popen.c Directory Properties: stable/10/ (props changed) Changes in other areas also in this revision: Modified: stable/11/usr.bin/mail/cmd1.c stable/11/usr.bin/mail/cmd2.c stable/11/usr.bin/mail/cmd3.c stable/11/usr.bin/mail/edit.c stable/11/usr.bin/mail/extern.h stable/11/usr.bin/mail/fio.c stable/11/usr.bin/mail/getname.c stable/11/usr.bin/mail/popen.c Directory Properties: stable/11/ (props changed) Modified: stable/10/usr.bin/mail/cmd1.c ============================================================================== --- stable/10/usr.bin/mail/cmd1.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/cmd1.c Tue Jul 31 00:37:25 2018 (r336953) @@ -439,7 +439,7 @@ folders(void) } if ((cmd = value("LISTER")) == NULL) cmd = "ls"; - (void)run_command(cmd, 0, -1, -1, dirname, NULL); + (void)run_command(cmd, 0, -1, -1, dirname, NULL, NULL); return (0); } Modified: stable/10/usr.bin/mail/cmd2.c ============================================================================== --- stable/10/usr.bin/mail/cmd2.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/cmd2.c Tue Jul 31 00:37:25 2018 (r336953) @@ -130,9 +130,8 @@ hitit: * so we can discard when the user quits. */ int -save(void *v) +save(char str[]) { - char *str = v; return (save1(str, 1, "save", saveignore)); } @@ -141,9 +140,8 @@ save(void *v) * Copy a message to a file without affected its saved-ness */ int -copycmd(void *v) +copycmd(char str[]) { - char *str = v; return (save1(str, 0, "copy", saveignore)); } Modified: stable/10/usr.bin/mail/cmd3.c ============================================================================== --- stable/10/usr.bin/mail/cmd3.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/cmd3.c Tue Jul 31 00:37:25 2018 (r336953) @@ -79,7 +79,7 @@ dosh(char *str __unused) if ((sh = value("SHELL")) == NULL) sh = _PATH_CSHELL; - (void)run_command(sh, 0, -1, -1, NULL); + (void)run_command(sh, 0, -1, -1, NULL, NULL, NULL); (void)signal(SIGINT, sigint); printf("\n"); return (0); @@ -102,7 +102,7 @@ bangexp(char *str, size_t strsize) n = sizeof(bangbuf); while (*cp != '\0') { if (*cp == '!') { - if (n < (int)strlen(lastbang)) { + if (n < strlen(lastbang)) { overf: printf("Command buffer overflow\n"); return (-1); Modified: stable/10/usr.bin/mail/edit.c ============================================================================== --- stable/10/usr.bin/mail/edit.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/edit.c Tue Jul 31 00:37:25 2018 (r336953) @@ -180,7 +180,7 @@ run_editor(FILE *fp, off_t size, int type, int readonl nf = NULL; if ((edit = value(type == 'e' ? "EDITOR" : "VISUAL")) == NULL) edit = type == 'e' ? _PATH_EX : _PATH_VI; - if (run_command(edit, 0, -1, -1, tempname, NULL) < 0) { + if (run_command(edit, 0, -1, -1, tempname, NULL, NULL) < 0) { (void)rm(tempname); goto out; } Modified: stable/10/usr.bin/mail/extern.h ============================================================================== --- stable/10/usr.bin/mail/extern.h Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/extern.h Tue Jul 31 00:37:25 2018 (r336953) @@ -49,7 +49,7 @@ char *copyin(char *, char **); char *detract(struct name *, int); char *expand(char *); char *getdeadletter(void); -char *getname(uid_t); +char *getname(int); char *hfield(const char *, struct message *); FILE *infix(struct header *, FILE *); char *ishfield(char [], char *, const char *); @@ -95,7 +95,7 @@ void collhup(int); void collint(int); void collstop(int); void commands(void); -int copycmd(void *v); +int copycmd(char []); int core(void); int count(struct name *); int delete(int []); @@ -130,7 +130,7 @@ int getfold(char *, int); int gethfield(FILE *, char [], int, char **); int getmsglist(char *, int *, int); int getrawlist(char [], char **, int); -uid_t getuserid(char []); +int getuserid(char []); int grabh(struct header *, int); int group(char **); void hangup(int); @@ -198,8 +198,8 @@ int respond(int *); int retfield(char *[]); int rexit(int); int rm(char *); -int run_command(char *, sigset_t *, int, int, ...); -int save(void *v); +int run_command(char *, sigset_t *, int, int, char *, char *, char *); +int save(char []); int save1(char [], int, const char *, struct ignoretab *); void savedeadletter(FILE *); int saveigfield(char *[]); @@ -223,7 +223,7 @@ void sort(char **); int source(char **); void spreserve(void); void sreset(void); -int start_command(char *, sigset_t *, int, int, ...); +int start_command(char *, sigset_t *, int, int, char *, char *, char *); void statusput(struct message *, FILE *, char *); void stop(int); int stouch(int []); Modified: stable/10/usr.bin/mail/fio.c ============================================================================== --- stable/10/usr.bin/mail/fio.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/fio.c Tue Jul 31 00:37:25 2018 (r336953) @@ -235,7 +235,7 @@ makemessage(FILE *f, int omsgCount) size -= (omsgCount + 1) * sizeof(struct message); (void)fflush(f); (void)lseek(fileno(f), (off_t)sizeof(*message), 0); - if (read(fileno(f), (void *)&message[omsgCount], size) != size) + if (read(fileno(f), (char *)&message[omsgCount], size) != size) errx(1, "Message temporary file corrupted"); message[msgCount].m_size = 0; message[msgCount].m_lines = 0; Modified: stable/10/usr.bin/mail/getname.c ============================================================================== --- stable/10/usr.bin/mail/getname.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/getname.c Tue Jul 31 00:37:25 2018 (r336953) @@ -45,7 +45,7 @@ __FBSDID("$FreeBSD$"); * Search the passwd file for a uid. Return name on success, NULL on failure. */ char * -getname(uid_t uid) +getname(int uid) { struct passwd *pw; @@ -58,7 +58,7 @@ getname(uid_t uid) * Convert the passed name to a user id and return it. Return -1 * on error. */ -uid_t +int getuserid(char name[]) { struct passwd *pw; Modified: stable/10/usr.bin/mail/popen.c ============================================================================== --- stable/10/usr.bin/mail/popen.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/10/usr.bin/mail/popen.c Tue Jul 31 00:37:25 2018 (r336953) @@ -38,8 +38,6 @@ __FBSDID("$FreeBSD$"); #include "rcv.h" #include #include -#include -#include #include "extern.h" #define READ 0 @@ -48,23 +46,22 @@ __FBSDID("$FreeBSD$"); struct fp { FILE *fp; int pipe; - pid_t pid; + int pid; struct fp *link; }; static struct fp *fp_head; struct child { - pid_t pid; + int pid; char done; char free; int status; struct child *link; }; -static struct child *child, *child_freelist = NULL; - +static struct child *child; +static struct child *findchild(int); static void delchild(struct child *); -static pid_t file_pid(FILE *); -static pid_t start_commandv(char *, sigset_t *, int, int, va_list); +static int file_pid(FILE *); FILE * Fopen(const char *path, const char *mode) @@ -93,7 +90,6 @@ Fdopen(int fd, const char *mode) int Fclose(FILE *fp) { - unregister_file(fp); return (fclose(fp)); } @@ -103,7 +99,7 @@ Popen(char *cmd, const char *mode) { int p[2]; int myside, hisside, fd0, fd1; - pid_t pid; + int pid; sigset_t nset; FILE *fp; @@ -113,15 +109,15 @@ Popen(char *cmd, const char *mode) (void)fcntl(p[WRITE], F_SETFD, 1); if (*mode == 'r') { myside = p[READ]; - hisside = fd0 = fd1 = p[WRITE]; + fd0 = -1; + hisside = fd1 = p[WRITE]; } else { myside = p[WRITE]; hisside = fd0 = p[READ]; fd1 = -1; } (void)sigemptyset(&nset); - pid = start_command(value("SHELL"), &nset, fd0, fd1, "-c", cmd, NULL); - if (pid < 0) { + if ((pid = start_command(cmd, &nset, fd0, fd1, NULL, NULL, NULL)) < 0) { (void)close(p[READ]); (void)close(p[WRITE]); return (NULL); @@ -162,7 +158,7 @@ close_all_files(void) } void -register_file(FILE *fp, int pipe, pid_t pid) +register_file(FILE *fp, int pipe, int pid) { struct fp *fpp; @@ -190,7 +186,7 @@ unregister_file(FILE *fp) /*NOTREACHED*/ } -pid_t +int file_pid(FILE *fp) { struct fp *p; @@ -204,17 +200,30 @@ file_pid(FILE *fp) /* * Run a command without a shell, with optional arguments and splicing - * of stdin (-1 means none) and stdout. The command name can be a sequence - * of words. + * of stdin and stdout. The command name can be a sequence of words. * Signals must be handled by the caller. - * "nset" contains the signals to ignore in the new process. - * SIGINT is enabled unless it's in "nset". + * "Mask" contains the signals to ignore in the new process. + * SIGINT is enabled unless it's in the mask. */ -static pid_t -start_commandv(char *cmd, sigset_t *nset, int infd, int outfd, va_list args) +/*VARARGS4*/ +int +run_command(char *cmd, sigset_t *mask, int infd, int outfd, char *a0, + char *a1, char *a2) { - pid_t pid; + int pid; + if ((pid = start_command(cmd, mask, infd, outfd, a0, a1, a2)) < 0) + return (-1); + return (wait_command(pid)); +} + +/*VARARGS4*/ +int +start_command(char *cmd, sigset_t *mask, int infd, int outfd, char *a0, + char *a1, char *a2) +{ + int pid; + if ((pid = fork()) < 0) { warn("fork"); return (-1); @@ -223,10 +232,11 @@ start_commandv(char *cmd, sigset_t *nset, int infd, in char *argv[100]; int i = getrawlist(cmd, argv, sizeof(argv) / sizeof(*argv)); - while ((argv[i++] = va_arg(args, char *))) - ; - argv[i] = NULL; - prepare_child(nset, infd, outfd); + if ((argv[i++] = a0) != NULL && + (argv[i++] = a1) != NULL && + (argv[i++] = a2) != NULL) + argv[i] = NULL; + prepare_child(mask, infd, outfd); execvp(argv[0], argv); warn("%s", argv[0]); _exit(1); @@ -234,32 +244,6 @@ start_commandv(char *cmd, sigset_t *nset, int infd, in return (pid); } -int -run_command(char *cmd, sigset_t *nset, int infd, int outfd, ...) -{ - pid_t pid; - va_list args; - - va_start(args, outfd); - pid = start_commandv(cmd, nset, infd, outfd, args); - va_end(args); - if (pid < 0) - return -1; - return wait_command(pid); -} - -int -start_command(char *cmd, sigset_t *nset, int infd, int outfd, ...) -{ - va_list args; - int r; - - va_start(args, outfd); - r = start_commandv(cmd, nset, infd, outfd, args); - va_end(args); - return r; -} - void prepare_child(sigset_t *nset, int infd, int outfd) { @@ -284,7 +268,7 @@ prepare_child(sigset_t *nset, int infd, int outfd) } int -wait_command(pid_t pid) +wait_command(int pid) { if (wait_child(pid) < 0) { @@ -295,7 +279,7 @@ wait_command(pid_t pid) } static struct child * -findchild(pid_t pid, int dont_alloc) +findchild(int pid) { struct child **cpp; @@ -303,16 +287,9 @@ findchild(pid_t pid, int dont_alloc) cpp = &(*cpp)->link) ; if (*cpp == NULL) { - if (dont_alloc) - return(NULL); - if (child_freelist) { - *cpp = child_freelist; - child_freelist = (*cpp)->link; - } else { - *cpp = malloc(sizeof(struct child)); - if (*cpp == NULL) - err(1, "malloc"); - } + *cpp = malloc(sizeof(struct child)); + if (*cpp == NULL) + err(1, "Out of memory"); (*cpp)->pid = pid; (*cpp)->done = (*cpp)->free = 0; (*cpp)->link = NULL; @@ -328,22 +305,19 @@ delchild(struct child *cp) for (cpp = &child; *cpp != cp; cpp = &(*cpp)->link) ; *cpp = cp->link; - cp->link = child_freelist; - child_freelist = cp; + (void)free(cp); } /*ARGSUSED*/ void sigchild(int signo __unused) { - pid_t pid; + int pid; int status; struct child *cp; - int save_errno; - save_errno = errno; while ((pid = waitpid(-1, &status, WNOHANG)) > 0) { - cp = findchild(pid, 1); + cp = findchild(pid); if (cp->free) delchild(cp); else { @@ -351,7 +325,6 @@ sigchild(int signo __unused) cp->status = status; } } - errno = save_errno; } int wait_status; @@ -360,51 +333,41 @@ int wait_status; * Wait for a specific child to die. */ int -wait_child(pid_t pid) +wait_child(int pid) { - struct child *cp; sigset_t nset, oset; - pid_t rv = 0; + struct child *cp; (void)sigemptyset(&nset); (void)sigaddset(&nset, SIGCHLD); - (void)sigprocmask(SIG_BLOCK, &nset, &oset); - /* - * If we have not already waited on the pid (via sigchild) - * wait on it now. Otherwise, use the wait status stashed - * by sigchild. - */ - cp = findchild(pid, 1); - if (cp == NULL || !cp->done) - rv = waitpid(pid, &wait_status, 0); - else - wait_status = cp->status; - if (cp != NULL) - delchild(cp); + (void)sigprocmask(SIG_BLOCK, &nset, &oset); + + cp = findchild(pid); + + while (!cp->done) + (void)sigsuspend(&oset); + wait_status = cp->status; + delchild(cp); (void)sigprocmask(SIG_SETMASK, &oset, NULL); - if (rv == -1 || (WIFEXITED(wait_status) && WEXITSTATUS(wait_status))) - return -1; - else - return 0; + return ((WIFEXITED(wait_status) && WEXITSTATUS(wait_status)) ? -1 : 0); } /* * Mark a child as don't care. */ void -free_child(pid_t pid) +free_child(int pid) { - struct child *cp; sigset_t nset, oset; + struct child *cp = findchild(pid); (void)sigemptyset(&nset); (void)sigaddset(&nset, SIGCHLD); - (void)sigprocmask(SIG_BLOCK, &nset, &oset); - if ((cp = findchild(pid, 0)) != NULL) { - if (cp->done) - delchild(cp); - else - cp->free = 1; - } + (void)sigprocmask(SIG_BLOCK, &nset, &oset); + + if (cp->done) + delchild(cp); + else + cp->free = 1; (void)sigprocmask(SIG_SETMASK, &oset, NULL); } From owner-svn-src-all@freebsd.org Tue Jul 31 00:37:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7674510663FD; Tue, 31 Jul 2018 00:37:29 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2752F8F2A5; Tue, 31 Jul 2018 00:37:29 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E10A543DE; Tue, 31 Jul 2018 00:37:28 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0bSOY099940; Tue, 31 Jul 2018 00:37:28 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0bRwX099930; Tue, 31 Jul 2018 00:37:27 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201807310037.w6V0bRwX099930@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Tue, 31 Jul 2018 00:37:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336953 - in stable: 10/usr.bin/mail 11/usr.bin/mail X-SVN-Group: stable-11 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in stable: 10/usr.bin/mail 11/usr.bin/mail X-SVN-Commit-Revision: 336953 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:37:29 -0000 Author: markj Date: Tue Jul 31 00:37:25 2018 New Revision: 336953 URL: https://svnweb.freebsd.org/changeset/base/336953 Log: Revert r335693, r335694, r335695 by eadler. They cause mail(1) to crash in certain scenarios. PR: 230196 Reported by: Pete French Modified: stable/11/usr.bin/mail/cmd1.c stable/11/usr.bin/mail/cmd2.c stable/11/usr.bin/mail/cmd3.c stable/11/usr.bin/mail/edit.c stable/11/usr.bin/mail/extern.h stable/11/usr.bin/mail/fio.c stable/11/usr.bin/mail/getname.c stable/11/usr.bin/mail/popen.c Directory Properties: stable/11/ (props changed) Changes in other areas also in this revision: Modified: stable/10/usr.bin/mail/cmd1.c stable/10/usr.bin/mail/cmd2.c stable/10/usr.bin/mail/cmd3.c stable/10/usr.bin/mail/edit.c stable/10/usr.bin/mail/extern.h stable/10/usr.bin/mail/fio.c stable/10/usr.bin/mail/getname.c stable/10/usr.bin/mail/popen.c Directory Properties: stable/10/ (props changed) Modified: stable/11/usr.bin/mail/cmd1.c ============================================================================== --- stable/11/usr.bin/mail/cmd1.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/cmd1.c Tue Jul 31 00:37:25 2018 (r336953) @@ -439,7 +439,7 @@ folders(void) } if ((cmd = value("LISTER")) == NULL) cmd = "ls"; - (void)run_command(cmd, 0, -1, -1, dirname, NULL); + (void)run_command(cmd, 0, -1, -1, dirname, NULL, NULL); return (0); } Modified: stable/11/usr.bin/mail/cmd2.c ============================================================================== --- stable/11/usr.bin/mail/cmd2.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/cmd2.c Tue Jul 31 00:37:25 2018 (r336953) @@ -130,9 +130,8 @@ hitit: * so we can discard when the user quits. */ int -save(void *v) +save(char str[]) { - char *str = v; return (save1(str, 1, "save", saveignore)); } @@ -141,9 +140,8 @@ save(void *v) * Copy a message to a file without affected its saved-ness */ int -copycmd(void *v) +copycmd(char str[]) { - char *str = v; return (save1(str, 0, "copy", saveignore)); } Modified: stable/11/usr.bin/mail/cmd3.c ============================================================================== --- stable/11/usr.bin/mail/cmd3.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/cmd3.c Tue Jul 31 00:37:25 2018 (r336953) @@ -79,7 +79,7 @@ dosh(char *str __unused) if ((sh = value("SHELL")) == NULL) sh = _PATH_CSHELL; - (void)run_command(sh, 0, -1, -1, NULL); + (void)run_command(sh, 0, -1, -1, NULL, NULL, NULL); (void)signal(SIGINT, sigint); printf("\n"); return (0); @@ -102,7 +102,7 @@ bangexp(char *str, size_t strsize) n = sizeof(bangbuf); while (*cp != '\0') { if (*cp == '!') { - if (n < (int)strlen(lastbang)) { + if (n < strlen(lastbang)) { overf: printf("Command buffer overflow\n"); return (-1); Modified: stable/11/usr.bin/mail/edit.c ============================================================================== --- stable/11/usr.bin/mail/edit.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/edit.c Tue Jul 31 00:37:25 2018 (r336953) @@ -180,7 +180,7 @@ run_editor(FILE *fp, off_t size, int type, int readonl nf = NULL; if ((edit = value(type == 'e' ? "EDITOR" : "VISUAL")) == NULL) edit = type == 'e' ? _PATH_EX : _PATH_VI; - if (run_command(edit, 0, -1, -1, tempname, NULL) < 0) { + if (run_command(edit, 0, -1, -1, tempname, NULL, NULL) < 0) { (void)rm(tempname); goto out; } Modified: stable/11/usr.bin/mail/extern.h ============================================================================== --- stable/11/usr.bin/mail/extern.h Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/extern.h Tue Jul 31 00:37:25 2018 (r336953) @@ -49,7 +49,7 @@ char *copyin(char *, char **); char *detract(struct name *, int); char *expand(char *); char *getdeadletter(void); -char *getname(uid_t); +char *getname(int); char *hfield(const char *, struct message *); FILE *infix(struct header *, FILE *); char *ishfield(char [], char *, const char *); @@ -95,7 +95,7 @@ void collhup(int); void collint(int); void collstop(int); void commands(void); -int copycmd(void *v); +int copycmd(char []); int core(void); int count(struct name *); int delete(int []); @@ -130,7 +130,7 @@ int getfold(char *, int); int gethfield(FILE *, char [], int, char **); int getmsglist(char *, int *, int); int getrawlist(char [], char **, int); -uid_t getuserid(char []); +int getuserid(char []); int grabh(struct header *, int); int group(char **); void hangup(int); @@ -198,8 +198,8 @@ int respond(int *); int retfield(char *[]); int rexit(int); int rm(char *); -int run_command(char *, sigset_t *, int, int, ...); -int save(void *v); +int run_command(char *, sigset_t *, int, int, char *, char *, char *); +int save(char []); int save1(char [], int, const char *, struct ignoretab *); void savedeadletter(FILE *); int saveigfield(char *[]); @@ -223,7 +223,7 @@ void sort(char **); int source(char **); void spreserve(void); void sreset(void); -int start_command(char *, sigset_t *, int, int, ...); +int start_command(char *, sigset_t *, int, int, char *, char *, char *); void statusput(struct message *, FILE *, char *); void stop(int); int stouch(int []); Modified: stable/11/usr.bin/mail/fio.c ============================================================================== --- stable/11/usr.bin/mail/fio.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/fio.c Tue Jul 31 00:37:25 2018 (r336953) @@ -235,7 +235,7 @@ makemessage(FILE *f, int omsgCount) size -= (omsgCount + 1) * sizeof(struct message); (void)fflush(f); (void)lseek(fileno(f), (off_t)sizeof(*message), 0); - if (read(fileno(f), (void *)&message[omsgCount], size) != size) + if (read(fileno(f), (char *)&message[omsgCount], size) != size) errx(1, "Message temporary file corrupted"); message[msgCount].m_size = 0; message[msgCount].m_lines = 0; Modified: stable/11/usr.bin/mail/getname.c ============================================================================== --- stable/11/usr.bin/mail/getname.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/getname.c Tue Jul 31 00:37:25 2018 (r336953) @@ -45,7 +45,7 @@ __FBSDID("$FreeBSD$"); * Search the passwd file for a uid. Return name on success, NULL on failure. */ char * -getname(uid_t uid) +getname(int uid) { struct passwd *pw; @@ -58,7 +58,7 @@ getname(uid_t uid) * Convert the passed name to a user id and return it. Return -1 * on error. */ -uid_t +int getuserid(char name[]) { struct passwd *pw; Modified: stable/11/usr.bin/mail/popen.c ============================================================================== --- stable/11/usr.bin/mail/popen.c Tue Jul 31 00:34:39 2018 (r336952) +++ stable/11/usr.bin/mail/popen.c Tue Jul 31 00:37:25 2018 (r336953) @@ -38,8 +38,6 @@ __FBSDID("$FreeBSD$"); #include "rcv.h" #include #include -#include -#include #include "extern.h" #define READ 0 @@ -48,23 +46,22 @@ __FBSDID("$FreeBSD$"); struct fp { FILE *fp; int pipe; - pid_t pid; + int pid; struct fp *link; }; static struct fp *fp_head; struct child { - pid_t pid; + int pid; char done; char free; int status; struct child *link; }; -static struct child *child, *child_freelist = NULL; - +static struct child *child; +static struct child *findchild(int); static void delchild(struct child *); -static pid_t file_pid(FILE *); -static pid_t start_commandv(char *, sigset_t *, int, int, va_list); +static int file_pid(FILE *); FILE * Fopen(const char *path, const char *mode) @@ -93,7 +90,6 @@ Fdopen(int fd, const char *mode) int Fclose(FILE *fp) { - unregister_file(fp); return (fclose(fp)); } @@ -103,7 +99,7 @@ Popen(char *cmd, const char *mode) { int p[2]; int myside, hisside, fd0, fd1; - pid_t pid; + int pid; sigset_t nset; FILE *fp; @@ -113,15 +109,15 @@ Popen(char *cmd, const char *mode) (void)fcntl(p[WRITE], F_SETFD, 1); if (*mode == 'r') { myside = p[READ]; - hisside = fd0 = fd1 = p[WRITE]; + fd0 = -1; + hisside = fd1 = p[WRITE]; } else { myside = p[WRITE]; hisside = fd0 = p[READ]; fd1 = -1; } (void)sigemptyset(&nset); - pid = start_command(value("SHELL"), &nset, fd0, fd1, "-c", cmd, NULL); - if (pid < 0) { + if ((pid = start_command(cmd, &nset, fd0, fd1, NULL, NULL, NULL)) < 0) { (void)close(p[READ]); (void)close(p[WRITE]); return (NULL); @@ -162,7 +158,7 @@ close_all_files(void) } void -register_file(FILE *fp, int pipe, pid_t pid) +register_file(FILE *fp, int pipe, int pid) { struct fp *fpp; @@ -190,7 +186,7 @@ unregister_file(FILE *fp) /*NOTREACHED*/ } -pid_t +int file_pid(FILE *fp) { struct fp *p; @@ -204,17 +200,30 @@ file_pid(FILE *fp) /* * Run a command without a shell, with optional arguments and splicing - * of stdin (-1 means none) and stdout. The command name can be a sequence - * of words. + * of stdin and stdout. The command name can be a sequence of words. * Signals must be handled by the caller. - * "nset" contains the signals to ignore in the new process. - * SIGINT is enabled unless it's in "nset". + * "Mask" contains the signals to ignore in the new process. + * SIGINT is enabled unless it's in the mask. */ -static pid_t -start_commandv(char *cmd, sigset_t *nset, int infd, int outfd, va_list args) +/*VARARGS4*/ +int +run_command(char *cmd, sigset_t *mask, int infd, int outfd, char *a0, + char *a1, char *a2) { - pid_t pid; + int pid; + if ((pid = start_command(cmd, mask, infd, outfd, a0, a1, a2)) < 0) + return (-1); + return (wait_command(pid)); +} + +/*VARARGS4*/ +int +start_command(char *cmd, sigset_t *mask, int infd, int outfd, char *a0, + char *a1, char *a2) +{ + int pid; + if ((pid = fork()) < 0) { warn("fork"); return (-1); @@ -223,10 +232,11 @@ start_commandv(char *cmd, sigset_t *nset, int infd, in char *argv[100]; int i = getrawlist(cmd, argv, sizeof(argv) / sizeof(*argv)); - while ((argv[i++] = va_arg(args, char *))) - ; - argv[i] = NULL; - prepare_child(nset, infd, outfd); + if ((argv[i++] = a0) != NULL && + (argv[i++] = a1) != NULL && + (argv[i++] = a2) != NULL) + argv[i] = NULL; + prepare_child(mask, infd, outfd); execvp(argv[0], argv); warn("%s", argv[0]); _exit(1); @@ -234,32 +244,6 @@ start_commandv(char *cmd, sigset_t *nset, int infd, in return (pid); } -int -run_command(char *cmd, sigset_t *nset, int infd, int outfd, ...) -{ - pid_t pid; - va_list args; - - va_start(args, outfd); - pid = start_commandv(cmd, nset, infd, outfd, args); - va_end(args); - if (pid < 0) - return -1; - return wait_command(pid); -} - -int -start_command(char *cmd, sigset_t *nset, int infd, int outfd, ...) -{ - va_list args; - int r; - - va_start(args, outfd); - r = start_commandv(cmd, nset, infd, outfd, args); - va_end(args); - return r; -} - void prepare_child(sigset_t *nset, int infd, int outfd) { @@ -284,7 +268,7 @@ prepare_child(sigset_t *nset, int infd, int outfd) } int -wait_command(pid_t pid) +wait_command(int pid) { if (wait_child(pid) < 0) { @@ -295,7 +279,7 @@ wait_command(pid_t pid) } static struct child * -findchild(pid_t pid, int dont_alloc) +findchild(int pid) { struct child **cpp; @@ -303,16 +287,9 @@ findchild(pid_t pid, int dont_alloc) cpp = &(*cpp)->link) ; if (*cpp == NULL) { - if (dont_alloc) - return(NULL); - if (child_freelist) { - *cpp = child_freelist; - child_freelist = (*cpp)->link; - } else { - *cpp = malloc(sizeof(struct child)); - if (*cpp == NULL) - err(1, "malloc"); - } + *cpp = malloc(sizeof(struct child)); + if (*cpp == NULL) + err(1, "Out of memory"); (*cpp)->pid = pid; (*cpp)->done = (*cpp)->free = 0; (*cpp)->link = NULL; @@ -328,22 +305,19 @@ delchild(struct child *cp) for (cpp = &child; *cpp != cp; cpp = &(*cpp)->link) ; *cpp = cp->link; - cp->link = child_freelist; - child_freelist = cp; + (void)free(cp); } /*ARGSUSED*/ void sigchild(int signo __unused) { - pid_t pid; + int pid; int status; struct child *cp; - int save_errno; - save_errno = errno; while ((pid = waitpid(-1, &status, WNOHANG)) > 0) { - cp = findchild(pid, 1); + cp = findchild(pid); if (cp->free) delchild(cp); else { @@ -351,7 +325,6 @@ sigchild(int signo __unused) cp->status = status; } } - errno = save_errno; } int wait_status; @@ -360,51 +333,41 @@ int wait_status; * Wait for a specific child to die. */ int -wait_child(pid_t pid) +wait_child(int pid) { - struct child *cp; sigset_t nset, oset; - pid_t rv = 0; + struct child *cp; (void)sigemptyset(&nset); (void)sigaddset(&nset, SIGCHLD); - (void)sigprocmask(SIG_BLOCK, &nset, &oset); - /* - * If we have not already waited on the pid (via sigchild) - * wait on it now. Otherwise, use the wait status stashed - * by sigchild. - */ - cp = findchild(pid, 1); - if (cp == NULL || !cp->done) - rv = waitpid(pid, &wait_status, 0); - else - wait_status = cp->status; - if (cp != NULL) - delchild(cp); + (void)sigprocmask(SIG_BLOCK, &nset, &oset); + + cp = findchild(pid); + + while (!cp->done) + (void)sigsuspend(&oset); + wait_status = cp->status; + delchild(cp); (void)sigprocmask(SIG_SETMASK, &oset, NULL); - if (rv == -1 || (WIFEXITED(wait_status) && WEXITSTATUS(wait_status))) - return -1; - else - return 0; + return ((WIFEXITED(wait_status) && WEXITSTATUS(wait_status)) ? -1 : 0); } /* * Mark a child as don't care. */ void -free_child(pid_t pid) +free_child(int pid) { - struct child *cp; sigset_t nset, oset; + struct child *cp = findchild(pid); (void)sigemptyset(&nset); (void)sigaddset(&nset, SIGCHLD); - (void)sigprocmask(SIG_BLOCK, &nset, &oset); - if ((cp = findchild(pid, 0)) != NULL) { - if (cp->done) - delchild(cp); - else - cp->free = 1; - } + (void)sigprocmask(SIG_BLOCK, &nset, &oset); + + if (cp->done) + delchild(cp); + else + cp->free = 1; (void)sigprocmask(SIG_SETMASK, &oset, NULL); } From owner-svn-src-all@freebsd.org Tue Jul 31 00:37:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5ACFE1066477; Tue, 31 Jul 2018 00:37:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F0BA8F484; Tue, 31 Jul 2018 00:37:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E461243E0; Tue, 31 Jul 2018 00:37:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0bjPZ000114; Tue, 31 Jul 2018 00:37:45 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0bjAw000113; Tue, 31 Jul 2018 00:37:45 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310037.w6V0bjAw000113@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:37:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336954 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 336954 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:37:46 -0000 Author: mav Date: Tue Jul 31 00:37:45 2018 New Revision: 336954 URL: https://svnweb.freebsd.org/changeset/base/336954 Log: MFV r336952: 9192 explicitly pass good_writes to vdev_uberblock/label_sync Currently vdev_label_sync and vdev_uberblock_sync take a zio_t and assume that its io_private is a pointer to the good_writes count. They should instead accept this argument explicitly. illumos/illumos-gate@a3b5583021b7b45676bf1f0cc68adf7a97900b56 Reviewed by: Pavel Zakharov Reviewed by: George Wilson Approved by: Richard Lowe Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Tue Jul 31 00:37:25 2018 (r336953) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Tue Jul 31 00:37:45 2018 (r336954) @@ -1175,10 +1175,13 @@ vdev_uberblock_sync_done(zio_t *zio) * Write the uberblock to all labels of all leaves of the specified vdev. */ static void -vdev_uberblock_sync(zio_t *zio, uberblock_t *ub, vdev_t *vd, int flags) +vdev_uberblock_sync(zio_t *zio, uint64_t *good_writes, + uberblock_t *ub, vdev_t *vd, int flags) { - for (uint64_t c = 0; c < vd->vdev_children; c++) - vdev_uberblock_sync(zio, ub, vd->vdev_child[c], flags); + for (uint64_t c = 0; c < vd->vdev_children; c++) { + vdev_uberblock_sync(zio, good_writes, + ub, vd->vdev_child[c], flags); + } if (!vd->vdev_ops->vdev_op_leaf) return; @@ -1196,7 +1199,7 @@ vdev_uberblock_sync(zio_t *zio, uberblock_t *ub, vdev_ for (int l = 0; l < VDEV_LABELS; l++) vdev_label_write(zio, vd, l, ub_abd, VDEV_UBERBLOCK_OFFSET(vd, n), VDEV_UBERBLOCK_SIZE(vd), - vdev_uberblock_sync_done, zio->io_private, + vdev_uberblock_sync_done, good_writes, flags | ZIO_FLAG_DONT_PROPAGATE); abd_free(ub_abd); @@ -1210,10 +1213,10 @@ vdev_uberblock_sync_list(vdev_t **svd, int svdcount, u zio_t *zio; uint64_t good_writes = 0; - zio = zio_root(spa, NULL, &good_writes, flags); + zio = zio_root(spa, NULL, NULL, flags); for (int v = 0; v < svdcount; v++) - vdev_uberblock_sync(zio, ub, svd[v], flags); + vdev_uberblock_sync(zio, &good_writes, ub, svd[v], flags); (void) zio_wait(zio); @@ -1274,7 +1277,8 @@ vdev_label_sync_ignore_done(zio_t *zio) * Write all even or odd labels to all leaves of the specified vdev. */ static void -vdev_label_sync(zio_t *zio, vdev_t *vd, int l, uint64_t txg, int flags) +vdev_label_sync(zio_t *zio, uint64_t *good_writes, + vdev_t *vd, int l, uint64_t txg, int flags) { nvlist_t *label; vdev_phys_t *vp; @@ -1282,8 +1286,10 @@ vdev_label_sync(zio_t *zio, vdev_t *vd, int l, uint64_ char *buf; size_t buflen; - for (int c = 0; c < vd->vdev_children; c++) - vdev_label_sync(zio, vd->vdev_child[c], l, txg, flags); + for (int c = 0; c < vd->vdev_children; c++) { + vdev_label_sync(zio, good_writes, + vd->vdev_child[c], l, txg, flags); + } if (!vd->vdev_ops->vdev_op_leaf) return; @@ -1308,7 +1314,7 @@ vdev_label_sync(zio_t *zio, vdev_t *vd, int l, uint64_ vdev_label_write(zio, vd, l, vp_abd, offsetof(vdev_label_t, vl_vdev_phys), sizeof (vdev_phys_t), - vdev_label_sync_done, zio->io_private, + vdev_label_sync_done, good_writes, flags | ZIO_FLAG_DONT_PROPAGATE); } } @@ -1340,7 +1346,7 @@ vdev_label_sync_list(spa_t *spa, int l, uint64_t txg, (vd->vdev_islog || vd->vdev_aux != NULL) ? vdev_label_sync_ignore_done : vdev_label_sync_top_done, good_writes, flags); - vdev_label_sync(vio, vd, l, txg, flags); + vdev_label_sync(vio, good_writes, vd, l, txg, flags); zio_nowait(vio); } From owner-svn-src-all@freebsd.org Tue Jul 31 00:42:34 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBCA31066846; Tue, 31 Jul 2018 00:42:33 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 912408FA7D; Tue, 31 Jul 2018 00:42:33 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 725FD456D; Tue, 31 Jul 2018 00:42:33 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0gX01004801; Tue, 31 Jul 2018 00:42:33 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0gW98004795; Tue, 31 Jul 2018 00:42:32 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310042.w6V0gW98004795@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:42:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336955 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Commit-Revision: 336955 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:42:34 -0000 Author: mav Date: Tue Jul 31 00:42:31 2018 New Revision: 336955 URL: https://svnweb.freebsd.org/changeset/base/336955 Log: 9236 nuke spa_dbgmsg We should use zfs_dbgmsg instead of spa_dbgmsg. Or at least, metaslab_condense() should call zfs_dbgmsg because it's important and rare enough to always log. It's possible that the message in zio_dva_allocate() would be too high-frequency for zfs_dbgmsg. illumos/illumos-gate@21f7c81cc1156e9202ce3412d3ecaa697c3b2222 Reviewed by: Serapheim Dimitropoulos Reviewed by: Pavel Zakharov Reviewed by: George Wilson Reviewed by: Richard Elling Approved by: Richard Lowe Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_debug.h vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/ztest/ztest.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:37:45 2018 (r336954) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:42:31 2018 (r336955) @@ -1733,7 +1733,7 @@ metaslab_set_fragmentation(metaslab_t *msp) if (spa_writeable(spa) && txg < spa_final_dirty_txg(spa)) { msp->ms_condense_wanted = B_TRUE; vdev_dirty(vd, VDD_METASLAB, msp, txg + 1); - spa_dbgmsg(spa, "txg %llu, requesting force condense: " + zfs_dbgmsg("txg %llu, requesting force condense: " "ms_id %llu, vdev_id %llu", txg, msp->ms_id, vd->vdev_id); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:37:45 2018 (r336954) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:42:31 2018 (r336955) @@ -245,7 +245,7 @@ int spa_mode_global; * Everything except dprintf, spa, and indirect_remap is on by default * in debug builds. */ -int zfs_flags = ~(ZFS_DEBUG_DPRINTF | ZFS_DEBUG_SPA | ZFS_DEBUG_INDIRECT_REMAP); +int zfs_flags = ~(ZFS_DEBUG_DPRINTF | ZFS_DEBUG_INDIRECT_REMAP); #else int zfs_flags = 0; #endif @@ -707,8 +707,6 @@ spa_add(const char *name, nvlist_t *config, const char kstat_install(spa->spa_iokstat); } - spa->spa_debug = ((zfs_flags & ZFS_DEBUG_SPA) != 0); - spa->spa_min_ashift = INT_MAX; spa->spa_max_ashift = 0; @@ -2135,12 +2133,6 @@ spa_scan_get_stats(spa_t *spa, pool_scan_stat_t *ps) ps->pss_pass_scrub_spent_paused = spa->spa_scan_pass_scrub_spent_paused; return (0); -} - -boolean_t -spa_debug_enabled(spa_t *spa) -{ - return (spa->spa_debug); } int Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Tue Jul 31 00:37:45 2018 (r336954) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Tue Jul 31 00:42:31 2018 (r336955) @@ -927,13 +927,6 @@ _NOTE(CONSTCOND) } while (0) #define dprintf_bp(bp, fmt, ...) #endif -extern boolean_t spa_debug_enabled(spa_t *spa); -#define spa_dbgmsg(spa, ...) \ -{ \ - if (spa_debug_enabled(spa)) \ - zfs_dbgmsg(__VA_ARGS__); \ -} - extern int spa_mode_global; /* mode, e.g. FREAD | FWRITE */ #ifdef __cplusplus Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Tue Jul 31 00:37:45 2018 (r336954) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Tue Jul 31 00:42:31 2018 (r336955) @@ -328,7 +328,6 @@ struct spa { kcondvar_t spa_suspend_cv; /* notification of resume */ uint8_t spa_suspended; /* pool is suspended */ uint8_t spa_claiming; /* pool is doing zil_claim() */ - boolean_t spa_debug; /* debug enabled? */ boolean_t spa_is_root; /* pool is root */ int spa_minref; /* num refs when first opened */ int spa_mode; /* FREAD | FWRITE */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_debug.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_debug.h Tue Jul 31 00:37:45 2018 (r336954) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_debug.h Tue Jul 31 00:42:31 2018 (r336955) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2016 by Delphix. All rights reserved. + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. */ #ifndef _SYS_ZFS_DEBUG_H @@ -55,7 +55,7 @@ extern boolean_t zfs_free_leak_on_eio; #define ZFS_DEBUG_DNODE_VERIFY (1 << 2) #define ZFS_DEBUG_SNAPNAMES (1 << 3) #define ZFS_DEBUG_MODIFY (1 << 4) -#define ZFS_DEBUG_SPA (1 << 5) +/* 1<<5 was previously used, try not to reuse */ #define ZFS_DEBUG_ZIO_FREE (1 << 6) #define ZFS_DEBUG_HISTOGRAM_VERIFY (1 << 7) #define ZFS_DEBUG_METASLAB_VERIFY (1 << 8) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Tue Jul 31 00:37:45 2018 (r336954) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Tue Jul 31 00:42:31 2018 (r336955) @@ -2932,7 +2932,7 @@ zio_dva_allocate(zio_t *zio) &zio->io_alloc_list, zio, zio->io_allocator); if (error != 0) { - spa_dbgmsg(spa, "%s: metaslab allocation failure: zio %p, " + zfs_dbgmsg("%s: metaslab allocation failure: zio %p, " "size %llu, error %d", spa_name(spa), zio, zio->io_size, error); if (error == ENOSPC && zio->io_size > SPA_MINBLOCKSIZE) From owner-svn-src-all@freebsd.org Tue Jul 31 00:42:34 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 595EB1066849; Tue, 31 Jul 2018 00:42:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BE0E8FA7E; Tue, 31 Jul 2018 00:42:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C8C6E456E; Tue, 31 Jul 2018 00:42:33 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0gXaj004807; Tue, 31 Jul 2018 00:42:33 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0gXnf004806; Tue, 31 Jul 2018 00:42:33 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310042.w6V0gXnf004806@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:42:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336955 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/ztest X-SVN-Commit-Revision: 336955 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:42:34 -0000 Author: mav Date: Tue Jul 31 00:42:31 2018 New Revision: 336955 URL: https://svnweb.freebsd.org/changeset/base/336955 Log: 9236 nuke spa_dbgmsg We should use zfs_dbgmsg instead of spa_dbgmsg. Or at least, metaslab_condense() should call zfs_dbgmsg because it's important and rare enough to always log. It's possible that the message in zio_dva_allocate() would be too high-frequency for zfs_dbgmsg. illumos/illumos-gate@21f7c81cc1156e9202ce3412d3ecaa697c3b2222 Reviewed by: Serapheim Dimitropoulos Reviewed by: Pavel Zakharov Reviewed by: George Wilson Reviewed by: Richard Elling Approved by: Richard Lowe Author: Matthew Ahrens Modified: vendor/illumos/dist/cmd/ztest/ztest.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zfs_debug.h vendor-sys/illumos/dist/uts/common/fs/zfs/zio.c Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Tue Jul 31 00:37:45 2018 (r336954) +++ vendor/illumos/dist/cmd/ztest/ztest.c Tue Jul 31 00:42:31 2018 (r336955) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2016 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. * Copyright (c) 2014 Integros [integros.com] @@ -5919,7 +5919,6 @@ ztest_run(ztest_shared_t *zs) */ kernel_init(FREAD | FWRITE); VERIFY0(spa_open(ztest_opts.zo_pool, &spa, FTAG)); - spa->spa_debug = B_TRUE; metaslab_preload_limit = ztest_random(20) + 1; ztest_spa = spa; @@ -6076,7 +6075,6 @@ ztest_freeze(void) kernel_init(FREAD | FWRITE); VERIFY3U(0, ==, spa_open(ztest_opts.zo_pool, &spa, FTAG)); VERIFY3U(0, ==, ztest_dataset_open(0)); - spa->spa_debug = B_TRUE; ztest_spa = spa; /* @@ -6147,7 +6145,6 @@ ztest_freeze(void) VERIFY3U(0, ==, ztest_dataset_open(0)); ztest_dataset_close(0); - spa->spa_debug = B_TRUE; ztest_spa = spa; txg_wait_synced(spa_get_dsl(spa), 0); ztest_reguid(NULL, 0); From owner-svn-src-all@freebsd.org Tue Jul 31 00:47:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E60F106695A; Tue, 31 Jul 2018 00:47:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 136168FD50; Tue, 31 Jul 2018 00:47:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E3CC54571; Tue, 31 Jul 2018 00:47:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0lSJK005097; Tue, 31 Jul 2018 00:47:28 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0lRAq005090; Tue, 31 Jul 2018 00:47:27 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310047.w6V0lRAq005090@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:47:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336956 - in head: cddl/contrib/opensolaris/cmd/ztest sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/ztest sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys X-SVN-Commit-Revision: 336956 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:47:29 -0000 Author: mav Date: Tue Jul 31 00:47:27 2018 New Revision: 336956 URL: https://svnweb.freebsd.org/changeset/base/336956 Log: MFV r336955: 9236 nuke spa_dbgmsg We should use zfs_dbgmsg instead of spa_dbgmsg. Or at least, metaslab_condense() should call zfs_dbgmsg because it's important and rare enough to always log. It's possible that the message in zio_dva_allocate() would be too high-frequency for zfs_dbgmsg. illumos/illumos-gate@21f7c81cc1156e9202ce3412d3ecaa697c3b2222 Reviewed by: Serapheim Dimitropoulos Reviewed by: Pavel Zakharov Reviewed by: George Wilson Reviewed by: Richard Elling Approved by: Richard Lowe Author: Matthew Ahrens Modified: head/cddl/contrib/opensolaris/cmd/ztest/ztest.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_debug.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/ztest/ztest.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Jul 31 00:42:31 2018 (r336955) +++ head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Jul 31 00:47:27 2018 (r336956) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2016 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2012 Martin Matuska . All rights reserved. * Copyright (c) 2013 Steven Hartland. All rights reserved. @@ -5921,7 +5921,6 @@ ztest_run(ztest_shared_t *zs) */ kernel_init(FREAD | FWRITE); VERIFY0(spa_open(ztest_opts.zo_pool, &spa, FTAG)); - spa->spa_debug = B_TRUE; metaslab_preload_limit = ztest_random(20) + 1; ztest_spa = spa; @@ -6078,7 +6077,6 @@ ztest_freeze(void) kernel_init(FREAD | FWRITE); VERIFY3U(0, ==, spa_open(ztest_opts.zo_pool, &spa, FTAG)); VERIFY3U(0, ==, ztest_dataset_open(0)); - spa->spa_debug = B_TRUE; ztest_spa = spa; /* @@ -6149,7 +6147,6 @@ ztest_freeze(void) VERIFY3U(0, ==, ztest_dataset_open(0)); ztest_dataset_close(0); - spa->spa_debug = B_TRUE; ztest_spa = spa; txg_wait_synced(spa_get_dsl(spa), 0); ztest_reguid(NULL, 0); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:42:31 2018 (r336955) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Tue Jul 31 00:47:27 2018 (r336956) @@ -1747,7 +1747,7 @@ metaslab_set_fragmentation(metaslab_t *msp) if (spa_writeable(spa) && txg < spa_final_dirty_txg(spa)) { msp->ms_condense_wanted = B_TRUE; vdev_dirty(vd, VDD_METASLAB, msp, txg + 1); - spa_dbgmsg(spa, "txg %llu, requesting force condense: " + zfs_dbgmsg("txg %llu, requesting force condense: " "ms_id %llu, vdev_id %llu", txg, msp->ms_id, vd->vdev_id); } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:42:31 2018 (r336955) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Tue Jul 31 00:47:27 2018 (r336956) @@ -252,7 +252,7 @@ int spa_mode_global; * Everything except dprintf, spa, and indirect_remap is on by default * in debug builds. */ -int zfs_flags = ~(ZFS_DEBUG_DPRINTF | ZFS_DEBUG_SPA | ZFS_DEBUG_INDIRECT_REMAP); +int zfs_flags = ~(ZFS_DEBUG_DPRINTF | ZFS_DEBUG_INDIRECT_REMAP); #else int zfs_flags = 0; #endif @@ -821,8 +821,6 @@ spa_add(const char *name, nvlist_t *config, const char KM_SLEEP) == 0); } - spa->spa_debug = ((zfs_flags & ZFS_DEBUG_SPA) != 0); - spa->spa_min_ashift = INT_MAX; spa->spa_max_ashift = 0; @@ -2278,12 +2276,6 @@ spa_scan_get_stats(spa_t *spa, pool_scan_stat_t *ps) ps->pss_pass_scrub_spent_paused = spa->spa_scan_pass_scrub_spent_paused; return (0); -} - -boolean_t -spa_debug_enabled(spa_t *spa) -{ - return (spa->spa_debug); } int Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h Tue Jul 31 00:42:31 2018 (r336955) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h Tue Jul 31 00:47:27 2018 (r336956) @@ -947,13 +947,6 @@ _NOTE(CONSTCOND) } while (0) #define dprintf_bp(bp, fmt, ...) #endif -extern boolean_t spa_debug_enabled(spa_t *spa); -#define spa_dbgmsg(spa, ...) \ -{ \ - if (spa_debug_enabled(spa)) \ - zfs_dbgmsg(__VA_ARGS__); \ -} - extern int spa_mode_global; /* mode, e.g. FREAD | FWRITE */ #ifdef __cplusplus Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h Tue Jul 31 00:42:31 2018 (r336955) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h Tue Jul 31 00:47:27 2018 (r336956) @@ -332,7 +332,6 @@ struct spa { kcondvar_t spa_suspend_cv; /* notification of resume */ uint8_t spa_suspended; /* pool is suspended */ uint8_t spa_claiming; /* pool is doing zil_claim() */ - boolean_t spa_debug; /* debug enabled? */ boolean_t spa_is_root; /* pool is root */ int spa_minref; /* num refs when first opened */ int spa_mode; /* FREAD | FWRITE */ Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_debug.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_debug.h Tue Jul 31 00:42:31 2018 (r336955) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_debug.h Tue Jul 31 00:47:27 2018 (r336956) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2016 by Delphix. All rights reserved. + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. */ #ifndef _SYS_ZFS_DEBUG_H @@ -57,7 +57,7 @@ extern boolean_t zfs_free_leak_on_eio; #define ZFS_DEBUG_DNODE_VERIFY (1 << 2) #define ZFS_DEBUG_SNAPNAMES (1 << 3) #define ZFS_DEBUG_MODIFY (1 << 4) -#define ZFS_DEBUG_SPA (1 << 5) +/* 1<<5 was previously used, try not to reuse */ #define ZFS_DEBUG_ZIO_FREE (1 << 6) #define ZFS_DEBUG_HISTOGRAM_VERIFY (1 << 7) #define ZFS_DEBUG_METASLAB_VERIFY (1 << 8) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Tue Jul 31 00:42:31 2018 (r336955) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Tue Jul 31 00:47:27 2018 (r336956) @@ -3037,7 +3037,7 @@ zio_dva_allocate(zio_t *zio) &zio->io_alloc_list, zio, zio->io_allocator); if (error != 0) { - spa_dbgmsg(spa, "%s: metaslab allocation failure: zio %p, " + zfs_dbgmsg("%s: metaslab allocation failure: zio %p, " "size %llu, error %d", spa_name(spa), zio, zio->io_size, error); if (error == ENOSPC && zio->io_size > SPA_MINBLOCKSIZE) From owner-svn-src-all@freebsd.org Tue Jul 31 00:48:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E89FF10669BA; Tue, 31 Jul 2018 00:48:08 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F6D98FEC9; Tue, 31 Jul 2018 00:48:08 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 806C94572; Tue, 31 Jul 2018 00:48:08 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0m88g005169; Tue, 31 Jul 2018 00:48:08 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0m8Et005168; Tue, 31 Jul 2018 00:48:08 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201807310048.w6V0m8Et005168@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Tue, 31 Jul 2018 00:48:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336957 - head/tests/sys/kern X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/tests/sys/kern X-SVN-Commit-Revision: 336957 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:48:09 -0000 Author: markj Date: Tue Jul 31 00:48:08 2018 New Revision: 336957 URL: https://svnweb.freebsd.org/changeset/base/336957 Log: Add a regression test related to PR 131876. If an error occurs while copying a SCM_RIGHTS message to userspace, we free the mbuf containing externalized rights, leaking them. PR: 131876 MFC after: 1 week Sponsored by: The FreeBSD Foundation Modified: head/tests/sys/kern/unix_passfd_test.c Modified: head/tests/sys/kern/unix_passfd_test.c ============================================================================== --- head/tests/sys/kern/unix_passfd_test.c Tue Jul 31 00:47:27 2018 (r336956) +++ head/tests/sys/kern/unix_passfd_test.c Tue Jul 31 00:48:08 2018 (r336957) @@ -447,6 +447,47 @@ ATF_TC_BODY(truncated_rights, tc) closesocketpair(fd); } +ATF_TC_WITHOUT_HEAD(copyout_rights_error); +ATF_TC_BODY(copyout_rights_error, tc) +{ + struct iovec iovec; + struct msghdr msghdr; + char buf[16]; + ssize_t len; + int fd[2], error, nfds, putfd; + + atf_tc_expect_fail("PR 131876: " + "FD leak when copyout of rights returns an error"); + + memset(buf, 0, sizeof(buf)); + domainsocketpair(fd); + devnull(&putfd); + nfds = getnfds(); + + sendfd_payload(fd[0], putfd, buf, sizeof(buf)); + + bzero(&msghdr, sizeof(msghdr)); + + iovec.iov_base = buf; + iovec.iov_len = sizeof(buf); + msghdr.msg_control = (char *)-1; /* trigger EFAULT */ + msghdr.msg_controllen = CMSG_SPACE(sizeof(int)); + msghdr.msg_iov = &iovec; + msghdr.msg_iovlen = 1; + + len = recvmsg(fd[1], &msghdr, 0); + error = errno; + ATF_REQUIRE_MSG(len == -1, "recvmsg succeeded: %zd", len); + ATF_REQUIRE_MSG(errno == EFAULT, "expected EFAULT, got %d (%s)", + error, strerror(errno)); + + /* Verify that no FDs were leaked. */ + ATF_REQUIRE(getnfds() == nfds); + + close(putfd); + closesocketpair(fd); +} + ATF_TP_ADD_TCS(tp) { @@ -459,6 +500,7 @@ ATF_TP_ADD_TCS(tp) ATF_TP_ADD_TC(tp, devfs_orphan); ATF_TP_ADD_TC(tp, rights_creds_payload); ATF_TP_ADD_TC(tp, truncated_rights); + ATF_TP_ADD_TC(tp, copyout_rights_error); return (atf_no_error()); } From owner-svn-src-all@freebsd.org Tue Jul 31 00:56:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD9FD1066D58; Tue, 31 Jul 2018 00:56:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CB787051D; Tue, 31 Jul 2018 00:56:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 373AF4704; Tue, 31 Jul 2018 00:56:43 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0uhKd010293; Tue, 31 Jul 2018 00:56:43 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0ufMF010286; Tue, 31 Jul 2018 00:56:41 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310056.w6V0ufMF010286@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:56:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336958 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 336958 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:56:44 -0000 Author: mav Date: Tue Jul 31 00:56:41 2018 New Revision: 336958 URL: https://svnweb.freebsd.org/changeset/base/336958 Log: 9337 zfs get all is slow due to uncached metadata This project's goal is to make read-heavy channel programs and zfs(1m) administrative commands faster by caching all the metadata that they will need in the dbuf layer. This will prevent the data from being evicted, so that any future call to i.e. zfs get all won't have to go to disk (very much). illumos/illumos-gate@adb52d9262f45a04318fc6e188fe2b7f59d989a5 Reviewed by: Prakash Surya Reviewed by: George Wilson Reviewed by: Thomas Caputi Approved by: Richard Lowe Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Tue Jul 31 00:48:08 2018 (r336957) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Tue Jul 31 00:56:41 2018 (r336958) @@ -49,6 +49,7 @@ #include #include #include +#include uint_t zfs_dbuf_evict_key; @@ -74,24 +75,58 @@ static kcondvar_t dbuf_evict_cv; static boolean_t dbuf_evict_thread_exit; /* - * LRU cache of dbufs. The dbuf cache maintains a list of dbufs that - * are not currently held but have been recently released. These dbufs - * are not eligible for arc eviction until they are aged out of the cache. - * Dbufs are added to the dbuf cache once the last hold is released. If a - * dbuf is later accessed and still exists in the dbuf cache, then it will - * be removed from the cache and later re-added to the head of the cache. - * Dbufs that are aged out of the cache will be immediately destroyed and - * become eligible for arc eviction. + * There are two dbuf caches; each dbuf can only be in one of them at a time. + * + * 1. Cache of metadata dbufs, to help make read-heavy administrative commands + * from /sbin/zfs run faster. The "metadata cache" specifically stores dbufs + * that represent the metadata that describes filesystems/snapshots/ + * bookmarks/properties/etc. We only evict from this cache when we export a + * pool, to short-circuit as much I/O as possible for all administrative + * commands that need the metadata. There is no eviction policy for this + * cache, because we try to only include types in it which would occupy a + * very small amount of space per object but create a large impact on the + * performance of these commands. Instead, after it reaches a maximum size + * (which should only happen on very small memory systems with a very large + * number of filesystem objects), we stop taking new dbufs into the + * metadata cache, instead putting them in the normal dbuf cache. + * + * 2. LRU cache of dbufs. The "dbuf cache" maintains a list of dbufs that + * are not currently held but have been recently released. These dbufs + * are not eligible for arc eviction until they are aged out of the cache. + * Dbufs that are aged out of the cache will be immediately destroyed and + * become eligible for arc eviction. + * + * Dbufs are added to these caches once the last hold is released. If a dbuf is + * later accessed and still exists in the dbuf cache, then it will be removed + * from the cache and later re-added to the head of the cache. + * + * If a given dbuf meets the requirements for the metadata cache, it will go + * there, otherwise it will be considered for the generic LRU dbuf cache. The + * caches and the refcounts tracking their sizes are stored in an array indexed + * by those caches' matching enum values (from dbuf_cached_state_t). */ -static multilist_t *dbuf_cache; -static refcount_t dbuf_cache_size; -uint64_t dbuf_cache_max_bytes = 0; +typedef struct dbuf_cache { + multilist_t *cache; + refcount_t size; +} dbuf_cache_t; +dbuf_cache_t dbuf_caches[DB_CACHE_MAX]; -/* Set the default size of the dbuf cache to log2 fraction of arc size. */ +/* Size limits for the caches */ +uint64_t dbuf_cache_max_bytes = 0; +uint64_t dbuf_metadata_cache_max_bytes = 0; +/* Set the default sizes of the caches to log2 fraction of arc size */ int dbuf_cache_shift = 5; +int dbuf_metadata_cache_shift = 6; /* - * The dbuf cache uses a three-stage eviction policy: + * For diagnostic purposes, this is incremented whenever we can't add + * something to the metadata cache because it's full, and instead put + * the data in the regular dbuf cache. + */ +uint64_t dbuf_metadata_cache_overflow; + +/* + * The LRU dbuf cache uses a three-stage eviction policy: * - A low water marker designates when the dbuf eviction thread * should stop evicting from the dbuf cache. * - When we reach the maximum size (aka mid water mark), we @@ -394,6 +429,41 @@ dbuf_is_metadata(dmu_buf_impl_t *db) } /* + * This returns whether this dbuf should be stored in the metadata cache, which + * is based on whether it's from one of the dnode types that store data related + * to traversing dataset hierarchies. + */ +static boolean_t +dbuf_include_in_metadata_cache(dmu_buf_impl_t *db) +{ + DB_DNODE_ENTER(db); + dmu_object_type_t type = DB_DNODE(db)->dn_type; + DB_DNODE_EXIT(db); + + /* Check if this dbuf is one of the types we care about */ + if (DMU_OT_IS_METADATA_CACHED(type)) { + /* If we hit this, then we set something up wrong in dmu_ot */ + ASSERT(DMU_OT_IS_METADATA(type)); + + /* + * Sanity check for small-memory systems: don't allocate too + * much memory for this purpose. + */ + if (refcount_count(&dbuf_caches[DB_DBUF_METADATA_CACHE].size) > + dbuf_metadata_cache_max_bytes) { + dbuf_metadata_cache_overflow++; + DTRACE_PROBE1(dbuf__metadata__cache__overflow, + dmu_buf_impl_t *, db); + return (B_FALSE); + } + + return (B_TRUE); + } + + return (B_FALSE); +} + +/* * This function *must* return indices evenly distributed between all * sublists of the multilist. This is needed due to how the dbuf eviction * code is laid out; dbuf_evict_thread() assumes dbufs are evenly @@ -428,7 +498,7 @@ dbuf_cache_above_hiwater(void) uint64_t dbuf_cache_hiwater_bytes = (dbuf_cache_max_bytes * dbuf_cache_hiwater_pct) / 100; - return (refcount_count(&dbuf_cache_size) > + return (refcount_count(&dbuf_caches[DB_DBUF_CACHE].size) > dbuf_cache_max_bytes + dbuf_cache_hiwater_bytes); } @@ -438,7 +508,7 @@ dbuf_cache_above_lowater(void) uint64_t dbuf_cache_lowater_bytes = (dbuf_cache_max_bytes * dbuf_cache_lowater_pct) / 100; - return (refcount_count(&dbuf_cache_size) > + return (refcount_count(&dbuf_caches[DB_DBUF_CACHE].size) > dbuf_cache_max_bytes - dbuf_cache_lowater_bytes); } @@ -448,8 +518,9 @@ dbuf_cache_above_lowater(void) static void dbuf_evict_one(void) { - int idx = multilist_get_random_index(dbuf_cache); - multilist_sublist_t *mls = multilist_sublist_lock(dbuf_cache, idx); + int idx = multilist_get_random_index(dbuf_caches[DB_DBUF_CACHE].cache); + multilist_sublist_t *mls = multilist_sublist_lock( + dbuf_caches[DB_DBUF_CACHE].cache, idx); ASSERT(!MUTEX_HELD(&dbuf_evict_lock)); @@ -472,8 +543,10 @@ dbuf_evict_one(void) if (db != NULL) { multilist_sublist_remove(mls, db); multilist_sublist_unlock(mls); - (void) refcount_remove_many(&dbuf_cache_size, + (void) refcount_remove_many(&dbuf_caches[DB_DBUF_CACHE].size, db->db.db_size, db); + ASSERT3U(db->db_caching_status, ==, DB_DBUF_CACHE); + db->db_caching_status = DB_NO_CACHE; dbuf_destroy(db); } else { multilist_sublist_unlock(mls); @@ -560,7 +633,8 @@ dbuf_evict_notify(void) * because it's OK to occasionally make the wrong decision here, * and grabbing the lock results in massive lock contention. */ - if (refcount_count(&dbuf_cache_size) > dbuf_cache_max_bytes) { + if (refcount_count(&dbuf_caches[DB_DBUF_CACHE].size) > + dbuf_cache_max_bytes) { if (dbuf_cache_above_hiwater()) dbuf_evict_one(); cv_signal(&dbuf_evict_cv); @@ -600,15 +674,21 @@ retry: mutex_init(&h->hash_mutexes[i], NULL, MUTEX_DEFAULT, NULL); /* - * Setup the parameters for the dbuf cache. We set the size of the - * dbuf cache to 1/32nd (default) of the size of the ARC. If the value - * has been set in /etc/system and it's not greater than the size of - * the ARC, then we honor that value. + * Setup the parameters for the dbuf caches. We set the sizes of the + * dbuf cache and the metadata cache to 1/32nd and 1/16th (default) + * of the size of the ARC, respectively. If the values are set in + * /etc/system and they're not greater than the size of the ARC, then + * we honor that value. */ if (dbuf_cache_max_bytes == 0 || dbuf_cache_max_bytes >= arc_max_bytes()) { dbuf_cache_max_bytes = arc_max_bytes() >> dbuf_cache_shift; } + if (dbuf_metadata_cache_max_bytes == 0 || + dbuf_metadata_cache_max_bytes >= arc_max_bytes()) { + dbuf_metadata_cache_max_bytes = + arc_max_bytes() >> dbuf_metadata_cache_shift; + } /* * All entries are queued via taskq_dispatch_ent(), so min/maxalloc @@ -616,10 +696,13 @@ retry: */ dbu_evict_taskq = taskq_create("dbu_evict", 1, minclsyspri, 0, 0, 0); - dbuf_cache = multilist_create(sizeof (dmu_buf_impl_t), - offsetof(dmu_buf_impl_t, db_cache_link), - dbuf_cache_multilist_index_func); - refcount_create(&dbuf_cache_size); + for (dbuf_cached_state_t dcs = 0; dcs < DB_CACHE_MAX; dcs++) { + dbuf_caches[dcs].cache = + multilist_create(sizeof (dmu_buf_impl_t), + offsetof(dmu_buf_impl_t, db_cache_link), + dbuf_cache_multilist_index_func); + refcount_create(&dbuf_caches[dcs].size); + } tsd_create(&zfs_dbuf_evict_key, NULL); dbuf_evict_thread_exit = B_FALSE; @@ -653,8 +736,10 @@ dbuf_fini(void) mutex_destroy(&dbuf_evict_lock); cv_destroy(&dbuf_evict_cv); - refcount_destroy(&dbuf_cache_size); - multilist_destroy(dbuf_cache); + for (dbuf_cached_state_t dcs = 0; dcs < DB_CACHE_MAX; dcs++) { + refcount_destroy(&dbuf_caches[dcs].size); + multilist_destroy(dbuf_caches[dcs].cache); + } } /* @@ -2037,9 +2122,15 @@ dbuf_destroy(dmu_buf_impl_t *db) dbuf_clear_data(db); if (multilist_link_active(&db->db_cache_link)) { - multilist_remove(dbuf_cache, db); - (void) refcount_remove_many(&dbuf_cache_size, + ASSERT(db->db_caching_status == DB_DBUF_CACHE || + db->db_caching_status == DB_DBUF_METADATA_CACHE); + + multilist_remove(dbuf_caches[db->db_caching_status].cache, db); + (void) refcount_remove_many( + &dbuf_caches[db->db_caching_status].size, db->db.db_size, db); + + db->db_caching_status = DB_NO_CACHE; } ASSERT(db->db_state == DB_UNCACHED || db->db_state == DB_NOFILL); @@ -2093,6 +2184,7 @@ dbuf_destroy(dmu_buf_impl_t *db) ASSERT(db->db_hash_next == NULL); ASSERT(db->db_blkptr == NULL); ASSERT(db->db_data_pending == NULL); + ASSERT3U(db->db_caching_status, ==, DB_NO_CACHE); ASSERT(!multilist_link_active(&db->db_cache_link)); kmem_cache_free(dbuf_kmem_cache, db); @@ -2231,6 +2323,7 @@ dbuf_create(dnode_t *dn, uint8_t level, uint64_t blkid ASSERT3U(db->db.db_size, >=, dn->dn_bonuslen); db->db.db_offset = DMU_BONUS_BLKID; db->db_state = DB_UNCACHED; + db->db_caching_status = DB_NO_CACHE; /* the bonus dbuf is not placed in the hash table */ arc_space_consume(sizeof (dmu_buf_impl_t), ARC_SPACE_OTHER); return (db); @@ -2263,6 +2356,7 @@ dbuf_create(dnode_t *dn, uint8_t level, uint64_t blkid avl_add(&dn->dn_dbufs, db); db->db_state = DB_UNCACHED; + db->db_caching_status = DB_NO_CACHE; mutex_exit(&dn->dn_dbufs_mtx); arc_space_consume(sizeof (dmu_buf_impl_t), ARC_SPACE_OTHER); @@ -2597,9 +2691,15 @@ top: if (multilist_link_active(&db->db_cache_link)) { ASSERT(refcount_is_zero(&db->db_holds)); - multilist_remove(dbuf_cache, db); - (void) refcount_remove_many(&dbuf_cache_size, + ASSERT(db->db_caching_status == DB_DBUF_CACHE || + db->db_caching_status == DB_DBUF_METADATA_CACHE); + + multilist_remove(dbuf_caches[db->db_caching_status].cache, db); + (void) refcount_remove_many( + &dbuf_caches[db->db_caching_status].size, db->db.db_size, db); + + db->db_caching_status = DB_NO_CACHE; } (void) refcount_add(&db->db_holds, tag); DBUF_VERIFY(db); @@ -2816,12 +2916,22 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag) db->db_pending_evict) { dbuf_destroy(db); } else if (!multilist_link_active(&db->db_cache_link)) { - multilist_insert(dbuf_cache, db); - (void) refcount_add_many(&dbuf_cache_size, + ASSERT3U(db->db_caching_status, ==, + DB_NO_CACHE); + + dbuf_cached_state_t dcs = + dbuf_include_in_metadata_cache(db) ? + DB_DBUF_METADATA_CACHE : DB_DBUF_CACHE; + db->db_caching_status = dcs; + + multilist_insert(dbuf_caches[dcs].cache, db); + (void) refcount_add_many(&dbuf_caches[dcs].size, db->db.db_size, db); mutex_exit(&db->db_mtx); - dbuf_evict_notify(); + if (db->db_caching_status == DB_DBUF_CACHE) { + dbuf_evict_notify(); + } } if (do_arc_evict) Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Tue Jul 31 00:48:08 2018 (r336957) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Tue Jul 31 00:56:41 2018 (r336958) @@ -73,60 +73,60 @@ uint32_t zfs_per_txg_dirty_frees_percent = 30; int zfs_object_remap_one_indirect_delay_ticks = 0; const dmu_object_type_info_t dmu_ot[DMU_OT_NUMTYPES] = { - { DMU_BSWAP_UINT8, TRUE, "unallocated" }, - { DMU_BSWAP_ZAP, TRUE, "object directory" }, - { DMU_BSWAP_UINT64, TRUE, "object array" }, - { DMU_BSWAP_UINT8, TRUE, "packed nvlist" }, - { DMU_BSWAP_UINT64, TRUE, "packed nvlist size" }, - { DMU_BSWAP_UINT64, TRUE, "bpobj" }, - { DMU_BSWAP_UINT64, TRUE, "bpobj header" }, - { DMU_BSWAP_UINT64, TRUE, "SPA space map header" }, - { DMU_BSWAP_UINT64, TRUE, "SPA space map" }, - { DMU_BSWAP_UINT64, TRUE, "ZIL intent log" }, - { DMU_BSWAP_DNODE, TRUE, "DMU dnode" }, - { DMU_BSWAP_OBJSET, TRUE, "DMU objset" }, - { DMU_BSWAP_UINT64, TRUE, "DSL directory" }, - { DMU_BSWAP_ZAP, TRUE, "DSL directory child map"}, - { DMU_BSWAP_ZAP, TRUE, "DSL dataset snap map" }, - { DMU_BSWAP_ZAP, TRUE, "DSL props" }, - { DMU_BSWAP_UINT64, TRUE, "DSL dataset" }, - { DMU_BSWAP_ZNODE, TRUE, "ZFS znode" }, - { DMU_BSWAP_OLDACL, TRUE, "ZFS V0 ACL" }, - { DMU_BSWAP_UINT8, FALSE, "ZFS plain file" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS directory" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS master node" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS delete queue" }, - { DMU_BSWAP_UINT8, FALSE, "zvol object" }, - { DMU_BSWAP_ZAP, TRUE, "zvol prop" }, - { DMU_BSWAP_UINT8, FALSE, "other uint8[]" }, - { DMU_BSWAP_UINT64, FALSE, "other uint64[]" }, - { DMU_BSWAP_ZAP, TRUE, "other ZAP" }, - { DMU_BSWAP_ZAP, TRUE, "persistent error log" }, - { DMU_BSWAP_UINT8, TRUE, "SPA history" }, - { DMU_BSWAP_UINT64, TRUE, "SPA history offsets" }, - { DMU_BSWAP_ZAP, TRUE, "Pool properties" }, - { DMU_BSWAP_ZAP, TRUE, "DSL permissions" }, - { DMU_BSWAP_ACL, TRUE, "ZFS ACL" }, - { DMU_BSWAP_UINT8, TRUE, "ZFS SYSACL" }, - { DMU_BSWAP_UINT8, TRUE, "FUID table" }, - { DMU_BSWAP_UINT64, TRUE, "FUID table size" }, - { DMU_BSWAP_ZAP, TRUE, "DSL dataset next clones"}, - { DMU_BSWAP_ZAP, TRUE, "scan work queue" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS user/group used" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS user/group quota" }, - { DMU_BSWAP_ZAP, TRUE, "snapshot refcount tags"}, - { DMU_BSWAP_ZAP, TRUE, "DDT ZAP algorithm" }, - { DMU_BSWAP_ZAP, TRUE, "DDT statistics" }, - { DMU_BSWAP_UINT8, TRUE, "System attributes" }, - { DMU_BSWAP_ZAP, TRUE, "SA master node" }, - { DMU_BSWAP_ZAP, TRUE, "SA attr registration" }, - { DMU_BSWAP_ZAP, TRUE, "SA attr layouts" }, - { DMU_BSWAP_ZAP, TRUE, "scan translations" }, - { DMU_BSWAP_UINT8, FALSE, "deduplicated block" }, - { DMU_BSWAP_ZAP, TRUE, "DSL deadlist map" }, - { DMU_BSWAP_UINT64, TRUE, "DSL deadlist map hdr" }, - { DMU_BSWAP_ZAP, TRUE, "DSL dir clones" }, - { DMU_BSWAP_UINT64, TRUE, "bpobj subobj" } + { DMU_BSWAP_UINT8, TRUE, FALSE, "unallocated" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "object directory" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "object array" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "packed nvlist" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "packed nvlist size" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "bpobj" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "bpobj header" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "SPA space map header" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "SPA space map" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "ZIL intent log" }, + { DMU_BSWAP_DNODE, TRUE, FALSE, "DMU dnode" }, + { DMU_BSWAP_OBJSET, TRUE, TRUE, "DMU objset" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "DSL directory" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL directory child map" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL dataset snap map" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL props" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "DSL dataset" }, + { DMU_BSWAP_ZNODE, TRUE, FALSE, "ZFS znode" }, + { DMU_BSWAP_OLDACL, TRUE, FALSE, "ZFS V0 ACL" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "ZFS plain file" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS directory" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS master node" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS delete queue" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "zvol object" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "zvol prop" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "other uint8[]" }, + { DMU_BSWAP_UINT64, FALSE, FALSE, "other uint64[]" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "other ZAP" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "persistent error log" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "SPA history" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "SPA history offsets" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "Pool properties" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL permissions" }, + { DMU_BSWAP_ACL, TRUE, FALSE, "ZFS ACL" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "ZFS SYSACL" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "FUID table" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "FUID table size" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL dataset next clones" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "scan work queue" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS user/group used" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS user/group quota" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "snapshot refcount tags" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "DDT ZAP algorithm" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "DDT statistics" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "System attributes" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "SA master node" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "SA attr registration" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "SA attr layouts" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "scan translations" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "deduplicated block" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL deadlist map" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "DSL deadlist map hdr" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL dir clones" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "bpobj subobj" } }; const dmu_object_byteswap_info_t dmu_ot_byteswap[DMU_BSWAP_NUMFUNCS] = { Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Tue Jul 31 00:48:08 2018 (r336957) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Tue Jul 31 00:56:41 2018 (r336958) @@ -496,6 +496,14 @@ dmu_objset_open_impl(spa_t *spa, dsl_dataset_t *ds, bl os->os_primary_cache = ZFS_CACHE_ALL; os->os_secondary_cache = ZFS_CACHE_ALL; } + /* + * These properties will be filled in by the logic in zfs_get_zplprop() + * when they are queried for the first time. + */ + os->os_version = OBJSET_PROP_UNINITIALIZED; + os->os_normalization = OBJSET_PROP_UNINITIALIZED; + os->os_utf8only = OBJSET_PROP_UNINITIALIZED; + os->os_casesensitivity = OBJSET_PROP_UNINITIALIZED; if (ds == NULL || !ds->ds_is_snapshot) os->os_zil_header = os->os_phys->os_zil_header; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Tue Jul 31 00:48:08 2018 (r336957) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Tue Jul 31 00:56:41 2018 (r336958) @@ -83,6 +83,13 @@ typedef enum dbuf_states { DB_EVICTING } dbuf_states_t; +typedef enum dbuf_cached_state { + DB_NO_CACHE = -1, + DB_DBUF_CACHE, + DB_DBUF_METADATA_CACHE, + DB_CACHE_MAX +} dbuf_cached_state_t; + struct dnode; struct dmu_tx; @@ -229,10 +236,11 @@ typedef struct dmu_buf_impl { */ avl_node_t db_link; - /* - * Link in dbuf_cache. - */ + /* Link in dbuf_cache or dbuf_metadata_cache */ multilist_node_t db_cache_link; + + /* Tells us which dbuf cache this dbuf is in, if any */ + dbuf_cached_state_t db_caching_status; /* Data which is unique to data (leaf) blocks: */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Tue Jul 31 00:48:08 2018 (r336957) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Tue Jul 31 00:56:41 2018 (r336958) @@ -109,7 +109,8 @@ typedef enum dmu_object_byteswap { /* * Defines a uint8_t object type. Object types specify if the data * in the object is metadata (boolean) and how to byteswap the data - * (dmu_object_byteswap_t). + * (dmu_object_byteswap_t). All of the types created by this method + * are cached in the dbuf metadata cache. */ #define DMU_OT(byteswap, metadata) \ (DMU_OT_NEWTYPE | \ @@ -124,6 +125,9 @@ typedef enum dmu_object_byteswap { ((ot) & DMU_OT_METADATA) : \ dmu_ot[(ot)].ot_metadata) +#define DMU_OT_IS_METADATA_CACHED(ot) (((ot) & DMU_OT_NEWTYPE) ? \ + B_TRUE : dmu_ot[(ot)].ot_dbuf_metadata_cache) + /* * These object types use bp_fill != 1 for their L0 bp's. Therefore they can't * have their data embedded (i.e. use a BP_IS_EMBEDDED() bp), because bp_fill @@ -799,6 +803,7 @@ typedef void arc_byteswap_func_t(void *buf, size_t siz typedef struct dmu_object_type_info { dmu_object_byteswap_t ot_byteswap; boolean_t ot_metadata; + boolean_t ot_dbuf_metadata_cache; char *ot_name; } dmu_object_type_info_t; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Tue Jul 31 00:48:08 2018 (r336957) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_objset.h Tue Jul 31 00:56:41 2018 (r336958) @@ -39,6 +39,7 @@ #include #include #include +#include #ifdef __cplusplus extern "C" { @@ -69,6 +70,7 @@ typedef struct objset_phys { dnode_phys_t os_groupused_dnode; } objset_phys_t; +#define OBJSET_PROP_UNINITIALIZED ((uint64_t)-1) struct objset { /* Immutable: */ struct dsl_dataset *os_dsl_dataset; @@ -100,6 +102,16 @@ struct objset { zfs_sync_type_t os_sync; zfs_redundant_metadata_type_t os_redundant_metadata; int os_recordsize; + /* + * The next four values are used as a cache of whatever's on disk, and + * are initialized the first time these properties are queried. Before + * being initialized with their real values, their values are + * OBJSET_PROP_UNINITIALIZED. + */ + uint64_t os_version; + uint64_t os_normalization; + uint64_t os_utf8only; + uint64_t os_casesensitivity; /* * Pointer is constant; the blkptr it points to is protected by Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c Tue Jul 31 00:48:08 2018 (r336957) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_vfsops.c Tue Jul 31 00:56:41 2018 (r336958) @@ -2232,6 +2232,7 @@ zfs_set_version(zfsvfs_t *zfsvfs, uint64_t newvers) dmu_tx_commit(tx); zfsvfs->z_version = newvers; + os->os_version = newvers; zfs_set_fuid_feature(zfsvfs); @@ -2244,17 +2245,47 @@ zfs_set_version(zfsvfs_t *zfsvfs, uint64_t newvers) int zfs_get_zplprop(objset_t *os, zfs_prop_t prop, uint64_t *value) { - const char *pname; - int error = ENOENT; + uint64_t *cached_copy = NULL; /* - * Look up the file system's value for the property. For the - * version property, we look up a slightly different string. + * Figure out where in the objset_t the cached copy would live, if it + * is available for the requested property. */ - if (prop == ZFS_PROP_VERSION) + if (os != NULL) { + switch (prop) { + case ZFS_PROP_VERSION: + cached_copy = &os->os_version; + break; + case ZFS_PROP_NORMALIZE: + cached_copy = &os->os_normalization; + break; + case ZFS_PROP_UTF8ONLY: + cached_copy = &os->os_utf8only; + break; + case ZFS_PROP_CASE: + cached_copy = &os->os_casesensitivity; + break; + default: + break; + } + } + if (cached_copy != NULL && *cached_copy != OBJSET_PROP_UNINITIALIZED) { + *value = *cached_copy; + return (0); + } + + /* + * If the property wasn't cached, look up the file system's value for + * the property. For the version property, we look up a slightly + * different string. + */ + const char *pname; + int error = ENOENT; + if (prop == ZFS_PROP_VERSION) { pname = ZPL_VERSION_STR; - else + } else { pname = zfs_prop_to_name(prop); + } if (os != NULL) { ASSERT3U(os->os_phys->os_type, ==, DMU_OST_ZFS); @@ -2279,6 +2310,15 @@ zfs_get_zplprop(objset_t *os, zfs_prop_t prop, uint64_ } error = 0; } + + /* + * If one of the methods for getting the property value above worked, + * copy it into the objset_t's cache. + */ + if (error == 0 && cached_copy != NULL) { + *cached_copy = *value; + } + return (error); } From owner-svn-src-all@freebsd.org Tue Jul 31 00:58:24 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 119951066DED; Tue, 31 Jul 2018 00:58:24 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9BF770676; Tue, 31 Jul 2018 00:58:23 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9A9504706; Tue, 31 Jul 2018 00:58:23 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V0wNtn010417; Tue, 31 Jul 2018 00:58:23 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V0wMZw010410; Tue, 31 Jul 2018 00:58:22 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310058.w6V0wMZw010410@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 00:58:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336959 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 336959 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 00:58:24 -0000 Author: mav Date: Tue Jul 31 00:58:21 2018 New Revision: 336959 URL: https://svnweb.freebsd.org/changeset/base/336959 Log: MFV r336958: 9337 zfs get all is slow due to uncached metadata This project's goal is to make read-heavy channel programs and zfs(1m) administrative commands faster by caching all the metadata that they will need in the dbuf layer. This will prevent the data from being evicted, so that any future call to i.e. zfs get all won't have to go to disk (very much). illumos/illumos-gate@adb52d9262f45a04318fc6e188fe2b7f59d989a5 Reviewed by: Prakash Surya Reviewed by: George Wilson Reviewed by: Thomas Caputi Approved by: Richard Lowe Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_objset.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c Tue Jul 31 00:56:41 2018 (r336958) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c Tue Jul 31 00:58:21 2018 (r336959) @@ -49,6 +49,7 @@ #include #include #include +#include uint_t zfs_dbuf_evict_key; @@ -74,24 +75,58 @@ static kcondvar_t dbuf_evict_cv; static boolean_t dbuf_evict_thread_exit; /* - * LRU cache of dbufs. The dbuf cache maintains a list of dbufs that - * are not currently held but have been recently released. These dbufs - * are not eligible for arc eviction until they are aged out of the cache. - * Dbufs are added to the dbuf cache once the last hold is released. If a - * dbuf is later accessed and still exists in the dbuf cache, then it will - * be removed from the cache and later re-added to the head of the cache. - * Dbufs that are aged out of the cache will be immediately destroyed and - * become eligible for arc eviction. + * There are two dbuf caches; each dbuf can only be in one of them at a time. + * + * 1. Cache of metadata dbufs, to help make read-heavy administrative commands + * from /sbin/zfs run faster. The "metadata cache" specifically stores dbufs + * that represent the metadata that describes filesystems/snapshots/ + * bookmarks/properties/etc. We only evict from this cache when we export a + * pool, to short-circuit as much I/O as possible for all administrative + * commands that need the metadata. There is no eviction policy for this + * cache, because we try to only include types in it which would occupy a + * very small amount of space per object but create a large impact on the + * performance of these commands. Instead, after it reaches a maximum size + * (which should only happen on very small memory systems with a very large + * number of filesystem objects), we stop taking new dbufs into the + * metadata cache, instead putting them in the normal dbuf cache. + * + * 2. LRU cache of dbufs. The "dbuf cache" maintains a list of dbufs that + * are not currently held but have been recently released. These dbufs + * are not eligible for arc eviction until they are aged out of the cache. + * Dbufs that are aged out of the cache will be immediately destroyed and + * become eligible for arc eviction. + * + * Dbufs are added to these caches once the last hold is released. If a dbuf is + * later accessed and still exists in the dbuf cache, then it will be removed + * from the cache and later re-added to the head of the cache. + * + * If a given dbuf meets the requirements for the metadata cache, it will go + * there, otherwise it will be considered for the generic LRU dbuf cache. The + * caches and the refcounts tracking their sizes are stored in an array indexed + * by those caches' matching enum values (from dbuf_cached_state_t). */ -static multilist_t *dbuf_cache; -static refcount_t dbuf_cache_size; -uint64_t dbuf_cache_max_bytes = 0; +typedef struct dbuf_cache { + multilist_t *cache; + refcount_t size; +} dbuf_cache_t; +dbuf_cache_t dbuf_caches[DB_CACHE_MAX]; -/* Set the default size of the dbuf cache to log2 fraction of arc size. */ +/* Size limits for the caches */ +uint64_t dbuf_cache_max_bytes = 0; +uint64_t dbuf_metadata_cache_max_bytes = 0; +/* Set the default sizes of the caches to log2 fraction of arc size */ int dbuf_cache_shift = 5; +int dbuf_metadata_cache_shift = 6; /* - * The dbuf cache uses a three-stage eviction policy: + * For diagnostic purposes, this is incremented whenever we can't add + * something to the metadata cache because it's full, and instead put + * the data in the regular dbuf cache. + */ +uint64_t dbuf_metadata_cache_overflow; + +/* + * The LRU dbuf cache uses a three-stage eviction policy: * - A low water marker designates when the dbuf eviction thread * should stop evicting from the dbuf cache. * - When we reach the maximum size (aka mid water mark), we @@ -404,6 +439,41 @@ dbuf_is_metadata(dmu_buf_impl_t *db) } /* + * This returns whether this dbuf should be stored in the metadata cache, which + * is based on whether it's from one of the dnode types that store data related + * to traversing dataset hierarchies. + */ +static boolean_t +dbuf_include_in_metadata_cache(dmu_buf_impl_t *db) +{ + DB_DNODE_ENTER(db); + dmu_object_type_t type = DB_DNODE(db)->dn_type; + DB_DNODE_EXIT(db); + + /* Check if this dbuf is one of the types we care about */ + if (DMU_OT_IS_METADATA_CACHED(type)) { + /* If we hit this, then we set something up wrong in dmu_ot */ + ASSERT(DMU_OT_IS_METADATA(type)); + + /* + * Sanity check for small-memory systems: don't allocate too + * much memory for this purpose. + */ + if (refcount_count(&dbuf_caches[DB_DBUF_METADATA_CACHE].size) > + dbuf_metadata_cache_max_bytes) { + dbuf_metadata_cache_overflow++; + DTRACE_PROBE1(dbuf__metadata__cache__overflow, + dmu_buf_impl_t *, db); + return (B_FALSE); + } + + return (B_TRUE); + } + + return (B_FALSE); +} + +/* * This function *must* return indices evenly distributed between all * sublists of the multilist. This is needed due to how the dbuf eviction * code is laid out; dbuf_evict_thread() assumes dbufs are evenly @@ -438,7 +508,7 @@ dbuf_cache_above_hiwater(void) uint64_t dbuf_cache_hiwater_bytes = (dbuf_cache_max_bytes * dbuf_cache_hiwater_pct) / 100; - return (refcount_count(&dbuf_cache_size) > + return (refcount_count(&dbuf_caches[DB_DBUF_CACHE].size) > dbuf_cache_max_bytes + dbuf_cache_hiwater_bytes); } @@ -448,7 +518,7 @@ dbuf_cache_above_lowater(void) uint64_t dbuf_cache_lowater_bytes = (dbuf_cache_max_bytes * dbuf_cache_lowater_pct) / 100; - return (refcount_count(&dbuf_cache_size) > + return (refcount_count(&dbuf_caches[DB_DBUF_CACHE].size) > dbuf_cache_max_bytes - dbuf_cache_lowater_bytes); } @@ -458,8 +528,9 @@ dbuf_cache_above_lowater(void) static void dbuf_evict_one(void) { - int idx = multilist_get_random_index(dbuf_cache); - multilist_sublist_t *mls = multilist_sublist_lock(dbuf_cache, idx); + int idx = multilist_get_random_index(dbuf_caches[DB_DBUF_CACHE].cache); + multilist_sublist_t *mls = multilist_sublist_lock( + dbuf_caches[DB_DBUF_CACHE].cache, idx); ASSERT(!MUTEX_HELD(&dbuf_evict_lock)); @@ -482,8 +553,10 @@ dbuf_evict_one(void) if (db != NULL) { multilist_sublist_remove(mls, db); multilist_sublist_unlock(mls); - (void) refcount_remove_many(&dbuf_cache_size, + (void) refcount_remove_many(&dbuf_caches[DB_DBUF_CACHE].size, db->db.db_size, db); + ASSERT3U(db->db_caching_status, ==, DB_DBUF_CACHE); + db->db_caching_status = DB_NO_CACHE; dbuf_destroy(db); } else { multilist_sublist_unlock(mls); @@ -570,7 +643,8 @@ dbuf_evict_notify(void) * because it's OK to occasionally make the wrong decision here, * and grabbing the lock results in massive lock contention. */ - if (refcount_count(&dbuf_cache_size) > dbuf_cache_max_bytes) { + if (refcount_count(&dbuf_caches[DB_DBUF_CACHE].size) > + dbuf_cache_max_bytes) { if (dbuf_cache_above_hiwater()) dbuf_evict_one(); cv_signal(&dbuf_evict_cv); @@ -610,15 +684,21 @@ retry: mutex_init(&h->hash_mutexes[i], NULL, MUTEX_DEFAULT, NULL); /* - * Setup the parameters for the dbuf cache. We set the size of the - * dbuf cache to 1/32nd (default) of the size of the ARC. If the value - * has been set in /etc/system and it's not greater than the size of - * the ARC, then we honor that value. + * Setup the parameters for the dbuf caches. We set the sizes of the + * dbuf cache and the metadata cache to 1/32nd and 1/16th (default) + * of the size of the ARC, respectively. If the values are set in + * /etc/system and they're not greater than the size of the ARC, then + * we honor that value. */ if (dbuf_cache_max_bytes == 0 || dbuf_cache_max_bytes >= arc_max_bytes()) { dbuf_cache_max_bytes = arc_max_bytes() >> dbuf_cache_shift; } + if (dbuf_metadata_cache_max_bytes == 0 || + dbuf_metadata_cache_max_bytes >= arc_max_bytes()) { + dbuf_metadata_cache_max_bytes = + arc_max_bytes() >> dbuf_metadata_cache_shift; + } /* * All entries are queued via taskq_dispatch_ent(), so min/maxalloc @@ -626,10 +706,13 @@ retry: */ dbu_evict_taskq = taskq_create("dbu_evict", 1, minclsyspri, 0, 0, 0); - dbuf_cache = multilist_create(sizeof (dmu_buf_impl_t), - offsetof(dmu_buf_impl_t, db_cache_link), - dbuf_cache_multilist_index_func); - refcount_create(&dbuf_cache_size); + for (dbuf_cached_state_t dcs = 0; dcs < DB_CACHE_MAX; dcs++) { + dbuf_caches[dcs].cache = + multilist_create(sizeof (dmu_buf_impl_t), + offsetof(dmu_buf_impl_t, db_cache_link), + dbuf_cache_multilist_index_func); + refcount_create(&dbuf_caches[dcs].size); + } tsd_create(&zfs_dbuf_evict_key, NULL); dbuf_evict_thread_exit = B_FALSE; @@ -663,8 +746,10 @@ dbuf_fini(void) mutex_destroy(&dbuf_evict_lock); cv_destroy(&dbuf_evict_cv); - refcount_destroy(&dbuf_cache_size); - multilist_destroy(dbuf_cache); + for (dbuf_cached_state_t dcs = 0; dcs < DB_CACHE_MAX; dcs++) { + refcount_destroy(&dbuf_caches[dcs].size); + multilist_destroy(dbuf_caches[dcs].cache); + } } /* @@ -2051,9 +2136,15 @@ dbuf_destroy(dmu_buf_impl_t *db) dbuf_clear_data(db); if (multilist_link_active(&db->db_cache_link)) { - multilist_remove(dbuf_cache, db); - (void) refcount_remove_many(&dbuf_cache_size, + ASSERT(db->db_caching_status == DB_DBUF_CACHE || + db->db_caching_status == DB_DBUF_METADATA_CACHE); + + multilist_remove(dbuf_caches[db->db_caching_status].cache, db); + (void) refcount_remove_many( + &dbuf_caches[db->db_caching_status].size, db->db.db_size, db); + + db->db_caching_status = DB_NO_CACHE; } ASSERT(db->db_state == DB_UNCACHED || db->db_state == DB_NOFILL); @@ -2107,6 +2198,7 @@ dbuf_destroy(dmu_buf_impl_t *db) ASSERT(db->db_hash_next == NULL); ASSERT(db->db_blkptr == NULL); ASSERT(db->db_data_pending == NULL); + ASSERT3U(db->db_caching_status, ==, DB_NO_CACHE); ASSERT(!multilist_link_active(&db->db_cache_link)); kmem_cache_free(dbuf_kmem_cache, db); @@ -2245,6 +2337,7 @@ dbuf_create(dnode_t *dn, uint8_t level, uint64_t blkid ASSERT3U(db->db.db_size, >=, dn->dn_bonuslen); db->db.db_offset = DMU_BONUS_BLKID; db->db_state = DB_UNCACHED; + db->db_caching_status = DB_NO_CACHE; /* the bonus dbuf is not placed in the hash table */ arc_space_consume(sizeof (dmu_buf_impl_t), ARC_SPACE_OTHER); return (db); @@ -2277,6 +2370,7 @@ dbuf_create(dnode_t *dn, uint8_t level, uint64_t blkid avl_add(&dn->dn_dbufs, db); db->db_state = DB_UNCACHED; + db->db_caching_status = DB_NO_CACHE; mutex_exit(&dn->dn_dbufs_mtx); arc_space_consume(sizeof (dmu_buf_impl_t), ARC_SPACE_OTHER); @@ -2619,9 +2713,15 @@ top: if (multilist_link_active(&db->db_cache_link)) { ASSERT(refcount_is_zero(&db->db_holds)); - multilist_remove(dbuf_cache, db); - (void) refcount_remove_many(&dbuf_cache_size, + ASSERT(db->db_caching_status == DB_DBUF_CACHE || + db->db_caching_status == DB_DBUF_METADATA_CACHE); + + multilist_remove(dbuf_caches[db->db_caching_status].cache, db); + (void) refcount_remove_many( + &dbuf_caches[db->db_caching_status].size, db->db.db_size, db); + + db->db_caching_status = DB_NO_CACHE; } (void) refcount_add(&db->db_holds, tag); DBUF_VERIFY(db); @@ -2838,12 +2938,22 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag) db->db_pending_evict) { dbuf_destroy(db); } else if (!multilist_link_active(&db->db_cache_link)) { - multilist_insert(dbuf_cache, db); - (void) refcount_add_many(&dbuf_cache_size, + ASSERT3U(db->db_caching_status, ==, + DB_NO_CACHE); + + dbuf_cached_state_t dcs = + dbuf_include_in_metadata_cache(db) ? + DB_DBUF_METADATA_CACHE : DB_DBUF_CACHE; + db->db_caching_status = dcs; + + multilist_insert(dbuf_caches[dcs].cache, db); + (void) refcount_add_many(&dbuf_caches[dcs].size, db->db.db_size, db); mutex_exit(&db->db_mtx); - dbuf_evict_notify(); + if (db->db_caching_status == DB_DBUF_CACHE) { + dbuf_evict_notify(); + } } if (do_arc_evict) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c Tue Jul 31 00:56:41 2018 (r336958) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c Tue Jul 31 00:58:21 2018 (r336959) @@ -79,60 +79,60 @@ SYSCTL_INT(_vfs_zfs, OID_AUTO, per_txg_dirty_frees_per int zfs_object_remap_one_indirect_delay_ticks = 0; const dmu_object_type_info_t dmu_ot[DMU_OT_NUMTYPES] = { - { DMU_BSWAP_UINT8, TRUE, "unallocated" }, - { DMU_BSWAP_ZAP, TRUE, "object directory" }, - { DMU_BSWAP_UINT64, TRUE, "object array" }, - { DMU_BSWAP_UINT8, TRUE, "packed nvlist" }, - { DMU_BSWAP_UINT64, TRUE, "packed nvlist size" }, - { DMU_BSWAP_UINT64, TRUE, "bpobj" }, - { DMU_BSWAP_UINT64, TRUE, "bpobj header" }, - { DMU_BSWAP_UINT64, TRUE, "SPA space map header" }, - { DMU_BSWAP_UINT64, TRUE, "SPA space map" }, - { DMU_BSWAP_UINT64, TRUE, "ZIL intent log" }, - { DMU_BSWAP_DNODE, TRUE, "DMU dnode" }, - { DMU_BSWAP_OBJSET, TRUE, "DMU objset" }, - { DMU_BSWAP_UINT64, TRUE, "DSL directory" }, - { DMU_BSWAP_ZAP, TRUE, "DSL directory child map"}, - { DMU_BSWAP_ZAP, TRUE, "DSL dataset snap map" }, - { DMU_BSWAP_ZAP, TRUE, "DSL props" }, - { DMU_BSWAP_UINT64, TRUE, "DSL dataset" }, - { DMU_BSWAP_ZNODE, TRUE, "ZFS znode" }, - { DMU_BSWAP_OLDACL, TRUE, "ZFS V0 ACL" }, - { DMU_BSWAP_UINT8, FALSE, "ZFS plain file" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS directory" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS master node" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS delete queue" }, - { DMU_BSWAP_UINT8, FALSE, "zvol object" }, - { DMU_BSWAP_ZAP, TRUE, "zvol prop" }, - { DMU_BSWAP_UINT8, FALSE, "other uint8[]" }, - { DMU_BSWAP_UINT64, FALSE, "other uint64[]" }, - { DMU_BSWAP_ZAP, TRUE, "other ZAP" }, - { DMU_BSWAP_ZAP, TRUE, "persistent error log" }, - { DMU_BSWAP_UINT8, TRUE, "SPA history" }, - { DMU_BSWAP_UINT64, TRUE, "SPA history offsets" }, - { DMU_BSWAP_ZAP, TRUE, "Pool properties" }, - { DMU_BSWAP_ZAP, TRUE, "DSL permissions" }, - { DMU_BSWAP_ACL, TRUE, "ZFS ACL" }, - { DMU_BSWAP_UINT8, TRUE, "ZFS SYSACL" }, - { DMU_BSWAP_UINT8, TRUE, "FUID table" }, - { DMU_BSWAP_UINT64, TRUE, "FUID table size" }, - { DMU_BSWAP_ZAP, TRUE, "DSL dataset next clones"}, - { DMU_BSWAP_ZAP, TRUE, "scan work queue" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS user/group used" }, - { DMU_BSWAP_ZAP, TRUE, "ZFS user/group quota" }, - { DMU_BSWAP_ZAP, TRUE, "snapshot refcount tags"}, - { DMU_BSWAP_ZAP, TRUE, "DDT ZAP algorithm" }, - { DMU_BSWAP_ZAP, TRUE, "DDT statistics" }, - { DMU_BSWAP_UINT8, TRUE, "System attributes" }, - { DMU_BSWAP_ZAP, TRUE, "SA master node" }, - { DMU_BSWAP_ZAP, TRUE, "SA attr registration" }, - { DMU_BSWAP_ZAP, TRUE, "SA attr layouts" }, - { DMU_BSWAP_ZAP, TRUE, "scan translations" }, - { DMU_BSWAP_UINT8, FALSE, "deduplicated block" }, - { DMU_BSWAP_ZAP, TRUE, "DSL deadlist map" }, - { DMU_BSWAP_UINT64, TRUE, "DSL deadlist map hdr" }, - { DMU_BSWAP_ZAP, TRUE, "DSL dir clones" }, - { DMU_BSWAP_UINT64, TRUE, "bpobj subobj" } + { DMU_BSWAP_UINT8, TRUE, FALSE, "unallocated" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "object directory" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "object array" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "packed nvlist" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "packed nvlist size" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "bpobj" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "bpobj header" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "SPA space map header" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "SPA space map" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "ZIL intent log" }, + { DMU_BSWAP_DNODE, TRUE, FALSE, "DMU dnode" }, + { DMU_BSWAP_OBJSET, TRUE, TRUE, "DMU objset" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "DSL directory" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL directory child map" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL dataset snap map" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL props" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "DSL dataset" }, + { DMU_BSWAP_ZNODE, TRUE, FALSE, "ZFS znode" }, + { DMU_BSWAP_OLDACL, TRUE, FALSE, "ZFS V0 ACL" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "ZFS plain file" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS directory" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS master node" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS delete queue" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "zvol object" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "zvol prop" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "other uint8[]" }, + { DMU_BSWAP_UINT64, FALSE, FALSE, "other uint64[]" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "other ZAP" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "persistent error log" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "SPA history" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "SPA history offsets" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "Pool properties" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL permissions" }, + { DMU_BSWAP_ACL, TRUE, FALSE, "ZFS ACL" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "ZFS SYSACL" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "FUID table" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "FUID table size" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL dataset next clones" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "scan work queue" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS user/group used" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "ZFS user/group quota" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "snapshot refcount tags" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "DDT ZAP algorithm" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "DDT statistics" }, + { DMU_BSWAP_UINT8, TRUE, FALSE, "System attributes" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "SA master node" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "SA attr registration" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "SA attr layouts" }, + { DMU_BSWAP_ZAP, TRUE, FALSE, "scan translations" }, + { DMU_BSWAP_UINT8, FALSE, FALSE, "deduplicated block" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL deadlist map" }, + { DMU_BSWAP_UINT64, TRUE, TRUE, "DSL deadlist map hdr" }, + { DMU_BSWAP_ZAP, TRUE, TRUE, "DSL dir clones" }, + { DMU_BSWAP_UINT64, TRUE, FALSE, "bpobj subobj" } }; const dmu_object_byteswap_info_t dmu_ot_byteswap[DMU_BSWAP_NUMFUNCS] = { Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c Tue Jul 31 00:56:41 2018 (r336958) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c Tue Jul 31 00:58:21 2018 (r336959) @@ -498,6 +498,14 @@ dmu_objset_open_impl(spa_t *spa, dsl_dataset_t *ds, bl os->os_primary_cache = ZFS_CACHE_ALL; os->os_secondary_cache = ZFS_CACHE_ALL; } + /* + * These properties will be filled in by the logic in zfs_get_zplprop() + * when they are queried for the first time. + */ + os->os_version = OBJSET_PROP_UNINITIALIZED; + os->os_normalization = OBJSET_PROP_UNINITIALIZED; + os->os_utf8only = OBJSET_PROP_UNINITIALIZED; + os->os_casesensitivity = OBJSET_PROP_UNINITIALIZED; if (ds == NULL || !ds->ds_is_snapshot) os->os_zil_header = os->os_phys->os_zil_header; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h Tue Jul 31 00:56:41 2018 (r336958) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h Tue Jul 31 00:58:21 2018 (r336959) @@ -83,6 +83,13 @@ typedef enum dbuf_states { DB_EVICTING } dbuf_states_t; +typedef enum dbuf_cached_state { + DB_NO_CACHE = -1, + DB_DBUF_CACHE, + DB_DBUF_METADATA_CACHE, + DB_CACHE_MAX +} dbuf_cached_state_t; + struct dnode; struct dmu_tx; @@ -229,10 +236,11 @@ typedef struct dmu_buf_impl { */ avl_node_t db_link; - /* - * Link in dbuf_cache. - */ + /* Link in dbuf_cache or dbuf_metadata_cache */ multilist_node_t db_cache_link; + + /* Tells us which dbuf cache this dbuf is in, if any */ + dbuf_cached_state_t db_caching_status; /* Data which is unique to data (leaf) blocks: */ Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Tue Jul 31 00:56:41 2018 (r336958) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Tue Jul 31 00:58:21 2018 (r336959) @@ -109,7 +109,8 @@ typedef enum dmu_object_byteswap { /* * Defines a uint8_t object type. Object types specify if the data * in the object is metadata (boolean) and how to byteswap the data - * (dmu_object_byteswap_t). + * (dmu_object_byteswap_t). All of the types created by this method + * are cached in the dbuf metadata cache. */ #define DMU_OT(byteswap, metadata) \ (DMU_OT_NEWTYPE | \ @@ -124,6 +125,9 @@ typedef enum dmu_object_byteswap { ((ot) & DMU_OT_METADATA) : \ dmu_ot[(ot)].ot_metadata) +#define DMU_OT_IS_METADATA_CACHED(ot) (((ot) & DMU_OT_NEWTYPE) ? \ + B_TRUE : dmu_ot[(ot)].ot_dbuf_metadata_cache) + /* * These object types use bp_fill != 1 for their L0 bp's. Therefore they can't * have their data embedded (i.e. use a BP_IS_EMBEDDED() bp), because bp_fill @@ -810,6 +814,7 @@ typedef void arc_byteswap_func_t(void *buf, size_t siz typedef struct dmu_object_type_info { dmu_object_byteswap_t ot_byteswap; boolean_t ot_metadata; + boolean_t ot_dbuf_metadata_cache; char *ot_name; } dmu_object_type_info_t; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_objset.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_objset.h Tue Jul 31 00:56:41 2018 (r336958) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_objset.h Tue Jul 31 00:58:21 2018 (r336959) @@ -39,6 +39,7 @@ #include #include #include +#include #ifdef __cplusplus extern "C" { @@ -69,6 +70,7 @@ typedef struct objset_phys { dnode_phys_t os_groupused_dnode; } objset_phys_t; +#define OBJSET_PROP_UNINITIALIZED ((uint64_t)-1) struct objset { /* Immutable: */ struct dsl_dataset *os_dsl_dataset; @@ -100,6 +102,16 @@ struct objset { zfs_sync_type_t os_sync; zfs_redundant_metadata_type_t os_redundant_metadata; int os_recordsize; + /* + * The next four values are used as a cache of whatever's on disk, and + * are initialized the first time these properties are queried. Before + * being initialized with their real values, their values are + * OBJSET_PROP_UNINITIALIZED. + */ + uint64_t os_version; + uint64_t os_normalization; + uint64_t os_utf8only; + uint64_t os_casesensitivity; /* * Pointer is constant; the blkptr it points to is protected by Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c Tue Jul 31 00:56:41 2018 (r336958) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c Tue Jul 31 00:58:21 2018 (r336959) @@ -2630,6 +2630,7 @@ zfs_set_version(zfsvfs_t *zfsvfs, uint64_t newvers) dmu_tx_commit(tx); zfsvfs->z_version = newvers; + os->os_version = newvers; zfs_set_fuid_feature(zfsvfs); @@ -2642,17 +2643,47 @@ zfs_set_version(zfsvfs_t *zfsvfs, uint64_t newvers) int zfs_get_zplprop(objset_t *os, zfs_prop_t prop, uint64_t *value) { - const char *pname; - int error = ENOENT; + uint64_t *cached_copy = NULL; /* - * Look up the file system's value for the property. For the - * version property, we look up a slightly different string. + * Figure out where in the objset_t the cached copy would live, if it + * is available for the requested property. */ - if (prop == ZFS_PROP_VERSION) + if (os != NULL) { + switch (prop) { + case ZFS_PROP_VERSION: + cached_copy = &os->os_version; + break; + case ZFS_PROP_NORMALIZE: + cached_copy = &os->os_normalization; + break; + case ZFS_PROP_UTF8ONLY: + cached_copy = &os->os_utf8only; + break; + case ZFS_PROP_CASE: + cached_copy = &os->os_casesensitivity; + break; + default: + break; + } + } + if (cached_copy != NULL && *cached_copy != OBJSET_PROP_UNINITIALIZED) { + *value = *cached_copy; + return (0); + } + + /* + * If the property wasn't cached, look up the file system's value for + * the property. For the version property, we look up a slightly + * different string. + */ + const char *pname; + int error = ENOENT; + if (prop == ZFS_PROP_VERSION) { pname = ZPL_VERSION_STR; - else + } else { pname = zfs_prop_to_name(prop); + } if (os != NULL) { ASSERT3U(os->os_phys->os_type, ==, DMU_OST_ZFS); @@ -2677,6 +2708,15 @@ zfs_get_zplprop(objset_t *os, zfs_prop_t prop, uint64_ } error = 0; } + + /* + * If one of the methods for getting the property value above worked, + * copy it into the objset_t's cache. + */ + if (error == 0 && cached_copy != NULL) { + *cached_copy = *value; + } + return (error); } From owner-svn-src-all@freebsd.org Tue Jul 31 01:00:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11A741066EBB; Tue, 31 Jul 2018 01:00:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7DE9709F3; Tue, 31 Jul 2018 01:00:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 868F8471D; Tue, 31 Jul 2018 01:00:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V10lKs010774; Tue, 31 Jul 2018 01:00:47 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V10lH3010773; Tue, 31 Jul 2018 01:00:47 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310100.w6V10lH3010773@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 01:00:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336960 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 336960 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 01:00:49 -0000 Author: mav Date: Tue Jul 31 01:00:46 2018 New Revision: 336960 URL: https://svnweb.freebsd.org/changeset/base/336960 Log: 9256 zfs send space estimation off by > 10% on some datasets illumos/illummos-gate@df477c0afa111b5205c872dab36dbfde391656de Reviewed by: Matt Ahrens Reviewed by: John Kennedy Approved by: Richard Lowe Author: Paul Dagnelie Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c Tue Jul 31 00:58:21 2018 (r336959) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c Tue Jul 31 01:00:46 2018 (r336960) @@ -66,6 +66,11 @@ int zfs_send_set_freerecords_bit = B_TRUE; static char *dmu_recv_tag = "dmu_recv_tag"; const char *recv_clone_name = "%recv"; +/* + * Use this to override the recordsize calculation for fast zfs send estimates. + */ +uint64_t zfs_override_estimate_recordsize = 0; + #define BP_SPAN(datablkszsec, indblkshift, level) \ (((uint64_t)datablkszsec) << (SPA_MINBLOCKSHIFT + \ (level) * (indblkshift - SPA_BLKPTRSHIFT))) @@ -1093,7 +1098,7 @@ static int dmu_adjust_send_estimate_for_indirects(dsl_dataset_t *ds, uint64_t uncompressed, uint64_t compressed, boolean_t stream_compressed, uint64_t *sizep) { - int err; + int err = 0; uint64_t size; /* * Assume that space (both on-disk and in-stream) is dominated by @@ -1106,7 +1111,9 @@ dmu_adjust_send_estimate_for_indirects(dsl_dataset_t * VERIFY0(dmu_objset_from_ds(ds, &os)); /* Assume all (uncompressed) blocks are recordsize. */ - if (os->os_phys->os_type == DMU_OST_ZVOL) { + if (zfs_override_estimate_recordsize != 0) { + recordsize = zfs_override_estimate_recordsize; + } else if (os->os_phys->os_type == DMU_OST_ZVOL) { err = dsl_prop_get_int_ds(ds, zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), &recordsize); } else { From owner-svn-src-all@freebsd.org Tue Jul 31 01:02:23 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FB881067061; Tue, 31 Jul 2018 01:02:23 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D398270C82; Tue, 31 Jul 2018 01:02:22 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B4C684886; Tue, 31 Jul 2018 01:02:22 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V12M3R015498; Tue, 31 Jul 2018 01:02:22 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V12MBt015497; Tue, 31 Jul 2018 01:02:22 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807310102.w6V12MBt015497@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 01:02:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336961 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 336961 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 01:02:23 -0000 Author: mav Date: Tue Jul 31 01:02:22 2018 New Revision: 336961 URL: https://svnweb.freebsd.org/changeset/base/336961 Log: MFV r336960: 9256 zfs send space estimation off by > 10% on some datasets illumos/illummos-gate@df477c0afa111b5205c872dab36dbfde391656de Reviewed by: Matt Ahrens Reviewed by: John Kennedy Approved by: Richard Lowe Author: Paul Dagnelie Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c Tue Jul 31 01:00:46 2018 (r336960) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c Tue Jul 31 01:02:22 2018 (r336961) @@ -76,6 +76,11 @@ TUNABLE_INT("vfs.zfs.send_set_freerecords_bit", &zfs_s static char *dmu_recv_tag = "dmu_recv_tag"; const char *recv_clone_name = "%recv"; +/* + * Use this to override the recordsize calculation for fast zfs send estimates. + */ +uint64_t zfs_override_estimate_recordsize = 0; + #define BP_SPAN(datablkszsec, indblkshift, level) \ (((uint64_t)datablkszsec) << (SPA_MINBLOCKSHIFT + \ (level) * (indblkshift - SPA_BLKPTRSHIFT))) @@ -1131,7 +1136,7 @@ static int dmu_adjust_send_estimate_for_indirects(dsl_dataset_t *ds, uint64_t uncompressed, uint64_t compressed, boolean_t stream_compressed, uint64_t *sizep) { - int err; + int err = 0; uint64_t size; /* * Assume that space (both on-disk and in-stream) is dominated by @@ -1144,7 +1149,9 @@ dmu_adjust_send_estimate_for_indirects(dsl_dataset_t * VERIFY0(dmu_objset_from_ds(ds, &os)); /* Assume all (uncompressed) blocks are recordsize. */ - if (os->os_phys->os_type == DMU_OST_ZVOL) { + if (zfs_override_estimate_recordsize != 0) { + recordsize = zfs_override_estimate_recordsize; + } else if (os->os_phys->os_type == DMU_OST_ZVOL) { err = dsl_prop_get_int_ds(ds, zfs_prop_to_name(ZFS_PROP_VOLBLOCKSIZE), &recordsize); } else { From owner-svn-src-all@freebsd.org Tue Jul 31 05:28:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06C87104E1EB for ; Tue, 31 Jul 2018 05:28:01 +0000 (UTC) (envelope-from eadler@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB26779EC3 for ; Tue, 31 Jul 2018 05:28:00 +0000 (UTC) (envelope-from eadler@freebsd.org) Received: from mail-yb0-f173.google.com (mail-yb0-f173.google.com [209.85.213.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) (Authenticated sender: eadler) by smtp.freebsd.org (Postfix) with ESMTPSA id 6BD2C10B1F for ; Tue, 31 Jul 2018 05:28:00 +0000 (UTC) (envelope-from eadler@freebsd.org) Received: by mail-yb0-f173.google.com with SMTP id x15-v6so5698207ybm.2 for ; Mon, 30 Jul 2018 22:28:00 -0700 (PDT) X-Gm-Message-State: AOUpUlGGT69TMvG3gfKYLxdPnpj/WlwekQaJesfo358Bhf9GkIzkplJk vwI8w1+0h96jdy7Zp4zN+l9Wox7y/9gyM+qu+OsjUg== X-Google-Smtp-Source: AAOMgpdC9iOSKC89nDEQE9hGUjdBon1/ltAxc4E87grs2TrjoPddrW9s4+IW7Jxvj/K8F/0glZJvd6eHrNTA+dFstxE= X-Received: by 2002:a25:4546:: with SMTP id s67-v6mr10971845yba.89.1533014879723; Mon, 30 Jul 2018 22:27:59 -0700 (PDT) MIME-Version: 1.0 References: <201807280737.w6S7b12F025452@repo.freebsd.org> <20180728233405.GD40119@kib.kiev.ua> In-Reply-To: From: Eitan Adler Date: Mon, 30 Jul 2018 22:27:33 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r336805 - in stable/11: share/man/man4 sys/dev/vt/hw/vga To: John Baldwin Cc: Kostik Belousov , src-committers , svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 05:28:01 -0000 On Mon, 30 Jul 2018 at 15:38, John Baldwin wrote: > > On 7/28/18 10:12 PM, Eitan Adler wrote: > > On Sat, 28 Jul 2018 at 16:34, Konstantin Belousov wrote: > >> > >> On Sat, Jul 28, 2018 at 07:37:01AM +0000, Eitan Adler wrote: > >>> Author: eadler > >>> Date: Sat Jul 28 07:37:01 2018 > >>> New Revision: 336805 > >>> URL: https://svnweb.freebsd.org/changeset/base/336805 > >>> > >>> Log: > >>> MFC r335629: > >>> > >>> vt: add option to ignore NO_VGA flag in ACPI > >>> > >>> To workaround buggy firmware that sets this flag when there's actually > >>> a VGA present. > >>> > >>> Ref D16003 > >>> PR: 229235 > >>> > >>> Modified: > >>> stable/11/share/man/man4/vt.4 > >>> stable/11/sys/dev/vt/hw/vga/vt_vga.c > >>> Directory Properties: > >>> stable/11/ (props changed) > >>> > >>> Modified: stable/11/share/man/man4/vt.4 > >>> ============================================================================== > >>> --- stable/11/share/man/man4/vt.4 Sat Jul 28 06:46:10 2018 (r336804) > >>> +++ stable/11/share/man/man4/vt.4 Sat Jul 28 07:37:01 2018 (r336805) > >>> @@ -44,6 +44,7 @@ > >>> In > >>> .Xr loader.conf 5 : > >>> .Cd hw.vga.textmode=1 > >>> +.Cd hw.vga.acpi_ignore_no_vga=1 > >>> .Cd kern.vty=vt > >>> .Cd kern.vt.color..rgb="" > >>> .Cd kern.vt.fb.default_mode="x" > >>> @@ -196,6 +197,10 @@ prompt or in > >>> Set to 1 to use virtual terminals in text mode instead of graphics mode. > >>> Features that require graphics mode, like loadable fonts, will be > >>> disabled. > >>> +.It Va hw.vga.acpi_ignore_no_vga > >>> +Set to 1 to force the usage of the VGA driver regardless of whether > >>> +ACPI IAPC_BOOT_ARCH signals no VGA support. > >>> +Can be used to workaround firmware bugs in the ACPI tables. > >>> .It Va kern.vty > >>> Set this value to > >>> .Ql vt > >>> > >>> Modified: stable/11/sys/dev/vt/hw/vga/vt_vga.c > >>> ============================================================================== > >>> --- stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 06:46:10 2018 (r336804) > >>> +++ stable/11/sys/dev/vt/hw/vga/vt_vga.c Sat Jul 28 07:37:01 2018 (r336805) > >>> @@ -1211,6 +1211,12 @@ vga_acpi_disabled(void) > >>> ACPI_TABLE_FADT *fadt; > >>> vm_paddr_t physaddr; > >>> uint16_t flags; > >>> + int ignore; > >>> + > >>> + TUNABLE_INT_FETCH("hw.vga.acpi_ignore_no_vga", &ignore); > >>> + > >>> + if (ignore) > >>> + return (false); > > > >> If the tunable is not set, this code evaluates the stack garbage. > >> Why did you merged this without asking the author and not looking > >> at the followups ? > > > > Please assume some level of good faith. > > > > I MFCed this because I was asked to here > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229235 over a month > > ago. During that time no one mentioned or linked any followups. I did > > look in my email for mentions of r335629 as well - of which there are > > none. > > > > Now that you mention it, I looked at the svn log and found r335631 > > which I will MFC as well. > > If MFC'ing a change by another developer you should proactively look at > the svn logs of the changed files to see if there are follow up fixes. I did this, but looked by bug ID and by revision ID. Could I have found r335631 if I had looked at svn logs too? yes. Is it appropriate to say I did not look for followups? no Anyways, -stable is good now so this is my last response to the thread. -- Eitan Adler Source, Ports, Doc committer Bugmeister, Ports Security teams From owner-svn-src-all@freebsd.org Tue Jul 31 05:47:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9F29104E82F for ; Tue, 31 Jul 2018 05:47:59 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yw0-x232.google.com (mail-yw0-x232.google.com [IPv6:2607:f8b0:4002:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 50FC97A7F1 for ; Tue, 31 Jul 2018 05:47:59 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yw0-x232.google.com with SMTP id j68-v6so5322549ywg.1 for ; Mon, 30 Jul 2018 22:47:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LqBDm/Qi3WGCXfWk/yXj+CRNPQJyzI5X66Um0p0OwwE=; b=hhoP6cZ2BywzKluxNNW1daoSVevKKoUY/ZlBjWRlf3YQHxO6L4fVCyMIULK9EJvR+h 2o9hRcwpqNfw+U1H8o3Yc87PjVrdj6Kdzg+EnCqXfJG6gCGD8lq3XjxjeLFBw6P5k9ed 6kQ2YCGMGfv4Y9JHQfKeDj3FYK4lTd+W1VIag= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LqBDm/Qi3WGCXfWk/yXj+CRNPQJyzI5X66Um0p0OwwE=; b=dK+0zlN2Ka2R4wqzFIwrhW/aMw+XiQGTB5R34/gxKPcMhl76yjY3d14D8qi55Hx+0d 71LhXnalTmkqzmPnSSX6QwkDmzGQRio5uTHxakwlkcH8gMep0yGvDCPdrc79Zh826SAO j+fGQh1bHjY983MtGemU3bnGI+zQBUjjC5FCr5IUZG3ugqqHSyGjz43vDnkzbFK9W95c 52TQgINNm6l+LbfNSqe/pfwLerjfD3t1UWU//cBi6eHYiHsktIn2mt134KUCWHcbez6s SPHSPwASMop6pkyx0y1WHemBVmxowZC3abhkujgWpZxhCvPaGAdZ0PxMVN3Kw2wJBpYq MmGA== X-Gm-Message-State: AOUpUlH7XcoFK7+wAwPiyJQp9r5NlU9FAjAk/RbEWN0I64Q3oD/eiZYg haAwCDB7/SqehHg4o69dY4a4O1nq/xnavaiuEesvYw== X-Google-Smtp-Source: AAOMgpe2em9BOKIIdKjlP9XgwJs3+iZj1+2yXjW1BlDaZKWfB6qbCnuhl/SYvbk2AVjvkEkVG+07uCoidPiijaU90YI= X-Received: by 2002:a81:af06:: with SMTP id n6-v6mr10266126ywh.113.1533016078553; Mon, 30 Jul 2018 22:47:58 -0700 (PDT) MIME-Version: 1.0 References: <201807310037.w6V0bPlm099916@repo.freebsd.org> In-Reply-To: <201807310037.w6V0bPlm099916@repo.freebsd.org> From: Eitan Adler Date: Mon, 30 Jul 2018 22:47:30 -0700 Message-ID: Subject: Re: svn commit: r336953 - in stable: 10/usr.bin/mail 11/usr.bin/mail To: Mark Johnston Cc: src-committers , svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 05:47:59 -0000 On Mon, 30 Jul 2018 at 17:37, Mark Johnston wrote: > > Author: markj > Date: Tue Jul 31 00:37:25 2018 > New Revision: 336953 > URL: https://svnweb.freebsd.org/changeset/base/336953 > > Log: > Revert r335693, r335694, r335695 by eadler. > > They cause mail(1) to crash in certain scenarios. Thanks! I was going to do this myself shortly. -- Eitan Adler From owner-svn-src-all@freebsd.org Tue Jul 31 06:27:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BBD63104F98A; Tue, 31 Jul 2018 06:27:06 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E9E67BBF3; Tue, 31 Jul 2018 06:27:06 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4F5377F56; Tue, 31 Jul 2018 06:27:06 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6V6R6WK079083; Tue, 31 Jul 2018 06:27:06 GMT (envelope-from tuexen@FreeBSD.org) Received: (from tuexen@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6V6R6nI079082; Tue, 31 Jul 2018 06:27:06 GMT (envelope-from tuexen@FreeBSD.org) Message-Id: <201807310627.w6V6R6nI079082@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tuexen set sender to tuexen@FreeBSD.org using -f From: Michael Tuexen Date: Tue, 31 Jul 2018 06:27:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336962 - head/sys/netinet X-SVN-Group: head X-SVN-Commit-Author: tuexen X-SVN-Commit-Paths: head/sys/netinet X-SVN-Commit-Revision: 336962 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 06:27:06 -0000 Author: tuexen Date: Tue Jul 31 06:27:05 2018 New Revision: 336962 URL: https://svnweb.freebsd.org/changeset/base/336962 Log: Fix INET only builds. r336940 introduced an "unused variable" warning on platforms which support INET, but not INET6, like MALTA and MALTA64 as reported by Mark Millard. Improve the #ifdefs to address this issue. Sponsored by: Netflix, Inc. Modified: head/sys/netinet/tcp_usrreq.c Modified: head/sys/netinet/tcp_usrreq.c ============================================================================== --- head/sys/netinet/tcp_usrreq.c Tue Jul 31 01:02:22 2018 (r336961) +++ head/sys/netinet/tcp_usrreq.c Tue Jul 31 06:27:05 2018 (r336962) @@ -902,7 +902,10 @@ tcp_usr_send(struct socket *so, int flags, struct mbuf struct tcpcb *tp = NULL; struct epoch_tracker net_et; #ifdef INET - struct sockaddr_in sin, *sinp; +#ifdef INET6 + struct sockaddr_in sin; +#endif + struct sockaddr_in *sinp; #endif #ifdef INET6 int isipv6; From owner-svn-src-all@freebsd.org Tue Jul 31 10:18:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BCB91054BFB; Tue, 31 Jul 2018 10:18:32 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3135A83411; Tue, 31 Jul 2018 10:18:32 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 136C1124D1; Tue, 31 Jul 2018 10:18:32 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VAIVw8098801; Tue, 31 Jul 2018 10:18:31 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VAIVeW098798; Tue, 31 Jul 2018 10:18:31 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807311018.w6VAIVeW098798@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 31 Jul 2018 10:18:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336963 - in stable/11/sys: amd64/amd64 i386/i386 i386/isa X-SVN-Group: stable-11 X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in stable/11/sys: amd64/amd64 i386/i386 i386/isa X-SVN-Commit-Revision: 336963 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 10:18:32 -0000 Author: kib Date: Tue Jul 31 10:18:30 2018 New Revision: 336963 URL: https://svnweb.freebsd.org/changeset/base/336963 Log: MFC r336683: Extend ranges of the critical sections to ensure that context switch code never sees FPU pcb flags not consistent with the hardware state. Modified: stable/11/sys/amd64/amd64/fpu.c stable/11/sys/amd64/amd64/machdep.c stable/11/sys/i386/i386/machdep.c stable/11/sys/i386/isa/npx.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/amd64/fpu.c ============================================================================== --- stable/11/sys/amd64/amd64/fpu.c Tue Jul 31 06:27:05 2018 (r336962) +++ stable/11/sys/amd64/amd64/fpu.c Tue Jul 31 10:18:30 2018 (r336963) @@ -744,22 +744,22 @@ fpugetregs(struct thread *td) int max_ext_n, i, owned; pcb = td->td_pcb; + critical_enter(); if ((pcb->pcb_flags & PCB_USERFPUINITDONE) == 0) { bcopy(fpu_initialstate, get_pcb_user_save_pcb(pcb), cpu_max_ext_state_size); get_pcb_user_save_pcb(pcb)->sv_env.en_cw = pcb->pcb_initial_fpucw; fpuuserinited(td); + critical_exit(); return (_MC_FPOWNED_PCB); } - critical_enter(); if (td == PCPU_GET(fpcurthread) && PCB_USER_FPU(pcb)) { fpusave(get_pcb_user_save_pcb(pcb)); owned = _MC_FPOWNED_FPU; } else { owned = _MC_FPOWNED_PCB; } - critical_exit(); if (use_xsave) { /* * Handle partially saved state. @@ -779,6 +779,7 @@ fpugetregs(struct thread *td) *xstate_bv |= bit; } } + critical_exit(); return (owned); } @@ -787,6 +788,7 @@ fpuuserinited(struct thread *td) { struct pcb *pcb; + CRITICAL_ASSERT(td); pcb = td->td_pcb; if (PCB_USER_FPU(pcb)) set_pcb_flags(pcb, @@ -845,26 +847,25 @@ fpusetregs(struct thread *td, struct savefpu *addr, ch addr->sv_env.en_mxcsr &= cpu_mxcsr_mask; pcb = td->td_pcb; + error = 0; critical_enter(); if (td == PCPU_GET(fpcurthread) && PCB_USER_FPU(pcb)) { error = fpusetxstate(td, xfpustate, xfpustate_size); - if (error != 0) { - critical_exit(); - return (error); + if (error == 0) { + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + fpurestore(get_pcb_user_save_td(td)); + set_pcb_flags(pcb, PCB_FPUINITDONE | + PCB_USERFPUINITDONE); } - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - fpurestore(get_pcb_user_save_td(td)); - critical_exit(); - set_pcb_flags(pcb, PCB_FPUINITDONE | PCB_USERFPUINITDONE); } else { - critical_exit(); error = fpusetxstate(td, xfpustate, xfpustate_size); - if (error != 0) - return (error); - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - fpuuserinited(td); + if (error == 0) { + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + fpuuserinited(td); + } } - return (0); + critical_exit(); + return (error); } /* @@ -1037,6 +1038,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx ctx->flags = FPU_KERN_CTX_DUMMY | FPU_KERN_CTX_INUSE; return (0); } + critical_enter(); KASSERT(!PCB_USER_FPU(pcb) || pcb->pcb_save == get_pcb_user_save_pcb(pcb), ("mangled pcb_save")); ctx->flags = FPU_KERN_CTX_INUSE; @@ -1047,6 +1049,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx pcb->pcb_save = fpu_kern_ctx_savefpu(ctx); set_pcb_flags(pcb, PCB_KERNFPU); clear_pcb_flags(pcb, PCB_FPUINITDONE); + critical_exit(); return (0); } @@ -1065,7 +1068,6 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx clear_pcb_flags(pcb, PCB_FPUNOSAVE | PCB_FPUINITDONE); start_emulating(); - critical_exit(); } else { KASSERT((ctx->flags & FPU_KERN_CTX_INUSE) != 0, ("leaving not inuse ctx")); @@ -1079,7 +1081,6 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx critical_enter(); if (curthread == PCPU_GET(fpcurthread)) fpudrop(); - critical_exit(); pcb->pcb_save = ctx->prev; } @@ -1096,6 +1097,7 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx clear_pcb_flags(pcb, PCB_FPUINITDONE); KASSERT(!PCB_USER_FPU(pcb), ("unpaired fpu_kern_leave")); } + critical_exit(); return (0); } Modified: stable/11/sys/amd64/amd64/machdep.c ============================================================================== --- stable/11/sys/amd64/amd64/machdep.c Tue Jul 31 06:27:05 2018 (r336962) +++ stable/11/sys/amd64/amd64/machdep.c Tue Jul 31 10:18:30 2018 (r336963) @@ -2158,8 +2158,10 @@ int set_fpregs(struct thread *td, struct fpreg *fpregs) { + critical_enter(); set_fpregs_xmm(fpregs, get_pcb_user_save_td(td)); fpuuserinited(td); + critical_exit(); return (0); } Modified: stable/11/sys/i386/i386/machdep.c ============================================================================== --- stable/11/sys/i386/i386/machdep.c Tue Jul 31 06:27:05 2018 (r336962) +++ stable/11/sys/i386/i386/machdep.c Tue Jul 31 10:18:30 2018 (r336963) @@ -3004,6 +3004,7 @@ int set_fpregs(struct thread *td, struct fpreg *fpregs) { + critical_enter(); if (cpu_fxsr) npx_set_fpregs_xmm((struct save87 *)fpregs, &get_pcb_user_save_td(td)->sv_xmm); @@ -3011,6 +3012,7 @@ set_fpregs(struct thread *td, struct fpreg *fpregs) bcopy(fpregs, &get_pcb_user_save_td(td)->sv_87, sizeof(*fpregs)); npxuserinited(td); + critical_exit(); return (0); } Modified: stable/11/sys/i386/isa/npx.c ============================================================================== --- stable/11/sys/i386/isa/npx.c Tue Jul 31 06:27:05 2018 (r336962) +++ stable/11/sys/i386/isa/npx.c Tue Jul 31 10:18:30 2018 (r336963) @@ -974,14 +974,15 @@ npxgetregs(struct thread *td) return (_MC_FPOWNED_NONE); pcb = td->td_pcb; + critical_enter(); if ((pcb->pcb_flags & PCB_NPXINITDONE) == 0) { bcopy(npx_initialstate, get_pcb_user_save_pcb(pcb), cpu_max_ext_state_size); SET_FPU_CW(get_pcb_user_save_pcb(pcb), pcb->pcb_initial_npxcw); npxuserinited(td); + critical_exit(); return (_MC_FPOWNED_PCB); } - critical_enter(); if (td == PCPU_GET(fpcurthread)) { fpusave(get_pcb_user_save_pcb(pcb)); if (!cpu_fxsr) @@ -995,7 +996,6 @@ npxgetregs(struct thread *td) } else { owned = _MC_FPOWNED_PCB; } - critical_exit(); if (use_xsave) { /* * Handle partially saved state. @@ -1018,6 +1018,7 @@ npxgetregs(struct thread *td) *xstate_bv |= bit; } } + critical_exit(); return (owned); } @@ -1026,6 +1027,7 @@ npxuserinited(struct thread *td) { struct pcb *pcb; + CRITICAL_ASSERT(td); pcb = td->td_pcb; if (PCB_USER_FPU(pcb)) pcb->pcb_flags |= PCB_NPXINITDONE; @@ -1083,28 +1085,26 @@ npxsetregs(struct thread *td, union savefpu *addr, cha if (cpu_fxsr) addr->sv_xmm.sv_env.en_mxcsr &= cpu_mxcsr_mask; pcb = td->td_pcb; + error = 0; critical_enter(); if (td == PCPU_GET(fpcurthread) && PCB_USER_FPU(pcb)) { error = npxsetxstate(td, xfpustate, xfpustate_size); - if (error != 0) { - critical_exit(); - return (error); + if (error == 0) { + if (!cpu_fxsr) + fnclex(); /* As in npxdrop(). */ + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + fpurstor(get_pcb_user_save_td(td)); + pcb->pcb_flags |= PCB_NPXUSERINITDONE | PCB_NPXINITDONE; } - if (!cpu_fxsr) - fnclex(); /* As in npxdrop(). */ - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - fpurstor(get_pcb_user_save_td(td)); - critical_exit(); - pcb->pcb_flags |= PCB_NPXUSERINITDONE | PCB_NPXINITDONE; } else { - critical_exit(); error = npxsetxstate(td, xfpustate, xfpustate_size); - if (error != 0) - return (error); - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - npxuserinited(td); + if (error == 0) { + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + npxuserinited(td); + } } - return (0); + critical_exit(); + return (error); } static void @@ -1373,6 +1373,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx return (0); } pcb = td->td_pcb; + critical_enter(); KASSERT(!PCB_USER_FPU(pcb) || pcb->pcb_save == get_pcb_user_save_pcb(pcb), ("mangled pcb_save")); ctx->flags = FPU_KERN_CTX_INUSE; @@ -1383,6 +1384,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx pcb->pcb_save = fpu_kern_ctx_savefpu(ctx); pcb->pcb_flags |= PCB_KERNNPX; pcb->pcb_flags &= ~PCB_NPXINITDONE; + critical_exit(); return (0); } @@ -1401,7 +1403,6 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx critical_enter(); if (curthread == PCPU_GET(fpcurthread)) npxdrop(); - critical_exit(); pcb->pcb_save = ctx->prev; if (pcb->pcb_save == get_pcb_user_save_pcb(pcb)) { if ((pcb->pcb_flags & PCB_NPXUSERINITDONE) != 0) @@ -1416,6 +1417,7 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx pcb->pcb_flags &= ~PCB_NPXINITDONE; KASSERT(!PCB_USER_FPU(pcb), ("unpaired fpu_kern_leave")); } + critical_exit(); return (0); } From owner-svn-src-all@freebsd.org Tue Jul 31 11:23:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 456431056C77; Tue, 31 Jul 2018 11:23:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE73186188; Tue, 31 Jul 2018 11:23:44 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CF02312FFF; Tue, 31 Jul 2018 11:23:44 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VBNiuW034205; Tue, 31 Jul 2018 11:23:44 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VBNikQ034204; Tue, 31 Jul 2018 11:23:44 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201807311123.w6VBNikQ034204@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Tue, 31 Jul 2018 11:23:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336964 - head/sys/ofed/include/rdma X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: head/sys/ofed/include/rdma X-SVN-Commit-Revision: 336964 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 11:23:45 -0000 Author: hselasky Date: Tue Jul 31 11:23:44 2018 New Revision: 336964 URL: https://svnweb.freebsd.org/changeset/base/336964 Log: Only NULL check the VNET pointer when VIMAGE is enabled in ibcore. Else a NULL VNET pointer should be ignored. This fixes address resolving when VIMAGE is disabled. MFC after: 3 days Sponsored by: Mellanox Technologies Modified: head/sys/ofed/include/rdma/ib_sa.h Modified: head/sys/ofed/include/rdma/ib_sa.h ============================================================================== --- head/sys/ofed/include/rdma/ib_sa.h Tue Jul 31 10:18:30 2018 (r336963) +++ head/sys/ofed/include/rdma/ib_sa.h Tue Jul 31 11:23:44 2018 (r336964) @@ -182,7 +182,11 @@ struct ib_sa_path_rec { static inline struct net_device *ib_get_ndev_from_path(struct ib_sa_path_rec *rec) { - return rec->net ? dev_get_by_index(rec->net, rec->ifindex) : NULL; +#ifdef VIMAGE + if (rec->net == NULL) + return NULL; +#endif + return dev_get_by_index(rec->net, rec->ifindex); } #define IB_SA_MCMEMBER_REC_MGID IB_SA_COMP_MASK( 0) From owner-svn-src-all@freebsd.org Tue Jul 31 12:36:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F36F10590AC; Tue, 31 Jul 2018 12:36:48 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4394188DE4; Tue, 31 Jul 2018 12:36:48 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1F0A013B31; Tue, 31 Jul 2018 12:36:48 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VCal16070187; Tue, 31 Jul 2018 12:36:47 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VCalrB070184; Tue, 31 Jul 2018 12:36:47 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807311236.w6VCalrB070184@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Tue, 31 Jul 2018 12:36:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336965 - head/sys/fs/cd9660 X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/sys/fs/cd9660 X-SVN-Commit-Revision: 336965 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 12:36:48 -0000 Author: emaste Date: Tue Jul 31 12:36:46 2018 New Revision: 336965 URL: https://svnweb.freebsd.org/changeset/base/336965 Log: cd9660: replace bcopy/bzero with C standard equivalents To reduce diffs against NetBSD. Modified: head/sys/fs/cd9660/cd9660_lookup.c head/sys/fs/cd9660/cd9660_rrip.c head/sys/fs/cd9660/cd9660_vfsops.c head/sys/fs/cd9660/cd9660_vnops.c Modified: head/sys/fs/cd9660/cd9660_lookup.c ============================================================================== --- head/sys/fs/cd9660/cd9660_lookup.c Tue Jul 31 11:23:44 2018 (r336964) +++ head/sys/fs/cd9660/cd9660_lookup.c Tue Jul 31 12:36:46 2018 (r336965) @@ -377,7 +377,7 @@ found: */ if (dp->i_number != i_ino) { ep2 = malloc(reclen, M_TEMP, M_WAITOK); - bcopy(ep, ep2, reclen); + memcpy(ep2, ep, reclen); ep = ep2; } brelse(bp); Modified: head/sys/fs/cd9660/cd9660_rrip.c ============================================================================== --- head/sys/fs/cd9660/cd9660_rrip.c Tue Jul 31 11:23:44 2018 (r336964) +++ head/sys/fs/cd9660/cd9660_rrip.c Tue Jul 31 12:36:46 2018 (r336965) @@ -204,7 +204,7 @@ cd9660_rrip_slink(p,ana) return 0; } - bcopy(inbuf,outbuf,wlen); + memcpy(outbuf, inbuf, wlen); outbuf += wlen; len += wlen; @@ -278,7 +278,7 @@ cd9660_rrip_altname(p,ana) return 0; } - bcopy(inbuf,ana->outbuf,wlen); + memcpy(ana->outbuf, inbuf, wlen); ana->outbuf += wlen; if (!cont) { @@ -356,7 +356,7 @@ cd9660_rrip_tstamp(p,ana) ISO_FTYPE_RRIP); ptime += 7; } else - bzero(&ana->inop->inode.iso_mtime,sizeof(struct timespec)); + memset(&ana->inop->inode.iso_mtime, 0, sizeof(struct timespec)); if (*p->flags&ISO_SUSP_TSTAMP_ACCESS) { cd9660_tstamp_conv7(ptime,&ana->inop->inode.iso_atime, @@ -379,7 +379,7 @@ cd9660_rrip_tstamp(p,ana) cd9660_tstamp_conv17(ptime,&ana->inop->inode.iso_mtime); ptime += 17; } else - bzero(&ana->inop->inode.iso_mtime,sizeof(struct timespec)); + memset(&ana->inop->inode.iso_mtime, 0, sizeof(struct timespec)); if (*p->flags&ISO_SUSP_TSTAMP_ACCESS) { cd9660_tstamp_conv17(ptime,&ana->inop->inode.iso_atime); Modified: head/sys/fs/cd9660/cd9660_vfsops.c ============================================================================== --- head/sys/fs/cd9660/cd9660_vfsops.c Tue Jul 31 11:23:44 2018 (r336964) +++ head/sys/fs/cd9660/cd9660_vfsops.c Tue Jul 31 12:36:46 2018 (r336965) @@ -363,7 +363,7 @@ iso_mountfs(devvp, mp) * filehandle validation. */ isomp->volume_space_size += ssector; - bcopy (rootp, isomp->root, sizeof isomp->root); + memcpy(isomp->root, rootp, sizeof isomp->root); isomp->root_extent = isonum_733 (rootp->extent); isomp->root_size = isonum_733 (rootp->size); @@ -465,7 +465,7 @@ iso_mountfs(devvp, mp) joliet_level); rootp = (struct iso_directory_record *) sup->root_directory_record; - bcopy (rootp, isomp->root, sizeof isomp->root); + memcpy(isomp->root, rootp, sizeof isomp->root); isomp->root_extent = isonum_733 (rootp->extent); isomp->root_size = isonum_733 (rootp->size); isomp->joliet_level = joliet_level; Modified: head/sys/fs/cd9660/cd9660_vnops.c ============================================================================== --- head/sys/fs/cd9660/cd9660_vnops.c Tue Jul 31 11:23:44 2018 (r336964) +++ head/sys/fs/cd9660/cd9660_vnops.c Tue Jul 31 12:36:46 2018 (r336965) @@ -446,10 +446,10 @@ iso_shipdir(idp) idp->current.d_reclen = GENERIC_DIRSIZ(&idp->current); if (assoc) { idp->assocoff = idp->curroff; - bcopy(&idp->current,&idp->assocent,idp->current.d_reclen); + memcpy(&idp->assocent, &idp->current, idp->current.d_reclen); } else { idp->saveoff = idp->curroff; - bcopy(&idp->current,&idp->saveent,idp->current.d_reclen); + memcpy(&idp->saveent, &idp->current, idp->current.d_reclen); } return (0); } From owner-svn-src-all@freebsd.org Tue Jul 31 12:44:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81ACE1059412; Tue, 31 Jul 2018 12:44:30 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 309468938C; Tue, 31 Jul 2018 12:44:30 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0B99113CC1; Tue, 31 Jul 2018 12:44:30 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VCiTPi075239; Tue, 31 Jul 2018 12:44:29 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VCiTDD075234; Tue, 31 Jul 2018 12:44:29 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807311244.w6VCiTDD075234@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Tue, 31 Jul 2018 12:44:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336966 - head/sys/fs/msdosfs X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/sys/fs/msdosfs X-SVN-Commit-Revision: 336966 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 12:44:30 -0000 Author: emaste Date: Tue Jul 31 12:44:28 2018 New Revision: 336966 URL: https://svnweb.freebsd.org/changeset/base/336966 Log: msdosfs: trim EOL whitespace Modified: head/sys/fs/msdosfs/msdosfs_conv.c head/sys/fs/msdosfs/msdosfs_denode.c head/sys/fs/msdosfs/msdosfs_lookup.c head/sys/fs/msdosfs/msdosfs_vfsops.c head/sys/fs/msdosfs/msdosfs_vnops.c Modified: head/sys/fs/msdosfs/msdosfs_conv.c ============================================================================== --- head/sys/fs/msdosfs/msdosfs_conv.c Tue Jul 31 12:36:46 2018 (r336965) +++ head/sys/fs/msdosfs/msdosfs_conv.c Tue Jul 31 12:44:28 2018 (r336966) @@ -70,7 +70,7 @@ static uint16_t unix2winchr(const u_char **, size_t *, /* * 0 - character disallowed in long file name. - * 1 - character should be replaced by '_' in DOS file name, + * 1 - character should be replaced by '_' in DOS file name, * and generation number inserted. * 2 - character ('.' and ' ') should be skipped in DOS file name, * and generation number inserted. @@ -353,7 +353,7 @@ unix2dosfn(const u_char *un, u_char dn[12], size_t unl * Note(2003/7): It seems recent Windows has * defferent rule than this code, that Windows * ignores all dots before extension, and use all - * chars as filename except for dots. + * chars as filename except for dots. */ dp = dp1 = NULL; for (cp = un + 1, i = unlen - 1; --i >= 0;) { Modified: head/sys/fs/msdosfs/msdosfs_denode.c ============================================================================== --- head/sys/fs/msdosfs/msdosfs_denode.c Tue Jul 31 12:36:46 2018 (r336965) +++ head/sys/fs/msdosfs/msdosfs_denode.c Tue Jul 31 12:44:28 2018 (r336966) @@ -383,7 +383,7 @@ detrunc(struct denode *dep, u_long length, int flags, dep->de_StartCluster = 0; eofentry = ~0; } else { - error = pcbmap(dep, de_clcount(pmp, length) - 1, 0, + error = pcbmap(dep, de_clcount(pmp, length) - 1, 0, &eofentry, 0); if (error) { #ifdef MSDOSFS_DEBUG Modified: head/sys/fs/msdosfs/msdosfs_lookup.c ============================================================================== --- head/sys/fs/msdosfs/msdosfs_lookup.c Tue Jul 31 12:36:46 2018 (r336965) +++ head/sys/fs/msdosfs/msdosfs_lookup.c Tue Jul 31 12:44:28 2018 (r336966) @@ -457,7 +457,7 @@ found: */ brelse(bp); bp = NULL; - + foundroot: /* * If we entered at foundroot, then we are looking for the . or .. @@ -1013,7 +1013,7 @@ uniqdosname(struct denode *dep, struct componentname * daddr_t bn; struct buf *bp; int error; - + if (pmp->pm_flags & MSDOSFSMNT_SHORTNAME) return (unix2dosfn((const u_char *)cnp->cn_nameptr, cp, cnp->cn_namelen, 0, pmp) ? 0 : EINVAL); Modified: head/sys/fs/msdosfs/msdosfs_vfsops.c ============================================================================== --- head/sys/fs/msdosfs/msdosfs_vfsops.c Tue Jul 31 12:36:46 2018 (r336965) +++ head/sys/fs/msdosfs/msdosfs_vfsops.c Tue Jul 31 12:44:28 2018 (r336966) @@ -320,7 +320,7 @@ msdosfs_mount(struct mount *mp) /* Now that the volume is modifiable, mark it dirty. */ error = markvoldirty(pmp, 1); if (error) - return (error); + return (error); } } /* Modified: head/sys/fs/msdosfs/msdosfs_vnops.c ============================================================================== --- head/sys/fs/msdosfs/msdosfs_vnops.c Tue Jul 31 12:36:46 2018 (r336965) +++ head/sys/fs/msdosfs/msdosfs_vnops.c Tue Jul 31 12:44:28 2018 (r336966) @@ -289,7 +289,7 @@ msdosfs_getattr(struct vop_getattr_args *ap) mode = S_IRWXU|S_IRWXG|S_IRWXO; if (dep->de_Attributes & ATTR_READONLY) mode &= ~(S_IWUSR|S_IWGRP|S_IWOTH); - vap->va_mode = mode & + vap->va_mode = mode & (ap->a_vp->v_type == VDIR ? pmp->pm_dirmask : pmp->pm_mask); vap->va_uid = pmp->pm_uid; vap->va_gid = pmp->pm_gid; From owner-svn-src-all@freebsd.org Tue Jul 31 12:53:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 015B1105982D; Tue, 31 Jul 2018 12:53:29 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8EA689B73; Tue, 31 Jul 2018 12:53:28 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8953C13E58; Tue, 31 Jul 2018 12:53:28 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VCrSHN080216; Tue, 31 Jul 2018 12:53:28 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VCrR6Y080209; Tue, 31 Jul 2018 12:53:27 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807311253.w6VCrR6Y080209@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Tue, 31 Jul 2018 12:53:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336967 - in head/sys: arm64/arm64 arm64/include dev/psci X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: in head/sys: arm64/arm64 arm64/include dev/psci X-SVN-Commit-Revision: 336967 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 12:53:29 -0000 Author: andrew Date: Tue Jul 31 12:53:27 2018 New Revision: 336967 URL: https://svnweb.freebsd.org/changeset/base/336967 Log: Implement the SSBD (CVE-2018-3639) workaround on arm64 This calls into the Arm Trusted Firmware to enable and disable the workaround for the Speculative Store Bypass Disable (SSBD) issue, also known as Spectre Variant 4. As this may have a large performance overhead, and how exploitable SSBD is is unknown we follow the Linux lead of allowing the administrator to select between always on, always off, or only enabled in the kernel, with the latter being the default. PR: 228955 Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D15819 Modified: head/sys/arm64/arm64/cpu_errata.c head/sys/arm64/arm64/exception.S head/sys/arm64/arm64/genassym.c head/sys/arm64/include/pcpu.h head/sys/dev/psci/smccc.c head/sys/dev/psci/smccc.h Modified: head/sys/arm64/arm64/cpu_errata.c ============================================================================== --- head/sys/arm64/arm64/cpu_errata.c Tue Jul 31 12:44:28 2018 (r336966) +++ head/sys/arm64/arm64/cpu_errata.c Tue Jul 31 12:53:27 2018 (r336967) @@ -38,6 +38,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include @@ -50,7 +51,14 @@ struct cpu_quirks { u_int midr_value; }; +static enum { + SSBD_FORCE_ON, + SSBD_FORCE_OFF, + SSBD_KERNEL, +} ssbd_method = SSBD_KERNEL; + static cpu_quirk_install install_psci_bp_hardening; +static cpu_quirk_install install_ssbd_workaround; static struct cpu_quirks cpu_quirks[] = { { @@ -79,6 +87,11 @@ static struct cpu_quirks cpu_quirks[] = { CPU_ID_RAW(CPU_IMPL_CAVIUM, CPU_PART_THUNDERX2, 0,0), .quirk_install = install_psci_bp_hardening, }, + { + .midr_mask = 0, + .midr_value = 0, + .quirk_install = install_ssbd_workaround, + }, }; static void @@ -89,6 +102,40 @@ install_psci_bp_hardening(void) return; PCPU_SET(bp_harden, smccc_arch_workaround_1); +} + +static void +install_ssbd_workaround(void) +{ + char *env; + + if (PCPU_GET(cpuid) == 0) { + env = kern_getenv("kern.cfg.ssbd"); + if (env != NULL) { + if (strcmp(env, "force-on") == 0) { + ssbd_method = SSBD_FORCE_ON; + } else if (strcmp(env, "force-off") == 0) { + ssbd_method = SSBD_FORCE_OFF; + } + } + } + + /* Enable the workaround on this CPU if it's enabled in the firmware */ + if (smccc_arch_features(SMCCC_ARCH_WORKAROUND_2) != SMCCC_RET_SUCCESS) + return; + + switch(ssbd_method) { + case SSBD_FORCE_ON: + smccc_arch_workaround_2(true); + break; + case SSBD_FORCE_OFF: + smccc_arch_workaround_2(false); + break; + case SSBD_KERNEL: + default: + PCPU_SET(ssbd, smccc_arch_workaround_2); + break; + } } void Modified: head/sys/arm64/arm64/exception.S ============================================================================== --- head/sys/arm64/arm64/exception.S Tue Jul 31 12:44:28 2018 (r336966) +++ head/sys/arm64/arm64/exception.S Tue Jul 31 12:53:27 2018 (r336967) @@ -66,6 +66,14 @@ __FBSDID("$FreeBSD$"); stp x18, lr, [sp, #(TF_SP)] mrs x18, tpidr_el1 add x29, sp, #(TF_SIZE) +.if \el == 0 + /* Apply the SSBD (CVE-2018-3639) workaround if needed */ + ldr x1, [x18, #PC_SSBD] + cbz x1, 1f + mov w0, #1 + blr x1 +1: +.endif .endm .macro restore_registers el @@ -75,6 +83,14 @@ __FBSDID("$FreeBSD$"); * Disable interrupts, x18 may change in the interrupt exception * handler. For EL0 exceptions, do_ast already did this. */ +.endif +.if \el == 0 + /* Remove the SSBD (CVE-2018-3639) workaround if needed */ + ldr x1, [x18, #PC_SSBD] + cbz x1, 1f + mov w0, #0 + blr x1 +1: .endif ldp x18, lr, [sp, #(TF_SP)] ldp x10, x11, [sp, #(TF_ELR)] Modified: head/sys/arm64/arm64/genassym.c ============================================================================== --- head/sys/arm64/arm64/genassym.c Tue Jul 31 12:44:28 2018 (r336966) +++ head/sys/arm64/arm64/genassym.c Tue Jul 31 12:53:27 2018 (r336967) @@ -43,6 +43,7 @@ ASSYM(TDF_NEEDRESCHED, TDF_NEEDRESCHED); ASSYM(PCPU_SIZE, sizeof(struct pcpu)); ASSYM(PC_CURPCB, offsetof(struct pcpu, pc_curpcb)); ASSYM(PC_CURTHREAD, offsetof(struct pcpu, pc_curthread)); +ASSYM(PC_SSBD, offsetof(struct pcpu, pc_ssbd)); /* Size of pcb, rounded to keep stack alignment */ ASSYM(PCB_SIZE, roundup2(sizeof(struct pcb), STACKALIGNBYTES + 1)); Modified: head/sys/arm64/include/pcpu.h ============================================================================== --- head/sys/arm64/include/pcpu.h Tue Jul 31 12:44:28 2018 (r336966) +++ head/sys/arm64/include/pcpu.h Tue Jul 31 12:53:27 2018 (r336967) @@ -36,13 +36,15 @@ #define ALT_STACK_SIZE 128 typedef int (*pcpu_bp_harden)(void); +typedef int (*pcpu_ssbd)(bool); #define PCPU_MD_FIELDS \ u_int pc_acpi_id; /* ACPI CPU id */ \ u_int pc_midr; /* stored MIDR value */ \ uint64_t pc_clock; \ pcpu_bp_harden pc_bp_harden; \ - char __pad[233] + pcpu_ssbd pc_ssbd; \ + char __pad[225] #ifdef _KERNEL Modified: head/sys/dev/psci/smccc.c ============================================================================== --- head/sys/dev/psci/smccc.c Tue Jul 31 12:44:28 2018 (r336966) +++ head/sys/dev/psci/smccc.c Tue Jul 31 12:53:27 2018 (r336967) @@ -91,3 +91,12 @@ smccc_arch_workaround_1(void) ("SMCCC arch workaround 1 called with an invalid SMCCC interface")); return (psci_call(SMCCC_ARCH_WORKAROUND_1, 0, 0, 0)); } + +int +smccc_arch_workaround_2(bool enable) +{ + + KASSERT(smccc_version != SMCCC_VERSION_1_0, + ("SMCCC arch workaround 2 called with an invalid SMCCC interface")); + return (psci_call(SMCCC_ARCH_WORKAROUND_2, enable ? 1 : 0, 0, 0)); +} Modified: head/sys/dev/psci/smccc.h ============================================================================== --- head/sys/dev/psci/smccc.h Tue Jul 31 12:44:28 2018 (r336966) +++ head/sys/dev/psci/smccc.h Tue Jul 31 12:53:27 2018 (r336967) @@ -59,6 +59,8 @@ SMCCC_FUNC_ID(SMCCC_FAST_CALL, SMCCC_32BIT_CALL, 0, 1) #define SMCCC_ARCH_WORKAROUND_1 \ SMCCC_FUNC_ID(SMCCC_FAST_CALL, SMCCC_32BIT_CALL, 0, 0x8000) +#define SMCCC_ARCH_WORKAROUND_2 \ + SMCCC_FUNC_ID(SMCCC_FAST_CALL, SMCCC_32BIT_CALL, 0, 0x7fff) /* The return values from ARM DEN 0070A. */ #define SMCCC_RET_SUCCESS 0 @@ -67,6 +69,7 @@ int32_t smccc_arch_features(uint32_t); int smccc_arch_workaround_1(void); +int smccc_arch_workaround_2(bool); #endif /* _PSCI_SMCCC_H_ */ From owner-svn-src-all@freebsd.org Tue Jul 31 13:32:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD044105A84F; Tue, 31 Jul 2018 13:32:37 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F90C8B2B6; Tue, 31 Jul 2018 13:32:37 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 71632144F2; Tue, 31 Jul 2018 13:32:37 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VDWbNB001096; Tue, 31 Jul 2018 13:32:37 GMT (envelope-from gjb@FreeBSD.org) Received: (from gjb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VDWbBH001094; Tue, 31 Jul 2018 13:32:37 GMT (envelope-from gjb@FreeBSD.org) Message-Id: <201807311332.w6VDWbBH001094@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gjb set sender to gjb@FreeBSD.org using -f From: Glen Barber Date: Tue, 31 Jul 2018 13:32:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336968 - stable/11/release/arm X-SVN-Group: stable-11 X-SVN-Commit-Author: gjb X-SVN-Commit-Paths: stable/11/release/arm X-SVN-Commit-Revision: 336968 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 13:32:38 -0000 Author: gjb Date: Tue Jul 31 13:32:36 2018 New Revision: 336968 URL: https://svnweb.freebsd.org/changeset/base/336968 Log: MFC r336721, r336750 [1]: r336721 (manu): Add OL_DIR forgot in r336600. r336750: Fix OL_DIR definition following r336721. [1] These commits were stated to have been merged in r336726, however are not present in context nor svn:mergeinfo. Sponsored by: The FreeBSD Foundation Modified: stable/11/release/arm/RPI-B.conf stable/11/release/arm/RPI2.conf Directory Properties: stable/11/ (props changed) Modified: stable/11/release/arm/RPI-B.conf ============================================================================== --- stable/11/release/arm/RPI-B.conf Tue Jul 31 12:53:27 2018 (r336967) +++ stable/11/release/arm/RPI-B.conf Tue Jul 31 13:32:36 2018 (r336968) @@ -13,13 +13,14 @@ IMAGE_SIZE="1G" KERNEL="RPI-B" MD_ARGS="-x 63 -y 255" NODOC=1 +UBOOT_DIR="/usr/local/share/u-boot/u-boot-rpi" +RPI_FIRMWARE_DIR="/usr/local/share/rpi-firmware" +OL_DIR="${RPI_FIRMWARE_DIR}/overlays" OVERLAYS="mmc.dtbo pi3-disable-bt.dtbo" PART_SCHEME="MBR" WORLD_FLAGS="${WORLD_FLAGS} UBLDR_LOADADDR=0x2000000" arm_install_uboot() { - UBOOT_DIR="/usr/local/share/u-boot/u-boot-rpi" - RPI_FIRMWARE_DIR="/usr/local/share/rpi-firmware" UBOOT_FILES="u-boot.bin" RPI_FIRMWARE_FILES="bootcode.bin config.txt \ fixup.dat fixup_cd.dat fixup_db.dat fixup_x.dat \ Modified: stable/11/release/arm/RPI2.conf ============================================================================== --- stable/11/release/arm/RPI2.conf Tue Jul 31 12:53:27 2018 (r336967) +++ stable/11/release/arm/RPI2.conf Tue Jul 31 13:32:36 2018 (r336968) @@ -13,13 +13,14 @@ IMAGE_SIZE="1536M" KERNEL="RPI2" MD_ARGS="-x 63 -y 255" NODOC=1 +UBOOT_DIR="/usr/local/share/u-boot/u-boot-rpi2" +RPI_FIRMWARE_DIR="/usr/local/share/rpi-firmware" +OL_DIR="${RPI_FIRMWARE_DIR}/overlays" OVERLAYS="mmc.dtbo" PART_SCHEME="MBR" WORLD_FLAGS="${WORLD_FLAGS} UBLDR_LOADADDR=0x2000000" arm_install_uboot() { - UBOOT_DIR="/usr/local/share/u-boot/u-boot-rpi2" - RPI_FIRMWARE_DIR="/usr/local/share/rpi-firmware" UBOOT_FILES="u-boot.bin" RPI_FIRMWARE_FILES="bootcode.bin config.txt \ fixup.dat fixup_cd.dat fixup_db.dat fixup_x.dat \ From owner-svn-src-all@freebsd.org Tue Jul 31 14:12:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47E16105B55B; Tue, 31 Jul 2018 14:12:10 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F22E88C474; Tue, 31 Jul 2018 14:12:09 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D22DE14A42; Tue, 31 Jul 2018 14:12:09 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VEC921020025; Tue, 31 Jul 2018 14:12:09 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VEC9Tj020024; Tue, 31 Jul 2018 14:12:09 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807311412.w6VEC9Tj020024@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Tue, 31 Jul 2018 14:12:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336969 - head/contrib/llvm/include/llvm/BinaryFormat X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/contrib/llvm/include/llvm/BinaryFormat X-SVN-Commit-Revision: 336969 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 14:12:10 -0000 Author: emaste Date: Tue Jul 31 14:12:09 2018 New Revision: 336969 URL: https://svnweb.freebsd.org/changeset/base/336969 Log: llvm: [ELF][ARM] Add Arm ABI names for float ABI ELF Header flags The ELF for the Arm architecture document defines, for EF_ARM_EABI_VER5 and above, the flags EF_ARM_ABI_FLOAT_HARD and EF_ARM_ABI_FLOAT_SOFT. These have been defined to be compatible with the existing EF_ARM_VFP_FLOAT and EF_ARM_SOFT_FLOAT used by gcc for EF_ARM_EABI_UNKNOWN. This patch adds the flags in addition to the existing ones so that any code depending on the old names will still work. Obtained from: llvm r338370 by Peter Smith Modified: head/contrib/llvm/include/llvm/BinaryFormat/ELF.h Modified: head/contrib/llvm/include/llvm/BinaryFormat/ELF.h ============================================================================== --- head/contrib/llvm/include/llvm/BinaryFormat/ELF.h Tue Jul 31 13:32:36 2018 (r336968) +++ head/contrib/llvm/include/llvm/BinaryFormat/ELF.h Tue Jul 31 14:12:09 2018 (r336969) @@ -418,8 +418,10 @@ enum { // ARM Specific e_flags enum : unsigned { - EF_ARM_SOFT_FLOAT = 0x00000200U, - EF_ARM_VFP_FLOAT = 0x00000400U, + EF_ARM_SOFT_FLOAT = 0x00000200U, // Legacy pre EABI_VER5 + EF_ARM_ABI_FLOAT_SOFT = 0x00000200U, // EABI_VER5 + EF_ARM_VFP_FLOAT = 0x00000400U, // Legacy pre EABI_VER5 + EF_ARM_ABI_FLOAT_HARD = 0x00000400U, // EABI_VER5 EF_ARM_EABI_UNKNOWN = 0x00000000U, EF_ARM_EABI_VER1 = 0x01000000U, EF_ARM_EABI_VER2 = 0x02000000U, From owner-svn-src-all@freebsd.org Tue Jul 31 14:14:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20FE8105B6A2; Tue, 31 Jul 2018 14:14:42 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAA148C6DC; Tue, 31 Jul 2018 14:14:41 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A7D1014B82; Tue, 31 Jul 2018 14:14:41 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VEEfbW021985; Tue, 31 Jul 2018 14:14:41 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VEEfwd021984; Tue, 31 Jul 2018 14:14:41 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807311414.w6VEEfwd021984@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Tue, 31 Jul 2018 14:14:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336970 - head/contrib/llvm/include/llvm/Support X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/contrib/llvm/include/llvm/Support X-SVN-Commit-Revision: 336970 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 14:14:42 -0000 Author: emaste Date: Tue Jul 31 14:14:41 2018 New Revision: 336970 URL: https://svnweb.freebsd.org/changeset/base/336970 Log: llvm: [ARM] Complete enumeration values for Tag_ABI_VFP_args The LLD implementation of Tag_ABI_VFP_args needs to check the rarely seen values of 3 (toolchain specific) and 4 compatible with both Base and VFP. Add the missing enumeration values so that LLD can refer to them without having to use the raw numbers. Obtained from: llvm r338373 by Peter Smith Modified: head/contrib/llvm/include/llvm/Support/ARMBuildAttributes.h Modified: head/contrib/llvm/include/llvm/Support/ARMBuildAttributes.h ============================================================================== --- head/contrib/llvm/include/llvm/Support/ARMBuildAttributes.h Tue Jul 31 14:12:09 2018 (r336969) +++ head/contrib/llvm/include/llvm/Support/ARMBuildAttributes.h Tue Jul 31 14:14:41 2018 (r336970) @@ -213,6 +213,8 @@ enum { // Tag_ABI_VFP_args, (=28), uleb128 BaseAAPCS = 0, HardFPAAPCS = 1, + ToolChainFPPCS = 2, + CompatibleFPAAPCS = 3, // Tag_FP_HP_extension, (=36), uleb128 AllowHPFP = 1, // Allow use of Half Precision FP From owner-svn-src-all@freebsd.org Tue Jul 31 15:08:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1ED7C105C42A; Tue, 31 Jul 2018 15:08:04 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCA268E0D5; Tue, 31 Jul 2018 15:08:03 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9953615370; Tue, 31 Jul 2018 15:08:03 +0000 (UTC) (envelope-from andrew@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VF83fc047842; Tue, 31 Jul 2018 15:08:03 GMT (envelope-from andrew@FreeBSD.org) Received: (from andrew@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VF82vN047838; Tue, 31 Jul 2018 15:08:02 GMT (envelope-from andrew@FreeBSD.org) Message-Id: <201807311508.w6VF82vN047838@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: andrew set sender to andrew@FreeBSD.org using -f From: Andrew Turner Date: Tue, 31 Jul 2018 15:08:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336971 - in head/sys: arm64/arm64 arm64/include dev/psci X-SVN-Group: head X-SVN-Commit-Author: andrew X-SVN-Commit-Paths: in head/sys: arm64/arm64 arm64/include dev/psci X-SVN-Commit-Revision: 336971 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 15:08:04 -0000 Author: andrew Date: Tue Jul 31 15:08:02 2018 New Revision: 336971 URL: https://svnweb.freebsd.org/changeset/base/336971 Log: Use int for the pcpu_ssbd argument. This is included from userland and may not include the needed headers to get the bool definition. Reported by: manu Pointy hat to: andrew Sponsored by: DARPA, AFRL Modified: head/sys/arm64/arm64/cpu_errata.c head/sys/arm64/include/pcpu.h head/sys/dev/psci/smccc.c head/sys/dev/psci/smccc.h Modified: head/sys/arm64/arm64/cpu_errata.c ============================================================================== --- head/sys/arm64/arm64/cpu_errata.c Tue Jul 31 14:14:41 2018 (r336970) +++ head/sys/arm64/arm64/cpu_errata.c Tue Jul 31 15:08:02 2018 (r336971) @@ -126,10 +126,10 @@ install_ssbd_workaround(void) switch(ssbd_method) { case SSBD_FORCE_ON: - smccc_arch_workaround_2(true); + smccc_arch_workaround_2(1); break; case SSBD_FORCE_OFF: - smccc_arch_workaround_2(false); + smccc_arch_workaround_2(0); break; case SSBD_KERNEL: default: Modified: head/sys/arm64/include/pcpu.h ============================================================================== --- head/sys/arm64/include/pcpu.h Tue Jul 31 14:14:41 2018 (r336970) +++ head/sys/arm64/include/pcpu.h Tue Jul 31 15:08:02 2018 (r336971) @@ -36,7 +36,7 @@ #define ALT_STACK_SIZE 128 typedef int (*pcpu_bp_harden)(void); -typedef int (*pcpu_ssbd)(bool); +typedef int (*pcpu_ssbd)(int); #define PCPU_MD_FIELDS \ u_int pc_acpi_id; /* ACPI CPU id */ \ Modified: head/sys/dev/psci/smccc.c ============================================================================== --- head/sys/dev/psci/smccc.c Tue Jul 31 14:14:41 2018 (r336970) +++ head/sys/dev/psci/smccc.c Tue Jul 31 15:08:02 2018 (r336971) @@ -93,10 +93,10 @@ smccc_arch_workaround_1(void) } int -smccc_arch_workaround_2(bool enable) +smccc_arch_workaround_2(int enable) { KASSERT(smccc_version != SMCCC_VERSION_1_0, ("SMCCC arch workaround 2 called with an invalid SMCCC interface")); - return (psci_call(SMCCC_ARCH_WORKAROUND_2, enable ? 1 : 0, 0, 0)); + return (psci_call(SMCCC_ARCH_WORKAROUND_2, enable, 0, 0)); } Modified: head/sys/dev/psci/smccc.h ============================================================================== --- head/sys/dev/psci/smccc.h Tue Jul 31 14:14:41 2018 (r336970) +++ head/sys/dev/psci/smccc.h Tue Jul 31 15:08:02 2018 (r336971) @@ -69,7 +69,7 @@ int32_t smccc_arch_features(uint32_t); int smccc_arch_workaround_1(void); -int smccc_arch_workaround_2(bool); +int smccc_arch_workaround_2(int); #endif /* _PSCI_SMCCC_H_ */ From owner-svn-src-all@freebsd.org Tue Jul 31 15:25:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7EE81105CAF5; Tue, 31 Jul 2018 15:25:04 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31BFE8EA79; Tue, 31 Jul 2018 15:25:04 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0E95A15696; Tue, 31 Jul 2018 15:25:04 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VFP36D058212; Tue, 31 Jul 2018 15:25:03 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VFP3vm058210; Tue, 31 Jul 2018 15:25:03 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807311525.w6VFP3vm058210@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Tue, 31 Jul 2018 15:25:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336972 - in head/contrib/llvm/tools/lld/ELF: . Arch X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: in head/contrib/llvm/tools/lld/ELF: . Arch X-SVN-Commit-Revision: 336972 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 15:25:04 -0000 Author: emaste Date: Tue Jul 31 15:25:03 2018 New Revision: 336972 URL: https://svnweb.freebsd.org/changeset/base/336972 Log: lld: [ELF][ARM] Implement support for Tag_ABI_VFP_args The Tag_ABI_VFP_args build attribute controls the procedure call standard used for floating point parameters on ARM. The values are: 0 - Base AAPCS (FP Parameters passed in Core (Integer) registers 1 - VFP AAPCS (FP Parameters passed in FP registers) 2 - Toolchain specific (Neither Base or VFP) 3 - Compatible with all (No use of floating point parameters) If the Tag_ABI_VFP_args build attribute is missing it has an implicit value of 0. We use the attribute in two ways: * Detect a clash in calling convention between Base, VFP and Toolchain. we follow ld.bfd's lead and do not error if there is a clash between an implicit Base AAPCS caused by a missing attribute. Many projects including the hard-float (VFP AAPCS) version of glibc contain assembler files that do not use floating point but do not have Tag_ABI_VFP_args. * Set the EF_ARM_ABI_FLOAT_SOFT or EF_ARM_ABI_FLOAT_HARD ELF header flag for Base or VFP AAPCS respectively. This flag is used by some ELF loaders. References: * Addenda to, and Errata in, the ABI for the ARM Architecture for Tag_ABI_VFP_args * Elf for the ARM Architecture for ELF header flags Fixes LLVM PR36009 PR: 229050 Obtained from: llvm r338377 by Peter Smith Modified: head/contrib/llvm/tools/lld/ELF/Arch/ARM.cpp head/contrib/llvm/tools/lld/ELF/Config.h head/contrib/llvm/tools/lld/ELF/InputFiles.cpp Modified: head/contrib/llvm/tools/lld/ELF/Arch/ARM.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Arch/ARM.cpp Tue Jul 31 15:08:02 2018 (r336971) +++ head/contrib/llvm/tools/lld/ELF/Arch/ARM.cpp Tue Jul 31 15:25:03 2018 (r336972) @@ -97,10 +97,19 @@ ARM::ARM() { } uint32_t ARM::calcEFlags() const { + // The ABIFloatType is used by loaders to detect the floating point calling + // convention. + uint32_t ABIFloatType = 0; + if (Config->ARMVFPArgs == ARMVFPArgKind::Base || + Config->ARMVFPArgs == ARMVFPArgKind::Default) + ABIFloatType = EF_ARM_ABI_FLOAT_SOFT; + else if (Config->ARMVFPArgs == ARMVFPArgKind::VFP) + ABIFloatType = EF_ARM_ABI_FLOAT_HARD; + // We don't currently use any features incompatible with EF_ARM_EABI_VER5, // but we don't have any firm guarantees of conformance. Linux AArch64 // kernels (as of 2016) require an EABI version to be set. - return EF_ARM_EABI_VER5; + return EF_ARM_EABI_VER5 | ABIFloatType; } RelExpr ARM::getRelExpr(RelType Type, const Symbol &S, Modified: head/contrib/llvm/tools/lld/ELF/Config.h ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Config.h Tue Jul 31 15:08:02 2018 (r336971) +++ head/contrib/llvm/tools/lld/ELF/Config.h Tue Jul 31 15:25:03 2018 (r336972) @@ -54,6 +54,9 @@ enum class SortSectionPolicy { Default, None, Alignmen // For --target2 enum class Target2Policy { Abs, Rel, GotRel }; +// For tracking ARM Float Argument PCS +enum class ARMVFPArgKind { Default, Base, VFP, ToolChain }; + struct SymbolVersion { llvm::StringRef Name; bool IsExternCpp; @@ -169,6 +172,7 @@ struct Configuration { StripPolicy Strip; UnresolvedPolicy UnresolvedSymbols; Target2Policy Target2; + ARMVFPArgKind ARMVFPArgs = ARMVFPArgKind::Default; BuildIdKind BuildId = BuildIdKind::None; ELFKind EKind = ELFNoneKind; uint16_t DefaultSymbolVersion = llvm::ELF::VER_NDX_GLOBAL; Modified: head/contrib/llvm/tools/lld/ELF/InputFiles.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/InputFiles.cpp Tue Jul 31 15:08:02 2018 (r336971) +++ head/contrib/llvm/tools/lld/ELF/InputFiles.cpp Tue Jul 31 15:25:03 2018 (r336972) @@ -441,6 +441,46 @@ void ObjFile::initializeSections( } } +// For ARM only, to set the EF_ARM_ABI_FLOAT_SOFT or EF_ARM_ABI_FLOAT_HARD +// flag in the ELF Header we need to look at Tag_ABI_VFP_args to find out how +// the input objects have been compiled. +static void updateARMVFPArgs(const ARMAttributeParser &Attributes, + const InputFile *F) { + if (!Attributes.hasAttribute(ARMBuildAttrs::ABI_VFP_args)) + // If an ABI tag isn't present then it is implicitly given the value of 0 + // which maps to ARMBuildAttrs::BaseAAPCS. However many assembler files, + // including some in glibc that don't use FP args (and should have value 3) + // don't have the attribute so we do not consider an implicit value of 0 + // as a clash. + return; + + unsigned VFPArgs = Attributes.getAttributeValue(ARMBuildAttrs::ABI_VFP_args); + ARMVFPArgKind Arg; + switch (VFPArgs) { + case ARMBuildAttrs::BaseAAPCS: + Arg = ARMVFPArgKind::Base; + break; + case ARMBuildAttrs::HardFPAAPCS: + Arg = ARMVFPArgKind::VFP; + break; + case ARMBuildAttrs::ToolChainFPPCS: + // Tool chain specific convention that conforms to neither AAPCS variant. + Arg = ARMVFPArgKind::ToolChain; + break; + case ARMBuildAttrs::CompatibleFPAAPCS: + // Object compatible with all conventions. + return; + default: + error(toString(F) + ": unknown Tag_ABI_VFP_args value: " + Twine(VFPArgs)); + return; + } + // Follow ld.bfd and error if there is a mix of calling conventions. + if (Config->ARMVFPArgs != Arg && Config->ARMVFPArgs != ARMVFPArgKind::Default) + error(toString(F) + ": incompatible Tag_ABI_VFP_args"); + else + Config->ARMVFPArgs = Arg; +} + // The ARM support in lld makes some use of instructions that are not available // on all ARM architectures. Namely: // - Use of BLX instruction for interworking between ARM and Thumb state. @@ -520,6 +560,8 @@ InputSectionBase *ObjFile::createInputSection(co ArrayRef Contents = check(this->getObj().getSectionContents(&Sec)); Attributes.Parse(Contents, /*isLittle*/ Config->EKind == ELF32LEKind); updateSupportedARMFeatures(Attributes); + updateARMVFPArgs(Attributes, this); + // FIXME: Retain the first attribute section we see. The eglibc ARM // dynamic loaders require the presence of an attribute section for dlopen // to work. In a full implementation we would merge all attribute sections. From owner-svn-src-all@freebsd.org Tue Jul 31 16:03:31 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15210105D80D; Tue, 31 Jul 2018 16:03:31 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF1768FFDE; Tue, 31 Jul 2018 16:03:30 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A0C6715D34; Tue, 31 Jul 2018 16:03:30 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VG3UoK078683; Tue, 31 Jul 2018 16:03:30 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VG3UiA078682; Tue, 31 Jul 2018 16:03:30 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807311603.w6VG3UiA078682@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Tue, 31 Jul 2018 16:03:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336973 - head/usr.sbin/config X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: head/usr.sbin/config X-SVN-Commit-Revision: 336973 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 16:03:31 -0000 Author: kevans Date: Tue Jul 31 16:03:30 2018 New Revision: 336973 URL: https://svnweb.freebsd.org/changeset/base/336973 Log: config(8): Strip comments from env lines Consolidates the small bits of logic required for preprocessing a line before inclusion into a file or nvlist. Modified: head/usr.sbin/config/mkmakefile.c Modified: head/usr.sbin/config/mkmakefile.c ============================================================================== --- head/usr.sbin/config/mkmakefile.c Tue Jul 31 15:25:03 2018 (r336972) +++ head/usr.sbin/config/mkmakefile.c Tue Jul 31 16:03:30 2018 (r336973) @@ -65,6 +65,7 @@ static void do_before_depend(FILE *); static int opteq(const char *, const char *); static void read_files(void); static void sanitize_envline(char *result, const char *src); +static bool preprocess(char *line, char *result); static void process_into_file(char *line, FILE *ofp); static void process_into_nvlist(char *line, nvlist_t *nvl); static void dump_nvlist(nvlist_t *nvl, FILE *ofp); @@ -243,32 +244,44 @@ sanitize_envline(char *result, const char *src) *dst = 0; } +/* + * Returns true if the caller may use the string. + */ +static bool +preprocess(char *line, char *result) +{ + + char result[BUFSIZ], *s; + + /* Strip any comments */ + if ((s = strchr(line, '#')) != NULL) + *s = '\0'; + sanitize_envline(result, line); + /* Return true if it's non-empty */ + return (*result != '\0'); +} + static void process_into_file(char *line, FILE *ofp) { char result[BUFSIZ]; - sanitize_envline(result, line); - /* anything left? */ - if (*result == '\0') - return; - fprintf(ofp, "\"%s\\0\"\n", result); + if (preprocess(line, result)) + fprintf(ofp, "\"%s\\0\"\n", result); } static void process_into_nvlist(char *line, nvlist_t *nvl) { - char result[BUFSIZ], *s; + char result[BUFSIZ]; - sanitize_envline(result, line); - /* anything left? */ - if (*result == '\0') - return; - s = strchr(result, '='); - *s = 0; - if (nvlist_exists(nvl, result)) - nvlist_free(nvl, result); - nvlist_add_string(nvl, result, s + 1); + if (preprocess(line, result)) { + s = strchr(result, '='); + *s = '\0'; + if (nvlist_exists(nvl, result)) + nvlist_free(nvl, result); + nvlist_add_string(nvl, result, s + 1); + } } static void From owner-svn-src-all@freebsd.org Tue Jul 31 16:04:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0BE0C105D8F7; Tue, 31 Jul 2018 16:04:50 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B64D6701CF; Tue, 31 Jul 2018 16:04:49 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9870915D35; Tue, 31 Jul 2018 16:04:49 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VG4nsH078778; Tue, 31 Jul 2018 16:04:49 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VG4n3s078777; Tue, 31 Jul 2018 16:04:49 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807311604.w6VG4n3s078777@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Tue, 31 Jul 2018 16:04:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336974 - head/usr.sbin/config X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: head/usr.sbin/config X-SVN-Commit-Revision: 336974 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 16:04:50 -0000 Author: kevans Date: Tue Jul 31 16:04:49 2018 New Revision: 336974 URL: https://svnweb.freebsd.org/changeset/base/336974 Log: Re-insert variable disappeared during mis-application of r336973 Modified: head/usr.sbin/config/mkmakefile.c Modified: head/usr.sbin/config/mkmakefile.c ============================================================================== --- head/usr.sbin/config/mkmakefile.c Tue Jul 31 16:03:30 2018 (r336973) +++ head/usr.sbin/config/mkmakefile.c Tue Jul 31 16:04:49 2018 (r336974) @@ -273,7 +273,7 @@ process_into_file(char *line, FILE *ofp) static void process_into_nvlist(char *line, nvlist_t *nvl) { - char result[BUFSIZ]; + char result[BUFSIZ], *s; if (preprocess(line, result)) { s = strchr(result, '='); From owner-svn-src-all@freebsd.org Tue Jul 31 16:08:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 706FD105DA17; Tue, 31 Jul 2018 16:08:18 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 151A07041E; Tue, 31 Jul 2018 16:08:18 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E626915D65; Tue, 31 Jul 2018 16:08:17 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VG8HPe078983; Tue, 31 Jul 2018 16:08:17 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VG8HaL078982; Tue, 31 Jul 2018 16:08:17 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201807311608.w6VG8HaL078982@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Tue, 31 Jul 2018 16:08:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336975 - head/usr.sbin/config X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: head/usr.sbin/config X-SVN-Commit-Revision: 336975 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 16:08:18 -0000 Author: kevans Date: Tue Jul 31 16:08:17 2018 New Revision: 336975 URL: https://svnweb.freebsd.org/changeset/base/336975 Log: Remove variable re-inserted during mis-application of r336973 Modified: head/usr.sbin/config/mkmakefile.c Modified: head/usr.sbin/config/mkmakefile.c ============================================================================== --- head/usr.sbin/config/mkmakefile.c Tue Jul 31 16:04:49 2018 (r336974) +++ head/usr.sbin/config/mkmakefile.c Tue Jul 31 16:08:17 2018 (r336975) @@ -250,8 +250,7 @@ sanitize_envline(char *result, const char *src) static bool preprocess(char *line, char *result) { - - char result[BUFSIZ], *s; + char *s; /* Strip any comments */ if ((s = strchr(line, '#')) != NULL) From owner-svn-src-all@freebsd.org Tue Jul 31 16:08:40 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7792F105DA49; Tue, 31 Jul 2018 16:08:40 +0000 (UTC) (envelope-from rpokala@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CF137054B; Tue, 31 Jul 2018 16:08:40 +0000 (UTC) (envelope-from rpokala@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E94C215D66; Tue, 31 Jul 2018 16:08:39 +0000 (UTC) (envelope-from rpokala@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VG8dAJ079049; Tue, 31 Jul 2018 16:08:39 GMT (envelope-from rpokala@FreeBSD.org) Received: (from rpokala@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VG8dsn079045; Tue, 31 Jul 2018 16:08:39 GMT (envelope-from rpokala@FreeBSD.org) Message-Id: <201807311608.w6VG8dsn079045@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rpokala set sender to rpokala@FreeBSD.org using -f From: Ravi Pokala Date: Tue, 31 Jul 2018 16:08:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r336976 - in stable/11: share/man/man4 sys/dev/jedec_ts X-SVN-Group: stable-11 X-SVN-Commit-Author: rpokala X-SVN-Commit-Paths: in stable/11: share/man/man4 sys/dev/jedec_ts X-SVN-Commit-Revision: 336976 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 16:08:40 -0000 Author: rpokala Date: Tue Jul 31 16:08:38 2018 New Revision: 336976 URL: https://svnweb.freebsd.org/changeset/base/336976 Log: MFC r336662,r336682 r336662: Deprecate jedec_ts(4) and point users to jedec_dimm(4) instead jedec_dimm(4) is a superset of the functionality of jedec_ts(4). Mark jedec_ts(4) as removed in FreeBSD 12, and include a pointer to the migration instructions in the jedec_dimm(4) manpage, in both the jedec_ts(4) code and the jedec_ts(4) manpage. Add a note to the jedec_dimm(4) manpage about the fact that it is a superset of jedec_ts(4). r336682: Update .Dd in light of r336662. Modified: stable/11/share/man/man4/jedec_dimm.4 stable/11/share/man/man4/jedec_ts.4 stable/11/sys/dev/jedec_ts/jedec_ts.c Directory Properties: stable/11/ (props changed) Modified: stable/11/share/man/man4/jedec_dimm.4 ============================================================================== --- stable/11/share/man/man4/jedec_dimm.4 Tue Jul 31 16:08:17 2018 (r336975) +++ stable/11/share/man/man4/jedec_dimm.4 Tue Jul 31 16:08:38 2018 (r336976) @@ -27,7 +27,7 @@ .\" .\" $FreeBSD$ .\" -.Dd February 22, 2018 +.Dd July 24, 2018 .Dt JEDEC_DIMM 4 .Os .Sh NAME @@ -163,6 +163,9 @@ dev.jedec_dimm.6.temp: 43.1C dev.jedec_dimm.6.type: DDR4 .Ed .Sh COMPATIBILITY +.Nm +implements a superset of the functionality of +.Xr jedec_ts 4 . Hints for .Xr jedec_ts 4 can be mechanically converted for use with Modified: stable/11/share/man/man4/jedec_ts.4 ============================================================================== --- stable/11/share/man/man4/jedec_ts.4 Tue Jul 31 16:08:17 2018 (r336975) +++ stable/11/share/man/man4/jedec_ts.4 Tue Jul 31 16:08:38 2018 (r336976) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd May 9, 2017 +.Dd July 24, 2018 .Dt JEDEC_TS 4 .Os .Sh NAME @@ -63,6 +63,18 @@ In .Cd hint.jedec_ts.7.at="smbus0" .Cd hint.jedec_ts.7.addr="0x3E" .Ed +.Sh DEPRECATION NOTICE +The +.Nm +driver is not present in +.Fx 12.0 +and later. +A superset of its functionality is available in the +.Xr jedec_dimm 4 +driver. +That driver's manpage includes instructions on updating +.Pa /boot/device.hints +accordingly. .Sh DESCRIPTION The .Nm Modified: stable/11/sys/dev/jedec_ts/jedec_ts.c ============================================================================== --- stable/11/sys/dev/jedec_ts/jedec_ts.c Tue Jul 31 16:08:17 2018 (r336975) +++ stable/11/sys/dev/jedec_ts/jedec_ts.c Tue Jul 31 16:08:38 2018 (r336976) @@ -247,6 +247,9 @@ ts_attach(device_t dev) CTLTYPE_INT | CTLFLAG_RD | CTLFLAG_MPSAFE, dev, 0, ts_temp_sysctl, "IK4", "Current temperature"); + gone_in_dev(dev, 12, + "jedec_ts(4) driver; see COMPATIBILITY section of jedec_dimm(4)"); + return (0); } From owner-svn-src-all@freebsd.org Tue Jul 31 16:39:39 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 988E6105E673; Tue, 31 Jul 2018 16:39:39 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ED77716ED; Tue, 31 Jul 2018 16:39:39 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 309F316223; Tue, 31 Jul 2018 16:39:39 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VGdcKP094640; Tue, 31 Jul 2018 16:39:38 GMT (envelope-from brd@FreeBSD.org) Received: (from brd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VGdcCr094637; Tue, 31 Jul 2018 16:39:38 GMT (envelope-from brd@FreeBSD.org) Message-Id: <201807311639.w6VGdcCr094637@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: brd set sender to brd@FreeBSD.org using -f From: Brad Davis Date: Tue, 31 Jul 2018 16:39:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336977 - in head: etc usr.sbin/blacklistd X-SVN-Group: head X-SVN-Commit-Author: brd X-SVN-Commit-Paths: in head: etc usr.sbin/blacklistd X-SVN-Commit-Revision: 336977 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 16:39:39 -0000 Author: brd Date: Tue Jul 31 16:39:38 2018 New Revision: 336977 URL: https://svnweb.freebsd.org/changeset/base/336977 Log: Move blacklistd.conf to usr.sbin/blacklistd/ This is prep for pkging base and helps tag and install config files with the correct packages. Approved by: bapt (mentor) Differential Revision: https://reviews.freebsd.org/D16493 Added: head/usr.sbin/blacklistd/blacklistd.conf - copied unchanged from r336976, head/etc/blacklistd.conf Deleted: head/etc/blacklistd.conf Modified: head/etc/Makefile head/usr.sbin/blacklistd/Makefile Modified: head/etc/Makefile ============================================================================== --- head/etc/Makefile Tue Jul 31 16:08:38 2018 (r336976) +++ head/etc/Makefile Tue Jul 31 16:39:38 2018 (r336977) @@ -74,10 +74,6 @@ BIN1+= amd.map BIN1+= auto_master .endif -.if ${MK_BLACKLIST} != "no" -BIN1+= blacklistd.conf -.endif - .if ${MK_FREEBSD_UPDATE} != "no" BIN1+= freebsd-update.conf .endif Modified: head/usr.sbin/blacklistd/Makefile ============================================================================== --- head/usr.sbin/blacklistd/Makefile Tue Jul 31 16:08:38 2018 (r336976) +++ head/usr.sbin/blacklistd/Makefile Tue Jul 31 16:39:38 2018 (r336977) @@ -3,6 +3,7 @@ BLACKLIST_DIR=${SRCTOP}/contrib/blacklist .PATH: ${BLACKLIST_DIR}/bin ${BLACKLIST_DIR}/port +CONFS= blacklistd.conf PROG= blacklistd SRCS= blacklistd.c conf.c run.c state.c support.c internal.c \ sockaddr_snprintf.c pidfile.c strtoi.c popenve.c Copied: head/usr.sbin/blacklistd/blacklistd.conf (from r336976, head/etc/blacklistd.conf) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/blacklistd/blacklistd.conf Tue Jul 31 16:39:38 2018 (r336977, copy of r336976, head/etc/blacklistd.conf) @@ -0,0 +1,17 @@ +# $FreeBSD$ +# +# Blacklist rule +# adr/mask:port type proto owner name nfail disable +[local] +ssh stream * * * 3 24h +ftp stream * * * 3 24h +smtp stream * * * 3 24h +submission stream * * * 3 24h +#6161 stream tcp6 christos * 2 10m +* * * * * 3 60 + +# adr/mask:port type proto owner name nfail disable +[remote] +#129.168.0.0/16 * * * = * * +#6161 = = = =/24 = = +#* stream tcp * = = = From owner-svn-src-all@freebsd.org Tue Jul 31 16:42:05 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A299105E933; Tue, 31 Jul 2018 16:42:05 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0031971BBC; Tue, 31 Jul 2018 16:42:05 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BDD91163D1; Tue, 31 Jul 2018 16:42:04 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VGg4ve097947; Tue, 31 Jul 2018 16:42:04 GMT (envelope-from brd@FreeBSD.org) Received: (from brd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VGg4jR097943; Tue, 31 Jul 2018 16:42:04 GMT (envelope-from brd@FreeBSD.org) Message-Id: <201807311642.w6VGg4jR097943@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: brd set sender to brd@FreeBSD.org using -f From: Brad Davis Date: Tue, 31 Jul 2018 16:42:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336978 - in head: etc etc/pkg usr.sbin/pkg X-SVN-Group: head X-SVN-Commit-Author: brd X-SVN-Commit-Paths: in head: etc etc/pkg usr.sbin/pkg X-SVN-Commit-Revision: 336978 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 16:42:05 -0000 Author: brd Date: Tue Jul 31 16:42:03 2018 New Revision: 336978 URL: https://svnweb.freebsd.org/changeset/base/336978 Log: Move pkg/FreeBSD.conf to usr.sbin/pkg/ Approved by: bapt (mentor) Differential Revision: https://reviews.freebsd.org/D16491 Added: head/usr.sbin/pkg/FreeBSD.conf - copied unchanged from r336977, head/etc/pkg/FreeBSD.conf Deleted: head/etc/pkg/ Modified: head/etc/Makefile head/usr.sbin/pkg/Makefile Modified: head/etc/Makefile ============================================================================== --- head/etc/Makefile Tue Jul 31 16:39:38 2018 (r336977) +++ head/etc/Makefile Tue Jul 31 16:42:03 2018 (r336978) @@ -214,9 +214,6 @@ distribution: ${_+_}cd ${.CURDIR}/ntp; ${MAKE} install .endif ${_+_}cd ${.CURDIR}/periodic; ${MAKE} install -.if ${MK_PKGBOOTSTRAP} != "no" - ${_+_}cd ${.CURDIR}/pkg; ${MAKE} install -.endif ${_+_}cd ${SRCTOP}/share/termcap; ${MAKE} etc-termcap ${_+_}cd ${.CURDIR}/syslog.d; ${MAKE} install ${_+_}cd ${SRCTOP}/usr.sbin/rmt; ${MAKE} etc-rmt Copied: head/usr.sbin/pkg/FreeBSD.conf (from r336977, head/etc/pkg/FreeBSD.conf) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/pkg/FreeBSD.conf Tue Jul 31 16:42:03 2018 (r336978, copy of r336977, head/etc/pkg/FreeBSD.conf) @@ -0,0 +1,16 @@ +# $FreeBSD$ +# +# To disable this repository, instead of modifying or removing this file, +# create a /usr/local/etc/pkg/repos/FreeBSD.conf file: +# +# mkdir -p /usr/local/etc/pkg/repos +# echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf +# + +FreeBSD: { + url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest", + mirror_type: "srv", + signature_type: "fingerprints", + fingerprints: "/usr/share/keys/pkg", + enabled: yes +} Modified: head/usr.sbin/pkg/Makefile ============================================================================== --- head/usr.sbin/pkg/Makefile Tue Jul 31 16:39:38 2018 (r336977) +++ head/usr.sbin/pkg/Makefile Tue Jul 31 16:42:03 2018 (r336978) @@ -1,5 +1,8 @@ # $FreeBSD$ +CONFS= FreeBSD.conf +CONFSDIR= /etc/pkg +CONFSMODE= 644 PROG= pkg SRCS= pkg.c dns_utils.c config.c MAN= pkg.7 From owner-svn-src-all@freebsd.org Tue Jul 31 17:15:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8540C105F1A6; Tue, 31 Jul 2018 17:15:32 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3AC8272CD4; Tue, 31 Jul 2018 17:15:32 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1BAD416901; Tue, 31 Jul 2018 17:15:32 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VHFVEo015215; Tue, 31 Jul 2018 17:15:31 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VHFV1C015213; Tue, 31 Jul 2018 17:15:31 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807311715.w6VHFV1C015213@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 31 Jul 2018 17:15:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336980 - head/sys/compat/freebsd32 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/compat/freebsd32 X-SVN-Commit-Revision: 336980 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 17:15:32 -0000 Author: kib Date: Tue Jul 31 17:15:31 2018 New Revision: 336980 URL: https://svnweb.freebsd.org/changeset/base/336980 Log: Provide compat32 shims for sched_rr_get_interval(2). The interface uses struct timespec, which needs a translation. Reported and reviewed by: asomers PR: 230175 Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D16525 Modified: head/sys/compat/freebsd32/freebsd32_misc.c head/sys/compat/freebsd32/syscalls.master Modified: head/sys/compat/freebsd32/freebsd32_misc.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_misc.c Tue Jul 31 17:06:31 2018 (r336979) +++ head/sys/compat/freebsd32/freebsd32_misc.c Tue Jul 31 17:15:31 2018 (r336980) @@ -3485,3 +3485,20 @@ freebsd32_ppoll(struct thread *td, struct freebsd32_pp return (kern_poll(td, uap->fds, uap->nfds, tsp, ssp)); } + +int +freebsd32_sched_rr_get_interval(struct thread *td, + struct freebsd32_sched_rr_get_interval_args *uap) +{ + struct timespec ts; + struct timespec32 ts32; + int error; + + error = kern_sched_rr_get_interval(td, uap->pid, &ts); + if (error == 0) { + CP(ts, ts32, tv_sec); + CP(ts, ts32, tv_nsec); + error = copyout(&ts32, uap->interval, sizeof(ts32)); + } + return (error); +} Modified: head/sys/compat/freebsd32/syscalls.master ============================================================================== --- head/sys/compat/freebsd32/syscalls.master Tue Jul 31 17:06:31 2018 (r336979) +++ head/sys/compat/freebsd32/syscalls.master Tue Jul 31 17:15:31 2018 (r336980) @@ -591,8 +591,9 @@ 331 AUE_NULL NOPROTO { int sched_yield (void); } 332 AUE_NULL NOPROTO { int sched_get_priority_max (int policy); } 333 AUE_NULL NOPROTO { int sched_get_priority_min (int policy); } -334 AUE_NULL NOPROTO { int sched_rr_get_interval (pid_t pid, \ - struct timespec *interval); } +334 AUE_NULL STD { int freebsd32_sched_rr_get_interval ( \ + pid_t pid, \ + struct timespec32 *interval); } 335 AUE_NULL NOPROTO { int utrace(const void *addr, size_t len); } 336 AUE_SENDFILE COMPAT4 { int freebsd32_sendfile(int fd, int s, \ uint32_t offset1, uint32_t offset2, \ From owner-svn-src-all@freebsd.org Tue Jul 31 17:16:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE2FC105F229; Tue, 31 Jul 2018 17:16:08 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 742DA72E3F; Tue, 31 Jul 2018 17:16:08 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 56C0F16909; Tue, 31 Jul 2018 17:16:08 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VHG8N5015405; Tue, 31 Jul 2018 17:16:08 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VHG7mG015300; Tue, 31 Jul 2018 17:16:07 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807311716.w6VHG7mG015300@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 31 Jul 2018 17:16:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336981 - head/sys/compat/freebsd32 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/compat/freebsd32 X-SVN-Commit-Revision: 336981 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 17:16:09 -0000 Author: kib Date: Tue Jul 31 17:16:06 2018 New Revision: 336981 URL: https://svnweb.freebsd.org/changeset/base/336981 Log: Regenerate after r336980. Modified: head/sys/compat/freebsd32/freebsd32_proto.h head/sys/compat/freebsd32/freebsd32_syscall.h head/sys/compat/freebsd32/freebsd32_syscalls.c head/sys/compat/freebsd32/freebsd32_sysent.c head/sys/compat/freebsd32/freebsd32_systrace_args.c Modified: head/sys/compat/freebsd32/freebsd32_proto.h ============================================================================== --- head/sys/compat/freebsd32/freebsd32_proto.h Tue Jul 31 17:15:31 2018 (r336980) +++ head/sys/compat/freebsd32/freebsd32_proto.h Tue Jul 31 17:16:06 2018 (r336981) @@ -279,6 +279,10 @@ struct freebsd32_aio_suspend_args { struct freebsd32_aio_error_args { char aiocbp_l_[PADL_(struct aiocb32 *)]; struct aiocb32 * aiocbp; char aiocbp_r_[PADR_(struct aiocb32 *)]; }; +struct freebsd32_sched_rr_get_interval_args { + char pid_l_[PADL_(pid_t)]; pid_t pid; char pid_r_[PADR_(pid_t)]; + char interval_l_[PADL_(struct timespec32 *)]; struct timespec32 * interval; char interval_r_[PADR_(struct timespec32 *)]; +}; struct freebsd32_jail_args { char jail_l_[PADL_(struct jail32 *)]; struct jail32 * jail; char jail_r_[PADR_(struct jail32 *)]; }; @@ -756,6 +760,7 @@ int freebsd32_kldstat(struct thread *, struct freebsd3 int freebsd32_aio_return(struct thread *, struct freebsd32_aio_return_args *); int freebsd32_aio_suspend(struct thread *, struct freebsd32_aio_suspend_args *); int freebsd32_aio_error(struct thread *, struct freebsd32_aio_error_args *); +int freebsd32_sched_rr_get_interval(struct thread *, struct freebsd32_sched_rr_get_interval_args *); int freebsd32_jail(struct thread *, struct freebsd32_jail_args *); int freebsd32_sigtimedwait(struct thread *, struct freebsd32_sigtimedwait_args *); int freebsd32_sigwaitinfo(struct thread *, struct freebsd32_sigwaitinfo_args *); @@ -1310,6 +1315,7 @@ int freebsd11_freebsd32_mknodat(struct thread *, struc #define FREEBSD32_SYS_AUE_freebsd6_freebsd32_aio_read AUE_AIO_READ #define FREEBSD32_SYS_AUE_freebsd6_freebsd32_aio_write AUE_AIO_WRITE #define FREEBSD32_SYS_AUE_freebsd6_freebsd32_lio_listio AUE_LIO_LISTIO +#define FREEBSD32_SYS_AUE_freebsd32_sched_rr_get_interval AUE_NULL #define FREEBSD32_SYS_AUE_freebsd4_freebsd32_sendfile AUE_SENDFILE #define FREEBSD32_SYS_AUE_freebsd32_jail AUE_JAIL #define FREEBSD32_SYS_AUE_freebsd4_freebsd32_sigaction AUE_SIGACTION Modified: head/sys/compat/freebsd32/freebsd32_syscall.h ============================================================================== --- head/sys/compat/freebsd32/freebsd32_syscall.h Tue Jul 31 17:15:31 2018 (r336980) +++ head/sys/compat/freebsd32/freebsd32_syscall.h Tue Jul 31 17:16:06 2018 (r336981) @@ -271,7 +271,7 @@ #define FREEBSD32_SYS_sched_yield 331 #define FREEBSD32_SYS_sched_get_priority_max 332 #define FREEBSD32_SYS_sched_get_priority_min 333 -#define FREEBSD32_SYS_sched_rr_get_interval 334 +#define FREEBSD32_SYS_freebsd32_sched_rr_get_interval 334 #define FREEBSD32_SYS_utrace 335 /* 336 is freebsd4 freebsd32_sendfile */ #define FREEBSD32_SYS_kldsym 337 Modified: head/sys/compat/freebsd32/freebsd32_syscalls.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_syscalls.c Tue Jul 31 17:15:31 2018 (r336980) +++ head/sys/compat/freebsd32/freebsd32_syscalls.c Tue Jul 31 17:16:06 2018 (r336981) @@ -343,7 +343,7 @@ const char *freebsd32_syscallnames[] = { "sched_yield", /* 331 = sched_yield */ "sched_get_priority_max", /* 332 = sched_get_priority_max */ "sched_get_priority_min", /* 333 = sched_get_priority_min */ - "sched_rr_get_interval", /* 334 = sched_rr_get_interval */ + "freebsd32_sched_rr_get_interval", /* 334 = freebsd32_sched_rr_get_interval */ "utrace", /* 335 = utrace */ "compat4.freebsd32_sendfile", /* 336 = freebsd4 freebsd32_sendfile */ "kldsym", /* 337 = kldsym */ Modified: head/sys/compat/freebsd32/freebsd32_sysent.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_sysent.c Tue Jul 31 17:15:31 2018 (r336980) +++ head/sys/compat/freebsd32/freebsd32_sysent.c Tue Jul 31 17:16:06 2018 (r336981) @@ -390,7 +390,7 @@ struct sysent freebsd32_sysent[] = { { 0, (sy_call_t *)sys_sched_yield, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 331 = sched_yield */ { AS(sched_get_priority_max_args), (sy_call_t *)sys_sched_get_priority_max, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 332 = sched_get_priority_max */ { AS(sched_get_priority_min_args), (sy_call_t *)sys_sched_get_priority_min, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 333 = sched_get_priority_min */ - { AS(sched_rr_get_interval_args), (sy_call_t *)sys_sched_rr_get_interval, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 334 = sched_rr_get_interval */ + { AS(freebsd32_sched_rr_get_interval_args), (sy_call_t *)freebsd32_sched_rr_get_interval, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 334 = freebsd32_sched_rr_get_interval */ { AS(utrace_args), (sy_call_t *)sys_utrace, AUE_NULL, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 335 = utrace */ { compat4(AS(freebsd4_freebsd32_sendfile_args),freebsd32_sendfile), AUE_SENDFILE, NULL, 0, 0, SYF_CAPENABLED, SY_THR_STATIC }, /* 336 = freebsd4 freebsd32_sendfile */ { AS(kldsym_args), (sy_call_t *)sys_kldsym, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 337 = kldsym */ Modified: head/sys/compat/freebsd32/freebsd32_systrace_args.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_systrace_args.c Tue Jul 31 17:15:31 2018 (r336980) +++ head/sys/compat/freebsd32/freebsd32_systrace_args.c Tue Jul 31 17:16:06 2018 (r336981) @@ -1534,11 +1534,11 @@ systrace_args(int sysnum, void *params, uint64_t *uarg *n_args = 1; break; } - /* sched_rr_get_interval */ + /* freebsd32_sched_rr_get_interval */ case 334: { - struct sched_rr_get_interval_args *p = params; + struct freebsd32_sched_rr_get_interval_args *p = params; iarg[0] = p->pid; /* pid_t */ - uarg[1] = (intptr_t) p->interval; /* struct timespec * */ + uarg[1] = (intptr_t) p->interval; /* struct timespec32 * */ *n_args = 2; break; } @@ -5669,14 +5669,14 @@ systrace_entry_setargdesc(int sysnum, int ndx, char *d break; }; break; - /* sched_rr_get_interval */ + /* freebsd32_sched_rr_get_interval */ case 334: switch(ndx) { case 0: p = "pid_t"; break; case 1: - p = "userland struct timespec *"; + p = "userland struct timespec32 *"; break; default: break; @@ -9658,7 +9658,7 @@ systrace_return_setargdesc(int sysnum, int ndx, char * if (ndx == 0 || ndx == 1) p = "int"; break; - /* sched_rr_get_interval */ + /* freebsd32_sched_rr_get_interval */ case 334: if (ndx == 0 || ndx == 1) p = "int"; From owner-svn-src-all@freebsd.org Tue Jul 31 17:18:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B561A105F3D9; Tue, 31 Jul 2018 17:18:59 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F69D7329D; Tue, 31 Jul 2018 17:18:59 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 11A5616911; Tue, 31 Jul 2018 17:18:59 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VHIwn9015657; Tue, 31 Jul 2018 17:18:58 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VHIwQ2015656; Tue, 31 Jul 2018 17:18:58 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201807311718.w6VHIwQ2015656@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Tue, 31 Jul 2018 17:18:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336983 - head/sys/dev/sound/pci/hda X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/dev/sound/pci/hda X-SVN-Commit-Revision: 336983 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 17:18:59 -0000 Author: jhibbits Date: Tue Jul 31 17:18:58 2018 New Revision: 336983 URL: https://svnweb.freebsd.org/changeset/base/336983 Log: snd_hda: Byteswap the buffer descriptor entries as needed The buffer descriptor list entries should be in little endian format. Byte swap them on BE. This is the last piece of the puzzle for snd_hda(4) to work on PowerPC. Modified: head/sys/dev/sound/pci/hda/hdac.c Modified: head/sys/dev/sound/pci/hda/hdac.c ============================================================================== --- head/sys/dev/sound/pci/hda/hdac.c Tue Jul 31 17:18:35 2018 (r336982) +++ head/sys/dev/sound/pci/hda/hdac.c Tue Jul 31 17:18:58 2018 (r336983) @@ -1930,10 +1930,10 @@ hdac_stream_start(device_t dev, device_t child, addr = (uint64_t)buf; bdle = (struct hdac_bdle *)sc->streams[ss].bdl.dma_vaddr; for (i = 0; i < blkcnt; i++, bdle++) { - bdle->addrl = (uint32_t)addr; - bdle->addrh = (uint32_t)(addr >> 32); - bdle->len = blksz; - bdle->ioc = 1; + bdle->addrl = htole32((uint32_t)addr); + bdle->addrh = htole32((uint32_t)(addr >> 32)); + bdle->len = htole32(blksz); + bdle->ioc = htole32(1); addr += blksz; } From owner-svn-src-all@freebsd.org Tue Jul 31 17:41:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3DB1105FDD5; Tue, 31 Jul 2018 17:41:49 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6817074021; Tue, 31 Jul 2018 17:41:49 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 494C116D8A; Tue, 31 Jul 2018 17:41:49 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VHfneP030811; Tue, 31 Jul 2018 17:41:49 GMT (envelope-from alc@FreeBSD.org) Received: (from alc@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VHfnu3030810; Tue, 31 Jul 2018 17:41:49 GMT (envelope-from alc@FreeBSD.org) Message-Id: <201807311741.w6VHfnu3030810@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: alc set sender to alc@FreeBSD.org using -f From: Alan Cox Date: Tue, 31 Jul 2018 17:41:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336984 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: alc X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 336984 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 17:41:49 -0000 Author: alc Date: Tue Jul 31 17:41:48 2018 New Revision: 336984 URL: https://svnweb.freebsd.org/changeset/base/336984 Log: Allow vm object coalescing to occur in the midst of a vm object when the OBJ_ONEMAPPING flag is set. In other words, allow recycling of existing but unused subranges of a vm object when the OBJ_ONEMAPPING flag is set. Such situations are increasingly common with jemalloc >= 5.0. This change has the expected effect of reducing the number of vm map entry and object allocations and increasing the number of superpage promotions. Reviewed by: kib, markj Tested by: pho MFC after: 6 weeks Differential Revision: https://reviews.freebsd.org/D16501 Modified: head/sys/vm/vm_object.c Modified: head/sys/vm/vm_object.c ============================================================================== --- head/sys/vm/vm_object.c Tue Jul 31 17:18:58 2018 (r336983) +++ head/sys/vm/vm_object.c Tue Jul 31 17:41:48 2018 (r336984) @@ -2142,8 +2142,9 @@ vm_object_coalesce(vm_object_t prev_object, vm_ooffset next_size >>= PAGE_SHIFT; next_pindex = OFF_TO_IDX(prev_offset) + prev_size; - if ((prev_object->ref_count > 1) && - (prev_object->size != next_pindex)) { + if (prev_object->ref_count > 1 && + prev_object->size != next_pindex && + (prev_object->flags & OBJ_ONEMAPPING) == 0) { VM_OBJECT_WUNLOCK(prev_object); return (FALSE); } From owner-svn-src-all@freebsd.org Tue Jul 31 18:00:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 19B561060554; Tue, 31 Jul 2018 18:00:48 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4D4D74D54; Tue, 31 Jul 2018 18:00:47 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A796416FA6; Tue, 31 Jul 2018 18:00:47 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VI0l70036692; Tue, 31 Jul 2018 18:00:47 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VI0l26036691; Tue, 31 Jul 2018 18:00:47 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201807311800.w6VI0l26036691@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 31 Jul 2018 18:00:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336987 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 336987 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 18:00:48 -0000 Author: kib Date: Tue Jul 31 18:00:47 2018 New Revision: 336987 URL: https://svnweb.freebsd.org/changeset/base/336987 Log: For compat32, emulate the same wraparound check as occurs on the real ILP32 system. Reported by and discussed with: asomers PR: 230162 Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D16525 Modified: head/sys/vm/vm_mmap.c Modified: head/sys/vm/vm_mmap.c ============================================================================== --- head/sys/vm/vm_mmap.c Tue Jul 31 17:53:24 2018 (r336986) +++ head/sys/vm/vm_mmap.c Tue Jul 31 18:00:47 2018 (r336987) @@ -600,6 +600,12 @@ kern_mprotect(struct thread *td, uintptr_t addr0, size addr -= pageoff; size += pageoff; size = (vm_size_t) round_page(size); +#ifdef COMPAT_FREEBSD32 + if (SV_PROC_FLAG(td->td_proc, SV_ILP32)) { + if (((addr + size) & 0xffffffff) < addr) + return (EINVAL); + } else +#endif if (addr + size < addr) return (EINVAL); From owner-svn-src-all@freebsd.org Tue Jul 31 18:49:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 298D9106173C; Tue, 31 Jul 2018 18:49:12 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D1D5E76C18; Tue, 31 Jul 2018 18:49:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B41C3177BF; Tue, 31 Jul 2018 18:49:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VInBCk065559; Tue, 31 Jul 2018 18:49:11 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VIn7Xt065532; Tue, 31 Jul 2018 18:49:07 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807311849.w6VIn7Xt065532@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 18:49:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336991 - vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/ill... X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendo... X-SVN-Commit-Revision: 336991 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 18:49:12 -0000 Author: mav Date: Tue Jul 31 18:49:07 2018 New Revision: 336991 URL: https://svnweb.freebsd.org/changeset/base/336991 Log: 9102 zfs should be able to initialize storage devices The first access to a disk block can incur a performance penalty on some platforms (e.g. AWS's EBS, VMware VMDKs). Therefore it is recommended that volumes be "thick provisioned", where supported by the platform (VMware). Thick provisioning is time consuming and often is ignored. If the thick provision step is omitted, customers will see suboptimal performance until we have written to all parts of the LUN. ZFS should be able to initialize any unused storage to remove any first-write penalty that exists. illumos/illumos-gate@094e47e980b0796b94b1b8f51f462a64d246e516 Reviewed by: John Wren Kennedy Reviewed by: Matthew Ahrens Reviewed by: Pavel Zakharov Reviewed by: Prakash Surya Approved by: Richard Lowe Modified: vendor-sys/illumos/dist/uts/common/Makefile.files vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_priority.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_missing.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_raidz.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_root.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c vendor/illumos/dist/lib/libzfs/common/libzfs_util.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h vendor/illumos/dist/man/man1m/zpool.1m Modified: vendor-sys/illumos/dist/uts/common/Makefile.files ============================================================================== --- vendor-sys/illumos/dist/uts/common/Makefile.files Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/Makefile.files Tue Jul 31 18:49:07 2018 (r336991) @@ -1398,6 +1398,7 @@ ZFS_COMMON_OBJS += \ vdev.o \ vdev_cache.o \ vdev_file.o \ + vdev_initialize.o \ vdev_label.o \ vdev_mirror.o \ vdev_missing.o \ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c Tue Jul 31 18:49:07 2018 (r336991) @@ -635,6 +635,8 @@ metaslab_group_create(metaslab_class_t *mc, vdev_t *vd mg = kmem_zalloc(sizeof (metaslab_group_t), KM_SLEEP); mutex_init(&mg->mg_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&mg->mg_ms_initialize_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&mg->mg_ms_initialize_cv, NULL, CV_DEFAULT, NULL); mg->mg_primaries = kmem_zalloc(allocators * sizeof (metaslab_t *), KM_SLEEP); mg->mg_secondaries = kmem_zalloc(allocators * sizeof (metaslab_t *), @@ -681,6 +683,8 @@ metaslab_group_destroy(metaslab_group_t *mg) kmem_free(mg->mg_secondaries, mg->mg_allocators * sizeof (metaslab_t *)); mutex_destroy(&mg->mg_lock); + mutex_destroy(&mg->mg_ms_initialize_lock); + cv_destroy(&mg->mg_ms_initialize_cv); for (int i = 0; i < mg->mg_allocators; i++) { refcount_destroy(&mg->mg_alloc_queue_depth[i]); @@ -1541,6 +1545,7 @@ metaslab_init(metaslab_group_t *mg, uint64_t id, uint6 mutex_init(&ms->ms_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&ms->ms_sync_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&ms->ms_load_cv, NULL, CV_DEFAULT, NULL); + ms->ms_id = id; ms->ms_start = id << vd->vdev_ms_shift; ms->ms_size = 1ULL << vd->vdev_ms_shift; @@ -2717,6 +2722,7 @@ metaslab_sync_done(metaslab_t *msp, uint64_t txg) * from it in 'metaslab_unload_delay' txgs, then unload it. */ if (msp->ms_loaded && + msp->ms_initializing == 0 && msp->ms_selected_txg + metaslab_unload_delay < txg) { for (int t = 1; t < TXG_CONCURRENT_STATES; t++) { VERIFY0(range_tree_space( @@ -2966,6 +2972,7 @@ metaslab_block_alloc(metaslab_t *msp, uint64_t size, u metaslab_class_t *mc = msp->ms_group->mg_class; VERIFY(!msp->ms_condensing); + VERIFY0(msp->ms_initializing); start = mc->mc_ops->msop_alloc(msp, size); if (start != -1ULL) { @@ -3026,9 +3033,10 @@ find_valid_metaslab(metaslab_group_t *mg, uint64_t act } /* - * If the selected metaslab is condensing, skip it. + * If the selected metaslab is condensing or being + * initialized, skip it. */ - if (msp->ms_condensing) + if (msp->ms_condensing || msp->ms_initializing > 0) continue; *was_active = msp->ms_allocator != -1; @@ -3193,11 +3201,20 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ /* * If this metaslab is currently condensing then pick again as * we can't manipulate this metaslab until it's committed - * to disk. + * to disk. If this metaslab is being initialized, we shouldn't + * allocate from it since the allocated region might be + * overwritten after allocation. */ if (msp->ms_condensing) { metaslab_trace_add(zal, mg, msp, asize, d, TRACE_CONDENSING, allocator); + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_ACTIVE_MASK); + mutex_exit(&msp->ms_lock); + continue; + } else if (msp->ms_initializing > 0) { + metaslab_trace_add(zal, mg, msp, asize, d, + TRACE_INITIALIZING, allocator); metaslab_passivate(msp, msp->ms_weight & ~METASLAB_ACTIVE_MASK); mutex_exit(&msp->ms_lock); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c Tue Jul 31 18:49:07 2018 (r336991) @@ -54,6 +54,7 @@ #include #include #include +#include #include #include #include @@ -413,8 +414,9 @@ spa_prop_get(spa_t *spa, nvlist_t **nvp) dp = spa_get_dsl(spa); dsl_pool_config_enter(dp, FTAG); - if (err = dsl_dataset_hold_obj(dp, - za.za_first_integer, FTAG, &ds)) { + err = dsl_dataset_hold_obj(dp, + za.za_first_integer, FTAG, &ds); + if (err != 0) { dsl_pool_config_exit(dp, FTAG); break; } @@ -569,7 +571,8 @@ spa_prop_validate(spa_t *spa, nvlist_t *props) break; } - if (error = dmu_objset_hold(strval, FTAG, &os)) + error = dmu_objset_hold(strval, FTAG, &os); + if (error != 0) break; /* @@ -1155,8 +1158,10 @@ spa_activate(spa_t *spa, int mode) spa_create_zio_taskqs(spa); } - for (size_t i = 0; i < TXG_SIZE; i++) - spa->spa_txg_zio[i] = zio_root(spa, NULL, NULL, 0); + for (size_t i = 0; i < TXG_SIZE; i++) { + spa->spa_txg_zio[i] = zio_root(spa, NULL, NULL, + ZIO_FLAG_CANFAIL); + } list_create(&spa->spa_config_dirty_list, sizeof (vdev_t), offsetof(vdev_t, vdev_config_dirty_node)); @@ -1315,6 +1320,11 @@ spa_unload(spa_t *spa) */ spa_async_suspend(spa); + if (spa->spa_root_vdev) { + vdev_initialize_stop_all(spa->spa_root_vdev, + VDEV_INITIALIZE_ACTIVE); + } + /* * Stop syncing. */ @@ -1330,10 +1340,10 @@ spa_unload(spa_t *spa) * calling taskq_wait(mg_taskq). */ if (spa->spa_root_vdev != NULL) { - spa_config_enter(spa, SCL_ALL, FTAG, RW_WRITER); + spa_config_enter(spa, SCL_ALL, spa, RW_WRITER); for (int c = 0; c < spa->spa_root_vdev->vdev_children; c++) vdev_metaslab_fini(spa->spa_root_vdev->vdev_child[c]); - spa_config_exit(spa, SCL_ALL, FTAG); + spa_config_exit(spa, SCL_ALL, spa); } /* @@ -1367,7 +1377,7 @@ spa_unload(spa_t *spa) bpobj_close(&spa->spa_deferred_bpobj); - spa_config_enter(spa, SCL_ALL, FTAG, RW_WRITER); + spa_config_enter(spa, SCL_ALL, spa, RW_WRITER); /* * Close all vdevs. @@ -1429,7 +1439,7 @@ spa_unload(spa_t *spa) spa->spa_comment = NULL; } - spa_config_exit(spa, SCL_ALL, FTAG); + spa_config_exit(spa, SCL_ALL, spa); } /* @@ -3866,6 +3876,10 @@ spa_load_impl(spa_t *spa, spa_import_type_t type, char spa_restart_removal(spa); spa_spawn_aux_threads(spa); + + spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); + vdev_initialize_restart(spa->spa_root_vdev); + spa_config_exit(spa, SCL_CONFIG, FTAG); } spa_load_note(spa, "LOADED"); @@ -5347,6 +5361,7 @@ spa_export_common(char *pool, int new_state, nvlist_t * in which case we can modify its state. */ if (spa->spa_state != POOL_STATE_UNINITIALIZED && spa->spa_sync_on) { + /* * Objsets may be open only because they're dirty, so we * have to force it to sync before checking spa_refcnt. @@ -5381,6 +5396,18 @@ spa_export_common(char *pool, int new_state, nvlist_t } /* + * We're about to export or destroy this pool. Make sure + * we stop all initializtion activity here before we + * set the spa_final_txg. This will ensure that all + * dirty data resulting from the initialization is + * committed to disk before we unload the pool. + */ + if (spa->spa_root_vdev != NULL) { + vdev_initialize_stop_all(spa->spa_root_vdev, + VDEV_INITIALIZE_ACTIVE); + } + + /* * We want this to be reflected on every label, * so mark them all dirty. spa_unload() will do the * final sync that pushes these changes out. @@ -6070,6 +6097,86 @@ spa_vdev_detach(spa_t *spa, uint64_t guid, uint64_t pg return (error); } +int +spa_vdev_initialize(spa_t *spa, uint64_t guid, uint64_t cmd_type) +{ + /* + * We hold the namespace lock through the whole function + * to prevent any changes to the pool while we're starting or + * stopping initialization. The config and state locks are held so that + * we can properly assess the vdev state before we commit to + * the initializing operation. + */ + mutex_enter(&spa_namespace_lock); + spa_config_enter(spa, SCL_CONFIG | SCL_STATE, FTAG, RW_READER); + + /* Look up vdev and ensure it's a leaf. */ + vdev_t *vd = spa_lookup_by_guid(spa, guid, B_FALSE); + if (vd == NULL || vd->vdev_detached) { + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(ENODEV)); + } else if (!vd->vdev_ops->vdev_op_leaf || !vdev_is_concrete(vd)) { + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(EINVAL)); + } else if (!vdev_writeable(vd)) { + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(EROFS)); + } + mutex_enter(&vd->vdev_initialize_lock); + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + + /* + * When we activate an initialize action we check to see + * if the vdev_initialize_thread is NULL. We do this instead + * of using the vdev_initialize_state since there might be + * a previous initialization process which has completed but + * the thread is not exited. + */ + if (cmd_type == POOL_INITIALIZE_DO && + (vd->vdev_initialize_thread != NULL || + vd->vdev_top->vdev_removing)) { + mutex_exit(&vd->vdev_initialize_lock); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(EBUSY)); + } else if (cmd_type == POOL_INITIALIZE_CANCEL && + (vd->vdev_initialize_state != VDEV_INITIALIZE_ACTIVE && + vd->vdev_initialize_state != VDEV_INITIALIZE_SUSPENDED)) { + mutex_exit(&vd->vdev_initialize_lock); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(ESRCH)); + } else if (cmd_type == POOL_INITIALIZE_SUSPEND && + vd->vdev_initialize_state != VDEV_INITIALIZE_ACTIVE) { + mutex_exit(&vd->vdev_initialize_lock); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(ESRCH)); + } + + switch (cmd_type) { + case POOL_INITIALIZE_DO: + vdev_initialize(vd); + break; + case POOL_INITIALIZE_CANCEL: + vdev_initialize_stop(vd, VDEV_INITIALIZE_CANCELED); + break; + case POOL_INITIALIZE_SUSPEND: + vdev_initialize_stop(vd, VDEV_INITIALIZE_SUSPENDED); + break; + default: + panic("invalid cmd_type %llu", (unsigned long long)cmd_type); + } + mutex_exit(&vd->vdev_initialize_lock); + + /* Sync out the initializing state */ + txg_wait_synced(spa->spa_dsl_pool, 0); + mutex_exit(&spa_namespace_lock); + + return (0); +} + + /* * Split a set of devices from their mirrors, and create a new pool from them. */ @@ -6277,6 +6384,19 @@ spa_vdev_split_mirror(spa_t *spa, char *newname, nvlis spa_activate(newspa, spa_mode_global); spa_async_suspend(newspa); + for (c = 0; c < children; c++) { + if (vml[c] != NULL) { + /* + * Temporarily stop the initializing activity. We set + * the state to ACTIVE so that we know to resume + * the initializing once the split has completed. + */ + mutex_enter(&vml[c]->vdev_initialize_lock); + vdev_initialize_stop(vml[c], VDEV_INITIALIZE_ACTIVE); + mutex_exit(&vml[c]->vdev_initialize_lock); + } + } + newspa->spa_config_source = SPA_CONFIG_SRC_SPLIT; /* create the new pool from the disks of the original pool */ @@ -6364,6 +6484,10 @@ out: if (vml[c] != NULL) vml[c]->vdev_offline = B_FALSE; } + + /* restart initializing disks as necessary */ + spa_async_request(spa, SPA_ASYNC_INITIALIZE_RESTART); + vdev_reopen(spa->spa_root_vdev); nvlist_free(spa->spa_config_splitting); @@ -6739,6 +6863,14 @@ spa_async_thread(void *arg) if (tasks & SPA_ASYNC_RESILVER) dsl_resilver_restart(spa->spa_dsl_pool, 0); + if (tasks & SPA_ASYNC_INITIALIZE_RESTART) { + mutex_enter(&spa_namespace_lock); + spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); + vdev_initialize_restart(spa->spa_root_vdev); + spa_config_exit(spa, SCL_CONFIG, FTAG); + mutex_exit(&spa_namespace_lock); + } + /* * Let the world know that we're done. */ @@ -7384,8 +7516,9 @@ spa_sync(spa_t *spa, uint64_t txg) * Wait for i/os issued in open context that need to complete * before this txg syncs. */ - VERIFY0(zio_wait(spa->spa_txg_zio[txg & TXG_MASK])); - spa->spa_txg_zio[txg & TXG_MASK] = zio_root(spa, NULL, NULL, 0); + (void) zio_wait(spa->spa_txg_zio[txg & TXG_MASK]); + spa->spa_txg_zio[txg & TXG_MASK] = zio_root(spa, NULL, NULL, + ZIO_FLAG_CANFAIL); /* * Lock out configuration changes. @@ -7674,7 +7807,8 @@ spa_sync(spa_t *spa, uint64_t txg) /* * Update usable space statistics. */ - while (vd = txg_list_remove(&spa->spa_vdev_txg_list, TXG_CLEAN(txg))) + while ((vd = txg_list_remove(&spa->spa_vdev_txg_list, TXG_CLEAN(txg))) + != NULL) vdev_sync_done(vd, txg); spa_update_dspace(spa); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Tue Jul 31 18:49:07 2018 (r336991) @@ -39,6 +39,7 @@ #include #include #include +#include #include #include #include @@ -1196,6 +1197,12 @@ spa_vdev_config_exit(spa_t *spa, vdev_t *vd, uint64_t if (vd != NULL) { ASSERT(!vd->vdev_detached || vd->vdev_dtl_sm == NULL); + if (vd->vdev_ops->vdev_op_leaf) { + mutex_enter(&vd->vdev_initialize_lock); + vdev_initialize_stop(vd, VDEV_INITIALIZE_CANCELED); + mutex_exit(&vd->vdev_initialize_lock); + } + spa_config_enter(spa, SCL_ALL, spa, RW_WRITER); vdev_free(vd); spa_config_exit(spa, SCL_ALL, spa); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h Tue Jul 31 18:49:07 2018 (r336991) @@ -68,7 +68,8 @@ typedef enum trace_alloc_type { TRACE_GROUP_FAILURE = -5ULL, TRACE_ENOSPC = -6ULL, TRACE_CONDENSING = -7ULL, - TRACE_VDEV_ERROR = -8ULL + TRACE_VDEV_ERROR = -8ULL, + TRACE_INITIALIZING = -9ULL } trace_alloc_type_t; #define METASLAB_WEIGHT_PRIMARY (1ULL << 63) @@ -270,6 +271,11 @@ struct metaslab_group { uint64_t mg_failed_allocations; uint64_t mg_fragmentation; uint64_t mg_histogram[RANGE_TREE_HISTOGRAM_SIZE]; + + int mg_ms_initializing; + boolean_t mg_initialize_updating; + kmutex_t mg_ms_initialize_lock; + kcondvar_t mg_ms_initialize_cv; }; /* @@ -359,6 +365,8 @@ struct metaslab { boolean_t ms_condensing; /* condensing? */ boolean_t ms_condense_wanted; uint64_t ms_condense_checked_txg; + + uint64_t ms_initializing; /* leaves initializing this ms */ /* * We must hold both ms_lock and ms_group->mg_lock in order to Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Tue Jul 31 18:49:07 2018 (r336991) @@ -650,6 +650,7 @@ extern int spa_scan_get_stats(spa_t *spa, pool_scan_st #define SPA_ASYNC_AUTOEXPAND 0x20 #define SPA_ASYNC_REMOVE_DONE 0x40 #define SPA_ASYNC_REMOVE_STOP 0x80 +#define SPA_ASYNC_INITIALIZE_RESTART 0x100 /* * Controls the behavior of spa_vdev_remove(). @@ -665,6 +666,7 @@ extern int spa_vdev_detach(spa_t *spa, uint64_t guid, int replace_done); extern int spa_vdev_remove(spa_t *spa, uint64_t guid, boolean_t unspare); extern boolean_t spa_vdev_remove_active(spa_t *spa); +extern int spa_vdev_initialize(spa_t *spa, uint64_t guid, uint64_t cmd_type); extern int spa_vdev_setpath(spa_t *spa, uint64_t guid, const char *newpath); extern int spa_vdev_setfru(spa_t *spa, uint64_t guid, const char *newfru); extern int spa_vdev_split_mirror(spa_t *spa, char *newname, nvlist_t *config, Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h Tue Jul 31 18:49:07 2018 (r336991) @@ -79,6 +79,12 @@ typedef void vdev_remap_cb_t(uint64_t inner_offset, vd uint64_t offset, uint64_t size, void *arg); typedef void vdev_remap_func_t(vdev_t *vd, uint64_t offset, uint64_t size, vdev_remap_cb_t callback, void *arg); +/* + * Given a target vdev, translates the logical range "in" to the physical + * range "res" + */ +typedef void vdev_xlation_func_t(vdev_t *cvd, const range_seg_t *in, + range_seg_t *res); typedef struct vdev_ops { vdev_open_func_t *vdev_op_open; @@ -90,6 +96,11 @@ typedef struct vdev_ops { vdev_hold_func_t *vdev_op_hold; vdev_rele_func_t *vdev_op_rele; vdev_remap_func_t *vdev_op_remap; + /* + * For translating ranges from non-leaf vdevs (e.g. raidz) to leaves. + * Used when initializing vdevs. Isn't used by leaf ops. + */ + vdev_xlation_func_t *vdev_op_xlate; char vdev_op_type[16]; boolean_t vdev_op_leaf; } vdev_ops_t; @@ -231,7 +242,25 @@ struct vdev { /* pool checkpoint related */ space_map_t *vdev_checkpoint_sm; /* contains reserved blocks */ + + boolean_t vdev_initialize_exit_wanted; + vdev_initializing_state_t vdev_initialize_state; + kthread_t *vdev_initialize_thread; + /* Protects vdev_initialize_thread and vdev_initialize_state. */ + kmutex_t vdev_initialize_lock; + kcondvar_t vdev_initialize_cv; + uint64_t vdev_initialize_offset[TXG_SIZE]; + uint64_t vdev_initialize_last_offset; + range_tree_t *vdev_initialize_tree; /* valid while initializing */ + uint64_t vdev_initialize_bytes_est; + uint64_t vdev_initialize_bytes_done; + time_t vdev_initialize_action_time; /* start and end time */ + /* for limiting outstanding I/Os */ + kmutex_t vdev_initialize_io_lock; + kcondvar_t vdev_initialize_io_cv; + uint64_t vdev_initialize_inflight; + /* * Values stored in the config for an indirect or removing vdev. */ @@ -434,6 +463,8 @@ extern vdev_ops_t vdev_indirect_ops; /* * Common size functions */ +extern void vdev_default_xlate(vdev_t *vd, const range_seg_t *in, + range_seg_t *out); extern uint64_t vdev_default_asize(vdev_t *vd, uint64_t psize); extern uint64_t vdev_get_min_asize(vdev_t *vd); extern void vdev_set_min_asize(vdev_t *vd); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_priority.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_priority.h Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_priority.h Tue Jul 31 18:49:07 2018 (r336991) @@ -13,7 +13,7 @@ * CDDL HEADER END */ /* - * Copyright (c) 2014 by Delphix. All rights reserved. + * Copyright (c) 2014, 2016 by Delphix. All rights reserved. */ #ifndef _ZIO_PRIORITY_H #define _ZIO_PRIORITY_H @@ -29,6 +29,7 @@ typedef enum zio_priority { ZIO_PRIORITY_ASYNC_WRITE, /* spa_sync() */ ZIO_PRIORITY_SCRUB, /* asynchronous scrub/resilver reads */ ZIO_PRIORITY_REMOVAL, /* reads/writes for vdev removal */ + ZIO_PRIORITY_INITIALIZING, /* initializing I/O */ ZIO_PRIORITY_NUM_QUEUEABLE, ZIO_PRIORITY_NOW /* non-queued i/os (e.g. free) */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Tue Jul 31 18:49:07 2018 (r336991) @@ -49,6 +49,7 @@ #include #include #include +#include /* * Virtual device management. @@ -183,6 +184,14 @@ vdev_getops(const char *type) return (ops); } +/* ARGSUSED */ +void +vdev_default_xlate(vdev_t *vd, const range_seg_t *in, range_seg_t *res) +{ + res->rs_start = in->rs_start; + res->rs_end = in->rs_end; +} + /* * Default asize function: return the MAX of psize with the asize of * all children. This is what's used by anything other than RAID-Z. @@ -453,6 +462,11 @@ vdev_alloc_common(spa_t *spa, uint_t id, uint64_t guid mutex_init(&vd->vdev_stat_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&vd->vdev_probe_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&vd->vdev_queue_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&vd->vdev_initialize_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&vd->vdev_initialize_io_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&vd->vdev_initialize_cv, NULL, CV_DEFAULT, NULL); + cv_init(&vd->vdev_initialize_io_cv, NULL, CV_DEFAULT, NULL); + for (int t = 0; t < DTL_TYPES; t++) { vd->vdev_dtl[t] = range_tree_create(NULL, NULL); } @@ -725,6 +739,7 @@ void vdev_free(vdev_t *vd) { spa_t *spa = vd->vdev_spa; + ASSERT3P(vd->vdev_initialize_thread, ==, NULL); /* * vdev_free() implies closing the vdev first. This is simpler than @@ -743,6 +758,7 @@ vdev_free(vdev_t *vd) ASSERT(vd->vdev_child == NULL); ASSERT(vd->vdev_guid_sum == vd->vdev_guid); + ASSERT(vd->vdev_initialize_thread == NULL); /* * Discard allocation state. @@ -815,6 +831,10 @@ vdev_free(vdev_t *vd) mutex_destroy(&vd->vdev_dtl_lock); mutex_destroy(&vd->vdev_stat_lock); mutex_destroy(&vd->vdev_probe_lock); + mutex_destroy(&vd->vdev_initialize_lock); + mutex_destroy(&vd->vdev_initialize_io_lock); + cv_destroy(&vd->vdev_initialize_io_cv); + cv_destroy(&vd->vdev_initialize_cv); if (vd == spa->spa_root_vdev) spa->spa_root_vdev = NULL; @@ -2841,7 +2861,8 @@ vdev_sync_done(vdev_t *vd, uint64_t txg) ASSERT(vdev_is_concrete(vd)); - while (msp = txg_list_remove(&vd->vdev_ms_list, TXG_CLEAN(txg))) + while ((msp = txg_list_remove(&vd->vdev_ms_list, TXG_CLEAN(txg))) + != NULL) metaslab_sync_done(msp, txg); if (reassess) @@ -3067,6 +3088,15 @@ vdev_online(spa_t *spa, uint64_t guid, uint64_t flags, spa_async_request(spa, SPA_ASYNC_CONFIG_UPDATE); } + /* Restart initializing if necessary */ + mutex_enter(&vd->vdev_initialize_lock); + if (vdev_writeable(vd) && + vd->vdev_initialize_thread == NULL && + vd->vdev_initialize_state == VDEV_INITIALIZE_ACTIVE) { + (void) vdev_initialize(vd); + } + mutex_exit(&vd->vdev_initialize_lock); + if (wasoffline || (oldstate < VDEV_STATE_DEGRADED && vd->vdev_state >= VDEV_STATE_DEGRADED)) @@ -3361,8 +3391,18 @@ vdev_get_stats(vdev_t *vd, vdev_stat_t *vs) vs->vs_timestamp = gethrtime() - vs->vs_timestamp; vs->vs_state = vd->vdev_state; vs->vs_rsize = vdev_get_min_asize(vd); - if (vd->vdev_ops->vdev_op_leaf) + if (vd->vdev_ops->vdev_op_leaf) { vs->vs_rsize += VDEV_LABEL_START_SIZE + VDEV_LABEL_END_SIZE; + /* + * Report intializing progress. Since we don't have the + * initializing locks held, this is only an estimate (although a + * fairly accurate one). + */ + vs->vs_initialize_bytes_done = vd->vdev_initialize_bytes_done; + vs->vs_initialize_bytes_est = vd->vdev_initialize_bytes_est; + vs->vs_initialize_state = vd->vdev_initialize_state; + vs->vs_initialize_action_time = vd->vdev_initialize_action_time; + } /* * Report expandable space on top-level, non-auxillary devices only. * The expandable space is reported in terms of metaslab sized units Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c Tue Jul 31 18:49:07 2018 (r336991) @@ -840,6 +840,7 @@ vdev_ops_t vdev_disk_ops = { vdev_disk_hold, vdev_disk_rele, NULL, + vdev_default_xlate, VDEV_TYPE_DISK, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c Tue Jul 31 18:49:07 2018 (r336991) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2015 by Delphix. All rights reserved. + * Copyright (c) 2011, 2016 by Delphix. All rights reserved. */ #include @@ -263,6 +263,7 @@ vdev_ops_t vdev_file_ops = { vdev_file_hold, vdev_file_rele, NULL, + vdev_default_xlate, VDEV_TYPE_FILE, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; @@ -282,6 +283,7 @@ vdev_ops_t vdev_disk_ops = { vdev_file_hold, vdev_file_rele, NULL, + vdev_default_xlate, VDEV_TYPE_DISK, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c Tue Jul 31 18:49:07 2018 (r336991) @@ -1628,6 +1628,7 @@ vdev_ops_t vdev_indirect_ops = { NULL, NULL, vdev_indirect_remap, + NULL, VDEV_TYPE_INDIRECT, /* name of this vdev type */ B_FALSE /* leaf vdev */ }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c Tue Jul 31 18:49:07 2018 (r336991) @@ -564,6 +564,7 @@ vdev_ops_t vdev_mirror_ops = { NULL, NULL, NULL, + vdev_default_xlate, VDEV_TYPE_MIRROR, /* name of this vdev type */ B_FALSE /* not a leaf vdev */ }; @@ -578,6 +579,7 @@ vdev_ops_t vdev_replacing_ops = { NULL, NULL, NULL, + vdev_default_xlate, VDEV_TYPE_REPLACING, /* name of this vdev type */ B_FALSE /* not a leaf vdev */ }; @@ -592,6 +594,7 @@ vdev_ops_t vdev_spare_ops = { NULL, NULL, NULL, + vdev_default_xlate, VDEV_TYPE_SPARE, /* name of this vdev type */ B_FALSE /* not a leaf vdev */ }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_missing.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_missing.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_missing.c Tue Jul 31 18:49:07 2018 (r336991) @@ -24,7 +24,7 @@ */ /* - * Copyright (c) 2012, 2014 by Delphix. All rights reserved. + * Copyright (c) 2012, 2016 by Delphix. All rights reserved. */ /* @@ -89,6 +89,7 @@ vdev_ops_t vdev_missing_ops = { NULL, NULL, NULL, + NULL, VDEV_TYPE_MISSING, /* name of this vdev type */ B_TRUE /* leaf vdev */ }; @@ -99,6 +100,7 @@ vdev_ops_t vdev_hole_ops = { vdev_default_asize, vdev_missing_io_start, vdev_missing_io_done, + NULL, NULL, NULL, NULL, Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c Tue Jul 31 18:49:07 2018 (r336991) @@ -150,6 +150,8 @@ uint32_t zfs_vdev_scrub_min_active = 1; uint32_t zfs_vdev_scrub_max_active = 2; uint32_t zfs_vdev_removal_min_active = 1; uint32_t zfs_vdev_removal_max_active = 2; +uint32_t zfs_vdev_initializing_min_active = 1; +uint32_t zfs_vdev_initializing_max_active = 1; /* * When the pool has less than zfs_vdev_async_write_active_min_dirty_percent @@ -407,6 +409,8 @@ vdev_queue_class_min_active(zio_priority_t p) return (zfs_vdev_scrub_min_active); case ZIO_PRIORITY_REMOVAL: return (zfs_vdev_removal_min_active); + case ZIO_PRIORITY_INITIALIZING: + return (zfs_vdev_initializing_min_active); default: panic("invalid priority %u", p); return (0); @@ -468,6 +472,8 @@ vdev_queue_class_max_active(spa_t *spa, zio_priority_t return (zfs_vdev_scrub_max_active); case ZIO_PRIORITY_REMOVAL: return (zfs_vdev_removal_max_active); + case ZIO_PRIORITY_INITIALIZING: + return (zfs_vdev_initializing_max_active); default: panic("invalid priority %u", p); return (0); @@ -688,8 +694,8 @@ again: } /* - * For LBA-ordered queues (async / scrub), issue the i/o which follows - * the most recently issued i/o in LBA (offset) order. + * For LBA-ordered queues (async / scrub / initializing), issue the + * i/o which follows the most recently issued i/o in LBA (offset) order. * * For FIFO queues (sync), issue the i/o with the lowest timestamp. */ @@ -745,13 +751,15 @@ vdev_queue_io(zio_t *zio) if (zio->io_priority != ZIO_PRIORITY_SYNC_READ && zio->io_priority != ZIO_PRIORITY_ASYNC_READ && zio->io_priority != ZIO_PRIORITY_SCRUB && - zio->io_priority != ZIO_PRIORITY_REMOVAL) + zio->io_priority != ZIO_PRIORITY_REMOVAL && + zio->io_priority != ZIO_PRIORITY_INITIALIZING) zio->io_priority = ZIO_PRIORITY_ASYNC_READ; } else { ASSERT(zio->io_type == ZIO_TYPE_WRITE); if (zio->io_priority != ZIO_PRIORITY_SYNC_WRITE && zio->io_priority != ZIO_PRIORITY_ASYNC_WRITE && - zio->io_priority != ZIO_PRIORITY_REMOVAL) + zio->io_priority != ZIO_PRIORITY_REMOVAL && + zio->io_priority != ZIO_PRIORITY_INITIALIZING) zio->io_priority = ZIO_PRIORITY_ASYNC_WRITE; } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_raidz.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_raidz.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_raidz.c Tue Jul 31 18:49:07 2018 (r336991) @@ -38,6 +38,10 @@ #include #include +#ifdef ZFS_DEBUG +#include /* vdev_xlate testing */ +#endif + /* * Virtual device vector for RAID-Z. * @@ -1884,6 +1888,39 @@ vdev_raidz_child_done(zio_t *zio) rc->rc_skipped = 0; } +static void +vdev_raidz_io_verify(zio_t *zio, raidz_map_t *rm, int col) +{ +#ifdef ZFS_DEBUG + vdev_t *vd = zio->io_vd; + vdev_t *tvd = vd->vdev_top; + + range_seg_t logical_rs, physical_rs; + logical_rs.rs_start = zio->io_offset; + logical_rs.rs_end = logical_rs.rs_start + + vdev_raidz_asize(zio->io_vd, zio->io_size); + + raidz_col_t *rc = &rm->rm_col[col]; + vdev_t *cvd = vd->vdev_child[rc->rc_devidx]; + + vdev_xlate(cvd, &logical_rs, &physical_rs); + ASSERT3U(rc->rc_offset, ==, physical_rs.rs_start); + ASSERT3U(rc->rc_offset, <, physical_rs.rs_end); + /* + * It would be nice to assert that rs_end is equal + * to rc_offset + rc_size but there might be an + * optional I/O at the end that is not accounted in + * rc_size. + */ + if (physical_rs.rs_end > rc->rc_offset + rc->rc_size) { + ASSERT3U(physical_rs.rs_end, ==, rc->rc_offset + + rc->rc_size + (1 << tvd->vdev_ashift)); + } else { + ASSERT3U(physical_rs.rs_end, ==, rc->rc_offset + rc->rc_size); + } +#endif +} + /* * Start an IO operation on a RAIDZ VDev * @@ -1926,6 +1963,12 @@ vdev_raidz_io_start(zio_t *zio) for (c = 0; c < rm->rm_cols; c++) { rc = &rm->rm_col[c]; cvd = vd->vdev_child[rc->rc_devidx]; + + /* + * Verify physical to logical translation. + */ + vdev_raidz_io_verify(zio, rm, c); + zio_nowait(zio_vdev_child_io(zio, NULL, cvd, rc->rc_offset, rc->rc_abd, rc->rc_size, zio->io_type, zio->io_priority, 0, @@ -2555,6 +2598,37 @@ vdev_raidz_state_change(vdev_t *vd, int faulted, int d vdev_set_state(vd, B_FALSE, VDEV_STATE_HEALTHY, VDEV_AUX_NONE); } +static void +vdev_raidz_xlate(vdev_t *cvd, const range_seg_t *in, range_seg_t *res) +{ + vdev_t *raidvd = cvd->vdev_parent; + ASSERT(raidvd->vdev_ops == &vdev_raidz_ops); + + uint64_t width = raidvd->vdev_children; + uint64_t tgt_col = cvd->vdev_id; + uint64_t ashift = raidvd->vdev_top->vdev_ashift; + + /* make sure the offsets are block-aligned */ + ASSERT0(in->rs_start % (1 << ashift)); + ASSERT0(in->rs_end % (1 << ashift)); + uint64_t b_start = in->rs_start >> ashift; + uint64_t b_end = in->rs_end >> ashift; + + uint64_t start_row = 0; + if (b_start > tgt_col) /* avoid underflow */ + start_row = ((b_start - tgt_col - 1) / width) + 1; + + uint64_t end_row = 0; + if (b_end > tgt_col) + end_row = ((b_end - tgt_col - 1) / width) + 1; + + res->rs_start = start_row << ashift; + res->rs_end = end_row << ashift; + + ASSERT3U(res->rs_start, <=, in->rs_start); + ASSERT3U(res->rs_end - res->rs_start, <=, in->rs_end - in->rs_start); +} + vdev_ops_t vdev_raidz_ops = { vdev_raidz_open, vdev_raidz_close, @@ -2565,6 +2639,7 @@ vdev_ops_t vdev_raidz_ops = { NULL, NULL, NULL, + vdev_raidz_xlate, VDEV_TYPE_RAIDZ, /* name of this vdev type */ B_FALSE /* not a leaf vdev */ }; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 18:49:07 2018 (r336991) @@ -44,6 +44,7 @@ #include #include #include +#include /* * This file contains the necessary logic to remove vdevs from a @@ -1021,6 +1022,7 @@ vdev_remove_complete(spa_t *spa) txg_wait_synced(spa->spa_dsl_pool, 0); txg = spa_vdev_enter(spa); vdev_t *vd = vdev_lookup_top(spa, spa->spa_vdev_removal->svr_vdev_id); + ASSERT3P(vd->vdev_initialize_thread, ==, NULL); sysevent_t *ev = spa_event_create(spa, vd, NULL, ESC_ZFS_VDEV_REMOVE_DEV); @@ -1659,6 +1661,9 @@ spa_vdev_remove_log(vdev_t *vd, uint64_t *txg) /* Make sure these changes are sync'ed */ spa_vdev_config_exit(spa, NULL, *txg, 0, FTAG); + /* Stop initializing */ + (void) vdev_initialize_stop_all(vd, VDEV_INITIALIZE_CANCELED); + *txg = spa_vdev_config_enter(spa); sysevent_t *ev = spa_event_create(spa, vd, NULL, @@ -1819,6 +1824,13 @@ spa_vdev_remove_top(vdev_t *vd, uint64_t *txg) */ error = spa_reset_logs(spa); + /* + * We stop any initializing that is currently in progress but leave + * the state as "active". This will allow the initializing to resume + * if the removal is canceled sometime later. + */ + vdev_initialize_stop_all(vd, VDEV_INITIALIZE_ACTIVE); + *txg = spa_vdev_config_enter(spa); /* @@ -1830,6 +1842,7 @@ spa_vdev_remove_top(vdev_t *vd, uint64_t *txg) if (error != 0) { metaslab_group_activate(mg); + spa_async_request(spa, SPA_ASYNC_INITIALIZE_RESTART); return (error); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_root.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_root.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_root.c Tue Jul 31 18:49:07 2018 (r336991) @@ -24,7 +24,7 @@ */ /* - * Copyright (c) 2012, 2014 by Delphix. All rights reserved. + * Copyright (c) 2012, 2016 by Delphix. All rights reserved. */ #include @@ -146,6 +146,7 @@ vdev_ops_t vdev_root_ops = { NULL, /* io_start - not applicable to the root */ NULL, /* io_done - not applicable to the root */ vdev_root_state_change, + NULL, NULL, NULL, NULL, Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Tue Jul 31 18:49:07 2018 (r336991) @@ -189,6 +189,8 @@ #include #include #include +#include +#include #include "zfs_namecheck.h" #include "zfs_prop.h" @@ -3707,6 +3709,80 @@ zfs_ioc_destroy(zfs_cmd_t *zc) } /* + * innvl: { + * vdevs: { + * guid 1, guid 2, ... + * }, + * func: POOL_INITIALIZE_{CANCEL|DO|SUSPEND} + * } + * + * outnvl: { + * [func: EINVAL (if provided command type didn't make sense)], + * [vdevs: { + * guid1: errno, (see function body for possible errnos) *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Tue Jul 31 18:49:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06FC31061743; Tue, 31 Jul 2018 18:49:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B030D76C1C; Tue, 31 Jul 2018 18:49:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 919A1177C0; Tue, 31 Jul 2018 18:49:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VInD0h065572; Tue, 31 Jul 2018 18:49:13 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VInBQN065564; Tue, 31 Jul 2018 18:49:11 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807311849.w6VInBQN065564@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 18:49:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r336991 - vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/ill... X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common vendor-sys/illumos/dist/uts/common/fs/zfs vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/cmd/zpool vendo... X-SVN-Commit-Revision: 336991 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 18:49:14 -0000 Author: mav Date: Tue Jul 31 18:49:07 2018 New Revision: 336991 URL: https://svnweb.freebsd.org/changeset/base/336991 Log: 9102 zfs should be able to initialize storage devices The first access to a disk block can incur a performance penalty on some platforms (e.g. AWS's EBS, VMware VMDKs). Therefore it is recommended that volumes be "thick provisioned", where supported by the platform (VMware). Thick provisioning is time consuming and often is ignored. If the thick provision step is omitted, customers will see suboptimal performance until we have written to all parts of the LUN. ZFS should be able to initialize any unused storage to remove any first-write penalty that exists. illumos/illumos-gate@094e47e980b0796b94b1b8f51f462a64d246e516 Reviewed by: John Wren Kennedy Reviewed by: Matthew Ahrens Reviewed by: Pavel Zakharov Reviewed by: Prakash Surya Approved by: Richard Lowe Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c vendor/illumos/dist/cmd/ztest/ztest.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c vendor/illumos/dist/lib/libzfs/common/libzfs_util.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h vendor/illumos/dist/man/man1m/zpool.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/Makefile.files vendor-sys/illumos/dist/uts/common/fs/zfs/metaslab.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/metaslab_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/zio_priority.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_disk.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_file.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_indirect.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_mirror.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_missing.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_queue.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_raidz.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_root.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor/illumos/dist/cmd/zpool/zpool_main.c ============================================================================== --- vendor/illumos/dist/cmd/zpool/zpool_main.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/cmd/zpool/zpool_main.c Tue Jul 31 18:49:07 2018 (r336991) @@ -84,6 +84,7 @@ static int zpool_do_detach(int, char **); static int zpool_do_replace(int, char **); static int zpool_do_split(int, char **); +static int zpool_do_initialize(int, char **); static int zpool_do_scrub(int, char **); static int zpool_do_import(int, char **); @@ -133,6 +134,7 @@ typedef enum { HELP_ONLINE, HELP_REPLACE, HELP_REMOVE, + HELP_INITIALIZE, HELP_SCRUB, HELP_STATUS, HELP_UPGRADE, @@ -184,6 +186,7 @@ static zpool_command_t command_table[] = { { "replace", zpool_do_replace, HELP_REPLACE }, { "split", zpool_do_split, HELP_SPLIT }, { NULL }, + { "initialize", zpool_do_initialize, HELP_INITIALIZE }, { "scrub", zpool_do_scrub, HELP_SCRUB }, { NULL }, { "import", zpool_do_import, HELP_IMPORT }, @@ -257,6 +260,8 @@ get_usage(zpool_help_t idx) return (gettext("\tremove [-nps] ...\n")); case HELP_REOPEN: return (gettext("\treopen \n")); + case HELP_INITIALIZE: + return (gettext("\tinitialize [-cs] [ ...]\n")); case HELP_SCRUB: return (gettext("\tscrub [-s | -p] ...\n")); case HELP_STATUS: @@ -1589,6 +1594,43 @@ print_status_config(zpool_handle_t *zhp, const char *n "resilvering" : "repairing"); } + if ((vs->vs_initialize_state == VDEV_INITIALIZE_ACTIVE || + vs->vs_initialize_state == VDEV_INITIALIZE_SUSPENDED || + vs->vs_initialize_state == VDEV_INITIALIZE_COMPLETE) && + !vs->vs_scan_removing) { + char zbuf[1024]; + char tbuf[256]; + struct tm zaction_ts; + + time_t t = vs->vs_initialize_action_time; + int initialize_pct = 100; + if (vs->vs_initialize_state != VDEV_INITIALIZE_COMPLETE) { + initialize_pct = (vs->vs_initialize_bytes_done * 100 / + (vs->vs_initialize_bytes_est + 1)); + } + + (void) localtime_r(&t, &zaction_ts); + (void) strftime(tbuf, sizeof (tbuf), "%c", &zaction_ts); + + switch (vs->vs_initialize_state) { + case VDEV_INITIALIZE_SUSPENDED: + (void) snprintf(zbuf, sizeof (zbuf), + ", suspended, started at %s", tbuf); + break; + case VDEV_INITIALIZE_ACTIVE: + (void) snprintf(zbuf, sizeof (zbuf), + ", started at %s", tbuf); + break; + case VDEV_INITIALIZE_COMPLETE: + (void) snprintf(zbuf, sizeof (zbuf), + ", completed at %s", tbuf); + break; + } + + (void) printf(gettext(" (%d%% initialized%s)"), + initialize_pct, zbuf); + } + (void) printf("\n"); for (c = 0; c < children; c++) { @@ -4163,6 +4205,119 @@ zpool_do_scrub(int argc, char **argv) } return (for_each_pool(argc, argv, B_TRUE, NULL, scrub_callback, &cb)); +} + +static void +zpool_collect_leaves(zpool_handle_t *zhp, nvlist_t *nvroot, nvlist_t *res) +{ + uint_t children = 0; + nvlist_t **child; + uint_t i; + + (void) nvlist_lookup_nvlist_array(nvroot, ZPOOL_CONFIG_CHILDREN, + &child, &children); + + if (children == 0) { + char *path = zpool_vdev_name(g_zfs, zhp, nvroot, B_FALSE); + fnvlist_add_boolean(res, path); + free(path); + return; + } + + for (i = 0; i < children; i++) { + zpool_collect_leaves(zhp, child[i], res); + } +} + +/* + * zpool initialize [-cs] [ ...] + * Initialize all unused blocks in the specified vdevs, or all vdevs in the pool + * if none specified. + * + * -c Cancel. Ends active initializing. + * -s Suspend. Initializing can then be restarted with no flags. + */ +int +zpool_do_initialize(int argc, char **argv) +{ + int c; + char *poolname; + zpool_handle_t *zhp; + nvlist_t *vdevs; + int err = 0; + + struct option long_options[] = { + {"cancel", no_argument, NULL, 'c'}, + {"suspend", no_argument, NULL, 's'}, + {0, 0, 0, 0} + }; + + pool_initialize_func_t cmd_type = POOL_INITIALIZE_DO; + while ((c = getopt_long(argc, argv, "cs", long_options, NULL)) != -1) { + switch (c) { + case 'c': + if (cmd_type != POOL_INITIALIZE_DO) { + (void) fprintf(stderr, gettext("-c cannot be " + "combined with other options\n")); + usage(B_FALSE); + } + cmd_type = POOL_INITIALIZE_CANCEL; + break; + case 's': + if (cmd_type != POOL_INITIALIZE_DO) { + (void) fprintf(stderr, gettext("-s cannot be " + "combined with other options\n")); + usage(B_FALSE); + } + cmd_type = POOL_INITIALIZE_SUSPEND; + break; + case '?': + if (optopt != 0) { + (void) fprintf(stderr, + gettext("invalid option '%c'\n"), optopt); + } else { + (void) fprintf(stderr, + gettext("invalid option '%s'\n"), + argv[optind - 1]); + } + usage(B_FALSE); + } + } + + argc -= optind; + argv += optind; + + if (argc < 1) { + (void) fprintf(stderr, gettext("missing pool name argument\n")); + usage(B_FALSE); + return (-1); + } + + poolname = argv[0]; + zhp = zpool_open(g_zfs, poolname); + if (zhp == NULL) + return (-1); + + vdevs = fnvlist_alloc(); + if (argc == 1) { + /* no individual leaf vdevs specified, so add them all */ + nvlist_t *config = zpool_get_config(zhp, NULL); + nvlist_t *nvroot = fnvlist_lookup_nvlist(config, + ZPOOL_CONFIG_VDEV_TREE); + zpool_collect_leaves(zhp, nvroot, vdevs); + } else { + int i; + for (i = 1; i < argc; i++) { + fnvlist_add_boolean(vdevs, argv[i]); + } + } + + err = zpool_initialize(zhp, cmd_type, vdevs); + + fnvlist_free(vdevs); + zpool_close(zhp); + + return (err); } typedef struct status_cbdata { Modified: vendor/illumos/dist/cmd/ztest/ztest.c ============================================================================== --- vendor/illumos/dist/cmd/ztest/ztest.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/cmd/ztest/ztest.c Tue Jul 31 18:49:07 2018 (r336991) @@ -103,6 +103,7 @@ #include #include #include +#include #include #include #include @@ -346,6 +347,7 @@ ztest_func_t ztest_spa_upgrade; ztest_func_t ztest_device_removal; ztest_func_t ztest_remap_blocks; ztest_func_t ztest_spa_checkpoint_create_discard; +ztest_func_t ztest_initialize; uint64_t zopt_always = 0ULL * NANOSEC; /* all the time */ uint64_t zopt_incessant = 1ULL * NANOSEC / 10; /* every 1/10 second */ @@ -389,7 +391,8 @@ ztest_info_t ztest_info[] = { &ztest_opts.zo_vdevtime }, { ztest_device_removal, 1, &zopt_sometimes }, { ztest_remap_blocks, 1, &zopt_sometimes }, - { ztest_spa_checkpoint_create_discard, 1, &zopt_rarely } + { ztest_spa_checkpoint_create_discard, 1, &zopt_rarely }, + { ztest_initialize, 1, &zopt_sometimes } }; #define ZTEST_FUNCS (sizeof (ztest_info) / sizeof (ztest_info_t)) @@ -5468,6 +5471,97 @@ ztest_spa_rename(ztest_ds_t *zd, uint64_t id) umem_free(newname, strlen(newname) + 1); rw_exit(&ztest_name_lock); +} + +static vdev_t * +ztest_random_concrete_vdev_leaf(vdev_t *vd) +{ + if (vd == NULL) + return (NULL); + + if (vd->vdev_children == 0) + return (vd); + + vdev_t *eligible[vd->vdev_children]; + int eligible_idx = 0, i; + for (i = 0; i < vd->vdev_children; i++) { + vdev_t *cvd = vd->vdev_child[i]; + if (cvd->vdev_top->vdev_removing) + continue; + if (cvd->vdev_children > 0 || + (vdev_is_concrete(cvd) && !cvd->vdev_detached)) { + eligible[eligible_idx++] = cvd; + } + } + VERIFY(eligible_idx > 0); + + uint64_t child_no = ztest_random(eligible_idx); + return (ztest_random_concrete_vdev_leaf(eligible[child_no])); +} + +/* ARGSUSED */ +void +ztest_initialize(ztest_ds_t *zd, uint64_t id) +{ + spa_t *spa = ztest_spa; + int error = 0; + + mutex_enter(&ztest_vdev_lock); + + spa_config_enter(spa, SCL_VDEV, FTAG, RW_READER); + + /* Random leaf vdev */ + vdev_t *rand_vd = ztest_random_concrete_vdev_leaf(spa->spa_root_vdev); + if (rand_vd == NULL) { + spa_config_exit(spa, SCL_VDEV, FTAG); + mutex_exit(&ztest_vdev_lock); + return; + } + + /* + * The random vdev we've selected may change as soon as we + * drop the spa_config_lock. We create local copies of things + * we're interested in. + */ + uint64_t guid = rand_vd->vdev_guid; + char *path = strdup(rand_vd->vdev_path); + boolean_t active = rand_vd->vdev_initialize_thread != NULL; + + zfs_dbgmsg("vd %p, guid %llu", rand_vd, guid); + spa_config_exit(spa, SCL_VDEV, FTAG); + + uint64_t cmd = ztest_random(POOL_INITIALIZE_FUNCS); + error = spa_vdev_initialize(spa, guid, cmd); + switch (cmd) { + case POOL_INITIALIZE_CANCEL: + if (ztest_opts.zo_verbose >= 4) { + (void) printf("Cancel initialize %s", path); + if (!active) + (void) printf(" failed (no initialize active)"); + (void) printf("\n"); + } + break; + case POOL_INITIALIZE_DO: + if (ztest_opts.zo_verbose >= 4) { + (void) printf("Start initialize %s", path); + if (active && error == 0) + (void) printf(" failed (already active)"); + else if (error != 0) + (void) printf(" failed (error %d)", error); + (void) printf("\n"); + } + break; + case POOL_INITIALIZE_SUSPEND: + if (ztest_opts.zo_verbose >= 4) { + (void) printf("Suspend initialize %s", path); + if (!active) + (void) printf(" failed (no initialize active)"); + (void) printf("\n"); + } + break; + } + free(path); + mutex_exit(&ztest_vdev_lock); } /* Modified: vendor/illumos/dist/lib/libzfs/common/libzfs.h ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs.h Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/lib/libzfs/common/libzfs.h Tue Jul 31 18:49:07 2018 (r336991) @@ -136,6 +136,9 @@ typedef enum zfs_error { EZFS_NO_CHECKPOINT, /* pool has no checkpoint */ EZFS_DEVRM_IN_PROGRESS, /* a device is currently being removed */ EZFS_VDEV_TOO_BIG, /* a device is too big to be used */ + EZFS_TOOMANY, /* argument list too long */ + EZFS_INITIALIZING, /* currently initializing */ + EZFS_NO_INITIALIZE, /* no active initialize */ EZFS_UNKNOWN } zfs_error_t; @@ -260,6 +263,8 @@ typedef struct splitflags { * Functions to manipulate pool and vdev state */ extern int zpool_scan(zpool_handle_t *, pool_scan_func_t, pool_scrub_cmd_t); +extern int zpool_initialize(zpool_handle_t *, pool_initialize_func_t, + nvlist_t *); extern int zpool_clear(zpool_handle_t *, const char *, nvlist_t *); extern int zpool_reguid(zpool_handle_t *); extern int zpool_reopen(zpool_handle_t *); Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_pool.c Tue Jul 31 18:49:07 2018 (r336991) @@ -1969,6 +1969,100 @@ zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func, } } +static int +xlate_init_err(int err) +{ + switch (err) { + case ENODEV: + return (EZFS_NODEVICE); + case EINVAL: + case EROFS: + return (EZFS_BADDEV); + case EBUSY: + return (EZFS_INITIALIZING); + case ESRCH: + return (EZFS_NO_INITIALIZE); + } + return (err); +} + +/* + * Begin, suspend, or cancel the initialization (initializing of all free + * blocks) for the given vdevs in the given pool. + */ +int +zpool_initialize(zpool_handle_t *zhp, pool_initialize_func_t cmd_type, + nvlist_t *vds) +{ + char msg[1024]; + libzfs_handle_t *hdl = zhp->zpool_hdl; + + nvlist_t *errlist; + + /* translate vdev names to guids */ + nvlist_t *vdev_guids = fnvlist_alloc(); + nvlist_t *guids_to_paths = fnvlist_alloc(); + boolean_t spare, cache; + nvlist_t *tgt; + nvpair_t *elem; + + for (elem = nvlist_next_nvpair(vds, NULL); elem != NULL; + elem = nvlist_next_nvpair(vds, elem)) { + char *vd_path = nvpair_name(elem); + tgt = zpool_find_vdev(zhp, vd_path, &spare, &cache, NULL); + + if ((tgt == NULL) || cache || spare) { + (void) snprintf(msg, sizeof (msg), + dgettext(TEXT_DOMAIN, "cannot initialize '%s'"), + vd_path); + int err = (tgt == NULL) ? EZFS_NODEVICE : + (spare ? EZFS_ISSPARE : EZFS_ISL2CACHE); + fnvlist_free(vdev_guids); + fnvlist_free(guids_to_paths); + return (zfs_error(hdl, err, msg)); + } + + uint64_t guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); + fnvlist_add_uint64(vdev_guids, vd_path, guid); + + (void) snprintf(msg, sizeof (msg), "%llu", guid); + fnvlist_add_string(guids_to_paths, msg, vd_path); + } + + int err = lzc_initialize(zhp->zpool_name, cmd_type, vdev_guids, + &errlist); + fnvlist_free(vdev_guids); + + if (err == 0) { + fnvlist_free(guids_to_paths); + return (0); + } + + nvlist_t *vd_errlist = NULL; + if (errlist != NULL) { + vd_errlist = fnvlist_lookup_nvlist(errlist, + ZPOOL_INITIALIZE_VDEVS); + } + + (void) snprintf(msg, sizeof (msg), + dgettext(TEXT_DOMAIN, "operation failed")); + + for (elem = nvlist_next_nvpair(vd_errlist, NULL); elem != NULL; + elem = nvlist_next_nvpair(vd_errlist, elem)) { + int64_t vd_error = xlate_init_err(fnvpair_value_int64(elem)); + char *path = fnvlist_lookup_string(guids_to_paths, + nvpair_name(elem)); + (void) zfs_error_fmt(hdl, vd_error, "cannot initialize '%s'", + path); + } + + fnvlist_free(guids_to_paths); + if (vd_errlist != NULL) + return (-1); + + return (zpool_standard_error(hdl, err, msg)); +} + /* * This provides a very minimal check whether a given string is likely a * c#t#d# style string. Users of this are expected to do their own Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_util.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_util.c Tue Jul 31 18:49:07 2018 (r336991) @@ -249,6 +249,13 @@ libzfs_error_description(libzfs_handle_t *hdl) return (dgettext(TEXT_DOMAIN, "device removal in progress")); case EZFS_VDEV_TOO_BIG: return (dgettext(TEXT_DOMAIN, "device exceeds supported size")); + case EZFS_TOOMANY: + return (dgettext(TEXT_DOMAIN, "argument list too long")); + case EZFS_INITIALIZING: + return (dgettext(TEXT_DOMAIN, "currently initializing")); + case EZFS_NO_INITIALIZE: + return (dgettext(TEXT_DOMAIN, "there is no active " + "initialization")); case EZFS_UNKNOWN: return (dgettext(TEXT_DOMAIN, "unknown error")); default: Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.c Tue Jul 31 18:49:07 2018 (r336991) @@ -1038,3 +1038,40 @@ lzc_channel_program_nosync(const char *pool, const cha return (lzc_channel_program_impl(pool, program, B_FALSE, timeout, memlimit, argnvl, outnvl)); } + +/* + * Changes initializing state. + * + * vdevs should be a list of (, guid) where guid is a uint64 vdev GUID. + * The key is ignored. + * + * If there are errors related to vdev arguments, per-vdev errors are returned + * in an nvlist with the key "vdevs". Each error is a (guid, errno) pair where + * guid is stringified with PRIu64, and errno is one of the following as + * an int64_t: + * - ENODEV if the device was not found + * - EINVAL if the devices is not a leaf or is not concrete (e.g. missing) + * - EROFS if the device is not writeable + * - EBUSY start requested but the device is already being initialized + * - ESRCH cancel/suspend requested but device is not being initialized + * + * If the errlist is empty, then return value will be: + * - EINVAL if one or more arguments was invalid + * - Other spa_open failures + * - 0 if the operation succeeded + */ +int +lzc_initialize(const char *poolname, pool_initialize_func_t cmd_type, + nvlist_t *vdevs, nvlist_t **errlist) +{ + int error; + nvlist_t *args = fnvlist_alloc(); + fnvlist_add_uint64(args, ZPOOL_INITIALIZE_COMMAND, (uint64_t)cmd_type); + fnvlist_add_nvlist(args, ZPOOL_INITIALIZE_VDEVS, vdevs); + + error = lzc_ioctl(ZFS_IOC_POOL_INITIALIZE, poolname, args, errlist); + + fnvlist_free(args); + + return (error); +} Modified: vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h ============================================================================== --- vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/lib/libzfs_core/common/libzfs_core.h Tue Jul 31 18:49:07 2018 (r336991) @@ -31,7 +31,9 @@ #include #include #include +#include + #ifdef __cplusplus extern "C" { #endif @@ -56,6 +58,8 @@ int lzc_destroy_snaps(nvlist_t *, boolean_t, nvlist_t int lzc_bookmark(nvlist_t *, nvlist_t **); int lzc_get_bookmarks(const char *, nvlist_t *, nvlist_t **); int lzc_destroy_bookmarks(nvlist_t *, nvlist_t **); +int lzc_initialize(const char *, pool_initialize_func_t, nvlist_t *, + nvlist_t **); int lzc_snaprange_space(const char *, const char *, uint64_t *); Modified: vendor/illumos/dist/man/man1m/zpool.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zpool.1m Tue Jul 31 18:32:57 2018 (r336990) +++ vendor/illumos/dist/man/man1m/zpool.1m Tue Jul 31 18:49:07 2018 (r336991) @@ -105,6 +105,11 @@ .Ar pool Ns | Ns Ar id .Op Ar newpool .Nm +.Cm initialize +.Op Fl cs +.Ar pool +.Op Ar device Ns ... +.Nm .Cm iostat .Op Fl v .Op Fl T Sy u Ns | Ns Sy d @@ -1333,6 +1338,32 @@ mounting option is enabled. In this case, the checkpointed state of the pool is opened and an administrator can see how the pool would look like if they were to fully rewind. +.El +.It Xo +.Nm +.Cm initialize +.Op Fl cs +.Ar pool +.Op Ar device Ns ... +.Xc +Begins initializing by writing to all unallocated regions on the specified +devices, or all eligible devices in the pool if no individual devices are +specified. +Only leaf data or log devices may be initialized. +.Bl -tag -width Ds +.It Fl c, -cancel +Cancel initializing on the specified devices, or all eligible devices if none +are specified. +If one or more target devices are invalid or are not currently being +initialized, the command will fail and no cancellation will occur on any device. +.It Fl s -suspend +Suspend initializing on the specified devices, or all eligible devices if none +are specified. +If one or more target devices are invalid or are not currently being +initialized, the command will fail and no suspension will occur on any device. +Initializing can then be resumed by running +.Nm zpool Cm initialize +with no flags on the relevant target devices. .El .It Xo .Nm From owner-svn-src-all@freebsd.org Tue Jul 31 18:52:44 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEA291061BA9; Tue, 31 Jul 2018 18:52:44 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A9B27721A; Tue, 31 Jul 2018 18:52:44 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5BCD017966; Tue, 31 Jul 2018 18:52:44 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VIqisP070385; Tue, 31 Jul 2018 18:52:44 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VIqiaS070384; Tue, 31 Jul 2018 18:52:44 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311852.w6VIqiaS070384@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 18:52:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336992 - head/release/arm X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release/arm X-SVN-Commit-Revision: 336992 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 18:52:45 -0000 Author: manu Date: Tue Jul 31 18:52:43 2018 New Revision: 336992 URL: https://svnweb.freebsd.org/changeset/base/336992 Log: release: arm: Use GENERIC kernel for IMX6 based boards Switch the IMX.6 based board to GENERIC the kernel config. Reviewed by: gjb Differential Revision: https://reviews.freebsd.org/D16511 Modified: head/release/arm/CUBOX-HUMMINGBOARD.conf head/release/arm/WANDBOARD.conf Modified: head/release/arm/CUBOX-HUMMINGBOARD.conf ============================================================================== --- head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 18:49:07 2018 (r336991) +++ head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 18:52:43 2018 (r336992) @@ -10,7 +10,7 @@ EMBEDDEDPORTS="sysutils/u-boot-cubox-hummingboard" FAT_SIZE="50m -b 16384" FAT_TYPE="16" IMAGE_SIZE="3072M" -KERNEL="IMX6" +KERNEL="GENERIC" MD_ARGS="-x 63 -y 255" NODOC=1 PART_SCHEME="MBR" Modified: head/release/arm/WANDBOARD.conf ============================================================================== --- head/release/arm/WANDBOARD.conf Tue Jul 31 18:49:07 2018 (r336991) +++ head/release/arm/WANDBOARD.conf Tue Jul 31 18:52:43 2018 (r336992) @@ -10,7 +10,7 @@ EMBEDDEDPORTS="sysutils/u-boot-wandboard" FAT_SIZE="50m -b 16384" FAT_TYPE="16" IMAGE_SIZE="3072M" -KERNEL="IMX6" +KERNEL="GENERIC" MD_ARGS="-x 63 -y 255" NODOC=1 PART_SCHEME="MBR" From owner-svn-src-all@freebsd.org Tue Jul 31 18:54:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A737F1061C31; Tue, 31 Jul 2018 18:54:07 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53EE677389; Tue, 31 Jul 2018 18:54:07 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3254A1796E; Tue, 31 Jul 2018 18:54:07 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VIs7cV070491; Tue, 31 Jul 2018 18:54:07 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VIs7fe070490; Tue, 31 Jul 2018 18:54:07 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311854.w6VIs7fe070490@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 18:54:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336993 - head/release/arm X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release/arm X-SVN-Commit-Revision: 336993 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 18:54:07 -0000 Author: manu Date: Tue Jul 31 18:54:06 2018 New Revision: 336993 URL: https://svnweb.freebsd.org/changeset/base/336993 Log: release: rpi-b: add BOARDNAME variable Reviewed by: gjb Differential Revision: https://reviews.freebsd.org/D16514 Modified: head/release/arm/RPI-B.conf Modified: head/release/arm/RPI-B.conf ============================================================================== --- head/release/arm/RPI-B.conf Tue Jul 31 18:52:43 2018 (r336992) +++ head/release/arm/RPI-B.conf Tue Jul 31 18:54:06 2018 (r336993) @@ -18,6 +18,7 @@ RPI_FIRMWARE_DIR="/usr/local/share/rpi-firmware" OL_DIR="${RPI_FIRMWARE_DIR}/overlays" OVERLAYS="mmc.dtbo pi3-disable-bt.dtbo" PART_SCHEME="MBR" +export BOARDNAME="RPI-B" arm_install_uboot() { UBOOT_FILES="u-boot.bin" From owner-svn-src-all@freebsd.org Tue Jul 31 18:55:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2DB2B1061CCE; Tue, 31 Jul 2018 18:55:45 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D904C7751D; Tue, 31 Jul 2018 18:55:44 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BA31C17978; Tue, 31 Jul 2018 18:55:44 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VItihP070622; Tue, 31 Jul 2018 18:55:44 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VItijE070621; Tue, 31 Jul 2018 18:55:44 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311855.w6VItijE070621@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 18:55:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336994 - head/release X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release X-SVN-Commit-Revision: 336994 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 18:55:45 -0000 Author: manu Date: Tue Jul 31 18:55:44 2018 New Revision: 336994 URL: https://svnweb.freebsd.org/changeset/base/336994 Log: release: arm: Use BOARDNAME in the image name Using KERNEL made sense when all boards had different kernel configuration. Now that all of them are using GENERIC use the board name instead Reviewed by: gjb Differential Revision: https://reviews.freebsd.org/D16512 Modified: head/release/release.sh Modified: head/release/release.sh ============================================================================== --- head/release/release.sh Tue Jul 31 18:54:06 2018 (r336993) +++ head/release/release.sh Tue Jul 31 18:55:44 2018 (r336994) @@ -392,7 +392,7 @@ chroot_arm_build_release() { export WORLDDIR="$(eval chroot ${CHROOTDIR} make ${MAKE_FLAGS} -C /usr/src/release -V WORLDDIR)" export OBJDIR="$(eval chroot ${CHROOTDIR} env WITH_UNIFIED_OBJDIR=1 make ${MAKE_FLAGS} -C /usr/src/release -V .OBJDIR)" export DESTDIR="${OBJDIR}/${KERNEL}" - export IMGBASE="${CHROOTDIR}/${OBJDIR}/${KERNEL}.img" + export IMGBASE="${CHROOTDIR}/${OBJDIR}/${BOARDNAME}.img" export OSRELEASE="$(eval chroot ${CHROOTDIR} make ${MAKE_FLAGS} -C /usr/src/release \ TARGET=${EMBEDDED_TARGET} TARGET_ARCH=${EMBEDDED_TARGET_ARCH} \ -V OSRELEASE)" @@ -406,11 +406,11 @@ chroot_arm_build_release() { arm_install_uboot mdconfig -d -u ${mddev} chroot ${CHROOTDIR} rmdir ${DESTDIR} - mv ${IMGBASE} ${CHROOTDIR}/${OBJDIR}/${OSRELEASE}-${KERNEL}.img + mv ${IMGBASE} ${CHROOTDIR}/${OBJDIR}/${OSRELEASE}-${BOARDNAME}.img chroot ${CHROOTDIR} mkdir -p /R - chroot ${CHROOTDIR} cp -p ${OBJDIR}/${OSRELEASE}-${KERNEL}.img \ - /R/${OSRELEASE}-${KERNEL}.img - chroot ${CHROOTDIR} xz -T ${XZ_THREADS} /R/${OSRELEASE}-${KERNEL}.img + chroot ${CHROOTDIR} cp -p ${OBJDIR}/${OSRELEASE}-${BOARDNAME}.img \ + /R/${OSRELEASE}-${BOARDNAME}.img + chroot ${CHROOTDIR} xz -T ${XZ_THREADS} /R/${OSRELEASE}-${BOARDNAME}.img cd ${CHROOTDIR}/R && sha512 ${OSRELEASE}* \ > CHECKSUM.SHA512 cd ${CHROOTDIR}/R && sha256 ${OSRELEASE}* \ From owner-svn-src-all@freebsd.org Tue Jul 31 18:57:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A23D11061D70; Tue, 31 Jul 2018 18:57:12 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53F427769F; Tue, 31 Jul 2018 18:57:12 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 32F411797F; Tue, 31 Jul 2018 18:57:12 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VIvChY070733; Tue, 31 Jul 2018 18:57:12 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VIvCDR070732; Tue, 31 Jul 2018 18:57:12 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311857.w6VIvCDR070732@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 18:57:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336995 - head/release X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release X-SVN-Commit-Revision: 336995 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 18:57:12 -0000 Author: manu Date: Tue Jul 31 18:57:11 2018 New Revision: 336995 URL: https://svnweb.freebsd.org/changeset/base/336995 Log: release: Deinstall u-boot ports before installing FORCE_PKG_REGISTER is broken so multiple invocation of release.sh for the same board will fails if /scratch isn't cleaned. Leave it but deinstall the package first. Reviewed by: gjb Differential Revision: https://reviews.freebsd.org/D16513 Modified: head/release/release.sh Modified: head/release/release.sh ============================================================================== --- head/release/release.sh Tue Jul 31 18:55:44 2018 (r336994) +++ head/release/release.sh Tue Jul 31 18:57:11 2018 (r336995) @@ -307,7 +307,7 @@ extra_chroot_setup() { for _PORT in ${EMBEDDEDPORTS}; do eval chroot ${CHROOTDIR} env ${PBUILD_FLAGS} make -C \ /usr/ports/${_PORT} \ - FORCE_PKG_REGISTER=1 install clean distclean + FORCE_PKG_REGISTER=1 deinstall install clean distclean done fi From owner-svn-src-all@freebsd.org Tue Jul 31 19:08:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11FF610621DA; Tue, 31 Jul 2018 19:08:27 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B87EF77EE7; Tue, 31 Jul 2018 19:08:26 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7F43017B59; Tue, 31 Jul 2018 19:08:26 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VJ8QKP076251; Tue, 31 Jul 2018 19:08:26 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VJ8Pmv076244; Tue, 31 Jul 2018 19:08:25 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311908.w6VJ8Pmv076244@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 19:08:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336996 - in head: share/man/man9 sys/arm/conf sys/arm64/conf sys/conf sys/dev/extres/nvmem X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: in head: share/man/man9 sys/arm/conf sys/arm64/conf sys/conf sys/dev/extres/nvmem X-SVN-Commit-Revision: 336996 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:08:27 -0000 Author: manu Date: Tue Jul 31 19:08:24 2018 New Revision: 336996 URL: https://svnweb.freebsd.org/changeset/base/336996 Log: nvmem: Add nvmem interface and helpers The nvmem interface helps provider of nvmem data to expose themselves to consumer. NVMEM is generally present on some embedded board in a form of eeprom or fuses. The nvmem api are helpers for consumer to read/write the cell data from a provider. Differential Revision: https://reviews.freebsd.org/D16419 Added: head/share/man/man9/nvmem.9 (contents, props changed) head/sys/dev/extres/nvmem/ head/sys/dev/extres/nvmem/nvmem.c (contents, props changed) head/sys/dev/extres/nvmem/nvmem.h (contents, props changed) head/sys/dev/extres/nvmem/nvmem_if.m (contents, props changed) Modified: head/sys/arm/conf/GENERIC head/sys/arm64/conf/GENERIC head/sys/conf/files Added: head/share/man/man9/nvmem.9 ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/man/man9/nvmem.9 Tue Jul 31 19:08:24 2018 (r336996) @@ -0,0 +1,157 @@ +.\" Copyright (c) 2018 Emmanuel Vadot +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE DEVELOPERS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES +.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +.\" IN NO EVENT SHALL THE DEVELOPERS BE LIABLE FOR ANY DIRECT, INDIRECT, +.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT +.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF +.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd July 24, 2018 +.Dt nvmem 9 +.Os +.Sh NAME +.Nm nvmem +.Nm nvmem_get_cell_len , +.Nm nvmem_read_cell_by_name , +.Nm nvmem_read_cell_by_idx , +.Nm nvmem_write_cell_by_name , +.Nm nvmem_write_cell_by_idx , +.Sh SYNOPSIS +.Cd "options EXT_RESOURCES" +.Cd "options FDT" +.Cd "device nvmem" +.In sys/extres/nvmem/nvmem.h +.Ft int +.Fn nvmem_get_cell_len "phandle_t node" "const char *name" +.Ft int +.Fn nvmem_read_cell_by_name "phandle_t node" "const char *name" "void *cell" "size_t buflen" +.Ft int +.Fn nvmem_read_cell_by_idx "phandle_t node" "int idx" "void *cell" "size_t buflen" +.Ft int +.Fn nvmem_write_cell_by_name "phandle_t node" "const char *name" "void *cell" "size_t buflen" +.Ft int +.Fn nvmem_write_cell_by_idx "phandle_t node" "int idx" "void *cell" "size_t buflen" +.Sh DESCRIPTION +On some embedded boards, the manufacturer stored some data on a NVMEM +(Non-Volatile Memory), this is generally stored in some eeprom or fuses. +.Pp +The +.Nm +API consist of helpers functions for consumer and device methods for +providers. +.Sh FUNCTIONS +.Bl -tag -width indent +.It Fn nvmem_get_cell_len "phandle_t node" "const char *name" +Get the size of the cell base on the reg property on the node. +Return the size or ENOENT if the cell name wasn't found +.It Fn nvmem_read_cell_by_name "phandle_t node" "const char *name" "void *cell" "size_t buflen" +Get the cell content based on the name. +Return 0 on sucess or ENOENT if the cell doesn't exists, ENXIO if no provider device was found, +EINVAL if the size isn't correct. +.It Fn nvmem_read_cell_by_idx "phandle_t node" "int idx" "void *cell" "size_t buflen" +Get the cell content based on the id. +Return 0 on sucess or ENOENT if the cell doesn't exists, ENXIO if no provider device was found, +EINVAL if the size isn't correct. +.It Fn nvmem_write_cell_by_name "phandle_t node" "const char *name" "void *cell" "size_t buflen" +Write the cell content based on the name. +Return 0 on sucess or ENOENT if the cell doesn't exists, ENXIO if no provider device was found, +EINVAL if the size isn't correct. +.It Fn nvmem_write_cell_by_idx "phandle_t node" "int idx" "void *cell" "size_t buflen" +Write the cell content based on the id. +Return 0 on sucess or ENOENT if the cell doesn't exists, ENXIO if no provider device was found, +EINVAL if the size isn't correct. +.El +.Sh DEVICE METHODS +.Bl -tag -width indent +.It Fn nvmem_read "device_t dev" "uint32_t offset" "uint32_t size" "uint8_t *buffer" +Provider device method to read a cell content. +.It Fn nvmem_write "device_t dev" "uint32_t offset" "uint32_t size" "uint8_t *buffer" +Provider device method to write a cell content. +.El +.Sh EXAMPLES +Consider this DTS +.Bd -literal +/* Provider */ +eeprom: eeprom@20000 { + board_id: id@0 { + reg = <0x0 0x4>; + }; +}; +/* Consumer */ +device@30000 { + ... + + nvmem-cells = <&board_id> + nvmem-cell-names = "boardid"; +}; +.Ed +.Pp +The device driver for eeprom@20000 needs to expose itself as a provider +.Bd -literal +#include "nvmem_if.h" + +int +foo_nvmem_read(device_t dev, uint32_t offset, uint32_t size, uint8_t *buffer) +{ + /* Read the data */ +} + +int +foo_attach(device_t dev) +{ + phandle_t node; + + node = ofw_bus_get_node(dev); + ... + /* Registering the device so the consumers can find us */ + OF_device_register_xref(OF_xref_from_node(node), dev); + + ... +} + +static device_method_t foo_methods[] = { + ... + + /* nvmem interface */ + DEVMETHOD(nvmem_read, foo_nvmem_read), + + /* Terminate method list */ + DEVMETHOD_END +}; +.Ed +.Pp +The consumer device driver for device@30000 can now read the nvmem data +.Bd -literal +int +bar_attach(device_t dev) +{ + phandle_t node; + uint32_t boardid; + + ... + node = ofw_bus_get_node(dev); + nvmem_read_cell_by_name(node, "boardid", (void *)&boardid, sizeof(boardid)); + ... +} +.Ed +.Sh HISTORY +The nvmem related function first appear in +.Fx 12.0 . +The nvmem interface and manual page was written by +.An Emmanuel Vadot Aq Mt manu@FreeBSD.org . Modified: head/sys/arm/conf/GENERIC ============================================================================== --- head/sys/arm/conf/GENERIC Tue Jul 31 18:57:11 2018 (r336995) +++ head/sys/arm/conf/GENERIC Tue Jul 31 19:08:24 2018 (r336996) @@ -72,6 +72,7 @@ options EXT_RESOURCES device clk device phy device hwreset +device nvmem device regulator device syscon Modified: head/sys/arm64/conf/GENERIC ============================================================================== --- head/sys/arm64/conf/GENERIC Tue Jul 31 18:57:11 2018 (r336995) +++ head/sys/arm64/conf/GENERIC Tue Jul 31 19:08:24 2018 (r336996) @@ -251,6 +251,7 @@ options EXT_RESOURCES device clk device phy device hwreset +device nvmem device regulator device syscon Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Tue Jul 31 18:57:11 2018 (r336995) +++ head/sys/conf/files Tue Jul 31 19:08:24 2018 (r336996) @@ -1750,6 +1750,8 @@ dev/extres/phy/phydev_if.m optional ext_resources phy dev/extres/phy/phynode_if.m optional ext_resources phy fdt dev/extres/hwreset/hwreset.c optional ext_resources hwreset fdt dev/extres/hwreset/hwreset_if.m optional ext_resources hwreset fdt +dev/extres/nvmem/nvmem.c optional ext_resources nvmem fdt +dev/extres/nvmem/nvmem_if.m optional ext_resources nvmem fdt dev/extres/regulator/regdev_if.m optional ext_resources regulator fdt dev/extres/regulator/regnode_if.m optional ext_resources regulator fdt dev/extres/regulator/regulator.c optional ext_resources regulator fdt Added: head/sys/dev/extres/nvmem/nvmem.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/extres/nvmem/nvmem.c Tue Jul 31 19:08:24 2018 (r336996) @@ -0,0 +1,199 @@ +/*- + * Copyright 2018 Emmanuel Vadot + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "nvmem.h" +#include "nvmem_if.h" + +static int +nvmem_get_cell_node(phandle_t node, int idx, phandle_t *cell) +{ + phandle_t *p_cell; + phandle_t cell_node; + int ncell; + + if (!OF_hasprop(node, "nvmem-cells") || + !OF_hasprop(node, "nvmem-cell-names")) + return (ENOENT); + + ncell = OF_getencprop_alloc_multi(node, "nvmem-cells", sizeof(*p_cell), (void **)&p_cell); + if (ncell <= 0) + return (ENOENT); + + cell_node = OF_node_from_xref(p_cell[idx]); + if (cell_node == p_cell[idx]) { + if (bootverbose) + printf("nvmem_get_node: Cannot resolve phandle %x\n", + p_cell[idx]); + OF_prop_free(p_cell); + return (ENOENT); + } + + OF_prop_free(p_cell); + *cell = cell_node; + + return (0); +} + +int +nvmem_get_cell_len(phandle_t node, const char *name) +{ + phandle_t cell_node; + uint32_t reg[2]; + int rv, idx; + + rv = ofw_bus_find_string_index(node, "nvmem-cell-names", name, &idx); + if (rv != 0) + return (rv); + + rv = nvmem_get_cell_node(node, idx, &cell_node); + if (rv != 0) + return (rv); + + if (OF_getencprop(cell_node, "reg", reg, sizeof(reg)) != sizeof(reg)) { + if (bootverbose) + printf("nvmem_get_cell_len: Cannot parse reg property of cell %s\n", + name); + return (ENOENT); + } + + return (reg[1]); +} + +int +nvmem_read_cell_by_idx(phandle_t node, int idx, void *cell, size_t buflen) +{ + phandle_t cell_node; + device_t provider; + uint32_t reg[2]; + int rv; + + rv = nvmem_get_cell_node(node, idx, &cell_node); + if (rv != 0) + return (rv); + + /* Validate the reg property */ + if (OF_getencprop(cell_node, "reg", reg, sizeof(reg)) != sizeof(reg)) { + if (bootverbose) + printf("nvmem_get_cell_by_name: Cannot parse reg property of cell %d\n", + idx); + return (ENOENT); + } + + if (buflen != reg[1]) + return (EINVAL); + + provider = OF_device_from_xref(OF_xref_from_node(OF_parent(cell_node))); + if (provider == NULL) { + if (bootverbose) + printf("nvmem_get_cell_by_idx: Cannot find the nvmem device\n"); + return (ENXIO); + } + + rv = NVMEM_READ(provider, reg[0], reg[1], cell); + if (rv != 0) { + return (rv); + } + + return (0); +} + +int +nvmem_read_cell_by_name(phandle_t node, const char *name, void *cell, size_t buflen) +{ + int rv, idx; + + rv = ofw_bus_find_string_index(node, "nvmem-cell-names", name, &idx); + if (rv != 0) + return (rv); + + return (nvmem_read_cell_by_idx(node, idx, cell, buflen)); +} + +int +nvmem_write_cell_by_idx(phandle_t node, int idx, void *cell, size_t buflen) +{ + phandle_t cell_node, prov_node; + device_t provider; + uint32_t reg[2]; + int rv; + + rv = nvmem_get_cell_node(node, idx, &cell_node); + if (rv != 0) + return (rv); + + prov_node = OF_parent(cell_node); + if (OF_hasprop(prov_node, "read-only")) + return (ENXIO); + + /* Validate the reg property */ + if (OF_getencprop(cell_node, "reg", reg, sizeof(reg)) != sizeof(reg)) { + if (bootverbose) + printf("nvmem_get_cell_by_idx: Cannot parse reg property of cell %d\n", + idx); + return (ENXIO); + } + + if (buflen != reg[1]) + return (EINVAL); + + provider = OF_device_from_xref(OF_xref_from_node(prov_node)); + if (provider == NULL) { + if (bootverbose) + printf("nvmem_get_cell_by_idx: Cannot find the nvmem device\n"); + return (ENXIO); + } + + rv = NVMEM_WRITE(provider, reg[0], reg[1], cell); + if (rv != 0) { + return (rv); + } + + return (0); +} + +int +nvmem_write_cell_by_name(phandle_t node, const char *name, void *cell, size_t buflen) +{ + int rv, idx; + + rv = ofw_bus_find_string_index(node, "nvmem-cell-names", name, &idx); + if (rv != 0) + return (rv); + + return (nvmem_write_cell_by_idx(node, idx, cell, buflen)); +} Added: head/sys/dev/extres/nvmem/nvmem.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/extres/nvmem/nvmem.h Tue Jul 31 19:08:24 2018 (r336996) @@ -0,0 +1,37 @@ +/*- + * Copyright 2018 Emmanuel Vadot + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _DEV_EXTRES_NVMEM_H_ +#define _DEV_EXTRES_NVMEM_H_ + +int nvmem_get_cell_len(phandle_t node, const char *name); +int nvmem_read_cell_by_name(phandle_t node, const char *name, void *cell, size_t buflen); +int nvmem_read_cell_by_idx(phandle_t node, int idx, void *cell, size_t buflen); +int nvmem_write_cell_by_name(phandle_t node, const char *name, void *cell, size_t buflen); +int nvmem_write_cell_by_idx(phandle_t node, int idx, void *cell, size_t buflen); + +#endif /* _DEV_EXTRES_NVMEM_H_ */ Added: head/sys/dev/extres/nvmem/nvmem_if.m ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sys/dev/extres/nvmem/nvmem_if.m Tue Jul 31 19:08:24 2018 (r336996) @@ -0,0 +1,67 @@ +#- +# Copyright (c) 2018 Emmanuel Vadot +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. +# +# $FreeBSD$ +# + +INTERFACE nvmem; + +# +# Default implementations of some methods. +# +CODE { + static int + null_nvmem_read(device_t dev __unused, uint32_t offset __unused, uint32_t size __unused, uint8_t *buffer __unused) + { + + return (ENXIO); + } + + static int + null_nvmem_write(device_t dev __unused, uint32_t offset __unused, uint32_t size __unused, uint8_t *buffer __unused) + { + + return (ENXIO); + } +}; + +# +# Read +# +METHOD int read { + device_t dev; + uint32_t offset; + uint32_t size; + uint8_t *buffer; +} DEFAULT null_nvmem_read; + +# +# Write +# +METHOD int write { + device_t dev; + uint32_t offset; + uint32_t size; + uint8_t *buffer; +} DEFAULT null_nvmem_write; From owner-svn-src-all@freebsd.org Tue Jul 31 19:10:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8ACC10623B8; Tue, 31 Jul 2018 19:10:52 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F0FB78138; Tue, 31 Jul 2018 19:10:52 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 51DD617B92; Tue, 31 Jul 2018 19:10:52 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VJAqKh077113; Tue, 31 Jul 2018 19:10:52 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VJAoNh077107; Tue, 31 Jul 2018 19:10:50 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311910.w6VJAoNh077107@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 19:10:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336997 - head/release/arm X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release/arm X-SVN-Commit-Revision: 336997 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:10:53 -0000 Author: manu Date: Tue Jul 31 19:10:50 2018 New Revision: 336997 URL: https://svnweb.freebsd.org/changeset/base/336997 Log: release: Restore copy of boot.scr for some board This is not a problem for 12-CURRENT as EFI boot works but it doesn't for 11. While here some board arm_install_uboot also copy ubldr.bin et create firstboot files but it's already done in arm_install_boot Reviewed by: gjb MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D16481 Modified: head/release/arm/BANANAPI.conf head/release/arm/BEAGLEBONE.conf head/release/arm/CUBIEBOARD.conf head/release/arm/CUBIEBOARD2.conf head/release/arm/CUBOX-HUMMINGBOARD.conf head/release/arm/PANDABOARD.conf head/release/arm/WANDBOARD.conf Modified: head/release/arm/BANANAPI.conf ============================================================================== --- head/release/arm/BANANAPI.conf Tue Jul 31 19:08:24 2018 (r336996) +++ head/release/arm/BANANAPI.conf Tue Jul 31 19:10:50 2018 (r336997) @@ -19,8 +19,15 @@ export BOARDNAME="BANANAPI" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-bananapi" UBOOT_FILES="u-boot-sunxi-with-spl.bin" + FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" + chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} + chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr + sync + umount_loop ${CHROOTDIR}/${FATMOUNT} + chroot ${CHROOTDIR} rmdir ${FATMOUNT} + return 0 } Modified: head/release/arm/BEAGLEBONE.conf ============================================================================== --- head/release/arm/BEAGLEBONE.conf Tue Jul 31 19:08:24 2018 (r336996) +++ head/release/arm/BEAGLEBONE.conf Tue Jul 31 19:10:50 2018 (r336997) @@ -27,6 +27,6 @@ arm_install_uboot() { sync umount_loop ${CHROOTDIR}/${FATMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - + return 0 } Modified: head/release/arm/CUBIEBOARD.conf ============================================================================== --- head/release/arm/CUBIEBOARD.conf Tue Jul 31 19:08:24 2018 (r336996) +++ head/release/arm/CUBIEBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) @@ -20,21 +20,14 @@ arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard" UBOOT_FILES="u-boot-sunxi-with-spl.bin" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: head/release/arm/CUBIEBOARD2.conf ============================================================================== --- head/release/arm/CUBIEBOARD2.conf Tue Jul 31 19:08:24 2018 (r336996) +++ head/release/arm/CUBIEBOARD2.conf Tue Jul 31 19:10:50 2018 (r336997) @@ -20,21 +20,14 @@ arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard2" UBOOT_FILES="u-boot-sunxi-with-spl.bin" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: head/release/arm/CUBOX-HUMMINGBOARD.conf ============================================================================== --- head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 19:08:24 2018 (r336996) +++ head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) @@ -20,21 +20,14 @@ arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubox-hummingboard" UBOOT_FILES="u-boot.imx" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=512 seek=2 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: head/release/arm/PANDABOARD.conf ============================================================================== --- head/release/arm/PANDABOARD.conf Tue Jul 31 19:08:24 2018 (r336996) +++ head/release/arm/PANDABOARD.conf Tue Jul 31 19:10:50 2018 (r336997) @@ -19,21 +19,14 @@ export BOARDNAME="PANDABOARD" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-pandaboard" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/MLO ${FATMOUNT}/MLO chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/u-boot.img ${FATMOUNT}/u-boot.img chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: head/release/arm/WANDBOARD.conf ============================================================================== --- head/release/arm/WANDBOARD.conf Tue Jul 31 19:08:24 2018 (r336996) +++ head/release/arm/WANDBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) @@ -19,8 +19,15 @@ export BOARDNAME="WANDBOARD" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-wandboard" UBOOT_FILES="u-boot.imx" + FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=512 seek=2 conv=sync - + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" + chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} + chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr + sync + umount_loop ${CHROOTDIR}/${FATMOUNT} + chroot ${CHROOTDIR} rmdir ${FATMOUNT} + return 0 } From owner-svn-src-all@freebsd.org Tue Jul 31 19:12:34 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 390E1106258B; Tue, 31 Jul 2018 19:12:34 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E34AA784F6; Tue, 31 Jul 2018 19:12:33 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C46B317D0B; Tue, 31 Jul 2018 19:12:33 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VJCXCM081192; Tue, 31 Jul 2018 19:12:33 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VJCVuL081184; Tue, 31 Jul 2018 19:12:31 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311912.w6VJCVuL081184@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 19:12:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336998 - head/release/arm X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release/arm X-SVN-Commit-Revision: 336998 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:12:34 -0000 Author: manu Date: Tue Jul 31 19:12:31 2018 New Revision: 336998 URL: https://svnweb.freebsd.org/changeset/base/336998 Log: release: arm: Always boot armv6/v7 image with EFI boot.scr is a u-boot script that loads and execute ubldr.bin If not present u-boot will automatically boot loader.efi which is already installed. This means that all armv6/armv7 images are now booted via EFI Tested-On: RPI-B Tested-On: RPI2 Tested-On: OrangePi One Tested-On: All lot of other boards MFC after: Never Relnotes: yes Modified: head/release/arm/BANANAPI.conf head/release/arm/BEAGLEBONE.conf head/release/arm/CUBIEBOARD.conf head/release/arm/CUBIEBOARD2.conf head/release/arm/CUBOX-HUMMINGBOARD.conf head/release/arm/PANDABOARD.conf head/release/arm/RPI-B.conf head/release/arm/RPI2.conf head/release/arm/WANDBOARD.conf Modified: head/release/arm/BANANAPI.conf ============================================================================== --- head/release/arm/BANANAPI.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/BANANAPI.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -19,15 +19,8 @@ export BOARDNAME="BANANAPI" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-bananapi" UBOOT_FILES="u-boot-sunxi-with-spl.bin" - FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - sync - umount_loop ${CHROOTDIR}/${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${FATMOUNT} return 0 } Modified: head/release/arm/BEAGLEBONE.conf ============================================================================== --- head/release/arm/BEAGLEBONE.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/BEAGLEBONE.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -23,7 +23,6 @@ arm_install_uboot() { chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/MLO ${FATMOUNT}/MLO chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/u-boot.img ${FATMOUNT}/u-boot.img - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr sync umount_loop ${CHROOTDIR}/${FATMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} Modified: head/release/arm/CUBIEBOARD.conf ============================================================================== --- head/release/arm/CUBIEBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/CUBIEBOARD.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -19,15 +19,8 @@ export BOARDNAME="CUBIEBOARD" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard" UBOOT_FILES="u-boot-sunxi-with-spl.bin" - FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - sync - umount_loop ${CHROOTDIR}/${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${FATMOUNT} return 0 } Modified: head/release/arm/CUBIEBOARD2.conf ============================================================================== --- head/release/arm/CUBIEBOARD2.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/CUBIEBOARD2.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -19,15 +19,8 @@ export BOARDNAME="CUBIEBOARD2" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard2" UBOOT_FILES="u-boot-sunxi-with-spl.bin" - FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - sync - umount_loop ${CHROOTDIR}/${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${FATMOUNT} return 0 } Modified: head/release/arm/CUBOX-HUMMINGBOARD.conf ============================================================================== --- head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -19,15 +19,8 @@ export BOARDNAME="CUBOX-HUMMINGBOARD" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubox-hummingboard" UBOOT_FILES="u-boot.imx" - FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=512 seek=2 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - sync - umount_loop ${CHROOTDIR}/${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${FATMOUNT} return 0 } Modified: head/release/arm/PANDABOARD.conf ============================================================================== --- head/release/arm/PANDABOARD.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/PANDABOARD.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -23,7 +23,6 @@ arm_install_uboot() { chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/MLO ${FATMOUNT}/MLO chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/u-boot.img ${FATMOUNT}/u-boot.img - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr sync umount_loop ${CHROOTDIR}/${FATMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} Modified: head/release/arm/RPI-B.conf ============================================================================== --- head/release/arm/RPI-B.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/RPI-B.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -43,7 +43,6 @@ arm_install_uboot() { chroot ${CHROOTDIR} cp -p ${OL_DIR}/${_OL} \ ${FATMOUNT}/overlays/${_OL} done - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr sync umount_loop ${CHROOTDIR}/${FATMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} Modified: head/release/arm/RPI2.conf ============================================================================== --- head/release/arm/RPI2.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/RPI2.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -42,7 +42,6 @@ arm_install_uboot() { chroot ${CHROOTDIR} cp -p ${OL_DIR}/${_OL} \ ${FATMOUNT}/overlays/${_OL} done - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr sync umount_loop ${CHROOTDIR}/${FATMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} Modified: head/release/arm/WANDBOARD.conf ============================================================================== --- head/release/arm/WANDBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) +++ head/release/arm/WANDBOARD.conf Tue Jul 31 19:12:31 2018 (r336998) @@ -19,15 +19,8 @@ export BOARDNAME="WANDBOARD" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-wandboard" UBOOT_FILES="u-boot.imx" - FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=512 seek=2 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - sync - umount_loop ${CHROOTDIR}/${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${FATMOUNT} return 0 } From owner-svn-src-all@freebsd.org Tue Jul 31 19:13:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28B13106260A; Tue, 31 Jul 2018 19:13:51 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3A8878673; Tue, 31 Jul 2018 19:13:50 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B4A4517D0D; Tue, 31 Jul 2018 19:13:50 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VJDo7h081288; Tue, 31 Jul 2018 19:13:50 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VJDoo5081287; Tue, 31 Jul 2018 19:13:50 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311913.w6VJDoo5081287@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 19:13:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r336999 - head/release/tools X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release/tools X-SVN-Commit-Revision: 336999 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:13:51 -0000 Author: manu Date: Tue Jul 31 19:13:50 2018 New Revision: 336999 URL: https://svnweb.freebsd.org/changeset/base/336999 Log: release: arm: Enable multicons for arm64 Since we have now EFI framebuffer enabled for ARM64 if we boot on a board with an screen, u-boot will set up a EFI GOP framebuffer and we won't boot using the serial console. Also on RPI3 the firmware always setup the framebuffer area resulting in u-boot always setup the EFI GOP and FreeBSD never using the serial console. Reviewed by: gjb, lwshu (previous version) Differential Revision: https://reviews.freebsd.org/D16472 Modified: head/release/tools/arm.subr Modified: head/release/tools/arm.subr ============================================================================== --- head/release/tools/arm.subr Tue Jul 31 19:12:31 2018 (r336998) +++ head/release/tools/arm.subr Tue Jul 31 19:13:50 2018 (r336999) @@ -124,7 +124,20 @@ arm_setup_usb_otg() { echo 'hw.usb.template=3' \ >> ${CHROOTDIR}/${DESTDIR}/boot/loader.conf echo 'umodem_load="YES"' \ + >> ${CHROOTDIR}/${DESTDIR}/boot/loader.conf +} + +arm64_setup_multicons() { + if [ "${EMBEDDED_TARGET_ARCH}" != "aarch64" ]; then + return + fi + + echo '# Multiple console (serial+efi gop) enabled.' \ >> ${CHROOTDIR}/${DESTDIR}/boot/loader.conf + echo 'boot_multicons="YES"' \ + >> ${CHROOTDIR}/${DESTDIR}/boot/loader.conf + echo 'boot_serial="YES"' \ + >> ${CHROOTDIR}/${DESTDIR}/boot/loader.conf } arm_install_base() { @@ -138,6 +151,7 @@ arm_install_base() { arm_create_user arm_setup_usb_otg + arm64_setup_multicons echo '# Custom /etc/fstab for FreeBSD embedded images' \ > ${CHROOTDIR}/${DESTDIR}/etc/fstab From owner-svn-src-all@freebsd.org Tue Jul 31 19:14:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E461106269C; Tue, 31 Jul 2018 19:14:53 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 137DB787F6; Tue, 31 Jul 2018 19:14:53 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E872717D17; Tue, 31 Jul 2018 19:14:52 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VJEqAJ081376; Tue, 31 Jul 2018 19:14:52 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VJEqWJ081375; Tue, 31 Jul 2018 19:14:52 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807311914.w6VJEqWJ081375@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 19:14:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337000 - head/release/arm64 X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/release/arm64 X-SVN-Commit-Revision: 337000 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:14:53 -0000 Author: manu Date: Tue Jul 31 19:14:52 2018 New Revision: 337000 URL: https://svnweb.freebsd.org/changeset/base/337000 Log: release: arm64: Add PINE64-LTS configuration file Pine64 isn't produced anymore but Pine64-LTS is. This image works on the LTS release and the Sopine module. Reviewed by: gjb Relnotes: yes Differential Revision: https://reviews.freebsd.org/D16487 Added: head/release/arm64/PINE64-LTS.conf (contents, props changed) Added: head/release/arm64/PINE64-LTS.conf ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/release/arm64/PINE64-LTS.conf Tue Jul 31 19:14:52 2018 (r337000) @@ -0,0 +1,26 @@ +#!/bin/sh +# +# $FreeBSD$ +# + +EMBEDDED_TARGET_ARCH="aarch64" +EMBEDDED_TARGET="arm64" +EMBEDDEDBUILD=1 +EMBEDDEDPORTS="sysutils/u-boot-sopine" +FAT_SIZE="54m -b 1m" +FAT_TYPE="16" +IMAGE_SIZE="2560M" +KERNEL="GENERIC" +MD_ARGS="-x 63 -y 255" +NODOC=1 +PART_SCHEME="MBR" +export BOARDNAME="PINE64-LTS" + +arm_install_uboot() { + UBOOT_DIR="/usr/local/share/u-boot/u-boot-sopine" + UBOOT_FILES="u-boot-sunxi-with-spl.bin" + chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ + of=/dev/${mddev} bs=1k seek=8 conv=sync + + return 0 +} From owner-svn-src-all@freebsd.org Tue Jul 31 19:23:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5459C1062AAE; Tue, 31 Jul 2018 19:23:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09BF8793F7; Tue, 31 Jul 2018 19:23:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DEDAC17EB2; Tue, 31 Jul 2018 19:23:27 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VJNRuV086560; Tue, 31 Jul 2018 19:23:27 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VJNR4h086559; Tue, 31 Jul 2018 19:23:27 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807311923.w6VJNR4h086559@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 19:23:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337001 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337001 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:23:28 -0000 Author: mav Date: Tue Jul 31 19:23:27 2018 New Revision: 337001 URL: https://svnweb.freebsd.org/changeset/base/337001 Log: Add new files missed in r336991. Added: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_initialize.h (contents, props changed) vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_initialize.c (contents, props changed) Added: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_initialize.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_initialize.h Tue Jul 31 19:23:27 2018 (r337001) @@ -0,0 +1,46 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2016 by Delphix. All rights reserved. + */ + +#ifndef _SYS_VDEV_INITIALIZE_H +#define _SYS_VDEV_INITIALIZE_H + +#ifdef __cplusplus +extern "C" { +#endif + +extern void vdev_initialize(vdev_t *vd); +extern void vdev_initialize_stop(vdev_t *vd, + vdev_initializing_state_t tgt_state); +extern void vdev_initialize_stop_all(vdev_t *vd, + vdev_initializing_state_t tgt_state); +extern void vdev_initialize_restart(vdev_t *vd); +extern void vdev_xlate(vdev_t *vd, const range_seg_t *logical_rs, + range_seg_t *physical_rs); + +#ifdef __cplusplus +} +#endif + +#endif /* _SYS_VDEV_INITIALIZE_H */ Added: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_initialize.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_initialize.c Tue Jul 31 19:23:27 2018 (r337001) @@ -0,0 +1,791 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2016 by Delphix. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * Maximum number of metaslabs per group that can be initialized + * simultaneously. + */ +int max_initialize_ms = 3; + +/* + * Value that is written to disk during initialization. + */ +uint64_t zfs_initialize_value = 0xdeadbeefdeadbeefULL; + +/* maximum number of I/Os outstanding per leaf vdev */ +int zfs_initialize_limit = 1; + +/* size of initializing writes; default 1MiB, see zfs_remove_max_segment */ +uint64_t zfs_initialize_chunk_size = 1024 * 1024; + +static boolean_t +vdev_initialize_should_stop(vdev_t *vd) +{ + return (vd->vdev_initialize_exit_wanted || !vdev_writeable(vd) || + vd->vdev_detached || vd->vdev_top->vdev_removing); +} + +static void +vdev_initialize_zap_update_sync(void *arg, dmu_tx_t *tx) +{ + /* + * We pass in the guid instead of the vdev_t since the vdev may + * have been freed prior to the sync task being processed. This + * happens when a vdev is detached as we call spa_config_vdev_exit(), + * stop the intializing thread, schedule the sync task, and free + * the vdev. Later when the scheduled sync task is invoked, it would + * find that the vdev has been freed. + */ + uint64_t guid = *(uint64_t *)arg; + uint64_t txg = dmu_tx_get_txg(tx); + kmem_free(arg, sizeof (uint64_t)); + + vdev_t *vd = spa_lookup_by_guid(tx->tx_pool->dp_spa, guid, B_FALSE); + if (vd == NULL || vd->vdev_top->vdev_removing || !vdev_is_concrete(vd)) + return; + + uint64_t last_offset = vd->vdev_initialize_offset[txg & TXG_MASK]; + vd->vdev_initialize_offset[txg & TXG_MASK] = 0; + + VERIFY(vd->vdev_leaf_zap != 0); + + objset_t *mos = vd->vdev_spa->spa_meta_objset; + + if (last_offset > 0) { + vd->vdev_initialize_last_offset = last_offset; + VERIFY0(zap_update(mos, vd->vdev_leaf_zap, + VDEV_LEAF_ZAP_INITIALIZE_LAST_OFFSET, + sizeof (last_offset), 1, &last_offset, tx)); + } + if (vd->vdev_initialize_action_time > 0) { + uint64_t val = (uint64_t)vd->vdev_initialize_action_time; + VERIFY0(zap_update(mos, vd->vdev_leaf_zap, + VDEV_LEAF_ZAP_INITIALIZE_ACTION_TIME, sizeof (val), + 1, &val, tx)); + } + + uint64_t initialize_state = vd->vdev_initialize_state; + VERIFY0(zap_update(mos, vd->vdev_leaf_zap, + VDEV_LEAF_ZAP_INITIALIZE_STATE, sizeof (initialize_state), 1, + &initialize_state, tx)); +} + +static void +vdev_initialize_change_state(vdev_t *vd, vdev_initializing_state_t new_state) +{ + ASSERT(MUTEX_HELD(&vd->vdev_initialize_lock)); + spa_t *spa = vd->vdev_spa; + + if (new_state == vd->vdev_initialize_state) + return; + + /* + * Copy the vd's guid, this will be freed by the sync task. + */ + uint64_t *guid = kmem_zalloc(sizeof (uint64_t), KM_SLEEP); + *guid = vd->vdev_guid; + + /* + * If we're suspending, then preserving the original start time. + */ + if (vd->vdev_initialize_state != VDEV_INITIALIZE_SUSPENDED) { + vd->vdev_initialize_action_time = gethrestime_sec(); + } + vd->vdev_initialize_state = new_state; + + dmu_tx_t *tx = dmu_tx_create_dd(spa_get_dsl(spa)->dp_mos_dir); + VERIFY0(dmu_tx_assign(tx, TXG_WAIT)); + dsl_sync_task_nowait(spa_get_dsl(spa), vdev_initialize_zap_update_sync, + guid, 2, ZFS_SPACE_CHECK_RESERVED, tx); + + switch (new_state) { + case VDEV_INITIALIZE_ACTIVE: + spa_history_log_internal(spa, "initialize", tx, + "vdev=%s activated", vd->vdev_path); + break; + case VDEV_INITIALIZE_SUSPENDED: + spa_history_log_internal(spa, "initialize", tx, + "vdev=%s suspended", vd->vdev_path); + break; + case VDEV_INITIALIZE_CANCELED: + spa_history_log_internal(spa, "initialize", tx, + "vdev=%s canceled", vd->vdev_path); + break; + case VDEV_INITIALIZE_COMPLETE: + spa_history_log_internal(spa, "initialize", tx, + "vdev=%s complete", vd->vdev_path); + break; + default: + panic("invalid state %llu", (unsigned long long)new_state); + } + + dmu_tx_commit(tx); +} + +static void +vdev_initialize_cb(zio_t *zio) +{ + vdev_t *vd = zio->io_vd; + mutex_enter(&vd->vdev_initialize_io_lock); + if (zio->io_error == ENXIO && !vdev_writeable(vd)) { + /* + * The I/O failed because the vdev was unavailable; roll the + * last offset back. (This works because spa_sync waits on + * spa_txg_zio before it runs sync tasks.) + */ + uint64_t *off = + &vd->vdev_initialize_offset[zio->io_txg & TXG_MASK]; + *off = MIN(*off, zio->io_offset); + } else { + /* + * Since initializing is best-effort, we ignore I/O errors and + * rely on vdev_probe to determine if the errors are more + * critical. + */ + if (zio->io_error != 0) + vd->vdev_stat.vs_initialize_errors++; + + vd->vdev_initialize_bytes_done += zio->io_orig_size; + } + ASSERT3U(vd->vdev_initialize_inflight, >, 0); + vd->vdev_initialize_inflight--; + cv_broadcast(&vd->vdev_initialize_io_cv); + mutex_exit(&vd->vdev_initialize_io_lock); + + spa_config_exit(vd->vdev_spa, SCL_STATE_ALL, vd); +} + +/* Takes care of physical writing and limiting # of concurrent ZIOs. */ +static int +vdev_initialize_write(vdev_t *vd, uint64_t start, uint64_t size, abd_t *data) +{ + spa_t *spa = vd->vdev_spa; + + /* Limit inflight initializing I/Os */ + mutex_enter(&vd->vdev_initialize_io_lock); + while (vd->vdev_initialize_inflight >= zfs_initialize_limit) { + cv_wait(&vd->vdev_initialize_io_cv, + &vd->vdev_initialize_io_lock); + } + vd->vdev_initialize_inflight++; + mutex_exit(&vd->vdev_initialize_io_lock); + + dmu_tx_t *tx = dmu_tx_create_dd(spa_get_dsl(spa)->dp_mos_dir); + VERIFY0(dmu_tx_assign(tx, TXG_WAIT)); + uint64_t txg = dmu_tx_get_txg(tx); + + spa_config_enter(spa, SCL_STATE_ALL, vd, RW_READER); + mutex_enter(&vd->vdev_initialize_lock); + + if (vd->vdev_initialize_offset[txg & TXG_MASK] == 0) { + uint64_t *guid = kmem_zalloc(sizeof (uint64_t), KM_SLEEP); + *guid = vd->vdev_guid; + + /* This is the first write of this txg. */ + dsl_sync_task_nowait(spa_get_dsl(spa), + vdev_initialize_zap_update_sync, guid, 2, + ZFS_SPACE_CHECK_RESERVED, tx); + } + + /* + * We know the vdev struct will still be around since all + * consumers of vdev_free must stop the initialization first. + */ + if (vdev_initialize_should_stop(vd)) { + mutex_enter(&vd->vdev_initialize_io_lock); + ASSERT3U(vd->vdev_initialize_inflight, >, 0); + vd->vdev_initialize_inflight--; + mutex_exit(&vd->vdev_initialize_io_lock); + spa_config_exit(vd->vdev_spa, SCL_STATE_ALL, vd); + mutex_exit(&vd->vdev_initialize_lock); + dmu_tx_commit(tx); + return (SET_ERROR(EINTR)); + } + mutex_exit(&vd->vdev_initialize_lock); + + vd->vdev_initialize_offset[txg & TXG_MASK] = start + size; + zio_nowait(zio_write_phys(spa->spa_txg_zio[txg & TXG_MASK], vd, start, + size, data, ZIO_CHECKSUM_OFF, vdev_initialize_cb, NULL, + ZIO_PRIORITY_INITIALIZING, ZIO_FLAG_CANFAIL, B_FALSE)); + /* vdev_initialize_cb releases SCL_STATE_ALL */ + + dmu_tx_commit(tx); + + return (0); +} + +/* + * Translate a logical range to the physical range for the specified vdev_t. + * This function is initially called with a leaf vdev and will walk each + * parent vdev until it reaches a top-level vdev. Once the top-level is + * reached the physical range is initialized and the recursive function + * begins to unwind. As it unwinds it calls the parent's vdev specific + * translation function to do the real conversion. + */ +void +vdev_xlate(vdev_t *vd, const range_seg_t *logical_rs, range_seg_t *physical_rs) +{ + /* + * Walk up the vdev tree + */ + if (vd != vd->vdev_top) { + vdev_xlate(vd->vdev_parent, logical_rs, physical_rs); + } else { + /* + * We've reached the top-level vdev, initialize the + * physical range to the logical range and start to + * unwind. + */ + physical_rs->rs_start = logical_rs->rs_start; + physical_rs->rs_end = logical_rs->rs_end; + return; + } + + vdev_t *pvd = vd->vdev_parent; + ASSERT3P(pvd, !=, NULL); + ASSERT3P(pvd->vdev_ops->vdev_op_xlate, !=, NULL); + + /* + * As this recursive function unwinds, translate the logical + * range into its physical components by calling the + * vdev specific translate function. + */ + range_seg_t intermediate = { 0 }; + pvd->vdev_ops->vdev_op_xlate(vd, physical_rs, &intermediate); + + physical_rs->rs_start = intermediate.rs_start; + physical_rs->rs_end = intermediate.rs_end; +} + +/* + * Callback to fill each ABD chunk with zfs_initialize_value. len must be + * divisible by sizeof (uint64_t), and buf must be 8-byte aligned. The ABD + * allocation will guarantee these for us. + */ +/* ARGSUSED */ +static int +vdev_initialize_block_fill(void *buf, size_t len, void *unused) +{ + ASSERT0(len % sizeof (uint64_t)); + for (uint64_t i = 0; i < len; i += sizeof (uint64_t)) { + *(uint64_t *)((char *)(buf) + i) = zfs_initialize_value; + } + return (0); +} + +static abd_t * +vdev_initialize_block_alloc() +{ + /* Allocate ABD for filler data */ + abd_t *data = abd_alloc_for_io(zfs_initialize_chunk_size, B_FALSE); + + ASSERT0(zfs_initialize_chunk_size % sizeof (uint64_t)); + (void) abd_iterate_func(data, 0, zfs_initialize_chunk_size, + vdev_initialize_block_fill, NULL); + + return (data); +} + +static void +vdev_initialize_block_free(abd_t *data) +{ + abd_free(data); +} + +static int +vdev_initialize_ranges(vdev_t *vd, abd_t *data) +{ + avl_tree_t *rt = &vd->vdev_initialize_tree->rt_root; + + for (range_seg_t *rs = avl_first(rt); rs != NULL; + rs = AVL_NEXT(rt, rs)) { + uint64_t size = rs->rs_end - rs->rs_start; + + /* Split range into legally-sized physical chunks */ + uint64_t writes_required = + ((size - 1) / zfs_initialize_chunk_size) + 1; + + for (uint64_t w = 0; w < writes_required; w++) { + int error; + + error = vdev_initialize_write(vd, + VDEV_LABEL_START_SIZE + rs->rs_start + + (w * zfs_initialize_chunk_size), + MIN(size - (w * zfs_initialize_chunk_size), + zfs_initialize_chunk_size), data); + if (error != 0) + return (error); + } + } + return (0); +} + +static void +vdev_initialize_ms_load(metaslab_t *msp) +{ + ASSERT(MUTEX_HELD(&msp->ms_lock)); + + metaslab_load_wait(msp); + if (!msp->ms_loaded) + VERIFY0(metaslab_load(msp)); +} + +static void +vdev_initialize_mg_wait(metaslab_group_t *mg) +{ + ASSERT(MUTEX_HELD(&mg->mg_ms_initialize_lock)); + while (mg->mg_initialize_updating) { + cv_wait(&mg->mg_ms_initialize_cv, &mg->mg_ms_initialize_lock); + } +} + +static void +vdev_initialize_mg_mark(metaslab_group_t *mg) +{ + ASSERT(MUTEX_HELD(&mg->mg_ms_initialize_lock)); + ASSERT(mg->mg_initialize_updating); + + while (mg->mg_ms_initializing >= max_initialize_ms) { + cv_wait(&mg->mg_ms_initialize_cv, &mg->mg_ms_initialize_lock); + } + mg->mg_ms_initializing++; + ASSERT3U(mg->mg_ms_initializing, <=, max_initialize_ms); +} + +/* + * Mark the metaslab as being initialized to prevent any allocations + * on this metaslab. We must also track how many metaslabs are currently + * being initialized within a metaslab group and limit them to prevent + * allocation failures from occurring because all metaslabs are being + * initialized. + */ +static void +vdev_initialize_ms_mark(metaslab_t *msp) +{ + ASSERT(!MUTEX_HELD(&msp->ms_lock)); + metaslab_group_t *mg = msp->ms_group; + + mutex_enter(&mg->mg_ms_initialize_lock); + + /* + * To keep an accurate count of how many threads are initializing + * a specific metaslab group, we only allow one thread to mark + * the metaslab group at a time. This ensures that the value of + * ms_initializing will be accurate when we decide to mark a metaslab + * group as being initialized. To do this we force all other threads + * to wait till the metaslab's mg_initialize_updating flag is no + * longer set. + */ + vdev_initialize_mg_wait(mg); + mg->mg_initialize_updating = B_TRUE; + if (msp->ms_initializing == 0) { + vdev_initialize_mg_mark(mg); + } + mutex_enter(&msp->ms_lock); + msp->ms_initializing++; + mutex_exit(&msp->ms_lock); + + mg->mg_initialize_updating = B_FALSE; + cv_broadcast(&mg->mg_ms_initialize_cv); + mutex_exit(&mg->mg_ms_initialize_lock); +} + +static void +vdev_initialize_ms_unmark(metaslab_t *msp) +{ + ASSERT(!MUTEX_HELD(&msp->ms_lock)); + metaslab_group_t *mg = msp->ms_group; + mutex_enter(&mg->mg_ms_initialize_lock); + mutex_enter(&msp->ms_lock); + if (--msp->ms_initializing == 0) { + mg->mg_ms_initializing--; + cv_broadcast(&mg->mg_ms_initialize_cv); + } + mutex_exit(&msp->ms_lock); + mutex_exit(&mg->mg_ms_initialize_lock); +} + +static void +vdev_initialize_calculate_progress(vdev_t *vd) +{ + ASSERT(spa_config_held(vd->vdev_spa, SCL_CONFIG, RW_READER) || + spa_config_held(vd->vdev_spa, SCL_CONFIG, RW_WRITER)); + ASSERT(vd->vdev_leaf_zap != 0); + + vd->vdev_initialize_bytes_est = 0; + vd->vdev_initialize_bytes_done = 0; + + for (uint64_t i = 0; i < vd->vdev_top->vdev_ms_count; i++) { + metaslab_t *msp = vd->vdev_top->vdev_ms[i]; + mutex_enter(&msp->ms_lock); + + uint64_t ms_free = msp->ms_size - + space_map_allocated(msp->ms_sm); + + if (vd->vdev_top->vdev_ops == &vdev_raidz_ops) + ms_free /= vd->vdev_top->vdev_children; + + /* + * Convert the metaslab range to a physical range + * on our vdev. We use this to determine if we are + * in the middle of this metaslab range. + */ + range_seg_t logical_rs, physical_rs; + logical_rs.rs_start = msp->ms_start; + logical_rs.rs_end = msp->ms_start + msp->ms_size; + vdev_xlate(vd, &logical_rs, &physical_rs); + + if (vd->vdev_initialize_last_offset <= physical_rs.rs_start) { + vd->vdev_initialize_bytes_est += ms_free; + mutex_exit(&msp->ms_lock); + continue; + } else if (vd->vdev_initialize_last_offset > + physical_rs.rs_end) { + vd->vdev_initialize_bytes_done += ms_free; + vd->vdev_initialize_bytes_est += ms_free; + mutex_exit(&msp->ms_lock); + continue; + } + + /* + * If we get here, we're in the middle of initializing this + * metaslab. Load it and walk the free tree for more accurate + * progress estimation. + */ + vdev_initialize_ms_load(msp); + + for (range_seg_t *rs = avl_first(&msp->ms_allocatable->rt_root); rs; + rs = AVL_NEXT(&msp->ms_allocatable->rt_root, rs)) { + logical_rs.rs_start = rs->rs_start; + logical_rs.rs_end = rs->rs_end; + vdev_xlate(vd, &logical_rs, &physical_rs); + + uint64_t size = physical_rs.rs_end - + physical_rs.rs_start; + vd->vdev_initialize_bytes_est += size; + if (vd->vdev_initialize_last_offset > + physical_rs.rs_end) { + vd->vdev_initialize_bytes_done += size; + } else if (vd->vdev_initialize_last_offset > + physical_rs.rs_start && + vd->vdev_initialize_last_offset < + physical_rs.rs_end) { + vd->vdev_initialize_bytes_done += + vd->vdev_initialize_last_offset - + physical_rs.rs_start; + } + } + mutex_exit(&msp->ms_lock); + } +} + +static void +vdev_initialize_load(vdev_t *vd) +{ + ASSERT(spa_config_held(vd->vdev_spa, SCL_CONFIG, RW_READER) || + spa_config_held(vd->vdev_spa, SCL_CONFIG, RW_WRITER)); + ASSERT(vd->vdev_leaf_zap != 0); + + if (vd->vdev_initialize_state == VDEV_INITIALIZE_ACTIVE || + vd->vdev_initialize_state == VDEV_INITIALIZE_SUSPENDED) { + int err = zap_lookup(vd->vdev_spa->spa_meta_objset, + vd->vdev_leaf_zap, VDEV_LEAF_ZAP_INITIALIZE_LAST_OFFSET, + sizeof (vd->vdev_initialize_last_offset), 1, + &vd->vdev_initialize_last_offset); + ASSERT(err == 0 || err == ENOENT); + } + + vdev_initialize_calculate_progress(vd); +} + + +/* + * Convert the logical range into a physcial range and add it to our + * avl tree. + */ +void +vdev_initialize_range_add(void *arg, uint64_t start, uint64_t size) +{ + vdev_t *vd = arg; + range_seg_t logical_rs, physical_rs; + logical_rs.rs_start = start; + logical_rs.rs_end = start + size; + + ASSERT(vd->vdev_ops->vdev_op_leaf); + vdev_xlate(vd, &logical_rs, &physical_rs); + + IMPLY(vd->vdev_top == vd, + logical_rs.rs_start == physical_rs.rs_start); + IMPLY(vd->vdev_top == vd, + logical_rs.rs_end == physical_rs.rs_end); + + /* Only add segments that we have not visited yet */ + if (physical_rs.rs_end <= vd->vdev_initialize_last_offset) + return; + + /* Pick up where we left off mid-range. */ + if (vd->vdev_initialize_last_offset > physical_rs.rs_start) { + zfs_dbgmsg("range write: vd %s changed (%llu, %llu) to " + "(%llu, %llu)", vd->vdev_path, + (u_longlong_t)physical_rs.rs_start, + (u_longlong_t)physical_rs.rs_end, + (u_longlong_t)vd->vdev_initialize_last_offset, + (u_longlong_t)physical_rs.rs_end); + ASSERT3U(physical_rs.rs_end, >, + vd->vdev_initialize_last_offset); + physical_rs.rs_start = vd->vdev_initialize_last_offset; + } + ASSERT3U(physical_rs.rs_end, >=, physical_rs.rs_start); + + /* + * With raidz, it's possible that the logical range does not live on + * this leaf vdev. We only add the physical range to this vdev's if it + * has a length greater than 0. + */ + if (physical_rs.rs_end > physical_rs.rs_start) { + range_tree_add(vd->vdev_initialize_tree, physical_rs.rs_start, + physical_rs.rs_end - physical_rs.rs_start); + } else { + ASSERT3U(physical_rs.rs_end, ==, physical_rs.rs_start); + } +} + +static void +vdev_initialize_thread(void *arg) +{ + vdev_t *vd = arg; + spa_t *spa = vd->vdev_spa; + int error = 0; + uint64_t ms_count = 0; + + ASSERT(vdev_is_concrete(vd)); + spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); + + vd->vdev_initialize_last_offset = 0; + vdev_initialize_load(vd); + + abd_t *deadbeef = vdev_initialize_block_alloc(); + + vd->vdev_initialize_tree = range_tree_create(NULL, NULL); + + for (uint64_t i = 0; !vd->vdev_detached && + i < vd->vdev_top->vdev_ms_count; i++) { + metaslab_t *msp = vd->vdev_top->vdev_ms[i]; + + /* + * If we've expanded the top-level vdev or it's our + * first pass, calculate our progress. + */ + if (vd->vdev_top->vdev_ms_count != ms_count) { + vdev_initialize_calculate_progress(vd); + ms_count = vd->vdev_top->vdev_ms_count; + } + + vdev_initialize_ms_mark(msp); + mutex_enter(&msp->ms_lock); + vdev_initialize_ms_load(msp); + + range_tree_walk(msp->ms_allocatable, vdev_initialize_range_add, + vd); + mutex_exit(&msp->ms_lock); + + spa_config_exit(spa, SCL_CONFIG, FTAG); + error = vdev_initialize_ranges(vd, deadbeef); + vdev_initialize_ms_unmark(msp); + spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); + + range_tree_vacate(vd->vdev_initialize_tree, NULL, NULL); + if (error != 0) + break; + } + + spa_config_exit(spa, SCL_CONFIG, FTAG); + mutex_enter(&vd->vdev_initialize_io_lock); + while (vd->vdev_initialize_inflight > 0) { + cv_wait(&vd->vdev_initialize_io_cv, + &vd->vdev_initialize_io_lock); + } + mutex_exit(&vd->vdev_initialize_io_lock); + + range_tree_destroy(vd->vdev_initialize_tree); + vdev_initialize_block_free(deadbeef); + vd->vdev_initialize_tree = NULL; + + mutex_enter(&vd->vdev_initialize_lock); + if (!vd->vdev_initialize_exit_wanted && vdev_writeable(vd)) { + vdev_initialize_change_state(vd, VDEV_INITIALIZE_COMPLETE); + } + ASSERT(vd->vdev_initialize_thread != NULL || + vd->vdev_initialize_inflight == 0); + + /* + * Drop the vdev_initialize_lock while we sync out the + * txg since it's possible that a device might be trying to + * come online and must check to see if it needs to restart an + * initialization. That thread will be holding the spa_config_lock + * which would prevent the txg_wait_synced from completing. + */ + mutex_exit(&vd->vdev_initialize_lock); + txg_wait_synced(spa_get_dsl(spa), 0); + mutex_enter(&vd->vdev_initialize_lock); + + vd->vdev_initialize_thread = NULL; + cv_broadcast(&vd->vdev_initialize_cv); + mutex_exit(&vd->vdev_initialize_lock); +} + +/* + * Initiates a device. Caller must hold vdev_initialize_lock. + * Device must be a leaf and not already be initializing. + */ +void +vdev_initialize(vdev_t *vd) +{ + ASSERT(MUTEX_HELD(&vd->vdev_initialize_lock)); + ASSERT(vd->vdev_ops->vdev_op_leaf); + ASSERT(vdev_is_concrete(vd)); + ASSERT3P(vd->vdev_initialize_thread, ==, NULL); + ASSERT(!vd->vdev_detached); + ASSERT(!vd->vdev_initialize_exit_wanted); + ASSERT(!vd->vdev_top->vdev_removing); + + vdev_initialize_change_state(vd, VDEV_INITIALIZE_ACTIVE); + vd->vdev_initialize_thread = thread_create(NULL, 0, + vdev_initialize_thread, vd, 0, &p0, TS_RUN, maxclsyspri); +} + +/* + * Stop initializng a device, with the resultant initialing state being + * tgt_state. Blocks until the initializing thread has exited. + * Caller must hold vdev_initialize_lock and must not be writing to the spa + * config, as the initializing thread may try to enter the config as a reader + * before exiting. + */ +void +vdev_initialize_stop(vdev_t *vd, vdev_initializing_state_t tgt_state) +{ + spa_t *spa = vd->vdev_spa; + ASSERT(!spa_config_held(spa, SCL_CONFIG | SCL_STATE, RW_WRITER)); + + ASSERT(MUTEX_HELD(&vd->vdev_initialize_lock)); + ASSERT(vd->vdev_ops->vdev_op_leaf); + ASSERT(vdev_is_concrete(vd)); + + /* + * Allow cancel requests to proceed even if the initialize thread + * has stopped. + */ + if (vd->vdev_initialize_thread == NULL && + tgt_state != VDEV_INITIALIZE_CANCELED) { + return; + } + + vdev_initialize_change_state(vd, tgt_state); + vd->vdev_initialize_exit_wanted = B_TRUE; + while (vd->vdev_initialize_thread != NULL) + cv_wait(&vd->vdev_initialize_cv, &vd->vdev_initialize_lock); + + ASSERT3P(vd->vdev_initialize_thread, ==, NULL); + vd->vdev_initialize_exit_wanted = B_FALSE; +} + +static void +vdev_initialize_stop_all_impl(vdev_t *vd, vdev_initializing_state_t tgt_state) +{ + if (vd->vdev_ops->vdev_op_leaf && vdev_is_concrete(vd)) { + mutex_enter(&vd->vdev_initialize_lock); + vdev_initialize_stop(vd, tgt_state); + mutex_exit(&vd->vdev_initialize_lock); + return; + } + + for (uint64_t i = 0; i < vd->vdev_children; i++) { + vdev_initialize_stop_all_impl(vd->vdev_child[i], tgt_state); + } +} + +/* + * Convenience function to stop initializing of a vdev tree and set all + * initialize thread pointers to NULL. + */ +void +vdev_initialize_stop_all(vdev_t *vd, vdev_initializing_state_t tgt_state) +{ + vdev_initialize_stop_all_impl(vd, tgt_state); + + if (vd->vdev_spa->spa_sync_on) { + /* Make sure that our state has been synced to disk */ + txg_wait_synced(spa_get_dsl(vd->vdev_spa), 0); + } +} + +void +vdev_initialize_restart(vdev_t *vd) +{ + ASSERT(MUTEX_HELD(&spa_namespace_lock)); + ASSERT(!spa_config_held(vd->vdev_spa, SCL_ALL, RW_WRITER)); + + if (vd->vdev_leaf_zap != 0) { + mutex_enter(&vd->vdev_initialize_lock); + uint64_t initialize_state = VDEV_INITIALIZE_NONE; + int err = zap_lookup(vd->vdev_spa->spa_meta_objset, + vd->vdev_leaf_zap, VDEV_LEAF_ZAP_INITIALIZE_STATE, + sizeof (initialize_state), 1, &initialize_state); + ASSERT(err == 0 || err == ENOENT); + vd->vdev_initialize_state = initialize_state; + + uint64_t timestamp = 0; + err = zap_lookup(vd->vdev_spa->spa_meta_objset, + vd->vdev_leaf_zap, VDEV_LEAF_ZAP_INITIALIZE_ACTION_TIME, + sizeof (timestamp), 1, ×tamp); + ASSERT(err == 0 || err == ENOENT); + vd->vdev_initialize_action_time = (time_t)timestamp; + + if (vd->vdev_initialize_state == VDEV_INITIALIZE_SUSPENDED || + vd->vdev_offline) { + /* load progress for reporting, but don't resume */ + vdev_initialize_load(vd); + } else if (vd->vdev_initialize_state == + VDEV_INITIALIZE_ACTIVE && vdev_writeable(vd)) { + vdev_initialize(vd); + } + + mutex_exit(&vd->vdev_initialize_lock); + } + + for (uint64_t i = 0; i < vd->vdev_children; i++) { + vdev_initialize_restart(vd->vdev_child[i]); + } +} From owner-svn-src-all@freebsd.org Tue Jul 31 19:30:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC0B41062D28; Tue, 31 Jul 2018 19:30:09 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82BB479691; Tue, 31 Jul 2018 19:30:09 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 64FF917EC7; Tue, 31 Jul 2018 19:30:09 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VJU9WZ086910; Tue, 31 Jul 2018 19:30:09 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VJU9H9086909; Tue, 31 Jul 2018 19:30:09 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807311930.w6VJU9H9086909@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 19:30:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337002 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337002 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:30:10 -0000 Author: mav Date: Tue Jul 31 19:30:08 2018 New Revision: 337002 URL: https://svnweb.freebsd.org/changeset/base/337002 Log: 9434 Speculative prefetch is blocked by device removal code illumos/illumos-gate@47b8d4b884f2dab4e4c6952b40fd740a8fc4ab32 Reviewed by: Matthew Ahrens Reviewed by: Prashanth Sreenivasa Approved by: Matt Ahrens Author: Alexander Motin Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 19:23:27 2018 (r337001) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Tue Jul 31 19:30:08 2018 (r337002) @@ -364,6 +364,7 @@ spa_remove_init(spa_t *spa) spa->spa_removing_phys.sr_state = DSS_NONE; spa->spa_removing_phys.sr_removing_vdev = -1; spa->spa_removing_phys.sr_prev_indirect_vdev = -1; + spa->spa_indirect_vdevs_loaded = B_TRUE; return (0); } else if (error != 0) { return (error); From owner-svn-src-all@freebsd.org Tue Jul 31 19:51:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B16D810633A4; Tue, 31 Jul 2018 19:51:33 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.blih.net", Issuer "mail.blih.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BDC07A212; Tue, 31 Jul 2018 19:51:32 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) by mail.blih.net (OpenSMTPD) with ESMTP id df6b5b98; Tue, 31 Jul 2018 21:51:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=bidouilliste.com; h=date :from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=mail; bh=nzcYLA3toM+3 VeZsSIgue4R6my8=; b=aJBXuH1YRplbcvZ8CVMKTZLTte+g7KjamPJAk+zukE9S ZNrsQLtMfBAZpWEIXA4zSePH3c4p5VJoyeYZL1AhE2QlTA+D5Vy8PyJON0bJCoMi 55kYgQ5F79KvzQn3UGTuBHK0rLeZZDQ1Vc+5tuGYCt45NHkRS25dlpBxWLAAEro= DomainKey-Signature: a=rsa-sha1; c=nofws; d=bidouilliste.com; h=date :from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=mail; b=ae6j7d BF+FOmy7hQ5S1YakvMw5XCFK2VhPmCBoJkNjmq4vZ2gda2Jzl5/lL702k2OCFHOQ Js7spAuZ3q8xmLqRQjlh99Waqmz4H2uKtQayZlYFxffhMVEjvayx21JKdNvalvFJ gvkcQv6qyinEMtK9aMaa/v6s8vzk1BN2jSmyA= Received: from skull.home.blih.net (ip-9.net-89-3-105.rev.numericable.fr [89.3.105.9]) by mail.blih.net (OpenSMTPD) with ESMTPSA id bcd7e18a TLS version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO; Tue, 31 Jul 2018 21:51:31 +0200 (CEST) Date: Tue, 31 Jul 2018 21:51:31 +0200 From: Emmanuel Vadot To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336998 - head/release/arm Message-Id: <20180731215131.ae657005f34a4bbb80ed39ce@bidouilliste.com> In-Reply-To: <201807311912.w6VJCVuL081184@repo.freebsd.org> References: <201807311912.w6VJCVuL081184@repo.freebsd.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd12.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 19:51:33 -0000 On Tue, 31 Jul 2018 19:12:31 +0000 (UTC) Emmanuel Vadot wrote: > Author: manu > Date: Tue Jul 31 19:12:31 2018 > New Revision: 336998 > URL: https://svnweb.freebsd.org/changeset/base/336998 > > Log: > release: arm: Always boot armv6/v7 image with EFI > > boot.scr is a u-boot script that loads and execute ubldr.bin > If not present u-boot will automatically boot loader.efi which > is already installed. > > This means that all armv6/armv7 images are now booted via EFI > > Tested-On: RPI-B > Tested-On: RPI2 > Tested-On: OrangePi One > Tested-On: All lot of other boards > MFC after: Never > Relnotes: yes Differencial Revision: https://reviews.freebsd.org/D16482 > Modified: > head/release/arm/BANANAPI.conf > head/release/arm/BEAGLEBONE.conf > head/release/arm/CUBIEBOARD.conf > head/release/arm/CUBIEBOARD2.conf > head/release/arm/CUBOX-HUMMINGBOARD.conf > head/release/arm/PANDABOARD.conf > head/release/arm/RPI-B.conf > head/release/arm/RPI2.conf > head/release/arm/WANDBOARD.conf > > Modified: head/release/arm/BANANAPI.conf > ============================================================================== > --- head/release/arm/BANANAPI.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/BANANAPI.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -19,15 +19,8 @@ export BOARDNAME="BANANAPI" > arm_install_uboot() { > UBOOT_DIR="/usr/local/share/u-boot/u-boot-bananapi" > UBOOT_FILES="u-boot-sunxi-with-spl.bin" > - FATMOUNT="${DESTDIR%${KERNEL}}/fat" > chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ > of=/dev/${mddev} bs=1k seek=8 conv=sync > - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" > - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > - sync > - umount_loop ${CHROOTDIR}/${FATMOUNT} > - chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > return 0 > } > > Modified: head/release/arm/BEAGLEBONE.conf > ============================================================================== > --- head/release/arm/BEAGLEBONE.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/BEAGLEBONE.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -23,7 +23,6 @@ arm_install_uboot() { > chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} > chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/MLO ${FATMOUNT}/MLO > chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/u-boot.img ${FATMOUNT}/u-boot.img > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > sync > umount_loop ${CHROOTDIR}/${FATMOUNT} > chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > Modified: head/release/arm/CUBIEBOARD.conf > ============================================================================== > --- head/release/arm/CUBIEBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/CUBIEBOARD.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -19,15 +19,8 @@ export BOARDNAME="CUBIEBOARD" > arm_install_uboot() { > UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard" > UBOOT_FILES="u-boot-sunxi-with-spl.bin" > - FATMOUNT="${DESTDIR%${KERNEL}}/fat" > chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ > of=/dev/${mddev} bs=1k seek=8 conv=sync > - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" > - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > - sync > - umount_loop ${CHROOTDIR}/${FATMOUNT} > - chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > return 0 > } > > Modified: head/release/arm/CUBIEBOARD2.conf > ============================================================================== > --- head/release/arm/CUBIEBOARD2.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/CUBIEBOARD2.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -19,15 +19,8 @@ export BOARDNAME="CUBIEBOARD2" > arm_install_uboot() { > UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard2" > UBOOT_FILES="u-boot-sunxi-with-spl.bin" > - FATMOUNT="${DESTDIR%${KERNEL}}/fat" > chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ > of=/dev/${mddev} bs=1k seek=8 conv=sync > - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" > - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > - sync > - umount_loop ${CHROOTDIR}/${FATMOUNT} > - chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > return 0 > } > > Modified: head/release/arm/CUBOX-HUMMINGBOARD.conf > ============================================================================== > --- head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/CUBOX-HUMMINGBOARD.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -19,15 +19,8 @@ export BOARDNAME="CUBOX-HUMMINGBOARD" > arm_install_uboot() { > UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubox-hummingboard" > UBOOT_FILES="u-boot.imx" > - FATMOUNT="${DESTDIR%${KERNEL}}/fat" > chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ > of=/dev/${mddev} bs=512 seek=2 conv=sync > - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" > - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > - sync > - umount_loop ${CHROOTDIR}/${FATMOUNT} > - chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > return 0 > } > > Modified: head/release/arm/PANDABOARD.conf > ============================================================================== > --- head/release/arm/PANDABOARD.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/PANDABOARD.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -23,7 +23,6 @@ arm_install_uboot() { > chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} > chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/MLO ${FATMOUNT}/MLO > chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/u-boot.img ${FATMOUNT}/u-boot.img > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > sync > umount_loop ${CHROOTDIR}/${FATMOUNT} > chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > Modified: head/release/arm/RPI-B.conf > ============================================================================== > --- head/release/arm/RPI-B.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/RPI-B.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -43,7 +43,6 @@ arm_install_uboot() { > chroot ${CHROOTDIR} cp -p ${OL_DIR}/${_OL} \ > ${FATMOUNT}/overlays/${_OL} > done > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > sync > umount_loop ${CHROOTDIR}/${FATMOUNT} > chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > Modified: head/release/arm/RPI2.conf > ============================================================================== > --- head/release/arm/RPI2.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/RPI2.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -42,7 +42,6 @@ arm_install_uboot() { > chroot ${CHROOTDIR} cp -p ${OL_DIR}/${_OL} \ > ${FATMOUNT}/overlays/${_OL} > done > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > sync > umount_loop ${CHROOTDIR}/${FATMOUNT} > chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > Modified: head/release/arm/WANDBOARD.conf > ============================================================================== > --- head/release/arm/WANDBOARD.conf Tue Jul 31 19:10:50 2018 (r336997) > +++ head/release/arm/WANDBOARD.conf Tue Jul 31 19:12:31 2018 (r336998) > @@ -19,15 +19,8 @@ export BOARDNAME="WANDBOARD" > arm_install_uboot() { > UBOOT_DIR="/usr/local/share/u-boot/u-boot-wandboard" > UBOOT_FILES="u-boot.imx" > - FATMOUNT="${DESTDIR%${KERNEL}}/fat" > chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ > of=/dev/${mddev} bs=512 seek=2 conv=sync > - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" > - chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} > - chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr > - sync > - umount_loop ${CHROOTDIR}/${FATMOUNT} > - chroot ${CHROOTDIR} rmdir ${FATMOUNT} > > return 0 > } -- Emmanuel Vadot From owner-svn-src-all@freebsd.org Tue Jul 31 20:50:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23C211064852; Tue, 31 Jul 2018 20:50:51 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CED277C387; Tue, 31 Jul 2018 20:50:50 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 972E518C43; Tue, 31 Jul 2018 20:50:50 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VKooV9027844; Tue, 31 Jul 2018 20:50:50 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VKoopE027843; Tue, 31 Jul 2018 20:50:50 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201807312050.w6VKoopE027843@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Tue, 31 Jul 2018 20:50:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337004 - head/sys/dev/ofw X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/sys/dev/ofw X-SVN-Commit-Revision: 337004 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 20:50:51 -0000 Author: manu Date: Tue Jul 31 20:50:50 2018 New Revision: 337004 URL: https://svnweb.freebsd.org/changeset/base/337004 Log: ofw_cpu: Add support for getting cpu clock via clock property Nominal Mhz is either expressed via the clock-frequency property or can be get via the clock property that holds the cpu clock. Add support for the later. Reviewed by: nwhitehorn Differential Revision: https://reviews.freebsd.org/D16346 Modified: head/sys/dev/ofw/ofw_cpu.c Modified: head/sys/dev/ofw/ofw_cpu.c ============================================================================== --- head/sys/dev/ofw/ofw_cpu.c Tue Jul 31 19:35:34 2018 (r337003) +++ head/sys/dev/ofw/ofw_cpu.c Tue Jul 31 20:50:50 2018 (r337004) @@ -45,6 +45,10 @@ __FBSDID("$FreeBSD$"); #include #include +#ifdef EXT_RESOURCES +#include +#endif + static int ofw_cpulist_probe(device_t); static int ofw_cpulist_attach(device_t); static const struct ofw_bus_devinfo *ofw_cpulist_get_devinfo(device_t dev, @@ -203,6 +207,10 @@ ofw_cpu_attach(device_t dev) phandle_t node; pcell_t cell; int rv; +#ifdef EXT_RESOURCES + clk_t cpuclk; + uint64_t freq; +#endif sc = device_get_softc(dev); psc = device_get_softc(device_get_parent(dev)); @@ -269,12 +277,28 @@ ofw_cpu_attach(device_t dev) sc->sc_cpu_pcpu = pcpu_find(device_get_unit(dev)); if (OF_getencprop(node, "clock-frequency", &cell, sizeof(cell)) < 0) { - if (bootverbose) - device_printf(dev, - "missing 'clock-frequency' property\n"); +#ifdef EXT_RESOURCES + rv = clk_get_by_ofw_index(dev, 0, 0, &cpuclk); + if (rv == 0) { + rv = clk_get_freq(cpuclk, &freq); + if (rv != 0 && bootverbose) + device_printf(dev, + "Cannot get freq of property clocks\n"); + else + sc->sc_nominal_mhz = freq / 1000000; + } else +#endif + { + if (bootverbose) + device_printf(dev, + "missing 'clock-frequency' property\n"); + } } else sc->sc_nominal_mhz = cell / 1000000; /* convert to MHz */ + if (sc->sc_nominal_mhz != 0 && bootverbose) + device_printf(dev, "Nominal frequency %dMhz\n", + sc->sc_nominal_mhz); bus_generic_probe(dev); return (bus_generic_attach(dev)); } From owner-svn-src-all@freebsd.org Tue Jul 31 20:57:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF6191064BCE; Tue, 31 Jul 2018 20:57:32 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8355D7C829; Tue, 31 Jul 2018 20:57:32 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6513B18DF2; Tue, 31 Jul 2018 20:57:32 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VKvWRv033215; Tue, 31 Jul 2018 20:57:32 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VKvW4k033214; Tue, 31 Jul 2018 20:57:32 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807312057.w6VKvW4k033214@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 31 Jul 2018 20:57:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337005 - stable/11/sys/fs/nfsserver X-SVN-Group: stable-11 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: stable/11/sys/fs/nfsserver X-SVN-Commit-Revision: 337005 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 20:57:33 -0000 Author: rmacklem Date: Tue Jul 31 20:57:31 2018 New Revision: 337005 URL: https://svnweb.freebsd.org/changeset/base/337005 Log: MFC: r336215 Ignore the cookie verifier for NFSv4.1 when the cookie is 0. RFC5661 states that the cookie verifier should be 0 when the cookie is 0. However, the wording is somewhat unclear and a recent discussion on the nfsv4@ietf.org mailing list indicated that the NFSv4 server should ignore the cookie verifier's value when the dirctory offset cookie is 0. This patch deletes the check for this that would return NFSERR_BAD_COOKIE when the verifier was not 0. This was found during testing of the ESXi client against the NFSv4.1 server. Modified: stable/11/sys/fs/nfsserver/nfs_nfsdport.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- stable/11/sys/fs/nfsserver/nfs_nfsdport.c Tue Jul 31 20:50:50 2018 (r337004) +++ stable/11/sys/fs/nfsserver/nfs_nfsdport.c Tue Jul 31 20:57:31 2018 (r337005) @@ -1876,6 +1876,7 @@ nfsrvd_readdirplus(struct nfsrv_descript *nd, int isdg } fullsiz = siz; nd->nd_repstat = getret = nfsvno_getattr(vp, &at, nd->nd_cred, p, 1); +#if 0 if (!nd->nd_repstat) { if (off && verf != at.na_filerev) { /* @@ -1884,17 +1885,14 @@ nfsrvd_readdirplus(struct nfsrv_descript *nd, int isdg * removed/added unless that offset cookies returned to * the client are no longer valid. */ -#if 0 if (nd->nd_flag & ND_NFSV4) { nd->nd_repstat = NFSERR_NOTSAME; } else { nd->nd_repstat = NFSERR_BAD_COOKIE; } -#endif - } else if ((nd->nd_flag & ND_NFSV4) && off == 0 && verf != 0) { - nd->nd_repstat = NFSERR_BAD_COOKIE; } } +#endif if (!nd->nd_repstat && vp->v_type != VDIR) nd->nd_repstat = NFSERR_NOTDIR; if (!nd->nd_repstat && cnt == 0) From owner-svn-src-all@freebsd.org Tue Jul 31 21:02:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED3921064E32; Tue, 31 Jul 2018 21:02:45 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95B5F7CCE8; Tue, 31 Jul 2018 21:02:45 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 72C6C18F8E; Tue, 31 Jul 2018 21:02:45 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VL2jUT038023; Tue, 31 Jul 2018 21:02:45 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VL2jDa038022; Tue, 31 Jul 2018 21:02:45 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201807312102.w6VL2jDa038022@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Tue, 31 Jul 2018 21:02:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337006 - stable/10/sys/fs/nfsserver X-SVN-Group: stable-10 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: stable/10/sys/fs/nfsserver X-SVN-Commit-Revision: 337006 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:02:46 -0000 Author: rmacklem Date: Tue Jul 31 21:02:45 2018 New Revision: 337006 URL: https://svnweb.freebsd.org/changeset/base/337006 Log: MFC: r336215 Ignore the cookie verifier for NFSv4.1 when the cookie is 0. RFC5661 states that the cookie verifier should be 0 when the cookie is 0. However, the wording is somewhat unclear and a recent discussion on the nfsv4@ietf.org mailing list indicated that the NFSv4 server should ignore the cookie verifier's value when the dirctory offset cookie is 0. This patch deletes the check for this that would return NFSERR_BAD_COOKIE when the verifier was not 0. This was found during testing of the ESXi client against the NFSv4.1 server. Modified: stable/10/sys/fs/nfsserver/nfs_nfsdport.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/fs/nfsserver/nfs_nfsdport.c ============================================================================== --- stable/10/sys/fs/nfsserver/nfs_nfsdport.c Tue Jul 31 20:57:31 2018 (r337005) +++ stable/10/sys/fs/nfsserver/nfs_nfsdport.c Tue Jul 31 21:02:45 2018 (r337006) @@ -1871,6 +1871,7 @@ nfsrvd_readdirplus(struct nfsrv_descript *nd, int isdg } fullsiz = siz; nd->nd_repstat = getret = nfsvno_getattr(vp, &at, nd->nd_cred, p, 1); +#if 0 if (!nd->nd_repstat) { if (off && verf != at.na_filerev) { /* @@ -1879,17 +1880,14 @@ nfsrvd_readdirplus(struct nfsrv_descript *nd, int isdg * removed/added unless that offset cookies returned to * the client are no longer valid. */ -#if 0 if (nd->nd_flag & ND_NFSV4) { nd->nd_repstat = NFSERR_NOTSAME; } else { nd->nd_repstat = NFSERR_BAD_COOKIE; } -#endif - } else if ((nd->nd_flag & ND_NFSV4) && off == 0 && verf != 0) { - nd->nd_repstat = NFSERR_BAD_COOKIE; } } +#endif if (!nd->nd_repstat && vp->v_type != VDIR) nd->nd_repstat = NFSERR_NOTDIR; if (!nd->nd_repstat && cnt == 0) From owner-svn-src-all@freebsd.org Tue Jul 31 21:06:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 767AE1064F26; Tue, 31 Jul 2018 21:06:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28E667CF78; Tue, 31 Jul 2018 21:06:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E400E18F9F; Tue, 31 Jul 2018 21:06:07 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VL67fj038239; Tue, 31 Jul 2018 21:06:07 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VL65J6038222; Tue, 31 Jul 2018 21:06:05 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312106.w6VL65J6038222@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 21:06:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337007 - in head: cddl/contrib/opensolaris/cmd/zpool cddl/contrib/opensolaris/cmd/ztest cddl/contrib/opensolaris/lib/libzfs/common cddl/contrib/opensolaris/lib/libzfs_core/common sys/c... X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/zpool cddl/contrib/opensolaris/cmd/ztest cddl/contrib/opensolaris/lib/libzfs/common cddl/contrib/opensolaris/lib/libzfs_core/common sys/cddl/contrib/opensolaris/u... X-SVN-Commit-Revision: 337007 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:06:08 -0000 Author: mav Date: Tue Jul 31 21:06:04 2018 New Revision: 337007 URL: https://svnweb.freebsd.org/changeset/base/337007 Log: MFV r336991, r337001: 9102 zfs should be able to initialize storage devices The first access to a disk block can incur a performance penalty on some platforms (e.g. AWS's EBS, VMware VMDKs). Therefore it is recommended that volumes be "thick provisioned", where supported by the platform (VMware). Thick provisioning is time consuming and often is ignored. If the thick provision step is omitted, customers will see suboptimal performance until we have written to all parts of the LUN. ZFS should be able to initialize any unused storage to remove any first-write penalty that exists. illumos/illumos-gate@094e47e980b0796b94b1b8f51f462a64d246e516 Reviewed by: John Wren Kennedy Reviewed by: Matthew Ahrens Reviewed by: Pavel Zakharov Reviewed by: Prakash Surya Approved by: Richard Lowe Author: George Wilson Added: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_initialize.h - copied unchanged from r337001, vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_initialize.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_initialize.c - copied, changed from r337001, vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_initialize.c Modified: head/cddl/contrib/opensolaris/cmd/zpool/zpool.8 head/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c head/cddl/contrib/opensolaris/cmd/ztest/ztest.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.c head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.h head/sys/cddl/contrib/opensolaris/uts/common/Makefile.files head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_priority.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_disk.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_file.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_indirect.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_root.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c head/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h head/sys/conf/files Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/cddl/contrib/opensolaris/lib/libzfs/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zpool/zpool.8 ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zpool/zpool.8 Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/cmd/zpool/zpool.8 Tue Jul 31 21:06:04 2018 (r337007) @@ -121,6 +121,11 @@ .Ar pool | id .Op Ar newpool .Nm +.Cm initialize +.Op Fl cs +.Ar pool +.Op Ar device Ns ... +.Nm .Cm iostat .Op Fl T Cm d Ns | Ns Cm u .Op Fl v @@ -1434,6 +1439,32 @@ mounting option is enabled. In this case, the checkpointed state of the pool is opened and an administrator can see how the pool would look like if they were to fully rewind. +.El +.It Xo +.Nm +.Cm initialize +.Op Fl cs +.Ar pool +.Op Ar device Ns ... +.Xc +Begins initializing by writing to all unallocated regions on the specified +devices, or all eligible devices in the pool if no individual devices are +specified. +Only leaf data or log devices may be initialized. +.Bl -tag -width Ds +.It Fl c, -cancel +Cancel initializing on the specified devices, or all eligible devices if none +are specified. +If one or more target devices are invalid or are not currently being +initialized, the command will fail and no cancellation will occur on any device. +.It Fl s -suspend +Suspend initializing on the specified devices, or all eligible devices if none +are specified. +If one or more target devices are invalid or are not currently being +initialized, the command will fail and no suspension will occur on any device. +Initializing can then be resumed by running +.Nm zpool Cm initialize +with no flags on the relevant target devices. .El .It Xo .Nm Modified: head/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c Tue Jul 31 21:06:04 2018 (r337007) @@ -87,6 +87,7 @@ static int zpool_do_detach(int, char **); static int zpool_do_replace(int, char **); static int zpool_do_split(int, char **); +static int zpool_do_initialize(int, char **); static int zpool_do_scrub(int, char **); static int zpool_do_import(int, char **); @@ -136,6 +137,7 @@ typedef enum { HELP_ONLINE, HELP_REPLACE, HELP_REMOVE, + HELP_INITIALIZE, HELP_SCRUB, HELP_STATUS, HELP_UPGRADE, @@ -187,6 +189,7 @@ static zpool_command_t command_table[] = { { "replace", zpool_do_replace, HELP_REPLACE }, { "split", zpool_do_split, HELP_SPLIT }, { NULL }, + { "initialize", zpool_do_initialize, HELP_INITIALIZE }, { "scrub", zpool_do_scrub, HELP_SCRUB }, { NULL }, { "import", zpool_do_import, HELP_IMPORT }, @@ -261,6 +264,8 @@ get_usage(zpool_help_t idx) return (gettext("\tremove [-nps] ...\n")); case HELP_REOPEN: return (gettext("\treopen \n")); + case HELP_INITIALIZE: + return (gettext("\tinitialize [-cs] [ ...]\n")); case HELP_SCRUB: return (gettext("\tscrub [-s | -p] ...\n")); case HELP_STATUS: @@ -1650,6 +1655,43 @@ print_status_config(zpool_handle_t *zhp, const char *n "resilvering" : "repairing"); } + if ((vs->vs_initialize_state == VDEV_INITIALIZE_ACTIVE || + vs->vs_initialize_state == VDEV_INITIALIZE_SUSPENDED || + vs->vs_initialize_state == VDEV_INITIALIZE_COMPLETE) && + !vs->vs_scan_removing) { + char zbuf[1024]; + char tbuf[256]; + struct tm zaction_ts; + + time_t t = vs->vs_initialize_action_time; + int initialize_pct = 100; + if (vs->vs_initialize_state != VDEV_INITIALIZE_COMPLETE) { + initialize_pct = (vs->vs_initialize_bytes_done * 100 / + (vs->vs_initialize_bytes_est + 1)); + } + + (void) localtime_r(&t, &zaction_ts); + (void) strftime(tbuf, sizeof (tbuf), "%c", &zaction_ts); + + switch (vs->vs_initialize_state) { + case VDEV_INITIALIZE_SUSPENDED: + (void) snprintf(zbuf, sizeof (zbuf), + ", suspended, started at %s", tbuf); + break; + case VDEV_INITIALIZE_ACTIVE: + (void) snprintf(zbuf, sizeof (zbuf), + ", started at %s", tbuf); + break; + case VDEV_INITIALIZE_COMPLETE: + (void) snprintf(zbuf, sizeof (zbuf), + ", completed at %s", tbuf); + break; + } + + (void) printf(gettext(" (%d%% initialized%s)"), + initialize_pct, zbuf); + } + (void) printf("\n"); for (c = 0; c < children; c++) { @@ -4236,6 +4278,119 @@ zpool_do_scrub(int argc, char **argv) } return (for_each_pool(argc, argv, B_TRUE, NULL, scrub_callback, &cb)); +} + +static void +zpool_collect_leaves(zpool_handle_t *zhp, nvlist_t *nvroot, nvlist_t *res) +{ + uint_t children = 0; + nvlist_t **child; + uint_t i; + + (void) nvlist_lookup_nvlist_array(nvroot, ZPOOL_CONFIG_CHILDREN, + &child, &children); + + if (children == 0) { + char *path = zpool_vdev_name(g_zfs, zhp, nvroot, B_FALSE); + fnvlist_add_boolean(res, path); + free(path); + return; + } + + for (i = 0; i < children; i++) { + zpool_collect_leaves(zhp, child[i], res); + } +} + +/* + * zpool initialize [-cs] [ ...] + * Initialize all unused blocks in the specified vdevs, or all vdevs in the pool + * if none specified. + * + * -c Cancel. Ends active initializing. + * -s Suspend. Initializing can then be restarted with no flags. + */ +int +zpool_do_initialize(int argc, char **argv) +{ + int c; + char *poolname; + zpool_handle_t *zhp; + nvlist_t *vdevs; + int err = 0; + + struct option long_options[] = { + {"cancel", no_argument, NULL, 'c'}, + {"suspend", no_argument, NULL, 's'}, + {0, 0, 0, 0} + }; + + pool_initialize_func_t cmd_type = POOL_INITIALIZE_DO; + while ((c = getopt_long(argc, argv, "cs", long_options, NULL)) != -1) { + switch (c) { + case 'c': + if (cmd_type != POOL_INITIALIZE_DO) { + (void) fprintf(stderr, gettext("-c cannot be " + "combined with other options\n")); + usage(B_FALSE); + } + cmd_type = POOL_INITIALIZE_CANCEL; + break; + case 's': + if (cmd_type != POOL_INITIALIZE_DO) { + (void) fprintf(stderr, gettext("-s cannot be " + "combined with other options\n")); + usage(B_FALSE); + } + cmd_type = POOL_INITIALIZE_SUSPEND; + break; + case '?': + if (optopt != 0) { + (void) fprintf(stderr, + gettext("invalid option '%c'\n"), optopt); + } else { + (void) fprintf(stderr, + gettext("invalid option '%s'\n"), + argv[optind - 1]); + } + usage(B_FALSE); + } + } + + argc -= optind; + argv += optind; + + if (argc < 1) { + (void) fprintf(stderr, gettext("missing pool name argument\n")); + usage(B_FALSE); + return (-1); + } + + poolname = argv[0]; + zhp = zpool_open(g_zfs, poolname); + if (zhp == NULL) + return (-1); + + vdevs = fnvlist_alloc(); + if (argc == 1) { + /* no individual leaf vdevs specified, so add them all */ + nvlist_t *config = zpool_get_config(zhp, NULL); + nvlist_t *nvroot = fnvlist_lookup_nvlist(config, + ZPOOL_CONFIG_VDEV_TREE); + zpool_collect_leaves(zhp, nvroot, vdevs); + } else { + int i; + for (i = 1; i < argc; i++) { + fnvlist_add_boolean(vdevs, argv[i]); + } + } + + err = zpool_initialize(zhp, cmd_type, vdevs); + + fnvlist_free(vdevs); + zpool_close(zhp); + + return (err); } typedef struct status_cbdata { Modified: head/cddl/contrib/opensolaris/cmd/ztest/ztest.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/cmd/ztest/ztest.c Tue Jul 31 21:06:04 2018 (r337007) @@ -104,6 +104,7 @@ #include #include #include +#include #include #include #include @@ -348,6 +349,7 @@ ztest_func_t ztest_spa_upgrade; ztest_func_t ztest_device_removal; ztest_func_t ztest_remap_blocks; ztest_func_t ztest_spa_checkpoint_create_discard; +ztest_func_t ztest_initialize; uint64_t zopt_always = 0ULL * NANOSEC; /* all the time */ uint64_t zopt_incessant = 1ULL * NANOSEC / 10; /* every 1/10 second */ @@ -391,7 +393,8 @@ ztest_info_t ztest_info[] = { &ztest_opts.zo_vdevtime }, { ztest_device_removal, 1, &zopt_sometimes }, { ztest_remap_blocks, 1, &zopt_sometimes }, - { ztest_spa_checkpoint_create_discard, 1, &zopt_rarely } + { ztest_spa_checkpoint_create_discard, 1, &zopt_rarely }, + { ztest_initialize, 1, &zopt_sometimes } }; #define ZTEST_FUNCS (sizeof (ztest_info) / sizeof (ztest_info_t)) @@ -5469,6 +5472,97 @@ ztest_spa_rename(ztest_ds_t *zd, uint64_t id) umem_free(newname, strlen(newname) + 1); rw_exit(&ztest_name_lock); +} + +static vdev_t * +ztest_random_concrete_vdev_leaf(vdev_t *vd) +{ + if (vd == NULL) + return (NULL); + + if (vd->vdev_children == 0) + return (vd); + + vdev_t *eligible[vd->vdev_children]; + int eligible_idx = 0, i; + for (i = 0; i < vd->vdev_children; i++) { + vdev_t *cvd = vd->vdev_child[i]; + if (cvd->vdev_top->vdev_removing) + continue; + if (cvd->vdev_children > 0 || + (vdev_is_concrete(cvd) && !cvd->vdev_detached)) { + eligible[eligible_idx++] = cvd; + } + } + VERIFY(eligible_idx > 0); + + uint64_t child_no = ztest_random(eligible_idx); + return (ztest_random_concrete_vdev_leaf(eligible[child_no])); +} + +/* ARGSUSED */ +void +ztest_initialize(ztest_ds_t *zd, uint64_t id) +{ + spa_t *spa = ztest_spa; + int error = 0; + + mutex_enter(&ztest_vdev_lock); + + spa_config_enter(spa, SCL_VDEV, FTAG, RW_READER); + + /* Random leaf vdev */ + vdev_t *rand_vd = ztest_random_concrete_vdev_leaf(spa->spa_root_vdev); + if (rand_vd == NULL) { + spa_config_exit(spa, SCL_VDEV, FTAG); + mutex_exit(&ztest_vdev_lock); + return; + } + + /* + * The random vdev we've selected may change as soon as we + * drop the spa_config_lock. We create local copies of things + * we're interested in. + */ + uint64_t guid = rand_vd->vdev_guid; + char *path = strdup(rand_vd->vdev_path); + boolean_t active = rand_vd->vdev_initialize_thread != NULL; + + zfs_dbgmsg("vd %p, guid %llu", rand_vd, guid); + spa_config_exit(spa, SCL_VDEV, FTAG); + + uint64_t cmd = ztest_random(POOL_INITIALIZE_FUNCS); + error = spa_vdev_initialize(spa, guid, cmd); + switch (cmd) { + case POOL_INITIALIZE_CANCEL: + if (ztest_opts.zo_verbose >= 4) { + (void) printf("Cancel initialize %s", path); + if (!active) + (void) printf(" failed (no initialize active)"); + (void) printf("\n"); + } + break; + case POOL_INITIALIZE_DO: + if (ztest_opts.zo_verbose >= 4) { + (void) printf("Start initialize %s", path); + if (active && error == 0) + (void) printf(" failed (already active)"); + else if (error != 0) + (void) printf(" failed (error %d)", error); + (void) printf("\n"); + } + break; + case POOL_INITIALIZE_SUSPEND: + if (ztest_opts.zo_verbose >= 4) { + (void) printf("Suspend initialize %s", path); + if (!active) + (void) printf(" failed (no initialize active)"); + (void) printf("\n"); + } + break; + } + free(path); + mutex_exit(&ztest_vdev_lock); } /* Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h Tue Jul 31 21:06:04 2018 (r337007) @@ -137,6 +137,9 @@ typedef enum zfs_error { EZFS_NO_CHECKPOINT, /* pool has no checkpoint */ EZFS_DEVRM_IN_PROGRESS, /* a device is currently being removed */ EZFS_VDEV_TOO_BIG, /* a device is too big to be used */ + EZFS_TOOMANY, /* argument list too long */ + EZFS_INITIALIZING, /* currently initializing */ + EZFS_NO_INITIALIZE, /* no active initialize */ EZFS_UNKNOWN } zfs_error_t; @@ -262,6 +265,8 @@ typedef struct splitflags { * Functions to manipulate pool and vdev state */ extern int zpool_scan(zpool_handle_t *, pool_scan_func_t, pool_scrub_cmd_t); +extern int zpool_initialize(zpool_handle_t *, pool_initialize_func_t, + nvlist_t *); extern int zpool_clear(zpool_handle_t *, const char *, nvlist_t *); extern int zpool_reguid(zpool_handle_t *); extern int zpool_reopen(zpool_handle_t *); Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c Tue Jul 31 21:06:04 2018 (r337007) @@ -1981,6 +1981,100 @@ zpool_scan(zpool_handle_t *zhp, pool_scan_func_t func, } } +static int +xlate_init_err(int err) +{ + switch (err) { + case ENODEV: + return (EZFS_NODEVICE); + case EINVAL: + case EROFS: + return (EZFS_BADDEV); + case EBUSY: + return (EZFS_INITIALIZING); + case ESRCH: + return (EZFS_NO_INITIALIZE); + } + return (err); +} + +/* + * Begin, suspend, or cancel the initialization (initializing of all free + * blocks) for the given vdevs in the given pool. + */ +int +zpool_initialize(zpool_handle_t *zhp, pool_initialize_func_t cmd_type, + nvlist_t *vds) +{ + char msg[1024]; + libzfs_handle_t *hdl = zhp->zpool_hdl; + + nvlist_t *errlist; + + /* translate vdev names to guids */ + nvlist_t *vdev_guids = fnvlist_alloc(); + nvlist_t *guids_to_paths = fnvlist_alloc(); + boolean_t spare, cache; + nvlist_t *tgt; + nvpair_t *elem; + + for (elem = nvlist_next_nvpair(vds, NULL); elem != NULL; + elem = nvlist_next_nvpair(vds, elem)) { + char *vd_path = nvpair_name(elem); + tgt = zpool_find_vdev(zhp, vd_path, &spare, &cache, NULL); + + if ((tgt == NULL) || cache || spare) { + (void) snprintf(msg, sizeof (msg), + dgettext(TEXT_DOMAIN, "cannot initialize '%s'"), + vd_path); + int err = (tgt == NULL) ? EZFS_NODEVICE : + (spare ? EZFS_ISSPARE : EZFS_ISL2CACHE); + fnvlist_free(vdev_guids); + fnvlist_free(guids_to_paths); + return (zfs_error(hdl, err, msg)); + } + + uint64_t guid = fnvlist_lookup_uint64(tgt, ZPOOL_CONFIG_GUID); + fnvlist_add_uint64(vdev_guids, vd_path, guid); + + (void) snprintf(msg, sizeof (msg), "%llu", guid); + fnvlist_add_string(guids_to_paths, msg, vd_path); + } + + int err = lzc_initialize(zhp->zpool_name, cmd_type, vdev_guids, + &errlist); + fnvlist_free(vdev_guids); + + if (err == 0) { + fnvlist_free(guids_to_paths); + return (0); + } + + nvlist_t *vd_errlist = NULL; + if (errlist != NULL) { + vd_errlist = fnvlist_lookup_nvlist(errlist, + ZPOOL_INITIALIZE_VDEVS); + } + + (void) snprintf(msg, sizeof (msg), + dgettext(TEXT_DOMAIN, "operation failed")); + + for (elem = nvlist_next_nvpair(vd_errlist, NULL); elem != NULL; + elem = nvlist_next_nvpair(vd_errlist, elem)) { + int64_t vd_error = xlate_init_err(fnvpair_value_int64(elem)); + char *path = fnvlist_lookup_string(guids_to_paths, + nvpair_name(elem)); + (void) zfs_error_fmt(hdl, vd_error, "cannot initialize '%s'", + path); + } + + fnvlist_free(guids_to_paths); + if (vd_errlist != NULL) + return (-1); + + return (zpool_standard_error(hdl, err, msg)); +} + #ifdef illumos /* * This provides a very minimal check whether a given string is likely a Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c Tue Jul 31 21:06:04 2018 (r337007) @@ -254,6 +254,13 @@ libzfs_error_description(libzfs_handle_t *hdl) return (dgettext(TEXT_DOMAIN, "device removal in progress")); case EZFS_VDEV_TOO_BIG: return (dgettext(TEXT_DOMAIN, "device exceeds supported size")); + case EZFS_TOOMANY: + return (dgettext(TEXT_DOMAIN, "argument list too long")); + case EZFS_INITIALIZING: + return (dgettext(TEXT_DOMAIN, "currently initializing")); + case EZFS_NO_INITIALIZE: + return (dgettext(TEXT_DOMAIN, "there is no active " + "initialization")); case EZFS_UNKNOWN: return (dgettext(TEXT_DOMAIN, "unknown error")); default: Modified: head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.c Tue Jul 31 21:06:04 2018 (r337007) @@ -1085,3 +1085,40 @@ lzc_channel_program_nosync(const char *pool, const cha return (lzc_channel_program_impl(pool, program, B_FALSE, timeout, memlimit, argnvl, outnvl)); } + +/* + * Changes initializing state. + * + * vdevs should be a list of (, guid) where guid is a uint64 vdev GUID. + * The key is ignored. + * + * If there are errors related to vdev arguments, per-vdev errors are returned + * in an nvlist with the key "vdevs". Each error is a (guid, errno) pair where + * guid is stringified with PRIu64, and errno is one of the following as + * an int64_t: + * - ENODEV if the device was not found + * - EINVAL if the devices is not a leaf or is not concrete (e.g. missing) + * - EROFS if the device is not writeable + * - EBUSY start requested but the device is already being initialized + * - ESRCH cancel/suspend requested but device is not being initialized + * + * If the errlist is empty, then return value will be: + * - EINVAL if one or more arguments was invalid + * - Other spa_open failures + * - 0 if the operation succeeded + */ +int +lzc_initialize(const char *poolname, pool_initialize_func_t cmd_type, + nvlist_t *vdevs, nvlist_t **errlist) +{ + int error; + nvlist_t *args = fnvlist_alloc(); + fnvlist_add_uint64(args, ZPOOL_INITIALIZE_COMMAND, (uint64_t)cmd_type); + fnvlist_add_nvlist(args, ZPOOL_INITIALIZE_VDEVS, vdevs); + + error = lzc_ioctl(ZFS_IOC_POOL_INITIALIZE, poolname, args, errlist); + + fnvlist_free(args); + + return (error); +} Modified: head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.h ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.h Tue Jul 31 21:02:45 2018 (r337006) +++ head/cddl/contrib/opensolaris/lib/libzfs_core/common/libzfs_core.h Tue Jul 31 21:06:04 2018 (r337007) @@ -31,7 +31,9 @@ #include #include #include +#include + #ifdef __cplusplus extern "C" { #endif @@ -56,6 +58,8 @@ int lzc_destroy_snaps(nvlist_t *, boolean_t, nvlist_t int lzc_bookmark(nvlist_t *, nvlist_t **); int lzc_get_bookmarks(const char *, nvlist_t *, nvlist_t **); int lzc_destroy_bookmarks(nvlist_t *, nvlist_t **); +int lzc_initialize(const char *, pool_initialize_func_t, nvlist_t *, + nvlist_t **); int lzc_snaprange_space(const char *, const char *, uint64_t *); Modified: head/sys/cddl/contrib/opensolaris/uts/common/Makefile.files ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/Makefile.files Tue Jul 31 21:02:45 2018 (r337006) +++ head/sys/cddl/contrib/opensolaris/uts/common/Makefile.files Tue Jul 31 21:06:04 2018 (r337007) @@ -124,6 +124,7 @@ ZFS_COMMON_OBJS += \ vdev_indirect.o \ vdev_indirect_births.o \ vdev_indirect_mapping.o \ + vdev_initialize.o \ vdev_label.o \ vdev_mirror.o \ vdev_missing.o \ Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/metaslab.c Tue Jul 31 21:06:04 2018 (r337007) @@ -725,6 +725,8 @@ metaslab_group_create(metaslab_class_t *mc, vdev_t *vd mg = kmem_zalloc(sizeof (metaslab_group_t), KM_SLEEP); mutex_init(&mg->mg_lock, NULL, MUTEX_DEFAULT, NULL); + mutex_init(&mg->mg_ms_initialize_lock, NULL, MUTEX_DEFAULT, NULL); + cv_init(&mg->mg_ms_initialize_cv, NULL, CV_DEFAULT, NULL); mg->mg_primaries = kmem_zalloc(allocators * sizeof (metaslab_t *), KM_SLEEP); mg->mg_secondaries = kmem_zalloc(allocators * sizeof (metaslab_t *), @@ -771,6 +773,8 @@ metaslab_group_destroy(metaslab_group_t *mg) kmem_free(mg->mg_secondaries, mg->mg_allocators * sizeof (metaslab_t *)); mutex_destroy(&mg->mg_lock); + mutex_destroy(&mg->mg_ms_initialize_lock); + cv_destroy(&mg->mg_ms_initialize_cv); for (int i = 0; i < mg->mg_allocators; i++) { refcount_destroy(&mg->mg_alloc_queue_depth[i]); @@ -1554,6 +1558,7 @@ metaslab_init(metaslab_group_t *mg, uint64_t id, uint6 mutex_init(&ms->ms_lock, NULL, MUTEX_DEFAULT, NULL); mutex_init(&ms->ms_sync_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&ms->ms_load_cv, NULL, CV_DEFAULT, NULL); + ms->ms_id = id; ms->ms_start = id << vd->vdev_ms_shift; ms->ms_size = 1ULL << vd->vdev_ms_shift; @@ -2731,6 +2736,7 @@ metaslab_sync_done(metaslab_t *msp, uint64_t txg) * from it in 'metaslab_unload_delay' txgs, then unload it. */ if (msp->ms_loaded && + msp->ms_initializing == 0 && msp->ms_selected_txg + metaslab_unload_delay < txg) { for (int t = 1; t < TXG_CONCURRENT_STATES; t++) { VERIFY0(range_tree_space( @@ -2980,6 +2986,7 @@ metaslab_block_alloc(metaslab_t *msp, uint64_t size, u metaslab_class_t *mc = msp->ms_group->mg_class; VERIFY(!msp->ms_condensing); + VERIFY0(msp->ms_initializing); start = mc->mc_ops->msop_alloc(msp, size); if (start != -1ULL) { @@ -3040,9 +3047,10 @@ find_valid_metaslab(metaslab_group_t *mg, uint64_t act } /* - * If the selected metaslab is condensing, skip it. + * If the selected metaslab is condensing or being + * initialized, skip it. */ - if (msp->ms_condensing) + if (msp->ms_condensing || msp->ms_initializing > 0) continue; *was_active = msp->ms_allocator != -1; @@ -3207,11 +3215,20 @@ metaslab_group_alloc_normal(metaslab_group_t *mg, zio_ /* * If this metaslab is currently condensing then pick again as * we can't manipulate this metaslab until it's committed - * to disk. + * to disk. If this metaslab is being initialized, we shouldn't + * allocate from it since the allocated region might be + * overwritten after allocation. */ if (msp->ms_condensing) { metaslab_trace_add(zal, mg, msp, asize, d, TRACE_CONDENSING, allocator); + metaslab_passivate(msp, msp->ms_weight & + ~METASLAB_ACTIVE_MASK); + mutex_exit(&msp->ms_lock); + continue; + } else if (msp->ms_initializing > 0) { + metaslab_trace_add(zal, mg, msp, asize, d, + TRACE_INITIALIZING, allocator); metaslab_passivate(msp, msp->ms_weight & ~METASLAB_ACTIVE_MASK); mutex_exit(&msp->ms_lock); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Tue Jul 31 21:06:04 2018 (r337007) @@ -55,6 +55,7 @@ #include #include #include +#include #include #include #include @@ -443,8 +444,9 @@ spa_prop_get(spa_t *spa, nvlist_t **nvp) dp = spa_get_dsl(spa); dsl_pool_config_enter(dp, FTAG); - if (err = dsl_dataset_hold_obj(dp, - za.za_first_integer, FTAG, &ds)) { + err = dsl_dataset_hold_obj(dp, + za.za_first_integer, FTAG, &ds); + if (err != 0) { dsl_pool_config_exit(dp, FTAG); break; } @@ -599,7 +601,8 @@ spa_prop_validate(spa_t *spa, nvlist_t *props) break; } - if (error = dmu_objset_hold(strval, FTAG, &os)) + error = dmu_objset_hold(strval, FTAG, &os); + if (error != 0) break; /* @@ -1215,8 +1218,10 @@ spa_activate(spa_t *spa, int mode) */ trim_thread_create(spa); - for (size_t i = 0; i < TXG_SIZE; i++) - spa->spa_txg_zio[i] = zio_root(spa, NULL, NULL, 0); + for (size_t i = 0; i < TXG_SIZE; i++) { + spa->spa_txg_zio[i] = zio_root(spa, NULL, NULL, + ZIO_FLAG_CANFAIL); + } list_create(&spa->spa_config_dirty_list, sizeof (vdev_t), offsetof(vdev_t, vdev_config_dirty_node)); @@ -1388,6 +1393,11 @@ spa_unload(spa_t *spa) */ spa_async_suspend(spa); + if (spa->spa_root_vdev) { + vdev_initialize_stop_all(spa->spa_root_vdev, + VDEV_INITIALIZE_ACTIVE); + } + /* * Stop syncing. */ @@ -1403,10 +1413,10 @@ spa_unload(spa_t *spa) * calling taskq_wait(mg_taskq). */ if (spa->spa_root_vdev != NULL) { - spa_config_enter(spa, SCL_ALL, FTAG, RW_WRITER); + spa_config_enter(spa, SCL_ALL, spa, RW_WRITER); for (int c = 0; c < spa->spa_root_vdev->vdev_children; c++) vdev_metaslab_fini(spa->spa_root_vdev->vdev_child[c]); - spa_config_exit(spa, SCL_ALL, FTAG); + spa_config_exit(spa, SCL_ALL, spa); } /* @@ -1440,7 +1450,7 @@ spa_unload(spa_t *spa) bpobj_close(&spa->spa_deferred_bpobj); - spa_config_enter(spa, SCL_ALL, FTAG, RW_WRITER); + spa_config_enter(spa, SCL_ALL, spa, RW_WRITER); /* * Close all vdevs. @@ -1502,7 +1512,7 @@ spa_unload(spa_t *spa) spa->spa_comment = NULL; } - spa_config_exit(spa, SCL_ALL, FTAG); + spa_config_exit(spa, SCL_ALL, spa); } /* @@ -3954,6 +3964,10 @@ spa_load_impl(spa_t *spa, spa_import_type_t type, char spa_restart_removal(spa); spa_spawn_aux_threads(spa); + + spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); + vdev_initialize_restart(spa->spa_root_vdev); + spa_config_exit(spa, SCL_CONFIG, FTAG); } spa_load_note(spa, "LOADED"); @@ -5675,6 +5689,7 @@ spa_export_common(char *pool, int new_state, nvlist_t * in which case we can modify its state. */ if (spa->spa_state != POOL_STATE_UNINITIALIZED && spa->spa_sync_on) { + /* * Objsets may be open only because they're dirty, so we * have to force it to sync before checking spa_refcnt. @@ -5709,6 +5724,18 @@ spa_export_common(char *pool, int new_state, nvlist_t } /* + * We're about to export or destroy this pool. Make sure + * we stop all initializtion activity here before we + * set the spa_final_txg. This will ensure that all + * dirty data resulting from the initialization is + * committed to disk before we unload the pool. + */ + if (spa->spa_root_vdev != NULL) { + vdev_initialize_stop_all(spa->spa_root_vdev, + VDEV_INITIALIZE_ACTIVE); + } + + /* * We want this to be reflected on every label, * so mark them all dirty. spa_unload() will do the * final sync that pushes these changes out. @@ -6398,6 +6425,86 @@ spa_vdev_detach(spa_t *spa, uint64_t guid, uint64_t pg return (error); } +int +spa_vdev_initialize(spa_t *spa, uint64_t guid, uint64_t cmd_type) +{ + /* + * We hold the namespace lock through the whole function + * to prevent any changes to the pool while we're starting or + * stopping initialization. The config and state locks are held so that + * we can properly assess the vdev state before we commit to + * the initializing operation. + */ + mutex_enter(&spa_namespace_lock); + spa_config_enter(spa, SCL_CONFIG | SCL_STATE, FTAG, RW_READER); + + /* Look up vdev and ensure it's a leaf. */ + vdev_t *vd = spa_lookup_by_guid(spa, guid, B_FALSE); + if (vd == NULL || vd->vdev_detached) { + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(ENODEV)); + } else if (!vd->vdev_ops->vdev_op_leaf || !vdev_is_concrete(vd)) { + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(EINVAL)); + } else if (!vdev_writeable(vd)) { + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(EROFS)); + } + mutex_enter(&vd->vdev_initialize_lock); + spa_config_exit(spa, SCL_CONFIG | SCL_STATE, FTAG); + + /* + * When we activate an initialize action we check to see + * if the vdev_initialize_thread is NULL. We do this instead + * of using the vdev_initialize_state since there might be + * a previous initialization process which has completed but + * the thread is not exited. + */ + if (cmd_type == POOL_INITIALIZE_DO && + (vd->vdev_initialize_thread != NULL || + vd->vdev_top->vdev_removing)) { + mutex_exit(&vd->vdev_initialize_lock); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(EBUSY)); + } else if (cmd_type == POOL_INITIALIZE_CANCEL && + (vd->vdev_initialize_state != VDEV_INITIALIZE_ACTIVE && + vd->vdev_initialize_state != VDEV_INITIALIZE_SUSPENDED)) { + mutex_exit(&vd->vdev_initialize_lock); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(ESRCH)); + } else if (cmd_type == POOL_INITIALIZE_SUSPEND && + vd->vdev_initialize_state != VDEV_INITIALIZE_ACTIVE) { + mutex_exit(&vd->vdev_initialize_lock); + mutex_exit(&spa_namespace_lock); + return (SET_ERROR(ESRCH)); + } + + switch (cmd_type) { + case POOL_INITIALIZE_DO: + vdev_initialize(vd); + break; + case POOL_INITIALIZE_CANCEL: + vdev_initialize_stop(vd, VDEV_INITIALIZE_CANCELED); + break; + case POOL_INITIALIZE_SUSPEND: + vdev_initialize_stop(vd, VDEV_INITIALIZE_SUSPENDED); + break; + default: + panic("invalid cmd_type %llu", (unsigned long long)cmd_type); + } + mutex_exit(&vd->vdev_initialize_lock); + + /* Sync out the initializing state */ + txg_wait_synced(spa->spa_dsl_pool, 0); + mutex_exit(&spa_namespace_lock); + + return (0); +} + + /* * Split a set of devices from their mirrors, and create a new pool from them. */ @@ -6605,6 +6712,19 @@ spa_vdev_split_mirror(spa_t *spa, char *newname, nvlis spa_activate(newspa, spa_mode_global); spa_async_suspend(newspa); + for (c = 0; c < children; c++) { + if (vml[c] != NULL) { + /* + * Temporarily stop the initializing activity. We set + * the state to ACTIVE so that we know to resume + * the initializing once the split has completed. + */ + mutex_enter(&vml[c]->vdev_initialize_lock); + vdev_initialize_stop(vml[c], VDEV_INITIALIZE_ACTIVE); + mutex_exit(&vml[c]->vdev_initialize_lock); + } + } + #ifndef illumos /* mark that we are creating new spa by splitting */ newspa->spa_splitting_newspa = B_TRUE; @@ -6699,6 +6819,10 @@ out: if (vml[c] != NULL) vml[c]->vdev_offline = B_FALSE; } + + /* restart initializing disks as necessary */ + spa_async_request(spa, SPA_ASYNC_INITIALIZE_RESTART); + vdev_reopen(spa->spa_root_vdev); nvlist_free(spa->spa_config_splitting); @@ -7063,6 +7187,14 @@ spa_async_thread(void *arg) if (tasks & SPA_ASYNC_RESILVER) dsl_resilver_restart(spa->spa_dsl_pool, 0); + if (tasks & SPA_ASYNC_INITIALIZE_RESTART) { + mutex_enter(&spa_namespace_lock); + spa_config_enter(spa, SCL_CONFIG, FTAG, RW_READER); + vdev_initialize_restart(spa->spa_root_vdev); + spa_config_exit(spa, SCL_CONFIG, FTAG); + mutex_exit(&spa_namespace_lock); + } + /* * Let the world know that we're done. */ @@ -7762,8 +7894,9 @@ spa_sync(spa_t *spa, uint64_t txg) * Wait for i/os issued in open context that need to complete * before this txg syncs. */ - VERIFY0(zio_wait(spa->spa_txg_zio[txg & TXG_MASK])); - spa->spa_txg_zio[txg & TXG_MASK] = zio_root(spa, NULL, NULL, 0); + (void) zio_wait(spa->spa_txg_zio[txg & TXG_MASK]); + spa->spa_txg_zio[txg & TXG_MASK] = zio_root(spa, NULL, NULL, + ZIO_FLAG_CANFAIL); /* * Lock out configuration changes. @@ -8065,7 +8198,8 @@ spa_sync(spa_t *spa, uint64_t txg) /* * Update usable space statistics. */ - while (vd = txg_list_remove(&spa->spa_vdev_txg_list, TXG_CLEAN(txg))) + while ((vd = txg_list_remove(&spa->spa_vdev_txg_list, TXG_CLEAN(txg))) + != NULL) vdev_sync_done(vd, txg); spa_update_dspace(spa); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Tue Jul 31 21:02:45 2018 (r337006) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Tue Jul 31 21:06:04 2018 (r337007) @@ -41,6 +41,7 @@ #include #include #include +#include #include #include #include @@ -1313,6 +1314,12 @@ spa_vdev_config_exit(spa_t *spa, vdev_t *vd, uint64_t if (vd != NULL) { ASSERT(!vd->vdev_detached || vd->vdev_dtl_sm == NULL); + if (vd->vdev_ops->vdev_op_leaf) { + mutex_enter(&vd->vdev_initialize_lock); + vdev_initialize_stop(vd, VDEV_INITIALIZE_CANCELED); + mutex_exit(&vd->vdev_initialize_lock); + } + spa_config_enter(spa, SCL_ALL, spa, RW_WRITER); vdev_free(vd); spa_config_exit(spa, SCL_ALL, spa); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h Tue Jul 31 21:02:45 2018 (r337006) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/metaslab_impl.h Tue Jul 31 21:06:04 2018 (r337007) @@ -68,7 +68,8 @@ typedef enum trace_alloc_type { TRACE_GROUP_FAILURE = -5ULL, TRACE_ENOSPC = -6ULL, TRACE_CONDENSING = -7ULL, - TRACE_VDEV_ERROR = -8ULL + TRACE_VDEV_ERROR = -8ULL, + TRACE_INITIALIZING = -9ULL } trace_alloc_type_t; #define METASLAB_WEIGHT_PRIMARY (1ULL << 63) @@ -271,6 +272,11 @@ struct metaslab_group { uint64_t mg_failed_allocations; uint64_t mg_fragmentation; uint64_t mg_histogram[RANGE_TREE_HISTOGRAM_SIZE]; + + int mg_ms_initializing; + boolean_t mg_initialize_updating; + kmutex_t mg_ms_initialize_lock; + kcondvar_t mg_ms_initialize_cv; }; /* @@ -360,6 +366,8 @@ struct metaslab { boolean_t ms_condensing; /* condensing? */ boolean_t ms_condense_wanted; uint64_t ms_condense_checked_txg; + + uint64_t ms_initializing; /* leaves initializing this ms */ *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Tue Jul 31 21:06:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 251431064F46; Tue, 31 Jul 2018 21:06:29 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D01B37D15E; Tue, 31 Jul 2018 21:06:28 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B169618FA1; Tue, 31 Jul 2018 21:06:28 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VL6SpC038342; Tue, 31 Jul 2018 21:06:28 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VL6S6l038341; Tue, 31 Jul 2018 21:06:28 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201807312106.w6VL6S6l038341@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Tue, 31 Jul 2018 21:06:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337009 - head/lib/clang/include/lld/Common X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/lib/clang/include/lld/Common X-SVN-Commit-Revision: 337009 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:06:29 -0000 Author: emaste Date: Tue Jul 31 21:06:28 2018 New Revision: 337009 URL: https://svnweb.freebsd.org/changeset/base/337009 Log: bump lld version number after r336972 arm(v7) VFP tag support Reported by: kevans Sponsored by: The FreeBSD Foundation Modified: head/lib/clang/include/lld/Common/Version.inc Modified: head/lib/clang/include/lld/Common/Version.inc ============================================================================== --- head/lib/clang/include/lld/Common/Version.inc Tue Jul 31 21:06:13 2018 (r337008) +++ head/lib/clang/include/lld/Common/Version.inc Tue Jul 31 21:06:28 2018 (r337009) @@ -7,4 +7,4 @@ #define LLD_REPOSITORY_STRING "FreeBSD" // - -#define LLD_REVISION_STRING "335540-1200003" +#define LLD_REVISION_STRING "335540-1200004" From owner-svn-src-all@freebsd.org Tue Jul 31 21:09:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB36910650A3; Tue, 31 Jul 2018 21:09:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8EDEE7D491; Tue, 31 Jul 2018 21:09:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 51AE918FB1; Tue, 31 Jul 2018 21:09:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VL9En5038588; Tue, 31 Jul 2018 21:09:14 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VL9Eev038587; Tue, 31 Jul 2018 21:09:14 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312109.w6VL9Eev038587@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 21:09:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337011 - head/sys/cddl/contrib/opensolaris X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris X-SVN-Commit-Revision: 337011 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:09:15 -0000 Author: mav Date: Tue Jul 31 21:09:13 2018 New Revision: 337011 URL: https://svnweb.freebsd.org/changeset/base/337011 Log: Record r337002 as already merged, since it came from us. Modified: Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) From owner-svn-src-all@freebsd.org Tue Jul 31 21:26:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AC651065855; Tue, 31 Jul 2018 21:26:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2B4F7E14B; Tue, 31 Jul 2018 21:26:03 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CEF5B19307; Tue, 31 Jul 2018 21:26:03 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VLQ3mM048755; Tue, 31 Jul 2018 21:26:03 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VLQ3qw048754; Tue, 31 Jul 2018 21:26:03 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312126.w6VLQ3qw048754@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 21:26:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337013 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337013 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:26:04 -0000 Author: mav Date: Tue Jul 31 21:26:03 2018 New Revision: 337013 URL: https://svnweb.freebsd.org/changeset/base/337013 Log: 9194 mechanism to override ashift at pool creation time illumos/illumos-gate@93a1902e519bf492c071b48ffb93e5c1c7b53fb9 Reviewed by: Serapheim Dimitropoulos Reviewed by: Prakash Surya Reviewed by: Allan Jude Reviewed by: Igor Kozhukhov Reviewed by: Richard Elling Approved by: Dan McDonald Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Tue Jul 31 21:09:32 2018 (r337012) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Tue Jul 31 21:26:03 2018 (r337013) @@ -96,6 +96,8 @@ int vdev_dtl_sm_blksz = (1 << 12); */ int vdev_standard_sm_blksz = (1 << 17); +int zfs_ashift_min; + /*PRINTFLIKE2*/ void vdev_dbgmsg(vdev_t *vd, const char *fmt, ...) @@ -1511,6 +1513,7 @@ vdev_open(vdev_t *vd) vd->vdev_asize = asize; vd->vdev_max_asize = max_asize; vd->vdev_ashift = MAX(ashift, vd->vdev_ashift); + vd->vdev_ashift = MAX(zfs_ashift_min, vd->vdev_ashift); } else { /* * Detect if the alignment requirement has increased. From owner-svn-src-all@freebsd.org Tue Jul 31 21:31:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 12C091065A6D; Tue, 31 Jul 2018 21:31:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC96F7E4BE; Tue, 31 Jul 2018 21:31:24 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9DC9B1944E; Tue, 31 Jul 2018 21:31:24 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VLVOeX053110; Tue, 31 Jul 2018 21:31:24 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VLVOtP053109; Tue, 31 Jul 2018 21:31:24 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312131.w6VLVOtP053109@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 21:31:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337014 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/zdb vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/zdb vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 337014 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:31:25 -0000 Author: mav Date: Tue Jul 31 21:31:24 2018 New Revision: 337014 URL: https://svnweb.freebsd.org/changeset/base/337014 Log: 9421 zdb should detect and print out the number of "leaked" objects 9422 zfs diff and zdb should explicitly mark objects that are on the deleted queue illumos/illumos-gate@20b5dafb425396adaebd0267d29e1026fc4dc413 Reviewed by: Matt Ahrens Reviewed by: Pavel Zakharov Approved by: Matt Ahrens Author: Paul Dagnelie Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Tue Jul 31 21:26:03 2018 (r337013) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Tue Jul 31 21:31:24 2018 (r337014) @@ -2036,6 +2036,17 @@ zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_h *path = '\0'; sa_hdl = hdl; + uint64_t deleteq_obj; + VERIFY0(zap_lookup(osp, MASTER_NODE_OBJ, + ZFS_UNLINKED_SET, sizeof (uint64_t), 1, &deleteq_obj)); + error = zap_lookup_int(osp, deleteq_obj, obj); + if (error == 0) { + return (ESTALE); + } else if (error != ENOENT) { + return (error); + } + error = 0; + for (;;) { uint64_t pobj; char component[MAXNAMELEN + 2]; From owner-svn-src-all@freebsd.org Tue Jul 31 21:31:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A0561065A71; Tue, 31 Jul 2018 21:31:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3ED497E4C0; Tue, 31 Jul 2018 21:31:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2013B1944F; Tue, 31 Jul 2018 21:31:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VLVPOp053118; Tue, 31 Jul 2018 21:31:25 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VLVOjR053115; Tue, 31 Jul 2018 21:31:24 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312131.w6VLVOjR053115@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 21:31:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337014 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/zdb vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/cmd/zdb vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 337014 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:31:25 -0000 Author: mav Date: Tue Jul 31 21:31:24 2018 New Revision: 337014 URL: https://svnweb.freebsd.org/changeset/base/337014 Log: 9421 zdb should detect and print out the number of "leaked" objects 9422 zfs diff and zdb should explicitly mark objects that are on the deleted queue illumos/illumos-gate@20b5dafb425396adaebd0267d29e1026fc4dc413 Reviewed by: Matt Ahrens Reviewed by: Pavel Zakharov Approved by: Matt Ahrens Author: Paul Dagnelie Modified: vendor/illumos/dist/cmd/zdb/zdb.c vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Tue Jul 31 21:26:03 2018 (r337013) +++ vendor/illumos/dist/cmd/zdb/zdb.c Tue Jul 31 21:31:24 2018 (r337014) @@ -108,6 +108,7 @@ uint64_t *zopt_object = NULL; static unsigned zopt_objects = 0; libzfs_handle_t *g_zfs; uint64_t max_inflight = 1000; +static int leaked_objects = 0; static void snprintf_blkptr_compact(char *, size_t, const blkptr_t *); @@ -1963,9 +1964,12 @@ dump_znode(objset_t *os, uint64_t object, void *data, if (dump_opt['d'] > 4) { error = zfs_obj_to_path(os, object, path, sizeof (path)); - if (error != 0) { + if (error == ESTALE) { + (void) snprintf(path, sizeof (path), "on delete queue"); + } else if (error != 0) { + leaked_objects++; (void) snprintf(path, sizeof (path), - "\?\?\?", (u_longlong_t)object); + "path not found, possibly leaked"); } (void) printf("\tpath %s\n", path); } @@ -2295,6 +2299,11 @@ dump_dir(objset_t *os) (void) fprintf(stderr, "dmu_object_next() = %d\n", error); abort(); } + if (leaked_objects != 0) { + (void) printf("%d potentially leaked objects detected\n", + leaked_objects); + leaked_objects = 0; + } } static void @@ -5364,5 +5373,5 @@ main(int argc, char **argv) libzfs_fini(g_zfs); kernel_fini(); - return (0); + return (error); } Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c Tue Jul 31 21:26:03 2018 (r337013) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_diff.c Tue Jul 31 21:31:24 2018 (r337014) @@ -22,7 +22,7 @@ /* * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2015 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2015 by Delphix. All rights reserved. + * Copyright (c) 2015, 2017 by Delphix. All rights reserved. * Copyright 2016 Joyent, Inc. * Copyright 2016 Igor Kozhukhov */ @@ -103,7 +103,10 @@ get_stats_for_obj(differ_info_t *di, const char *dsnam return (0); } - if (di->zerr == EPERM) { + if (di->zerr == ESTALE) { + (void) snprintf(pn, maxlen, "(on_delete_queue)"); + return (0); + } else if (di->zerr == EPERM) { (void) snprintf(di->errbuf, sizeof (di->errbuf), dgettext(TEXT_DOMAIN, "The sys_config privilege or diff delegated permission " From owner-svn-src-all@freebsd.org Tue Jul 31 21:40:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB2211065CAC; Tue, 31 Jul 2018 21:40:14 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 904F07E989; Tue, 31 Jul 2018 21:40:14 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6DF99194AB; Tue, 31 Jul 2018 21:40:14 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VLeEqY054479; Tue, 31 Jul 2018 21:40:14 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VLeERY054478; Tue, 31 Jul 2018 21:40:14 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201807312140.w6VLeERY054478@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Tue, 31 Jul 2018 21:40:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337015 - head/sys/amd64/conf X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/amd64/conf X-SVN-Commit-Revision: 337015 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:40:15 -0000 Author: markj Date: Tue Jul 31 21:40:13 2018 New Revision: 337015 URL: https://svnweb.freebsd.org/changeset/base/337015 Log: COMPAT_LINUX32 has not depended on COMPAT_43 in some time. MFC after: 3 days Modified: head/sys/amd64/conf/NOTES Modified: head/sys/amd64/conf/NOTES ============================================================================== --- head/sys/amd64/conf/NOTES Tue Jul 31 21:31:24 2018 (r337014) +++ head/sys/amd64/conf/NOTES Tue Jul 31 21:40:13 2018 (r337015) @@ -644,7 +644,7 @@ options COMPAT_CLOUDABI64 # Enable Linux ABI emulation #XXX#options COMPAT_LINUX -# Enable 32-bit Linux ABI emulation (requires COMPAT_43 and COMPAT_FREEBSD32) +# Enable 32-bit Linux ABI emulation (requires COMPAT_FREEBSD32). options COMPAT_LINUX32 # Enable the linux-like proc filesystem support (requires COMPAT_LINUX32 From owner-svn-src-all@freebsd.org Tue Jul 31 21:42:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A0AC1065E7D; Tue, 31 Jul 2018 21:42:32 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 409527F27F; Tue, 31 Jul 2018 21:42:32 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 22CC71962E; Tue, 31 Jul 2018 21:42:32 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VLgWTD059002; Tue, 31 Jul 2018 21:42:32 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VLgW1g059001; Tue, 31 Jul 2018 21:42:32 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312142.w6VLgW1g059001@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 21:42:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337016 - vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 337016 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 21:42:32 -0000 Author: mav Date: Tue Jul 31 21:42:31 2018 New Revision: 337016 URL: https://svnweb.freebsd.org/changeset/base/337016 Log: 9237 "zpool add" fails for very large pools illumos/illumos-gate@ae5ee1bd5e54409b4589217b160c8cf4b81e2fb8 Reviewed by: Matthew Ahrens Reviewed by: Andriy Gapon Approved by: Matt Ahrens Author: Alan Somers Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_import.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Tue Jul 31 21:40:13 2018 (r337015) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Tue Jul 31 21:42:31 2018 (r337016) @@ -895,6 +895,7 @@ zpool_read_label(int fd, nvlist_t **config) free(label); *config = NULL; + errno = ENOENT; return (-1); } From owner-svn-src-all@freebsd.org Tue Jul 31 22:18:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 697ED1066AFF; Tue, 31 Jul 2018 22:18:42 +0000 (UTC) (envelope-from danfe@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24A70809AB; Tue, 31 Jul 2018 22:18:42 +0000 (UTC) (envelope-from danfe@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1033) id 1D05C1C525; Tue, 31 Jul 2018 22:18:42 +0000 (UTC) Date: Tue, 31 Jul 2018 22:18:42 +0000 From: Alexey Dokuchaev To: Emmanuel Vadot Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r337004 - head/sys/dev/ofw Message-ID: <20180731221842.GA79074@FreeBSD.org> References: <201807312050.w6VKoopE027843@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201807312050.w6VKoopE027843@repo.freebsd.org> User-Agent: Mutt/1.9.5 (2018-04-13) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 22:18:42 -0000 On Tue, Jul 31, 2018 at 08:50:50PM +0000, Emmanuel Vadot wrote: > New Revision: 337004 > URL: https://svnweb.freebsd.org/changeset/base/337004 > > Log: > ofw_cpu: Add support for getting cpu clock via clock property > > Nominal Mhz is either expressed via the clock-frequency property > ... > +#ifdef EXT_RESOURCES > + rv = clk_get_by_ofw_index(dev, 0, 0, &cpuclk); > + if (rv == 0) { > + rv = clk_get_freq(cpuclk, &freq); > + if (rv != 0 && bootverbose) > + device_printf(dev, > + "Cannot get freq of property clocks\n"); Why informal "freq" contraction, esp. when it's properly spelled out few lines below? > + if (bootverbose) > + device_printf(dev, > + "missing 'clock-frequency' property\n"); > } else > sc->sc_nominal_mhz = cell / 1000000; /* convert to MHz */ > > + if (sc->sc_nominal_mhz != 0 && bootverbose) > + device_printf(dev, "Nominal frequency %dMhz\n", I think correct spelling is MHz, not Mhz. Notice that comment above uses this spelling. ./danfe From owner-svn-src-all@freebsd.org Tue Jul 31 22:50:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F6E51067613; Tue, 31 Jul 2018 22:50:51 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5410381DB3; Tue, 31 Jul 2018 22:50:51 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 36B9919FF3; Tue, 31 Jul 2018 22:50:51 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VMopGS090508; Tue, 31 Jul 2018 22:50:51 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VMooxZ090505; Tue, 31 Jul 2018 22:50:50 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312250.w6VMooxZ090505@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 22:50:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337017 - in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337017 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 22:50:51 -0000 Author: mav Date: Tue Jul 31 22:50:50 2018 New Revision: 337017 URL: https://svnweb.freebsd.org/changeset/base/337017 Log: MFV r337014: 9421 zdb should detect and print out the number of "leaked" objects 9422 zfs diff and zdb should explicitly mark objects that are on the deleted queue illumos/illumos-gate@20b5dafb425396adaebd0267d29e1026fc4dc413 Reviewed by: Matt Ahrens Reviewed by: Pavel Zakharov Approved by: Matt Ahrens Author: Paul Dagnelie Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_diff.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/cddl/contrib/opensolaris/cmd/zdb/ (props changed) head/cddl/contrib/opensolaris/lib/libzfs/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Tue Jul 31 21:42:31 2018 (r337016) +++ head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Tue Jul 31 22:50:50 2018 (r337017) @@ -108,6 +108,7 @@ static uint64_t *zopt_object = NULL; static unsigned zopt_objects = 0; static libzfs_handle_t *g_zfs; static uint64_t max_inflight = 1000; +static int leaked_objects = 0; static void snprintf_blkptr_compact(char *, size_t, const blkptr_t *); @@ -1988,9 +1989,12 @@ dump_znode(objset_t *os, uint64_t object, void *data, if (dump_opt['d'] > 4) { error = zfs_obj_to_path(os, object, path, sizeof (path)); - if (error != 0) { + if (error == ESTALE) { + (void) snprintf(path, sizeof (path), "on delete queue"); + } else if (error != 0) { + leaked_objects++; (void) snprintf(path, sizeof (path), - "\?\?\?", (u_longlong_t)object); + "path not found, possibly leaked"); } (void) printf("\tpath %s\n", path); } @@ -2320,6 +2324,12 @@ dump_dir(objset_t *os) } ASSERT3U(object_count, ==, usedobjs); + + if (leaked_objects != 0) { + (void) printf("%d potentially leaked objects detected\n", + leaked_objects); + leaked_objects = 0; + } } static void @@ -5405,5 +5415,5 @@ main(int argc, char **argv) libzfs_fini(g_zfs); kernel_fini(); - return (0); + return (error); } Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_diff.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_diff.c Tue Jul 31 21:42:31 2018 (r337016) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_diff.c Tue Jul 31 22:50:50 2018 (r337017) @@ -22,7 +22,7 @@ /* * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2015 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2015 by Delphix. All rights reserved. + * Copyright (c) 2015, 2017 by Delphix. All rights reserved. * Copyright 2016 Joyent, Inc. * Copyright 2016 Igor Kozhukhov */ @@ -101,7 +101,10 @@ get_stats_for_obj(differ_info_t *di, const char *dsnam return (0); } - if (di->zerr == EPERM) { + if (di->zerr == ESTALE) { + (void) snprintf(pn, maxlen, "(on_delete_queue)"); + return (0); + } else if (di->zerr == EPERM) { (void) snprintf(di->errbuf, sizeof (di->errbuf), dgettext(TEXT_DOMAIN, "The sys_config privilege or diff delegated permission " Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c Tue Jul 31 21:42:31 2018 (r337016) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c Tue Jul 31 22:50:50 2018 (r337017) @@ -2102,6 +2102,17 @@ zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_h *path = '\0'; sa_hdl = hdl; + uint64_t deleteq_obj; + VERIFY0(zap_lookup(osp, MASTER_NODE_OBJ, + ZFS_UNLINKED_SET, sizeof (uint64_t), 1, &deleteq_obj)); + error = zap_lookup_int(osp, deleteq_obj, obj); + if (error == 0) { + return (ESTALE); + } else if (error != ENOENT) { + return (error); + } + error = 0; + for (;;) { uint64_t pobj; char component[MAXNAMELEN + 2]; From owner-svn-src-all@freebsd.org Tue Jul 31 22:56:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76BA3106788C; Tue, 31 Jul 2018 22:56:09 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1BDB182113; Tue, 31 Jul 2018 22:56:09 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E26651A16A; Tue, 31 Jul 2018 22:56:08 +0000 (UTC) (envelope-from tuexen@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VMu8fM095211; Tue, 31 Jul 2018 22:56:08 GMT (envelope-from tuexen@FreeBSD.org) Received: (from tuexen@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VMu4SH095187; Tue, 31 Jul 2018 22:56:04 GMT (envelope-from tuexen@FreeBSD.org) Message-Id: <201807312256.w6VMu4SH095187@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tuexen set sender to tuexen@FreeBSD.org using -f From: Michael Tuexen Date: Tue, 31 Jul 2018 22:56:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337018 - in head: cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip cddl/lib/libdtrace cddl/usr.sbin/dtrace/tests/common/ip cddl/usr.sbin/dtrace/tests/tools cddl/usr.sbin/dwatch/l... X-SVN-Group: head X-SVN-Commit-Author: tuexen X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip cddl/lib/libdtrace cddl/usr.sbin/dtrace/tests/common/ip cddl/usr.sbin/dtrace/tests/tools cddl/usr.sbin/dwatch/libexec share/man/man4 sys... X-SVN-Commit-Revision: 337018 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 22:56:09 -0000 Author: tuexen Date: Tue Jul 31 22:56:03 2018 New Revision: 337018 URL: https://svnweb.freebsd.org/changeset/base/337018 Log: Add a dtrace provider for UDP-Lite. The dtrace provider for UDP-Lite is modeled after the UDP provider. This fixes the bug that UDP-Lite packets were triggering the UDP provider. Thanks to dteske@ for providing the dwatch module. Reviewed by: dteske@, markj@, rrs@ Relnotes: yes Differential Revision: https://reviews.freebsd.org/D16377 Added: head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4localudplite.ksh (contents, props changed) head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4localudplite.ksh.out head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4remoteudplite.ksh (contents, props changed) head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4remoteudplite.ksh.out head/cddl/lib/libdtrace/udplite.d (contents, props changed) head/cddl/usr.sbin/dwatch/libexec/udplite (contents, props changed) head/share/man/man4/dtrace_udplite.4 (contents, props changed) Modified: head/cddl/lib/libdtrace/Makefile head/cddl/lib/libdtrace/ip.d head/cddl/usr.sbin/dtrace/tests/common/ip/Makefile head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh head/cddl/usr.sbin/dwatch/libexec/Makefile head/share/man/man4/Makefile head/share/man/man4/dtrace_sctp.4 head/share/man/man4/dtrace_tcp.4 head/share/man/man4/dtrace_udp.4 head/sys/netinet/in_kdtrace.c head/sys/netinet/in_kdtrace.h head/sys/netinet/udp_usrreq.c head/sys/netinet/udplite.h head/sys/netinet6/udp6_usrreq.c Added: head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4localudplite.ksh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4localudplite.ksh Tue Jul 31 22:56:03 2018 (r337018) @@ -0,0 +1,125 @@ +#!/usr/bin/env ksh +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or http://www.opensolaris.org/os/licensing. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. +# + +# +# Test {ip,udplite}:::{send,receive} of IPv4 UDP-Lite to a local address. +# +# This may fail due to: +# +# 1. A change to the ip stack breaking expected probe behavior, +# which is the reason we are testing. +# 2. No physical network interface is plumbed and up. +# 3. No other hosts on this subnet are reachable and listening on rpcbind. +# 4. An unlikely race causes the unlocked global send/receive +# variables to be corrupted. +# +# This test sends a UDP-Lite message using perl and checks that at least the +# following counts were traced: +# +# 1 x ip:::send (UDPLite sent to UDP-Lite port 33434) +# 1 x udplite:::send (UDPLite sent to UDP-Lite port 33434) +# 1 x ip:::receive (UDP-Lite received) +# 1 x udplite:::receive (UDP-Lite received) +# +# A udplite:::receive event is expected even if the received UDP-Lite packet +# elicits an ICMP PORT_UNREACHABLE message since there is no UDP-Lite +# socket for receiving the packet. +# + +if (( $# != 1 )); then + print -u2 "expected one argument: " + exit 2 +fi + +dtrace=$1 +local=127.0.0.1 +port=33434 +DIR=/var/tmp/dtest.$$ + +mkdir $DIR +cd $DIR + +cat > test.pl <<-EOPERL + use IO::Socket; + my \$s = IO::Socket::INET->new( + Type => SOCK_DGRAM, + Proto => "udplite", + PeerAddr => "$local", + PeerPort => $port); + die "Could not create UDP-Lite socket $local port $port" unless \$s; + send \$s, "Hello", 0; + close \$s; + sleep(2); +EOPERL + +$dtrace -c 'perl test.pl' -qs /dev/stdin <ip_saddr == "$local" && args[2]->ip_daddr == "$local" && + args[4]->ipv4_protocol == IPPROTO_UDPLITE/ +{ + ipsend++; +} + +udplite:::send +/args[2]->ip_saddr == "$local" && args[2]->ip_daddr == "$local"/ +{ + udplitesend++; +} + +ip:::receive +/args[2]->ip_saddr == "$local" && args[2]->ip_daddr == "$local" && + args[4]->ipv4_protocol == IPPROTO_UDPLITE/ +{ + ipreceive++; +} + +udplite:::receive +/args[2]->ip_saddr == "$local" && args[2]->ip_daddr == "$local"/ +{ + udplitereceive++; +} + +END +{ + printf("Minimum UDP-Lite events seen\n\n"); + printf("ip:::send - %s\n", ipsend >= 1 ? "yes" : "no"); + printf("ip:::receive - %s\n", ipreceive >= 1 ? "yes" : "no"); + printf("udplite:::send - %s\n", udplitesend >= 1 ? "yes" : "no"); + printf("udplite:::receive - %s\n", udplitereceive >= 1 ? "yes" : "no"); +} +EODTRACE + +status=$? + +cd / +/bin/rm -rf $DIR + +exit $status Added: head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4localudplite.ksh.out ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4localudplite.ksh.out Tue Jul 31 22:56:03 2018 (r337018) @@ -0,0 +1,7 @@ +Minimum UDP-Lite events seen + +ip:::send - yes +ip:::receive - yes +udplite:::send - yes +udplite:::receive - yes + Added: head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4remoteudplite.ksh ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4remoteudplite.ksh Tue Jul 31 22:56:03 2018 (r337018) @@ -0,0 +1,113 @@ +#!/usr/bin/env ksh93 +# +# CDDL HEADER START +# +# The contents of this file are subject to the terms of the +# Common Development and Distribution License (the "License"). +# You may not use this file except in compliance with the License. +# +# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE +# or http://www.opensolaris.org/os/licensing. +# See the License for the specific language governing permissions +# and limitations under the License. +# +# When distributing Covered Code, include this CDDL HEADER in each +# file and include the License file at usr/src/OPENSOLARIS.LICENSE. +# If applicable, add the following below this CDDL HEADER, with the +# fields enclosed by brackets "[]" replaced with your own identifying +# information: Portions Copyright [yyyy] [name of copyright owner] +# +# CDDL HEADER END +# + +# +# Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. +# + +# +# Test {udplite,ip}:::{send,receive} of IPv4 UDP-Lite to a remote host. +# +# This may fail due to: +# +# 1. A change to the ip stack breaking expected probe behavior, +# which is the reason we are testing. +# 2. No physical network interface is plumbed and up. +# 3. No other hosts on this subnet are reachable and listening on rpcbind. +# 4. An unlikely race causes the unlocked global send/receive +# variables to be corrupted. +# +# This test sends a UDP-Lite message using perl and checks that at least the +# following counts were traced: +# +# 1 x ip:::send (UDP-Lite sent to UDP-Lite port 33434) +# 1 x udplite:::send (UDP-Lite sent to UDP-Lite port 33434) +# + +if (( $# != 1 )); then + print -u2 "expected one argument: " + exit 2 +fi + +dtrace=$1 +getaddr=./get.ipv4remote.pl +port=33434 +DIR=/var/tmp/dtest.$$ + +if [[ ! -x $getaddr ]]; then + print -u2 "could not find or execute sub program: $getaddr" + exit 3 +fi +$getaddr | read source dest +if (( $? != 0 )); then + exit 4 +fi + +mkdir $DIR +cd $DIR + +cat > test.pl <<-EOPERL + use IO::Socket; + my \$s = IO::Socket::INET->new( + Type => SOCK_DGRAM, + Proto => "udplite", + PeerAddr => "$dest", + PeerPort => $port); + die "Could not create UDP-Lite socket $dest port $port" unless \$s; + send \$s, "Hello", 0; + close \$s; + sleep(2); +EOPERL + +$dtrace -c 'perl test.pl' -qs /dev/stdin <ip_saddr == "$source" && args[2]->ip_daddr == "$dest" && + args[4]->ipv4_protocol == IPPROTO_UDPLITE/ +{ + ipsend++; +} + +udplite:::send +/args[2]->ip_saddr == "$source" && args[2]->ip_daddr == "$dest"/ +{ + udplitesend++; +} + +END +{ + printf("Minimum UDPLite events seen\n\n"); + printf("ip:::send - %s\n", ipsend >= 1 ? "yes" : "no"); + printf("udplite:::send - %s\n", udplitesend >= 1 ? "yes" : "no"); +} +EODTRACE + +status=$? + +cd / +/bin/rm -rf $DIR + +exit $status Added: head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4remoteudplite.ksh.out ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/ip/tst.ipv4remoteudplite.ksh.out Tue Jul 31 22:56:03 2018 (r337018) @@ -0,0 +1,5 @@ +Minimum UDP-Lite events seen + +ip:::send - yes +udplite:::send - yes + Modified: head/cddl/lib/libdtrace/Makefile ============================================================================== --- head/cddl/lib/libdtrace/Makefile Tue Jul 31 22:50:50 2018 (r337017) +++ head/cddl/lib/libdtrace/Makefile Tue Jul 31 22:56:03 2018 (r337018) @@ -56,6 +56,7 @@ DSRCS= errno.d \ tcp.d \ socket.d \ udp.d \ + udplite.d \ unistd.d FILES= ${DSRCS} Modified: head/cddl/lib/libdtrace/ip.d ============================================================================== --- head/cddl/lib/libdtrace/ip.d Tue Jul 31 22:50:50 2018 (r337017) +++ head/cddl/lib/libdtrace/ip.d Tue Jul 31 22:56:03 2018 (r337018) @@ -167,6 +167,8 @@ inline short IPPROTO_IPCOMP = 108; inline short IPPROTO_SCTP = 132; #pragma D binding "1.5" IPPROTO_RAW inline short IPPROTO_RAW = 255; +#pragma D binding "1.13" IPPROTO_UDPLITE +inline short IPPROTO_UDPLITE = 136; inline uint8_t INP_IPV4 = 0x01; inline uint8_t INP_IPV6 = 0x02; @@ -193,6 +195,7 @@ inline string protocols[int proto] = proto == IPPROTO_PIM ? "PIM" : proto == IPPROTO_IPCOMP ? "IPCOMP" : proto == IPPROTO_SCTP ? "SCTP" : + proto == IPPROTO_UDPLITE ? "UDPLITE" : proto == IPPROTO_RAW ? "RAW" : ""; Added: head/cddl/lib/libdtrace/udplite.d ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/cddl/lib/libdtrace/udplite.d Tue Jul 31 22:56:03 2018 (r337018) @@ -0,0 +1,77 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + * + * $FreeBSD$ + */ +/* + * Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2013 Mark Johnston + * Copyright (c) 2018 Michael Tuexen + */ + +#pragma D depends_on library ip.d +#pragma D depends_on module kernel +#pragma D depends_on provider udplite + +/* + * udplitesinfo contains stable UDPLite details. + */ +typedef struct udplitesinfo { + uintptr_t udplites_addr; + uint16_t udplites_lport; /* local port */ + uint16_t udplites_rport; /* remote port */ + string udplites_laddr; /* local address, as a string */ + string udplites_raddr; /* remote address, as a string */ +} udplitesinfo_t; + +/* + * udpliteinfo is the UDPLite header fields. + */ +typedef struct udpliteinfo { + uint16_t udplite_sport; /* source port */ + uint16_t udplite_dport; /* destination port */ + uint16_t udplite_coverage; /* checksum coverage */ + uint16_t udplite_checksum; /* headers + data checksum */ + struct udplitehdr *udplite_hdr; /* raw UDPLite header */ +} udpliteinfo_t; + +#pragma D binding "1.13" translator +translator udplitesinfo_t < struct inpcb *p > { + udplites_addr = (uintptr_t)p; + udplites_lport = p == NULL ? 0 : ntohs(p->inp_inc.inc_ie.ie_lport); + udplites_rport = p == NULL ? 0 : ntohs(p->inp_inc.inc_ie.ie_fport); + udplites_laddr = p == NULL ? "" : + p->inp_vflag == INP_IPV4 ? + inet_ntoa(&p->inp_inc.inc_ie.ie_dependladdr.id46_addr.ia46_addr4.s_addr) : + inet_ntoa6(&p->inp_inc.inc_ie.ie_dependladdr.id6_addr); + udplites_raddr = p == NULL ? "" : + p->inp_vflag == INP_IPV4 ? + inet_ntoa(&p->inp_inc.inc_ie.ie_dependfaddr.id46_addr.ia46_addr4.s_addr) : + inet_ntoa6(&p->inp_inc.inc_ie.ie_dependfaddr.id6_addr); +}; + +#pragma D binding "1.13" translator +translator udpliteinfo_t < struct udphdr *p > { + udplite_sport = p == NULL ? 0 : ntohs(p->uh_sport); + udplite_dport = p == NULL ? 0 : ntohs(p->uh_dport); + udplite_coverage = p == NULL ? 0 : ntohs(p->uh_ulen); + udplite_checksum = p == NULL ? 0 : ntohs(p->uh_sum); + udplite_hdr = (struct udplitehdr *)p; +}; Modified: head/cddl/usr.sbin/dtrace/tests/common/ip/Makefile ============================================================================== --- head/cddl/usr.sbin/dtrace/tests/common/ip/Makefile Tue Jul 31 22:50:50 2018 (r337017) +++ head/cddl/usr.sbin/dtrace/tests/common/ip/Makefile Tue Jul 31 22:56:03 2018 (r337018) @@ -13,12 +13,16 @@ ${PACKAGE}FILES= \ tst.ipv4localtcp.ksh.out \ tst.ipv4localudp.ksh \ tst.ipv4localudp.ksh.out \ + tst.ipv4localudplite.ksh \ + tst.ipv4localudplite.ksh.out \ tst.ipv4remoteicmp.ksh \ tst.ipv4remoteicmp.ksh.out \ tst.ipv4remotetcp.ksh \ tst.ipv4remotetcp.ksh.out \ tst.ipv4remoteudp.ksh \ tst.ipv4remoteudp.ksh.out \ + tst.ipv4remoteudplite.ksh \ + tst.ipv4remoteudplite.ksh.out \ tst.ipv6localicmp.ksh \ tst.ipv6localicmp.ksh.out \ tst.ipv6remoteicmp.ksh \ Modified: head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh ============================================================================== --- head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh Tue Jul 31 22:50:50 2018 (r337017) +++ head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh Tue Jul 31 22:56:03 2018 (r337018) @@ -119,6 +119,7 @@ exclude SKIP common/builtinvar/tst.ipl1.d # These tests rely on being able to find a host via broadcast pings. exclude EXFAIL common/ip/tst.ipv4remotetcp.ksh exclude EXFAIL common/ip/tst.ipv4remoteudp.ksh +exclude EXFAIL common/ip/tst.ipv4remoteudplite.ksh exclude EXFAIL common/ip/tst.ipv6remoteicmp.ksh exclude EXFAIL common/ip/tst.ipv4remoteicmp.ksh exclude EXFAIL common/ip/tst.remotetcpstate.ksh Modified: head/cddl/usr.sbin/dwatch/libexec/Makefile ============================================================================== --- head/cddl/usr.sbin/dwatch/libexec/Makefile Tue Jul 31 22:50:50 2018 (r337017) +++ head/cddl/usr.sbin/dwatch/libexec/Makefile Tue Jul 31 22:56:03 2018 (r337018) @@ -14,6 +14,7 @@ FILES= chmod \ sendrecv \ tcp \ udp \ + udplite \ vop_create \ vop_readdir \ vop_rename \ @@ -78,6 +79,8 @@ LINKS+= ${LIBEXECDIR}/dwatch/tcp ${LIBEXECDIR}/dwatch/ LINKS+= ${LIBEXECDIR}/dwatch/tcp ${LIBEXECDIR}/dwatch/tcp-status LINKS+= ${LIBEXECDIR}/dwatch/udp ${LIBEXECDIR}/dwatch/udp-receive LINKS+= ${LIBEXECDIR}/dwatch/udp ${LIBEXECDIR}/dwatch/udp-send +LINKS+= ${LIBEXECDIR}/dwatch/udplite ${LIBEXECDIR}/dwatch/udplite-receive +LINKS+= ${LIBEXECDIR}/dwatch/udplite ${LIBEXECDIR}/dwatch/udplite-send LINKS+= ${LIBEXECDIR}/dwatch/vop_create ${LIBEXECDIR}/dwatch/vop_lookup LINKS+= ${LIBEXECDIR}/dwatch/vop_create ${LIBEXECDIR}/dwatch/vop_mkdir LINKS+= ${LIBEXECDIR}/dwatch/vop_create ${LIBEXECDIR}/dwatch/vop_mknod Added: head/cddl/usr.sbin/dwatch/libexec/udplite ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/cddl/usr.sbin/dwatch/libexec/udplite Tue Jul 31 22:56:03 2018 (r337018) @@ -0,0 +1,89 @@ +# -*- tab-width: 4 -*- ;; Emacs +# vi: set filetype=sh tabstop=8 shiftwidth=8 noexpandtab :: Vi/ViM +############################################################ IDENT(1) +# +# $Title: dwatch(8) module for dtrace_udplite(4) $ +# $Copyright: 2014-2018 Devin Teske. All rights reserved. $ +# $FreeBSD$ +# +############################################################ DESCRIPTION +# +# Display local/remote UDP addresses/ports and bytes sent/received for UDP I/O +# +############################################################ PROBE + +case "$PROFILE" in +udplite) : ${PROBE:=udplite:::send, udplite:::receive} ;; + *) : ${PROBE:=udplite:::${PROFILE#udplite-}} +esac + +############################################################ ACTIONS + +exec 9<"); +} + /* + * dtrace_udplite(4) + */ + this->recv = probename == "receive" ? 1 : 0; + this->flow = this->recv ? "<-" : "->"; + + /* + * ipinfo_t * + */ + this->local = this->recv ? args[2]->ip_daddr : args[2]->ip_saddr; + this->remote = this->recv ? args[2]->ip_saddr : args[2]->ip_daddr; + + /* + * udpliteinfo_t * + */ + this->coverage = (uint16_t)args[4]->udplite_coverage; + this->lport = this->recv ? args[4]->udplite_dport : args[4]->udplite_sport; + this->rport = this->recv ? args[4]->udplite_sport : args[4]->udplite_dport; + + /* + * IPv6 support + */ + this->local6 = strstr(this->local, ":") != NULL ? 1 : 0; + this->remote6 = strstr(this->remote, ":") != NULL ? 1 : 0; + this->local = strjoin(strjoin(this->local6 ? "[" : "", + this->local), this->local6 ? "]" : ""); + this->remote = strjoin(strjoin(this->remote6 ? "[" : "", + this->remote), this->remote6 ? "]" : ""); +} +EOF +ACTIONS=$( cat <&9 ) +ID=$(( $ID + 1 )) + +############################################################ EVENT DETAILS + +if [ ! "$CUSTOM_DETAILS" ]; then +exec 9<local, this->lport, + this->flow, + this->remote, this->rport, + this->coverage, + this->coverage == 1 ? "" : "s"); +EOF +EVENT_DETAILS=$( cat <&9 ) +fi + +################################################################################ +# END +################################################################################ Modified: head/share/man/man4/Makefile ============================================================================== --- head/share/man/man4/Makefile Tue Jul 31 22:50:50 2018 (r337017) +++ head/share/man/man4/Makefile Tue Jul 31 22:56:03 2018 (r337018) @@ -897,7 +897,8 @@ _dtrace_provs= dtrace_io.4 \ dtrace_sched.4 \ dtrace_sctp.4 \ dtrace_tcp.4 \ - dtrace_udp.4 + dtrace_udp.4 \ + dtrace_udplite.4 .endif .if ${MK_ISCSI} != "no" Modified: head/share/man/man4/dtrace_sctp.4 ============================================================================== --- head/share/man/man4/dtrace_sctp.4 Tue Jul 31 22:50:50 2018 (r337017) +++ head/share/man/man4/dtrace_sctp.4 Tue Jul 31 22:56:03 2018 (r337018) @@ -23,7 +23,7 @@ .\" .\" $FreeBSD$ .\" -.Dd April 22, 2018 +.Dd August 1, 2018 .Dt DTRACE_SCTP 4 .Os .Sh NAME @@ -117,6 +117,7 @@ handshake. .Xr dtrace 1 , .Xr dtrace_ip 4 , .Xr dtrace_udp 4 , +.Xr dtrace_udplite 4 , .Xr sctp 4 , .Xr SDT 9 .\" .Sh HISTORY Modified: head/share/man/man4/dtrace_tcp.4 ============================================================================== --- head/share/man/man4/dtrace_tcp.4 Tue Jul 31 22:50:50 2018 (r337017) +++ head/share/man/man4/dtrace_tcp.4 Tue Jul 31 22:56:03 2018 (r337018) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd February 10, 2018 +.Dd August 1, 2018 .Dt DTRACE_TCP 4 .Os .Sh NAME @@ -374,7 +374,9 @@ provider in Solaris. .Sh SEE ALSO .Xr dtrace 1 , .Xr dtrace_ip 4 , +.Xr dtrace_sctp 4 , .Xr dtrace_udp 4 , +.Xr dtrace_udplite 4 , .Xr tcp 4 , .Xr SDT 9 .Sh HISTORY Modified: head/share/man/man4/dtrace_udp.4 ============================================================================== --- head/share/man/man4/dtrace_udp.4 Tue Jul 31 22:50:50 2018 (r337017) +++ head/share/man/man4/dtrace_udp.4 Tue Jul 31 22:56:03 2018 (r337018) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 20, 2018 +.Dd August 1, 2018 .Dt DTRACE_UDP 4 .Os .Sh NAME @@ -189,7 +189,9 @@ provider in Solaris. .Sh SEE ALSO .Xr dtrace 1 , .Xr dtrace_ip 4 , +.Xr dtrace_sctp 4 , .Xr dtrace_tcp 4 , +.Xr dtrace_udplite 4 , .Xr udp 4 , .Xr SDT 9 .Sh HISTORY Added: head/share/man/man4/dtrace_udplite.4 ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/man/man4/dtrace_udplite.4 Tue Jul 31 22:56:03 2018 (r337018) @@ -0,0 +1,204 @@ +.\" Copyright (c) 2015 Mark Johnston +.\" Copyright (c) 2018 Michael Tuexen +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.Dd August 1, 2018 +.Dt DTRACE_UDPLITE 4 +.Os +.Sh NAME +.Nm dtrace_udplite +.Nd a DTrace provider for tracing events related to the UDP-Lite protocol +.Sh SYNOPSIS +.Fn udplite:::receive "pktinfo_t *" "csinfo_t *" "ipinfo_t *" "udplitesinfo_t *" \ + "udpliteinfo_t *" +.Fn udplite:::send "pktinfo_t *" "csinfo_t *" "ipinfo_t *" "udplitesinfo_t *" \ + "udpliteinfo_t *" +.Sh DESCRIPTION +The DTrace +.Nm udplite +provider allows users to trace events in the +.Xr udplite 4 +protocol implementation. +The +.Fn udplite:::send +probe fires whenever the kernel prepares to transmit a UDP-Lite packet, and the +.Fn udplite:::receive +probe fires whenever the kernel receives a UDP-Lite packet, unless +the UDP-Lite header is incomplete, +the destination port is 0, +the length field is invalid, +or the checksum is wrong. +The arguments to these probes can be used to obtain detailed information about +the IP and UDP-Lite headers of the corresponding packet. +.Sh ARGUMENTS +The +.Vt pktinfo_t +argument is currently unimplemented and is included for compatibility with other +implementations of this provider. +Its fields are: +.Bl -tag -width "uintptr_t pkt_addr" -offset indent +.It Vt uintptr_t pkt_addr +Always set to 0. +.El +.Pp +The +.Vt csinfo_t +argument is currently unimplemented and is included for compatibility with other +implementations of this provider. +Its fields are: +.Bl -tag -width "uintptr_t cs_addr" -offset indent +.It Vt uintptr_t cs_addr +Always set to 0. +.It Vt uint64_t cs_cid +A pointer to the +.Vt struct inpcb +for this packet, or +.Dv NULL . +.It Vt pid_t cs_pid +Always set to 0. +.El +.Pp +The +.Vt ipinfo_t +argument contains IP fields common to both IPv4 and IPv6 packets. +Its fields are: +.Bl -tag -width "uint32_t ip_plength" -offset indent +.It Vt uint8_t ip_ver +IP version of the packet, 4 for IPv4 packets and 6 for IPv6 packets. +.It Vt uint32_t ip_plength +IP payload size. +This does not include the size of the IP header or IPv6 option headers. +.It Vt string ip_saddr +IP source address. +.It Vt string ip_daddr +IP destination address. +.El +.Pp +The +.Vt udplitesinfo_t +argument contains the state of the UDP-Lite connection associated with the packet. +Its fields are: +.Bl -tag -width "uintptr_t udplites_addr" -offset indent +.It Vt uintptr_t udplites_addr +Pointer to the +.Vt struct inpcb +containing the IP state for the associated socket. +.It Vt uint16_t udplites_lport +Local UDP-Lite port. +.It Vt uint16_t udplites_rport +Remote UDP-Lite port. +.It Vt string udplites_laddr +Local IPv4 or IPv6 address. +.It Vt string udplites_raddr +Remote IPv4 or IPv6 address. +.El +.Pp +The +.Vt udpliteinfo_t +argument is the raw UDP-Lite header of the packet, with all fields in host order. +Its fields are: +.Bl -tag -width "struct udplitehdr *udplite_hdr" -offset indent +.It Vt uint16_t udplite_sport +Source UDP-Lite port. +.It Vt uint16_t udplite_dport +Destination UDP-Lite port. +.It Vt uint16_t udplite_coverage +Checksum coverage of the UDP-Lite header, in bytes, or 0 for full coverage. +.It Vt uint16_t udplite_checksum +A checksum of the UDP-Lite header and payload, or 0 if no checksum was calculated. +.It Vt struct udplitehdr *udplite_hdr +A pointer to the raw UDP-Lite header. +.El +.Sh FILES +.Bl -tag -width "/usr/lib/dtrace/udplite.d" -compact +.It Pa /usr/lib/dtrace/udplite.d +DTrace type and translator definitions for the +.Nm udplite +provider. +.El +.Sh EXAMPLES +The following script counts transmitted packets by destination port. +.Bd -literal -offset indent +udplite:::send +{ + @num[args[4]->udplite_dport] = count(); +} +.Ed +.Pp +This script will print some details of each UDP-Lite packet as it is sent or received +by the kernel: +.Bd -literal -offset indent +#pragma D option quiet +#pragma D option switchrate=10Hz + +dtrace:::BEGIN +{ + printf(" %10s %36s %-36s %6s\\n", "DELTA(us)", "SOURCE", + "DEST", "COV"); + last = timestamp; +} + +udplite:::send +{ + this->elapsed = (timestamp - last) / 1000; + self->dest = strjoin(strjoin(args[2]->ip_daddr, ":"), + lltostr(args[4]->udplite_dport)); + printf(" %10d %30s:%-5d -> %-36s %6d\\n", this->elapsed, + args[2]->ip_saddr, args[4]->udplite_sport, + self->dest, args[4]->udplite_coverage); + last = timestamp; +} + +udplite:::receive +{ + this->elapsed = (timestamp - last) / 1000; + self->dest = strjoin(strjoin(args[2]->ip_saddr, ":"), + lltostr(args[4]->udplite_sport)); + printf(" %10d %30s:%-5d <- %-36s %6d\\n", this->elapsed, + args[2]->ip_daddr, args[4]->udplite_dport, + self->dest, args[4]->udplite_coverage); + last = timestamp; +} +.Ed +.Sh SEE ALSO +.Xr dtrace 1 , +.Xr dtrace_ip 4 , +.Xr dtrace_sctp 4 , +.Xr dtrace_tcp 4 , +.Xr dtrace_udp 4 , +.Xr udplite 4 , +.Xr SDT 9 +.Sh HISTORY +The +.Nm udplite +provider first appeared in +.Fx +12.0. +.Sh AUTHORS +This manual page was written by +.An Mark Johnston Aq Mt markj@FreeBSD.org +and +.An Michael Tuexen Aq Mt tuexen@FreeBSD.org . Modified: head/sys/netinet/in_kdtrace.c ============================================================================== --- head/sys/netinet/in_kdtrace.c Tue Jul 31 22:50:50 2018 (r337017) +++ head/sys/netinet/in_kdtrace.c Tue Jul 31 22:56:03 2018 (r337018) @@ -38,6 +38,7 @@ __FBSDID("$FreeBSD$"); SDT_PROVIDER_DEFINE(ip); SDT_PROVIDER_DEFINE(tcp); SDT_PROVIDER_DEFINE(udp); +SDT_PROVIDER_DEFINE(udplite); SDT_PROBE_DEFINE6_XLATE(ip, , , receive, "void *", "pktinfo_t *", @@ -155,3 +156,17 @@ SDT_PROBE_DEFINE5_XLATE(udp, , , send, "uint8_t *", "ipinfo_t *", "struct inpcb *", "udpsinfo_t *", "struct udphdr *", "udpinfo_t *"); + +SDT_PROBE_DEFINE5_XLATE(udplite, , , receive, + "void *", "pktinfo_t *", + "struct inpcb *", "csinfo_t *", + "uint8_t *", "ipinfo_t *", + "struct inpcb *", "udplitesinfo_t *", + "struct udphdr *", "udpliteinfo_t *"); + +SDT_PROBE_DEFINE5_XLATE(udplite, , , send, + "void *", "pktinfo_t *", + "struct inpcb *", "csinfo_t *", + "uint8_t *", "ipinfo_t *", + "struct inpcb *", "udplitesinfo_t *", + "struct udphdr *", "udpliteinfo_t *"); Modified: head/sys/netinet/in_kdtrace.h ============================================================================== --- head/sys/netinet/in_kdtrace.h Tue Jul 31 22:50:50 2018 (r337017) +++ head/sys/netinet/in_kdtrace.h Tue Jul 31 22:56:03 2018 (r337018) @@ -34,6 +34,8 @@ SDT_PROBE6(ip, , , probe, arg0, arg1, arg2, arg3, arg4, arg5) #define UDP_PROBE(probe, arg0, arg1, arg2, arg3, arg4) \ SDT_PROBE5(udp, , , probe, arg0, arg1, arg2, arg3, arg4) +#define UDPLITE_PROBE(probe, arg0, arg1, arg2, arg3, arg4) \ + SDT_PROBE5(udplite, , , probe, arg0, arg1, arg2, arg3, arg4) #define TCP_PROBE1(probe, arg0) \ SDT_PROBE1(tcp, , , probe, arg0) #define TCP_PROBE2(probe, arg0, arg1) \ @@ -50,6 +52,7 @@ SDT_PROVIDER_DECLARE(ip); SDT_PROVIDER_DECLARE(tcp); SDT_PROVIDER_DECLARE(udp); +SDT_PROVIDER_DECLARE(udplite); SDT_PROBE_DECLARE(ip, , , receive); SDT_PROBE_DECLARE(ip, , , send); @@ -71,5 +74,8 @@ SDT_PROBE_DECLARE(tcp, , , receive__autoresize); SDT_PROBE_DECLARE(udp, , , receive); SDT_PROBE_DECLARE(udp, , , send); + +SDT_PROBE_DECLARE(udplite, , , receive); +SDT_PROBE_DECLARE(udplite, , , send); #endif Modified: head/sys/netinet/udp_usrreq.c ============================================================================== --- head/sys/netinet/udp_usrreq.c Tue Jul 31 22:50:50 2018 (r337017) +++ head/sys/netinet/udp_usrreq.c Tue Jul 31 22:56:03 2018 (r337018) @@ -594,8 +594,12 @@ udp_input(struct mbuf **mp, int *offp, int proto) if ((n = m_copym(m, 0, M_COPYALL, M_NOWAIT)) != NULL) { - UDP_PROBE(receive, NULL, last, ip, - last, uh); + if (proto == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, last, ip, + last, uh); + else + UDP_PROBE(receive, NULL, last, ip, last, + uh); if (udp_append(last, ip, n, iphlen, udp_in)) { goto inp_lost; @@ -629,7 +633,10 @@ udp_input(struct mbuf **mp, int *offp, int proto) INP_INFO_RUNLOCK_ET(pcbinfo, et); goto badunlocked; } - UDP_PROBE(receive, NULL, last, ip, last, uh); + if (proto == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, last, ip, last, uh); + else + UDP_PROBE(receive, NULL, last, ip, last, uh); if (udp_append(last, ip, m, iphlen, udp_in) == 0) INP_RUNLOCK(last); inp_lost: @@ -685,7 +692,10 @@ udp_input(struct mbuf **mp, int *offp, int proto) inet_ntoa_r(ip->ip_dst, dst), ntohs(uh->uh_dport), inet_ntoa_r(ip->ip_src, src), ntohs(uh->uh_sport)); } - UDP_PROBE(receive, NULL, NULL, ip, NULL, uh); + if (proto == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, NULL, ip, NULL, uh); + else + UDP_PROBE(receive, NULL, NULL, ip, NULL, uh); UDPSTAT_INC(udps_noport); if (m->m_flags & (M_BCAST | M_MCAST)) { UDPSTAT_INC(udps_noportbcast); @@ -705,7 +715,10 @@ udp_input(struct mbuf **mp, int *offp, int proto) */ INP_RLOCK_ASSERT(inp); if (inp->inp_ip_minttl && inp->inp_ip_minttl > ip->ip_ttl) { - UDP_PROBE(receive, NULL, inp, ip, inp, uh); + if (proto == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, inp, ip, inp, uh); + else + UDP_PROBE(receive, NULL, inp, ip, inp, uh); INP_RUNLOCK(inp); m_freem(m); return (IPPROTO_DONE); @@ -721,7 +734,10 @@ udp_input(struct mbuf **mp, int *offp, int proto) } } - UDP_PROBE(receive, NULL, inp, ip, inp, uh); + if (proto == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, inp, ip, inp, uh); + else + UDP_PROBE(receive, NULL, inp, ip, inp, uh); if (udp_append(inp, ip, m, iphlen, udp_in) == 0) INP_RUNLOCK(inp); return (IPPROTO_DONE); @@ -1526,7 +1542,10 @@ udp_output(struct inpcb *inp, struct mbuf *m, struct s INP_HASH_WUNLOCK(pcbinfo); else if (unlock_udbinfo == UH_RLOCKED) INP_HASH_RUNLOCK_ET(pcbinfo, et); - UDP_PROBE(send, NULL, inp, &ui->ui_i, inp, &ui->ui_u); + if (pr == IPPROTO_UDPLITE) + UDPLITE_PROBE(send, NULL, inp, &ui->ui_i, inp, &ui->ui_u); + else + UDP_PROBE(send, NULL, inp, &ui->ui_i, inp, &ui->ui_u); error = ip_output(m, inp->inp_options, (unlock_inp == UH_WLOCKED ? &inp->inp_route : NULL), ipflags, inp->inp_moptions, inp); Modified: head/sys/netinet/udplite.h ============================================================================== --- head/sys/netinet/udplite.h Tue Jul 31 22:50:50 2018 (r337017) +++ head/sys/netinet/udplite.h Tue Jul 31 22:56:03 2018 (r337018) @@ -29,6 +29,17 @@ #ifndef _NETINET_UDPLITE_H_ #define _NETINET_UDPLITE_H_ +/* + * UDP-Lite protocol header. + * Per RFC 3828, July, 2004. + */ +struct udplitehdr { + u_short udplite_sport; /* UDO-Lite source port */ + u_short udplite_dport; /* UDP-Lite destination port */ + u_short udplite_coverage; /* UDP-Lite checksum coverage */ + u_short udplite_checksum; /* UDP-Lite checksum */ +}; + /* * User-settable options (used with setsockopt). */ Modified: head/sys/netinet6/udp6_usrreq.c ============================================================================== --- head/sys/netinet6/udp6_usrreq.c Tue Jul 31 22:50:50 2018 (r337017) +++ head/sys/netinet6/udp6_usrreq.c Tue Jul 31 22:56:03 2018 (r337018) @@ -388,8 +388,12 @@ udp6_input(struct mbuf **mp, int *offp, int proto) NULL) { INP_RLOCK(last); if (__predict_true(last->inp_flags2 & INP_FREED) == 0) { - UDP_PROBE(receive, NULL, last, ip6, - last, uh); + if (nxt == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, last, + ip6, last, uh); + else + UDP_PROBE(receive, NULL, last, + ip6, last, uh); if (udp6_append(last, n, off, fromsa)) goto inp_lost; } @@ -422,7 +426,10 @@ udp6_input(struct mbuf **mp, int *offp, int proto) } INP_RLOCK(last); if (__predict_true(last->inp_flags2 & INP_FREED) == 0) { - UDP_PROBE(receive, NULL, last, ip6, last, uh); + if (nxt == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, last, ip6, last, uh); + else + UDP_PROBE(receive, NULL, last, ip6, last, uh); if (udp6_append(last, m, off, fromsa) == 0) INP_RUNLOCK(last); } else @@ -483,7 +490,10 @@ udp6_input(struct mbuf **mp, int *offp, int proto) ip6_sprintf(ip6bufs, &ip6->ip6_src), ntohs(uh->uh_sport)); } - UDP_PROBE(receive, NULL, NULL, ip6, NULL, uh); + if (nxt == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, NULL, ip6, NULL, uh); + else + UDP_PROBE(receive, NULL, NULL, ip6, NULL, uh); UDPSTAT_INC(udps_noport); if (m->m_flags & M_MCAST) { printf("UDP6: M_MCAST is set in a unicast packet.\n"); @@ -504,7 +514,10 @@ udp6_input(struct mbuf **mp, int *offp, int proto) return (IPPROTO_DONE); } } - UDP_PROBE(receive, NULL, inp, ip6, inp, uh); + if (nxt == IPPROTO_UDPLITE) + UDPLITE_PROBE(receive, NULL, inp, ip6, inp, uh); + else + UDP_PROBE(receive, NULL, inp, ip6, inp, uh); if (udp6_append(inp, m, off, fromsa) == 0) INP_RUNLOCK(inp); return (IPPROTO_DONE); @@ -919,7 +932,10 @@ udp6_output(struct inpcb *inp, struct mbuf *m, struct flags |= IP_NODEFAULTFLOWID; #endif *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Tue Jul 31 22:57:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1B151067949; Tue, 31 Jul 2018 22:57:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 65574822FA; Tue, 31 Jul 2018 22:57:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2BF811A18F; Tue, 31 Jul 2018 22:57:28 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VMvRi5095323; Tue, 31 Jul 2018 22:57:27 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VMvRdN095322; Tue, 31 Jul 2018 22:57:27 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312257.w6VMvRdN095322@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 22:57:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337019 - head/cddl/contrib/opensolaris X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/cddl/contrib/opensolaris X-SVN-Commit-Revision: 337019 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 22:57:28 -0000 Author: mav Date: Tue Jul 31 22:57:27 2018 New Revision: 337019 URL: https://svnweb.freebsd.org/changeset/base/337019 Log: Record r337016 as already merged, since it came from us. Modified: Directory Properties: head/cddl/contrib/opensolaris/ (props changed) From owner-svn-src-all@freebsd.org Tue Jul 31 22:59:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D1851067A01; Tue, 31 Jul 2018 22:59:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D7C2782486; Tue, 31 Jul 2018 22:59:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B501B1A190; Tue, 31 Jul 2018 22:59:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VMxmu0095461; Tue, 31 Jul 2018 22:59:48 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VMxm5t095460; Tue, 31 Jul 2018 22:59:48 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312259.w6VMxm5t095460@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 22:59:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337020 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337020 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 22:59:49 -0000 Author: mav Date: Tue Jul 31 22:59:48 2018 New Revision: 337020 URL: https://svnweb.freebsd.org/changeset/base/337020 Log: 9443 panic when scrub a v10 pool illumos/illumos-gate@bb1f424574ac8e08069d0ba993c2a41ffe796794 Reviewed by: Serapheim Dimitropoulos Reviewed by: George Wilson Reviewed by: Andriy Gapon Reviewed by: Igor Kozhukhov Approved by: Dan McDonald Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 22:57:27 2018 (r337019) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 22:59:48 2018 (r337020) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2017 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright 2016 Gary Mills * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright 2017 Joyent, Inc. @@ -1185,7 +1185,8 @@ dsl_scan_visitds(dsl_scan_t *scn, uint64_t dsobj, dmu_ * block-sharing rules don't apply to it. */ if (DSL_SCAN_IS_SCRUB_RESILVER(scn) && !dsl_dataset_is_snapshot(ds) && - ds->ds_dir != dp->dp_origin_snap->ds_dir) { + (dp->dp_origin_snap == NULL || + ds->ds_dir != dp->dp_origin_snap->ds_dir)) { objset_t *os; if (dmu_objset_from_ds(ds, &os) != 0) { goto out; From owner-svn-src-all@freebsd.org Tue Jul 31 23:00:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67FC21067BC7; Tue, 31 Jul 2018 23:00:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E0CD8276C; Tue, 31 Jul 2018 23:00:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F33411A1B7; Tue, 31 Jul 2018 23:00:58 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VN0wTA097832; Tue, 31 Jul 2018 23:00:58 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VN0wVL097831; Tue, 31 Jul 2018 23:00:58 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312300.w6VN0wVL097831@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 23:00:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337021 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337021 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 23:00:59 -0000 Author: mav Date: Tue Jul 31 23:00:58 2018 New Revision: 337021 URL: https://svnweb.freebsd.org/changeset/base/337021 Log: MFV r337020:9443 panic when scrub a v10 pool illumos/illumos-gate@bb1f424574ac8e08069d0ba993c2a41ffe796794 Reviewed by: Serapheim Dimitropoulos Reviewed by: George Wilson Reviewed by: Andriy Gapon Reviewed by: Igor Kozhukhov Approved by: Dan McDonald Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 22:59:48 2018 (r337020) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Tue Jul 31 23:00:58 2018 (r337021) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2017 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright 2016 Gary Mills * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright 2017 Joyent, Inc. @@ -2164,7 +2164,8 @@ dsl_scan_visitds(dsl_scan_t *scn, uint64_t dsobj, dmu_ * block-sharing rules don't apply to it. */ if (DSL_SCAN_IS_SCRUB_RESILVER(scn) && !dsl_dataset_is_snapshot(ds) && - ds->ds_dir != dp->dp_origin_snap->ds_dir) { + (dp->dp_origin_snap == NULL || + ds->ds_dir != dp->dp_origin_snap->ds_dir)) { objset_t *os; if (dmu_objset_from_ds(ds, &os) != 0) { goto out; From owner-svn-src-all@freebsd.org Tue Jul 31 23:44:15 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E10C10687B9; Tue, 31 Jul 2018 23:44:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D585283DF8; Tue, 31 Jul 2018 23:44:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B5B731A99C; Tue, 31 Jul 2018 23:44:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w6VNiEI7021633; Tue, 31 Jul 2018 23:44:14 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w6VNiEem021630; Tue, 31 Jul 2018 23:44:14 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201807312344.w6VNiEem021630@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 31 Jul 2018 23:44:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337022 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337022 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jul 2018 23:44:15 -0000 Author: mav Date: Tue Jul 31 23:44:13 2018 New Revision: 337022 URL: https://svnweb.freebsd.org/changeset/base/337022 Log: 9403 assertion failed in arc_buf_destroy() when concurrently reading block with checksum error This assertion (VERIFY) failure was reported when reading a block. Turns out the problem is that if we get an i/o error (ECKSUM in this case), and there are multiple concurrent ARC reads of the same block (from different clones), then the ARC will put multiple buf's on the same ANON hdr, which isn't supposed to happen, and then causes a panic when we try to arc_buf_destroy() the buf. illumos/illumos-gate@fa98e487a9619b7902f218663be219e787a57dad Reviewed by: George Wilson Reviewed by: Paul Dagnelie Reviewed by: Pavel Zakharov Approved by: Matt Ahrens Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/zio_compress.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Tue Jul 31 23:00:58 2018 (r337021) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Tue Jul 31 23:44:13 2018 (r337022) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2018, Joyent, Inc. - * Copyright (c) 2011, 2017 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright (c) 2014 by Saso Kiselkov. All rights reserved. * Copyright 2017 Nexenta Systems, Inc. All rights reserved. */ @@ -4717,12 +4717,13 @@ void arc_getbuf_func(zio_t *zio, arc_buf_t *buf, void *arg) { arc_buf_t **bufp = arg; - if (zio && zio->io_error) { - arc_buf_destroy(buf, arg); + if (buf == NULL) { + ASSERT(zio == NULL || zio->io_error != 0); *bufp = NULL; } else { + ASSERT(zio == NULL || zio->io_error == 0); *bufp = buf; - ASSERT(buf->b_data); + ASSERT(buf->b_data != NULL); } } @@ -4821,12 +4822,38 @@ arc_read_done(zio_t *zio) /* This is a demand read since prefetches don't use callbacks */ callback_cnt++; - int error = arc_buf_alloc_impl(hdr, acb->acb_private, - acb->acb_compressed, no_zio_error, &acb->acb_buf); if (no_zio_error) { - zio->io_error = error; + int error = arc_buf_alloc_impl(hdr, acb->acb_private, + acb->acb_compressed, zio->io_error == 0, + &acb->acb_buf); + if (error != 0) { + /* + * Decompression failed. Set io_error + * so that when we call acb_done (below), + * we will indicate that the read failed. + * Note that in the unusual case where one + * callback is compressed and another + * uncompressed, we will mark all of them + * as failed, even though the uncompressed + * one can't actually fail. In this case, + * the hdr will not be anonymous, because + * if there are multiple callbacks, it's + * because multiple threads found the same + * arc buf in the hash table. + */ + zio->io_error = error; + } } } + /* + * If there are multiple callbacks, we must have the hash lock, + * because the only way for multiple threads to find this hdr is + * in the hash table. This ensures that if there are multiple + * callbacks, the hdr is not anonymous. If it were anonymous, + * we couldn't use arc_buf_destroy() in the error case below. + */ + ASSERT(callback_cnt < 2 || hash_lock != NULL); + hdr->b_l1hdr.b_acb = NULL; arc_hdr_clear_flags(hdr, ARC_FLAG_IO_IN_PROGRESS); if (callback_cnt == 0) { @@ -4871,8 +4898,18 @@ arc_read_done(zio_t *zio) /* execute each callback and free its structure */ while ((acb = callback_list) != NULL) { - if (acb->acb_done) + if (acb->acb_done != NULL) { + if (zio->io_error != 0 && acb->acb_buf != NULL) { + /* + * If arc_buf_alloc_impl() fails during + * decompression, the buf will still be + * allocated, and needs to be freed here. + */ + arc_buf_destroy(acb->acb_buf, acb->acb_private); + acb->acb_buf = NULL; + } acb->acb_done(zio, acb->acb_buf, acb->acb_private); + } if (acb->acb_zio_dummy != NULL) { acb->acb_zio_dummy->io_error = zio->io_error; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Tue Jul 31 23:00:58 2018 (r337021) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Tue Jul 31 23:44:13 2018 (r337022) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2012, 2017 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. @@ -989,22 +989,26 @@ dbuf_read_done(zio_t *zio, arc_buf_t *buf, void *vdb) ASSERT(refcount_count(&db->db_holds) > 0); ASSERT(db->db_buf == NULL); ASSERT(db->db.db_data == NULL); - if (db->db_level == 0 && db->db_freed_in_flight) { - /* we were freed in flight; disregard any error */ + if (buf == NULL) { + /* i/o error */ + ASSERT(zio == NULL || zio->io_error != 0); + ASSERT(db->db_blkid != DMU_BONUS_BLKID); + ASSERT3P(db->db_buf, ==, NULL); + db->db_state = DB_UNCACHED; + } else if (db->db_level == 0 && db->db_freed_in_flight) { + /* freed in flight */ + ASSERT(zio == NULL || zio->io_error == 0); arc_release(buf, db); bzero(buf->b_data, db->db.db_size); arc_buf_freeze(buf); db->db_freed_in_flight = FALSE; dbuf_set_data(db, buf); db->db_state = DB_CACHED; - } else if (zio == NULL || zio->io_error == 0) { + } else { + /* success */ + ASSERT(zio == NULL || zio->io_error == 0); dbuf_set_data(db, buf); db->db_state = DB_CACHED; - } else { - ASSERT(db->db_blkid != DMU_BONUS_BLKID); - ASSERT3P(db->db_buf, ==, NULL); - arc_buf_destroy(buf, db); - db->db_state = DB_UNCACHED; } cv_broadcast(&db->db_changed); dbuf_rele_and_unlock(db, NULL); @@ -2417,6 +2421,13 @@ dbuf_prefetch_indirect_done(zio_t *zio, arc_buf_t *abu ASSERT3S(dpa->dpa_zb.zb_level, <, dpa->dpa_curlevel); ASSERT3S(dpa->dpa_curlevel, >, 0); + if (abuf == NULL) { + ASSERT(zio == NULL || zio->io_error != 0); + kmem_free(dpa, sizeof (*dpa)); + return; + } + ASSERT(zio == NULL || zio->io_error == 0); + /* * The dpa_dnode is only valid if we are called with a NULL * zio. This indicates that the arc_read() returned without @@ -2455,7 +2466,7 @@ dbuf_prefetch_indirect_done(zio_t *zio, arc_buf_t *abu (dpa->dpa_epbs * (dpa->dpa_curlevel - dpa->dpa_zb.zb_level)); blkptr_t *bp = ((blkptr_t *)abuf->b_data) + P2PHASE(nextblkid, 1ULL << dpa->dpa_epbs); - if (BP_IS_HOLE(bp) || (zio != NULL && zio->io_error != 0)) { + if (BP_IS_HOLE(bp)) { kmem_free(dpa, sizeof (*dpa)); } else if (dpa->dpa_curlevel == dpa->dpa_zb.zb_level) { ASSERT3U(nextblkid, ==, dpa->dpa_zb.zb_blkid); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zio_compress.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zio_compress.c Tue Jul 31 23:00:58 2018 (r337021) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zio_compress.c Tue Jul 31 23:44:13 2018 (r337022) @@ -25,7 +25,7 @@ */ /* * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. - * Copyright (c) 2013, 2016 by Delphix. All rights reserved. + * Copyright (c) 2013, 2018 by Delphix. All rights reserved. */ #include @@ -36,6 +36,12 @@ #include /* + * If nonzero, every 1/X decompression attempts will fail, simulating + * an undetected memory error. + */ +uint64_t zio_decompress_fail_fraction = 0; + +/* * Compression vectors. */ zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS] = { @@ -145,6 +151,16 @@ zio_decompress_data(enum zio_compress c, abd_t *src, v void *tmp = abd_borrow_buf_copy(src, s_len); int ret = zio_decompress_data_buf(c, tmp, dst, s_len, d_len); abd_return_buf(src, tmp, s_len); + + /* + * Decompression shouldn't fail, because we've already verifyied + * the checksum. However, for extra protection (e.g. against bitflips + * in non-ECC RAM), we handle this error (and test it). + */ + ASSERT0(ret); + if (zio_decompress_fail_fraction != 0 && + spa_get_random(zio_decompress_fail_fraction) == 0) + ret = SET_ERROR(EINVAL); return (ret); } From owner-svn-src-all@freebsd.org Wed Aug 1 00:39:23 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49CBE106AF28; Wed, 1 Aug 2018 00:39:23 +0000 (UTC) (envelope-from araujo@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E8E418BE3E; Wed, 1 Aug 2018 00:39:22 +0000 (UTC) (envelope-from araujo@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A8E511B21D; Wed, 1 Aug 2018 00:39:22 +0000 (UTC) (envelope-from araujo@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w710dMgv060350; Wed, 1 Aug 2018 00:39:22 GMT (envelope-from araujo@FreeBSD.org) Received: (from araujo@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w710dM9o060348; Wed, 1 Aug 2018 00:39:22 GMT (envelope-from araujo@FreeBSD.org) Message-Id: <201808010039.w710dM9o060348@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: araujo set sender to araujo@FreeBSD.org using -f From: Marcelo Araujo Date: Wed, 1 Aug 2018 00:39:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337023 - in head: sys/amd64/vmm usr.sbin/jail X-SVN-Group: head X-SVN-Commit-Author: araujo X-SVN-Commit-Paths: in head: sys/amd64/vmm usr.sbin/jail X-SVN-Commit-Revision: 337023 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 00:39:23 -0000 Author: araujo Date: Wed Aug 1 00:39:21 2018 New Revision: 337023 URL: https://svnweb.freebsd.org/changeset/base/337023 Log: - Add the ability to run bhyve(8) within a jail(8). This patch adds a new sysctl(8) knob "security.jail.vmm_allowed", by default this option is disable. Submitted by: Shawn Webb Reviewed by: jamie@ and myself. Relnotes: Yes. Sponsored by: HardenedBSD and G2, Inc. Differential Revision: https://reviews.freebsd.org/D16057 Modified: head/sys/amd64/vmm/vmm_dev.c head/usr.sbin/jail/jail.8 Modified: head/sys/amd64/vmm/vmm_dev.c ============================================================================== --- head/sys/amd64/vmm/vmm_dev.c Tue Jul 31 23:44:13 2018 (r337022) +++ head/sys/amd64/vmm/vmm_dev.c Wed Aug 1 00:39:21 2018 (r337023) @@ -33,6 +33,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -43,6 +44,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -82,16 +84,29 @@ struct vmmdev_softc { static SLIST_HEAD(, vmmdev_softc) head; +static unsigned pr_allow_flag; static struct mtx vmmdev_mtx; static MALLOC_DEFINE(M_VMMDEV, "vmmdev", "vmmdev"); SYSCTL_DECL(_hw_vmm); +static int vmm_priv_check(struct ucred *ucred); static int devmem_create_cdev(const char *vmname, int id, char *devmem); static void devmem_destroy(void *arg); static int +vmm_priv_check(struct ucred *ucred) +{ + + if (jailed(ucred) && + !(ucred->cr_prison->pr_allow & pr_allow_flag)) + return (EPERM); + + return (0); +} + +static int vcpu_lock_one(struct vmmdev_softc *sc, int vcpu) { int error; @@ -177,6 +192,10 @@ vmmdev_rw(struct cdev *cdev, struct uio *uio, int flag void *hpa, *cookie; struct vmmdev_softc *sc; + error = vmm_priv_check(curthread->td_ucred); + if (error) + return (error); + sc = vmmdev_lookup2(cdev); if (sc == NULL) return (ENXIO); @@ -351,11 +370,14 @@ vmmdev_ioctl(struct cdev *cdev, u_long cmd, caddr_t da uint64_t *regvals; int *regnums; + error = vmm_priv_check(curthread->td_ucred); + if (error) + return (error); + sc = vmmdev_lookup2(cdev); if (sc == NULL) return (ENXIO); - error = 0; vcpu = -1; state_changed = 0; @@ -777,6 +799,10 @@ vmmdev_mmap_single(struct cdev *cdev, vm_ooffset_t *of int error, found, segid; bool sysmem; + error = vmm_priv_check(curthread->td_ucred); + if (error) + return (error); + first = *offset; last = first + mapsize; if ((nprot & PROT_EXEC) || first < 0 || first >= last) @@ -865,6 +891,10 @@ sysctl_vmm_destroy(SYSCTL_HANDLER_ARGS) struct vmmdev_softc *sc; struct cdev *cdev; + error = vmm_priv_check(req->td->td_ucred); + if (error) + return (error); + strlcpy(buf, "beavis", sizeof(buf)); error = sysctl_handle_string(oidp, buf, sizeof(buf), req); if (error != 0 || req->newptr == NULL) @@ -906,7 +936,8 @@ sysctl_vmm_destroy(SYSCTL_HANDLER_ARGS) destroy_dev_sched_cb(cdev, vmmdev_destroy, sc); return (0); } -SYSCTL_PROC(_hw_vmm, OID_AUTO, destroy, CTLTYPE_STRING | CTLFLAG_RW, +SYSCTL_PROC(_hw_vmm, OID_AUTO, destroy, + CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_PRISON, NULL, 0, sysctl_vmm_destroy, "A", NULL); static struct cdevsw vmmdevsw = { @@ -927,6 +958,10 @@ sysctl_vmm_create(SYSCTL_HANDLER_ARGS) struct vmmdev_softc *sc, *sc2; char buf[VM_MAX_NAMELEN]; + error = vmm_priv_check(req->td->td_ucred); + if (error) + return (error); + strlcpy(buf, "beavis", sizeof(buf)); error = sysctl_handle_string(oidp, buf, sizeof(buf), req); if (error != 0 || req->newptr == NULL) @@ -977,13 +1012,16 @@ sysctl_vmm_create(SYSCTL_HANDLER_ARGS) return (0); } -SYSCTL_PROC(_hw_vmm, OID_AUTO, create, CTLTYPE_STRING | CTLFLAG_RW, +SYSCTL_PROC(_hw_vmm, OID_AUTO, create, + CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_PRISON, NULL, 0, sysctl_vmm_create, "A", NULL); void vmmdev_init(void) { mtx_init(&vmmdev_mtx, "vmm device mutex", NULL, MTX_DEF); + pr_allow_flag = prison_add_allow(NULL, "vmm", NULL, + "Allow use of vmm in a jail."); } int Modified: head/usr.sbin/jail/jail.8 ============================================================================== --- head/usr.sbin/jail/jail.8 Tue Jul 31 23:44:13 2018 (r337022) +++ head/usr.sbin/jail/jail.8 Wed Aug 1 00:39:21 2018 (r337023) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 29, 2018 +.Dd July 30, 2018 .Dt JAIL 8 .Os .Sh NAME @@ -650,6 +650,12 @@ See .Xr zfs 8 for information on how to configure the ZFS filesystem to operate from within a jail. +.It Va allow.vmm +The jail may access +.Xr vmm 4 . +This flag is only available when the +.Xr vmm 4 +kernel module is loaded. .It Va linux Determine how a jail's Linux emulation environment appears. A value of @@ -1294,6 +1300,7 @@ environment of the first jail. .Xr ps 1 , .Xr quota 1 , .Xr jail_set 2 , +.Xr vmm 4 , .Xr devfs 5 , .Xr fdescfs 5 , .Xr jail.conf 5 , From owner-svn-src-all@freebsd.org Wed Aug 1 00:41:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EB49106B07C; Wed, 1 Aug 2018 00:41:04 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B72D18C217; Wed, 1 Aug 2018 00:41:03 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 99BAB1B26C; Wed, 1 Aug 2018 00:41:03 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w710f3sS060492; Wed, 1 Aug 2018 00:41:03 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w710f3Cg060489; Wed, 1 Aug 2018 00:41:03 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201808010041.w710f3Cg060489@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Wed, 1 Aug 2018 00:41:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337024 - in stable/11/sys: amd64/linux amd64/linux32 i386/linux X-SVN-Group: stable-11 X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: in stable/11/sys: amd64/linux amd64/linux32 i386/linux X-SVN-Commit-Revision: 337024 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 00:41:04 -0000 Author: jhb Date: Wed Aug 1 00:41:02 2018 New Revision: 337024 URL: https://svnweb.freebsd.org/changeset/base/337024 Log: MFC 330823,332335: Cosmetic cleanups to some Linuxulator files. 330823: Apply some style(9) to Linuxulator linux_sysvec.c comments 332335: linuxulator: add else case braces to reduce diffs between archs Modified: stable/11/sys/amd64/linux/linux_sysvec.c stable/11/sys/amd64/linux32/linux32_sysvec.c stable/11/sys/i386/linux/linux_sysvec.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/linux/linux_sysvec.c ============================================================================== --- stable/11/sys/amd64/linux/linux_sysvec.c Wed Aug 1 00:39:21 2018 (r337023) +++ stable/11/sys/amd64/linux/linux_sysvec.c Wed Aug 1 00:41:02 2018 (r337024) @@ -101,10 +101,9 @@ SYSCTL_PROC(_compat_linux, OID_AUTO, debug, #endif /* - * Allow the this functions to use the ldebug() facility - * even though they are not syscalls themselves. Map them - * to syscall 0. This is slightly less bogus than using - * ldebug(sigreturn). + * Allow the sendsig functions to use the ldebug() facility even though they + * are not syscalls themselves. Map them to syscall 0. This is slightly less + * bogus than using ldebug(sigreturn). */ #define LINUX_SYS_linux_rt_sendsig 0 @@ -253,8 +252,7 @@ linux_set_syscall_retval(struct thread *td, int error) /* * On Linux only %rcx and %r11 values are not preserved across - * the syscall. - * So, do not clobber %rdx and %r10 + * the syscall. So, do not clobber %rdx and %r10. */ td->td_retval[1] = frame->tf_rdx; if (error != EJUSTRETURN) @@ -336,9 +334,7 @@ linux_copyout_strings(struct image_params *imgp) size_t execpath_len; struct proc *p; - /* - * Calculate string base and vector table pointers. - */ + /* Calculate string base and vector table pointers. */ if (imgp->execpath != NULL && imgp->auxargs != NULL) execpath_len = strlen(imgp->execpath) + 1; else @@ -346,7 +342,7 @@ linux_copyout_strings(struct image_params *imgp) p = imgp->proc; arginfo = (struct ps_strings *)p->p_sysent->sv_psstrings; - destp = (caddr_t)arginfo - SPARE_USRSPACE - + destp = (caddr_t)arginfo - SPARE_USRSPACE - roundup(sizeof(canary), sizeof(char *)) - roundup(execpath_len, sizeof(char *)) - roundup(ARG_MAX - imgp->args->stringspace, sizeof(char *)); @@ -356,19 +352,14 @@ linux_copyout_strings(struct image_params *imgp) copyout(imgp->execpath, (void *)imgp->execpathp, execpath_len); } - /* - * Prepare the canary for SSP. - */ + /* Prepare the canary for SSP. */ arc4rand(canary, sizeof(canary), 0); imgp->canary = (uintptr_t)arginfo - roundup(execpath_len, sizeof(char *)) - roundup(sizeof(canary), sizeof(char *)); copyout(canary, (void *)imgp->canary, sizeof(canary)); - /* - * If we have a valid auxargs ptr, prepare some room - * on the stack. - */ + /* If we have a valid auxargs ptr, prepare some room on the stack. */ if (imgp->auxargs) { /* * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for @@ -394,29 +385,21 @@ linux_copyout_strings(struct image_params *imgp) imgp->args->envc + 2) * sizeof(char *)); } - /* - * vectp also becomes our initial stack base - */ + /* vectp also becomes our initial stack base. */ stack_base = (register_t *)vectp; stringp = imgp->args->begin_argv; argc = imgp->args->argc; envc = imgp->args->envc; - /* - * Copy out strings - arguments and environment. - */ + /* Copy out strings - arguments and environment. */ copyout(stringp, destp, ARG_MAX - imgp->args->stringspace); - /* - * Fill in "ps_strings" struct for ps, w, etc. - */ + /* Fill in "ps_strings" struct for ps, w, etc. */ suword(&arginfo->ps_argvstr, (long)(intptr_t)vectp); suword(&arginfo->ps_nargvstr, argc); - /* - * Fill in argument portion of vector table. - */ + /* Fill in argument portion of vector table. */ for (; argc > 0; --argc) { suword(vectp++, (long)(intptr_t)destp); while (*stringp++ != 0) @@ -424,15 +407,13 @@ linux_copyout_strings(struct image_params *imgp) destp++; } - /* a null vector table pointer separates the argp's from the envp's */ + /* A null vector table pointer separates the argp's from the envp's. */ suword(vectp++, 0); suword(&arginfo->ps_envstr, (long)(intptr_t)vectp); suword(&arginfo->ps_nenvstr, envc); - /* - * Fill in environment portion of vector table. - */ + /* Fill in environment portion of vector table. */ for (; envc > 0; --envc) { suword(vectp++, (long)(intptr_t)destp); while (*stringp++ != 0) @@ -440,7 +421,7 @@ linux_copyout_strings(struct image_params *imgp) destp++; } - /* end of vector table is a null pointer */ + /* The end of the vector table is a null pointer. */ suword(vectp, 0); return (stack_base); } @@ -685,12 +666,10 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse regs->tf_rdx = (register_t)&sfp->sf_sc; /* arg 3 in %rdx */ sf.sf_handler = catcher; - /* Fill in POSIX parts */ + /* Fill in POSIX parts. */ ksiginfo_to_lsiginfo(ksi, &sf.sf_si, sig); - /* - * Copy the sigframe out to the user's stack. - */ + /* Copy the sigframe out to the user's stack. */ if (copyout(&sf, sfp, sizeof(*sfp)) != 0) { #ifdef DEBUG printf("process %ld has trashed its stack\n", (long)p->p_pid); @@ -766,7 +745,7 @@ linux_vsyscall(struct thread *td) frame = td->td_frame; - /* Check %rip for vsyscall area */ + /* Check %rip for vsyscall area. */ if (__predict_true(frame->tf_rip < LINUX_VSYSCALL_START)) return (EINVAL); if ((frame->tf_rip & (LINUX_VSYSCALL_SZ - 1)) != 0) @@ -777,7 +756,7 @@ linux_vsyscall(struct thread *td) /* * vsyscall called as callq *(%rax), so we must - * use return address from %rsp and also fixup %rsp + * use return address from %rsp and also fixup %rsp. */ error = copyin((void *)frame->tf_rsp, &retqaddr, sizeof(retqaddr)); if (error) Modified: stable/11/sys/amd64/linux32/linux32_sysvec.c ============================================================================== --- stable/11/sys/amd64/linux32/linux32_sysvec.c Wed Aug 1 00:39:21 2018 (r337023) +++ stable/11/sys/amd64/linux32/linux32_sysvec.c Wed Aug 1 00:41:02 2018 (r337024) @@ -100,10 +100,9 @@ MODULE_VERSION(linux, 1); #endif /* - * Allow the sendsig functions to use the ldebug() facility - * even though they are not syscalls themselves. Map them - * to syscall 0. This is slightly less bogus than using - * ldebug(sigreturn). + * Allow the sendsig functions to use the ldebug() facility even though they + * are not syscalls themselves. Map them to syscall 0. This is slightly less + * bogus than using ldebug(sigreturn). */ #define LINUX32_SYS_linux_rt_sendsig 0 #define LINUX32_SYS_linux_sendsig 0 @@ -310,9 +309,7 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse printf(ARGS(rt_sendsig, "%p, %d, %p, %u"), catcher, sig, (void*)mask, code); #endif - /* - * Allocate space for the signal handler context. - */ + /* Allocate space for the signal handler context. */ if ((td->td_pflags & TDP_ALTSTACK) && !oonstack && SIGISMEMBER(psp->ps_sigonstack, sig)) { fp = (struct l_rt_sigframe *)((uintptr_t)td->td_sigstk.ss_sp + @@ -321,9 +318,7 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse fp = (struct l_rt_sigframe *)regs->tf_rsp - 1; mtx_unlock(&psp->ps_mtx); - /* - * Build the argument list for the signal handler. - */ + /* Build the argument list for the signal handler. */ sig = bsd_to_linux_signal(sig); bzero(&frame, sizeof(frame)); @@ -333,12 +328,11 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse frame.sf_siginfo = PTROUT(&fp->sf_si); frame.sf_ucontext = PTROUT(&fp->sf_sc); - /* Fill in POSIX parts */ + /* Fill in POSIX parts. */ ksiginfo_to_lsiginfo(ksi, &frame.sf_si, sig); /* - * Build the signal context to be used by sigreturn - * and libgcc unwind. + * Build the signal context to be used by sigreturn and libgcc unwind. */ frame.sf_sc.uc_flags = 0; /* XXX ??? */ frame.sf_sc.uc_link = 0; /* XXX ??? */ @@ -394,9 +388,7 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse sigexit(td, SIGILL); } - /* - * Build context to run handler in. - */ + /* Build context to run handler in. */ regs->tf_rsp = PTROUT(fp); regs->tf_rip = linux32_rt_sigcode; regs->tf_rflags &= ~(PSL_T | PSL_D); @@ -455,9 +447,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t catcher, sig, (void*)mask, code); #endif - /* - * Allocate space for the signal handler context. - */ + /* Allocate space for the signal handler context. */ if ((td->td_pflags & TDP_ALTSTACK) && !oonstack && SIGISMEMBER(psp->ps_sigonstack, sig)) { fp = (struct l_sigframe *)((uintptr_t)td->td_sigstk.ss_sp + @@ -467,9 +457,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t mtx_unlock(&psp->ps_mtx); PROC_UNLOCK(p); - /* - * Build the argument list for the signal handler. - */ + /* Build the argument list for the signal handler. */ sig = bsd_to_linux_signal(sig); bzero(&frame, sizeof(frame)); @@ -479,9 +467,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t bsd_to_linux_sigset(mask, &lmask); - /* - * Build the signal context to be used by sigreturn. - */ + /* Build the signal context to be used by sigreturn. */ frame.sf_sc.sc_mask = lmask.__mask; frame.sf_sc.sc_gs = regs->tf_gs; frame.sf_sc.sc_fs = regs->tf_fs; @@ -515,9 +501,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t sigexit(td, SIGILL); } - /* - * Build context to run handler in. - */ + /* Build context to run handler in. */ regs->tf_rsp = PTROUT(fp); regs->tf_rip = linux32_sigcode; regs->tf_rflags &= ~(PSL_T | PSL_D); @@ -567,9 +551,7 @@ linux_sigreturn(struct thread *td, struct linux_sigret if (copyin(args->sfp, &frame, sizeof(frame)) != 0) return (EFAULT); - /* - * Check for security violations. - */ + /* Check for security violations. */ #define EFLAGS_SECURE(ef, oef) ((((ef) ^ (oef)) & ~PSL_USERCHANGE) == 0) eflags = frame.sf_sc.sc_eflags; if (!EFLAGS_SECURE(eflags, regs->tf_rflags)) @@ -596,9 +578,7 @@ linux_sigreturn(struct thread *td, struct linux_sigret linux_to_bsd_sigset(&lmask, &bmask); kern_sigprocmask(td, SIG_SETMASK, &bmask, NULL, 0); - /* - * Restore signal context. - */ + /* Restore signal context. */ regs->tf_rdi = frame.sf_sc.sc_edi; regs->tf_rsi = frame.sf_sc.sc_esi; regs->tf_rbp = frame.sf_sc.sc_ebp; @@ -658,9 +638,7 @@ linux_rt_sigreturn(struct thread *td, struct linux_rt_ context = &uc.uc_mcontext; - /* - * Check for security violations. - */ + /* Check for security violations. */ #define EFLAGS_SECURE(ef, oef) ((((ef) ^ (oef)) & ~PSL_USERCHANGE) == 0) eflags = context->sc_eflags; if (!EFLAGS_SECURE(eflags, regs->tf_rflags)) @@ -852,9 +830,7 @@ linux_copyout_strings(struct image_params *imgp) char canary[LINUX_AT_RANDOM_LEN]; size_t execpath_len; - /* - * Calculate string base and vector table pointers. - */ + /* Calculate string base and vector table pointers. */ if (imgp->execpath != NULL && imgp->auxargs != NULL) execpath_len = strlen(imgp->execpath) + 1; else @@ -871,19 +847,14 @@ linux_copyout_strings(struct image_params *imgp) copyout(imgp->execpath, (void *)imgp->execpathp, execpath_len); } - /* - * Prepare the canary for SSP. - */ + /* Prepare the canary for SSP. */ arc4rand(canary, sizeof(canary), 0); imgp->canary = (uintptr_t)arginfo - roundup(execpath_len, sizeof(char *)) - roundup(sizeof(canary), sizeof(char *)); copyout(canary, (void *)imgp->canary, sizeof(canary)); - /* - * If we have a valid auxargs ptr, prepare some room - * on the stack. - */ + /* If we have a valid auxargs ptr, prepare some room on the stack. */ if (imgp->auxargs) { /* * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for @@ -900,36 +871,29 @@ linux_copyout_strings(struct image_params *imgp) imgp->args->envc + 2 + imgp->auxarg_size) * sizeof(u_int32_t)); - } else + } else { /* * The '+ 2' is for the null pointers at the end of each of * the arg and env vector sets */ vectp = (u_int32_t *)(destp - (imgp->args->argc + imgp->args->envc + 2) * sizeof(u_int32_t)); + } - /* - * vectp also becomes our initial stack base - */ + /* vectp also becomes our initial stack base. */ stack_base = vectp; stringp = imgp->args->begin_argv; argc = imgp->args->argc; envc = imgp->args->envc; - /* - * Copy out strings - arguments and environment. - */ + /* Copy out strings - arguments and environment. */ copyout(stringp, destp, ARG_MAX - imgp->args->stringspace); - /* - * Fill in "ps_strings" struct for ps, w, etc. - */ + /* Fill in "ps_strings" struct for ps, w, etc. */ suword32(&arginfo->ps_argvstr, (uint32_t)(intptr_t)vectp); suword32(&arginfo->ps_nargvstr, argc); - /* - * Fill in argument portion of vector table. - */ + /* Fill in argument portion of vector table. */ for (; argc > 0; --argc) { suword32(vectp++, (uint32_t)(intptr_t)destp); while (*stringp++ != 0) @@ -937,15 +901,13 @@ linux_copyout_strings(struct image_params *imgp) destp++; } - /* a null vector table pointer separates the argp's from the envp's */ + /* A null vector table pointer separates the argp's from the envp's. */ suword32(vectp++, 0); suword32(&arginfo->ps_envstr, (uint32_t)(intptr_t)vectp); suword32(&arginfo->ps_nenvstr, envc); - /* - * Fill in environment portion of vector table. - */ + /* Fill in environment portion of vector table. */ for (; envc > 0; --envc) { suword32(vectp++, (uint32_t)(intptr_t)destp); while (*stringp++ != 0) @@ -953,7 +915,7 @@ linux_copyout_strings(struct image_params *imgp) destp++; } - /* end of vector table is a null pointer */ + /* The end of the vector table is a null pointer. */ suword32(vectp, 0); return ((register_t *)stack_base); Modified: stable/11/sys/i386/linux/linux_sysvec.c ============================================================================== --- stable/11/sys/i386/linux/linux_sysvec.c Wed Aug 1 00:39:21 2018 (r337023) +++ stable/11/sys/i386/linux/linux_sysvec.c Wed Aug 1 00:41:02 2018 (r337024) @@ -90,10 +90,9 @@ SYSCTL_PROC(_compat_linux, OID_AUTO, debug, #endif /* - * Allow the sendsig functions to use the ldebug() facility - * even though they are not syscalls themselves. Map them - * to syscall 0. This is slightly less bogus than using - * ldebug(sigreturn). + * Allow the sendsig functions to use the ldebug() facility even though they + * are not syscalls themselves. Map them to syscall 0. This is slightly less + * bogus than using ldebug(sigreturn). */ #define LINUX_SYS_linux_rt_sendsig 0 #define LINUX_SYS_linux_sendsig 0 @@ -308,9 +307,7 @@ linux_copyout_strings(struct image_params *imgp) size_t execpath_len; struct proc *p; - /* - * Calculate string base and vector table pointers. - */ + /* Calculate string base and vector table pointers. */ p = imgp->proc; if (imgp->execpath != NULL && imgp->auxargs != NULL) execpath_len = strlen(imgp->execpath) + 1; @@ -322,9 +319,7 @@ linux_copyout_strings(struct image_params *imgp) roundup(execpath_len, sizeof(char *)) - roundup(ARG_MAX - imgp->args->stringspace, sizeof(char *)); - /* - * install LINUX_PLATFORM - */ + /* Install LINUX_PLATFORM. */ copyout(linux_kplatform, ((caddr_t)arginfo - linux_szplatform), linux_szplatform); @@ -334,19 +329,14 @@ linux_copyout_strings(struct image_params *imgp) copyout(imgp->execpath, (void *)imgp->execpathp, execpath_len); } - /* - * Prepare the canary for SSP. - */ + /* Prepare the canary for SSP. */ arc4rand(canary, sizeof(canary), 0); imgp->canary = (uintptr_t)arginfo - linux_szplatform - roundup(execpath_len, sizeof(char *)) - roundup(sizeof(canary), sizeof(char *)); copyout(canary, (void *)imgp->canary, sizeof(canary)); - /* - * If we have a valid auxargs ptr, prepare some room - * on the stack. - */ + /* If we have a valid auxargs ptr, prepare some room on the stack. */ if (imgp->auxargs) { /* * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for @@ -370,29 +360,21 @@ linux_copyout_strings(struct image_params *imgp) sizeof(char *)); } - /* - * vectp also becomes our initial stack base - */ + /* vectp also becomes our initial stack base. */ stack_base = (register_t *)vectp; stringp = imgp->args->begin_argv; argc = imgp->args->argc; envc = imgp->args->envc; - /* - * Copy out strings - arguments and environment. - */ + /* Copy out strings - arguments and environment. */ copyout(stringp, destp, ARG_MAX - imgp->args->stringspace); - /* - * Fill in "ps_strings" struct for ps, w, etc. - */ + /* Fill in "ps_strings" struct for ps, w, etc. */ suword(&arginfo->ps_argvstr, (long)(intptr_t)vectp); suword(&arginfo->ps_nargvstr, argc); - /* - * Fill in argument portion of vector table. - */ + /* Fill in argument portion of vector table. */ for (; argc > 0; --argc) { suword(vectp++, (long)(intptr_t)destp); while (*stringp++ != 0) @@ -400,15 +382,13 @@ linux_copyout_strings(struct image_params *imgp) destp++; } - /* a null vector table pointer separates the argp's from the envp's */ + /* A null vector table pointer separates the argp's from the envp's. */ suword(vectp++, 0); suword(&arginfo->ps_envstr, (long)(intptr_t)vectp); suword(&arginfo->ps_nenvstr, envc); - /* - * Fill in environment portion of vector table. - */ + /* Fill in environment portion of vector table. */ for (; envc > 0; --envc) { suword(vectp++, (long)(intptr_t)destp); while (*stringp++ != 0) @@ -416,7 +396,7 @@ linux_copyout_strings(struct image_params *imgp) destp++; } - /* end of vector table is a null pointer */ + /* The end of the vector table is a null pointer. */ suword(vectp, 0); return (stack_base); @@ -446,9 +426,7 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse printf(ARGS(rt_sendsig, "%p, %d, %p, %u"), catcher, sig, (void*)mask, code); #endif - /* - * Allocate space for the signal handler context. - */ + /* Allocate space for the signal handler context. */ if ((td->td_pflags & TDP_ALTSTACK) && !oonstack && SIGISMEMBER(psp->ps_sigonstack, sig)) { fp = (struct l_rt_sigframe *)((uintptr_t)td->td_sigstk.ss_sp + @@ -457,9 +435,7 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse fp = (struct l_rt_sigframe *)regs->tf_esp - 1; mtx_unlock(&psp->ps_mtx); - /* - * Build the argument list for the signal handler. - */ + /* Build the argument list for the signal handler. */ sig = bsd_to_linux_signal(sig); bzero(&frame, sizeof(frame)); @@ -469,12 +445,10 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse frame.sf_siginfo = &fp->sf_si; frame.sf_ucontext = &fp->sf_sc; - /* Fill in POSIX parts */ + /* Fill in POSIX parts. */ ksiginfo_to_lsiginfo(ksi, &frame.sf_si, sig); - /* - * Build the signal context to be used by sigreturn. - */ + /* Build the signal context to be used by sigreturn. */ frame.sf_sc.uc_flags = 0; /* XXX ??? */ frame.sf_sc.uc_link = NULL; /* XXX ??? */ @@ -529,9 +503,7 @@ linux_rt_sendsig(sig_t catcher, ksiginfo_t *ksi, sigse sigexit(td, SIGILL); } - /* - * Build context to run handler in. - */ + /* Build context to run handler in. */ regs->tf_esp = (int)fp; regs->tf_eip = linux_rt_sigcode; regs->tf_eflags &= ~(PSL_T | PSL_VM | PSL_D); @@ -586,9 +558,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t catcher, sig, (void*)mask, code); #endif - /* - * Allocate space for the signal handler context. - */ + /* Allocate space for the signal handler context. */ if ((td->td_pflags & TDP_ALTSTACK) && !oonstack && SIGISMEMBER(psp->ps_sigonstack, sig)) { fp = (struct l_sigframe *)((uintptr_t)td->td_sigstk.ss_sp + @@ -598,9 +568,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t mtx_unlock(&psp->ps_mtx); PROC_UNLOCK(p); - /* - * Build the argument list for the signal handler. - */ + /* Build the argument list for the signal handler. */ sig = bsd_to_linux_signal(sig); bzero(&frame, sizeof(frame)); @@ -610,9 +578,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t bsd_to_linux_sigset(mask, &lmask); - /* - * Build the signal context to be used by sigreturn. - */ + /* Build the signal context to be used by sigreturn. */ frame.sf_sc.sc_mask = lmask.__mask; frame.sf_sc.sc_gs = rgs(); frame.sf_sc.sc_fs = regs->tf_fs; @@ -646,9 +612,7 @@ linux_sendsig(sig_t catcher, ksiginfo_t *ksi, sigset_t sigexit(td, SIGILL); } - /* - * Build context to run handler in. - */ + /* Build context to run handler in. */ regs->tf_esp = (int)fp; regs->tf_eip = linux_sigcode; regs->tf_eflags &= ~(PSL_T | PSL_VM | PSL_D); @@ -695,9 +659,7 @@ linux_sigreturn(struct thread *td, struct linux_sigret if (copyin(args->sfp, &frame, sizeof(frame)) != 0) return (EFAULT); - /* - * Check for security violations. - */ + /* Check for security violations. */ #define EFLAGS_SECURE(ef, oef) ((((ef) ^ (oef)) & ~PSL_USERCHANGE) == 0) eflags = frame.sf_sc.sc_eflags; if (!EFLAGS_SECURE(eflags, regs->tf_eflags)) @@ -723,9 +685,7 @@ linux_sigreturn(struct thread *td, struct linux_sigret linux_to_bsd_sigset(&lmask, &bmask); kern_sigprocmask(td, SIG_SETMASK, &bmask, NULL, 0); - /* - * Restore signal context. - */ + /* Restore signal context. */ /* %gs was restored by the trampoline. */ regs->tf_fs = frame.sf_sc.sc_fs; regs->tf_es = frame.sf_sc.sc_es; @@ -784,9 +744,7 @@ linux_rt_sigreturn(struct thread *td, struct linux_rt_ context = &uc.uc_mcontext; - /* - * Check for security violations. - */ + /* Check for security violations. */ #define EFLAGS_SECURE(ef, oef) ((((ef) ^ (oef)) & ~PSL_USERCHANGE) == 0) eflags = context->sc_eflags; if (!EFLAGS_SECURE(eflags, regs->tf_eflags)) @@ -811,9 +769,7 @@ linux_rt_sigreturn(struct thread *td, struct linux_rt_ linux_to_bsd_sigset(&uc.uc_sigmask, &bmask); kern_sigprocmask(td, SIG_SETMASK, &bmask, NULL, 0); - /* - * Restore signal context - */ + /* Restore signal context. */ /* %gs was restored by the trampoline. */ regs->tf_fs = context->sc_fs; regs->tf_es = context->sc_es; @@ -831,9 +787,7 @@ linux_rt_sigreturn(struct thread *td, struct linux_rt_ regs->tf_esp = context->sc_esp_at_signal; regs->tf_ss = context->sc_ss; - /* - * call sigaltstack & ignore results.. - */ + /* Call sigaltstack & ignore results. */ lss = &uc.uc_stack; ss.ss_sp = lss->ss_sp; ss.ss_size = lss->ss_size; @@ -930,7 +884,7 @@ exec_linux_setregs(struct thread *td, struct image_par exec_setregs(td, imgp, stack); - /* Linux sets %gs to 0, we default to _udatasel */ + /* Linux sets %gs to 0, we default to _udatasel. */ pcb->pcb_gs = 0; load_gs(0); From owner-svn-src-all@freebsd.org Wed Aug 1 02:39:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD823106D469; Wed, 1 Aug 2018 02:39:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 607B48EFE9; Wed, 1 Aug 2018 02:39:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 419901C571; Wed, 1 Aug 2018 02:39:45 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w712djFp020808; Wed, 1 Aug 2018 02:39:45 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w712didb020805; Wed, 1 Aug 2018 02:39:44 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808010239.w712didb020805@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 02:39:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337025 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337025 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 02:39:45 -0000 Author: mav Date: Wed Aug 1 02:39:44 2018 New Revision: 337025 URL: https://svnweb.freebsd.org/changeset/base/337025 Log: MFV r337022: 9403 assertion failed in arc_buf_destroy() when concurrently reading block with checksum error This assertion (VERIFY) failure was reported when reading a block. Turns out the problem is that if we get an i/o error (ECKSUM in this case), and there are multiple concurrent ARC reads of the same block (from different clones), then the ARC will put multiple buf's on the same ANON hdr, which isn't supposed to happen, and then causes a panic when we try to arc_buf_destroy() the buf. illumos/illumos-gate@fa98e487a9619b7902f218663be219e787a57dad Reviewed by: George Wilson Reviewed by: Paul Dagnelie Reviewed by: Pavel Zakharov Approved by: Matt Ahrens Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Aug 1 00:41:02 2018 (r337024) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Wed Aug 1 02:39:44 2018 (r337025) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2018, Joyent, Inc. - * Copyright (c) 2011, 2017 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright (c) 2014 by Saso Kiselkov. All rights reserved. * Copyright 2017 Nexenta Systems, Inc. All rights reserved. */ @@ -5148,12 +5148,13 @@ arc_getbuf_func(zio_t *zio, const zbookmark_phys_t *zb arc_buf_t *buf, void *arg) { arc_buf_t **bufp = arg; - if (buf == NULL) { + ASSERT(zio == NULL || zio->io_error != 0); *bufp = NULL; } else { + ASSERT(zio == NULL || zio->io_error == 0); *bufp = buf; - ASSERT(buf->b_data); + ASSERT(buf->b_data != NULL); } } @@ -5181,6 +5182,7 @@ arc_read_done(zio_t *zio) arc_callback_t *callback_list; arc_callback_t *acb; boolean_t freeable = B_FALSE; + boolean_t no_zio_error = (zio->io_error == 0); /* * The hdr was inserted into hash-table and removed from lists @@ -5206,7 +5208,7 @@ arc_read_done(zio_t *zio) ASSERT3P(hash_lock, !=, NULL); } - if (zio->io_error == 0) { + if (no_zio_error) { /* byteswap if necessary */ if (BP_SHOULD_BYTESWAP(zio->io_bp)) { if (BP_GET_LEVEL(zio->io_bp) > 0) { @@ -5227,8 +5229,7 @@ arc_read_done(zio_t *zio) callback_list = hdr->b_l1hdr.b_acb; ASSERT3P(callback_list, !=, NULL); - if (hash_lock && zio->io_error == 0 && - hdr->b_l1hdr.b_state == arc_anon) { + if (hash_lock && no_zio_error && hdr->b_l1hdr.b_state == arc_anon) { /* * Only call arc_access on anonymous buffers. This is because * if we've issued an I/O for an evicted buffer, we've already @@ -5251,20 +5252,38 @@ arc_read_done(zio_t *zio) callback_cnt++; - if (zio->io_error != 0) - continue; - - int error = arc_buf_alloc_impl(hdr, acb->acb_private, - acb->acb_compressed, - B_TRUE, &acb->acb_buf); - if (error != 0) { - arc_buf_destroy(acb->acb_buf, acb->acb_private); - acb->acb_buf = NULL; + if (no_zio_error) { + int error = arc_buf_alloc_impl(hdr, acb->acb_private, + acb->acb_compressed, zio->io_error == 0, + &acb->acb_buf); + if (error != 0) { + /* + * Decompression failed. Set io_error + * so that when we call acb_done (below), + * we will indicate that the read failed. + * Note that in the unusual case where one + * callback is compressed and another + * uncompressed, we will mark all of them + * as failed, even though the uncompressed + * one can't actually fail. In this case, + * the hdr will not be anonymous, because + * if there are multiple callbacks, it's + * because multiple threads found the same + * arc buf in the hash table. + */ + zio->io_error = error; + } } - - if (zio->io_error == 0) - zio->io_error = error; } + /* + * If there are multiple callbacks, we must have the hash lock, + * because the only way for multiple threads to find this hdr is + * in the hash table. This ensures that if there are multiple + * callbacks, the hdr is not anonymous. If it were anonymous, + * we couldn't use arc_buf_destroy() in the error case below. + */ + ASSERT(callback_cnt < 2 || hash_lock != NULL); + hdr->b_l1hdr.b_acb = NULL; arc_hdr_clear_flags(hdr, ARC_FLAG_IO_IN_PROGRESS); if (callback_cnt == 0) { @@ -5276,7 +5295,7 @@ arc_read_done(zio_t *zio) ASSERT(refcount_is_zero(&hdr->b_l1hdr.b_refcnt) || callback_list != NULL); - if (zio->io_error == 0) { + if (no_zio_error) { arc_hdr_verify(hdr, zio->io_bp); } else { arc_hdr_set_flags(hdr, ARC_FLAG_IO_ERROR); @@ -5309,7 +5328,16 @@ arc_read_done(zio_t *zio) /* execute each callback and free its structure */ while ((acb = callback_list) != NULL) { - if (acb->acb_done) { + if (acb->acb_done != NULL) { + if (zio->io_error != 0 && acb->acb_buf != NULL) { + /* + * If arc_buf_alloc_impl() fails during + * decompression, the buf will still be + * allocated, and needs to be freed here. + */ + arc_buf_destroy(acb->acb_buf, acb->acb_private); + acb->acb_buf = NULL; + } acb->acb_done(zio, &zio->io_bookmark, zio->io_bp, acb->acb_buf, acb->acb_private); } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c Wed Aug 1 00:41:02 2018 (r337024) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c Wed Aug 1 02:39:44 2018 (r337025) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2012, 2017 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2013, Joyent, Inc. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. @@ -1000,8 +1000,15 @@ dbuf_read_done(zio_t *zio, const zbookmark_phys_t *zb, ASSERT(refcount_count(&db->db_holds) > 0); ASSERT(db->db_buf == NULL); ASSERT(db->db.db_data == NULL); - if (db->db_level == 0 && db->db_freed_in_flight) { - /* we were freed in flight; disregard any error */ + if (buf == NULL) { + /* i/o error */ + ASSERT(zio == NULL || zio->io_error != 0); + ASSERT(db->db_blkid != DMU_BONUS_BLKID); + ASSERT3P(db->db_buf, ==, NULL); + db->db_state = DB_UNCACHED; + } else if (db->db_level == 0 && db->db_freed_in_flight) { + /* freed in flight */ + ASSERT(zio == NULL || zio->io_error == 0); if (buf == NULL) { buf = arc_alloc_buf(db->db_objset->os_spa, db, DBUF_GET_BUFC_TYPE(db), db->db.db_size); @@ -1012,13 +1019,11 @@ dbuf_read_done(zio_t *zio, const zbookmark_phys_t *zb, db->db_freed_in_flight = FALSE; dbuf_set_data(db, buf); db->db_state = DB_CACHED; - } else if (buf != NULL) { + } else { + /* success */ + ASSERT(zio == NULL || zio->io_error == 0); dbuf_set_data(db, buf); db->db_state = DB_CACHED; - } else { - ASSERT(db->db_blkid != DMU_BONUS_BLKID); - ASSERT3P(db->db_buf, ==, NULL); - db->db_state = DB_UNCACHED; } cv_broadcast(&db->db_changed); dbuf_rele_and_unlock(db, NULL); @@ -2431,6 +2436,13 @@ dbuf_prefetch_indirect_done(zio_t *zio, const zbookmar ASSERT3S(dpa->dpa_zb.zb_level, <, dpa->dpa_curlevel); ASSERT3S(dpa->dpa_curlevel, >, 0); + + if (abuf == NULL) { + ASSERT(zio == NULL || zio->io_error != 0); + kmem_free(dpa, sizeof (*dpa)); + return; + } + ASSERT(zio == NULL || zio->io_error == 0); /* * The dpa_dnode is only valid if we are called with a NULL Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c Wed Aug 1 00:41:02 2018 (r337024) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio_compress.c Wed Aug 1 02:39:44 2018 (r337025) @@ -25,7 +25,7 @@ */ /* * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. - * Copyright (c) 2013, 2016 by Delphix. All rights reserved. + * Copyright (c) 2013, 2018 by Delphix. All rights reserved. */ #include @@ -56,6 +56,12 @@ static zcomp_stats_t zcomp_stats = { kstat_t *zcomp_ksp; /* + * If nonzero, every 1/X decompression attempts will fail, simulating + * an undetected memory error. + */ +uint64_t zio_decompress_fail_fraction = 0; + +/* * Compression vectors. */ zio_compress_info_t zio_compress_table[ZIO_COMPRESS_FUNCTIONS] = { @@ -171,6 +177,16 @@ zio_decompress_data(enum zio_compress c, abd_t *src, v void *tmp = abd_borrow_buf_copy(src, s_len); int ret = zio_decompress_data_buf(c, tmp, dst, s_len, d_len); abd_return_buf(src, tmp, s_len); + + /* + * Decompression shouldn't fail, because we've already verifyied + * the checksum. However, for extra protection (e.g. against bitflips + * in non-ECC RAM), we handle this error (and test it). + */ + ASSERT0(ret); + if (zio_decompress_fail_fraction != 0 && + spa_get_random(zio_decompress_fail_fraction) == 0) + ret = SET_ERROR(EINVAL); return (ret); } From owner-svn-src-all@freebsd.org Wed Aug 1 02:59:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 553E4106DB1E; Wed, 1 Aug 2018 02:59:57 +0000 (UTC) (envelope-from cy@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F14348F931; Wed, 1 Aug 2018 02:59:56 +0000 (UTC) (envelope-from cy@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CCE8A1C8C9; Wed, 1 Aug 2018 02:59:56 +0000 (UTC) (envelope-from cy@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w712xuoT031704; Wed, 1 Aug 2018 02:59:56 GMT (envelope-from cy@FreeBSD.org) Received: (from cy@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w712xuHe031703; Wed, 1 Aug 2018 02:59:56 GMT (envelope-from cy@FreeBSD.org) Message-Id: <201808010259.w712xuHe031703@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cy set sender to cy@FreeBSD.org using -f From: Cy Schubert Date: Wed, 1 Aug 2018 02:59:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337026 - head/sys/dev/bge X-SVN-Group: head X-SVN-Commit-Author: cy X-SVN-Commit-Paths: head/sys/dev/bge X-SVN-Commit-Revision: 337026 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 02:59:57 -0000 Author: cy Date: Wed Aug 1 02:59:56 2018 New Revision: 337026 URL: https://svnweb.freebsd.org/changeset/base/337026 Log: As non-sparc64 undconditionally returns 1, simplify #ifdefs. MFC after: 1 week Modified: head/sys/dev/bge/if_bge.c Modified: head/sys/dev/bge/if_bge.c ============================================================================== --- head/sys/dev/bge/if_bge.c Wed Aug 1 02:39:44 2018 (r337025) +++ head/sys/dev/bge/if_bge.c Wed Aug 1 02:59:56 2018 (r337026) @@ -6724,15 +6724,15 @@ bge_sysctl_mem_read(SYSCTL_HANDLER_ARGS) static int bge_get_eaddr_fw(struct bge_softc *sc, uint8_t ether_addr[]) { - +#ifdef __sparc64__ if (sc->bge_flags & BGE_FLAG_EADDR) return (1); -#ifdef __sparc64__ OF_getetheraddr(sc->bge_dev, ether_addr); return (0); -#endif +#else return (1); +#endif } static int From owner-svn-src-all@freebsd.org Wed Aug 1 03:03:17 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 210F4106DE5E; Wed, 1 Aug 2018 03:03:17 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C352B701A1; Wed, 1 Aug 2018 03:03:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A42D81CADE; Wed, 1 Aug 2018 03:03:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7133GPV037373; Wed, 1 Aug 2018 03:03:16 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7133FfN037368; Wed, 1 Aug 2018 03:03:15 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808010303.w7133FfN037368@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 03:03:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337027 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337027 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 03:03:17 -0000 Author: mav Date: Wed Aug 1 03:03:15 2018 New Revision: 337027 URL: https://svnweb.freebsd.org/changeset/base/337027 Log: 9328 zap code can take advantage of c99 9329 panic in zap_leaf_lookup() due to concurrent zapification illumos/illumos-gate@bf26014c5541b6119f34e0d95294b7f2eb105ac2 Reviewed by: Steve Gonczi Reviewed by: George Wilson Reviewed by: Pavel Zakharov Reviewed by: Brad Lewis Approved by: Dan McDonald Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c vendor-sys/illumos/dist/uts/common/fs/zfs/zap_leaf.c vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Wed Aug 1 02:59:56 2018 (r337026) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Wed Aug 1 03:03:15 2018 (r337027) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, 2015 by Delphix. All rights reserved. + * Copyright (c) 2013, 2017 by Delphix. All rights reserved. * Copyright 2014 HybridCluster. All rights reserved. */ @@ -204,12 +204,18 @@ dmu_object_zapify(objset_t *mos, uint64_t object, dmu_ } ASSERT3U(dn->dn_type, ==, old_type); ASSERT0(dn->dn_maxblkid); + + /* + * We must initialize the ZAP data before changing the type, + * so that concurrent calls to *_is_zapified() can determine if + * the object has been completely zapified by checking the type. + */ + mzap_create_impl(mos, object, 0, 0, tx); + dn->dn_next_type[tx->tx_txg & TXG_MASK] = dn->dn_type = DMU_OTN_ZAP_METADATA; dnode_setdirty(dn, tx); dnode_rele(dn, FTAG); - - mzap_create_impl(mos, object, 0, 0, tx); spa_feature_incr(dmu_objset_spa(mos), SPA_FEATURE_EXTENSIBLE_DATASET, tx); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c Wed Aug 1 02:59:56 2018 (r337026) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zap.c Wed Aug 1 03:03:15 2018 (r337027) @@ -58,10 +58,8 @@ static uint64_t zap_allocate_blocks(zap_t *zap, int nb void fzap_byteswap(void *vbuf, size_t size) { - uint64_t block_type; + uint64_t block_type = *(uint64_t *)vbuf; - block_type = *(uint64_t *)vbuf; - if (block_type == ZBT_LEAF || block_type == BSWAP_64(ZBT_LEAF)) zap_leaf_byteswap(vbuf, size); else { @@ -73,11 +71,6 @@ fzap_byteswap(void *vbuf, size_t size) void fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t flags) { - dmu_buf_t *db; - zap_leaf_t *l; - int i; - zap_phys_t *zp; - ASSERT(RW_WRITE_HELD(&zap->zap_rwlock)); zap->zap_ismicro = FALSE; @@ -87,7 +80,7 @@ fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t fla mutex_init(&zap->zap_f.zap_num_entries_mtx, 0, 0, 0); zap->zap_f.zap_block_shift = highbit64(zap->zap_dbuf->db_size) - 1; - zp = zap_f_phys(zap); + zap_phys_t *zp = zap_f_phys(zap); /* * explicitly zero it since it might be coming from an * initialized microzap @@ -106,17 +99,18 @@ fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t fla zp->zap_flags = flags; /* block 1 will be the first leaf */ - for (i = 0; i < (1<zap_ptrtbl.zt_shift); i++) + for (int i = 0; i < (1<zap_ptrtbl.zt_shift); i++) ZAP_EMBEDDED_PTRTBL_ENT(zap, i) = 1; /* * set up block 1 - the first leaf */ - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + dmu_buf_t *db; + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, 1<l_dbuf = db; zap_leaf_init(l, zp->zap_normflags != 0); @@ -146,9 +140,7 @@ zap_table_grow(zap_t *zap, zap_table_phys_t *tbl, void (*transfer_func)(const uint64_t *src, uint64_t *dst, int n), dmu_tx_t *tx) { - uint64_t b, newblk; - dmu_buf_t *db_old, *db_new; - int err; + uint64_t newblk; int bs = FZAP_BLOCK_SHIFT(zap); int hepb = 1<<(bs-4); /* hepb = half the number of entries in a block */ @@ -172,21 +164,23 @@ zap_table_grow(zap_t *zap, zap_table_phys_t *tbl, * Copy the ptrtbl from the old to new location. */ - b = tbl->zt_blks_copied; - err = dmu_buf_hold(zap->zap_objset, zap->zap_object, + uint64_t b = tbl->zt_blks_copied; + dmu_buf_t *db_old; + int err = dmu_buf_hold(zap->zap_objset, zap->zap_object, (tbl->zt_blk + b) << bs, FTAG, &db_old, DMU_READ_NO_PREFETCH); - if (err) + if (err != 0) return (err); /* first half of entries in old[b] go to new[2*b+0] */ - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + dmu_buf_t *db_new; + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, (newblk + 2*b+0) << bs, FTAG, &db_new, DMU_READ_NO_PREFETCH)); dmu_buf_will_dirty(db_new, tx); transfer_func(db_old->db_data, db_new->db_data, hepb); dmu_buf_rele(db_new, FTAG); /* second half of entries in old[b] go to new[2*b+1] */ - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, (newblk + 2*b+1) << bs, FTAG, &db_new, DMU_READ_NO_PREFETCH)); dmu_buf_will_dirty(db_new, tx); transfer_func((uint64_t *)db_old->db_data + hepb, @@ -221,22 +215,20 @@ static int zap_table_store(zap_t *zap, zap_table_phys_t *tbl, uint64_t idx, uint64_t val, dmu_tx_t *tx) { - int err; - uint64_t blk, off; int bs = FZAP_BLOCK_SHIFT(zap); - dmu_buf_t *db; ASSERT(RW_LOCK_HELD(&zap->zap_rwlock)); ASSERT(tbl->zt_blk != 0); dprintf("storing %llx at index %llx\n", val, idx); - blk = idx >> (bs-3); - off = idx & ((1<<(bs-3))-1); + uint64_t blk = idx >> (bs-3); + uint64_t off = idx & ((1<<(bs-3))-1); - err = dmu_buf_hold(zap->zap_objset, zap->zap_object, + dmu_buf_t *db; + int err = dmu_buf_hold(zap->zap_objset, zap->zap_object, (tbl->zt_blk + blk) << bs, FTAG, &db, DMU_READ_NO_PREFETCH); - if (err) + if (err != 0) return (err); dmu_buf_will_dirty(db, tx); @@ -249,7 +241,7 @@ zap_table_store(zap_t *zap, zap_table_phys_t *tbl, uin err = dmu_buf_hold(zap->zap_objset, zap->zap_object, (tbl->zt_nextblk + blk2) << bs, FTAG, &db2, DMU_READ_NO_PREFETCH); - if (err) { + if (err != 0) { dmu_buf_rele(db, FTAG); return (err); } @@ -268,27 +260,24 @@ zap_table_store(zap_t *zap, zap_table_phys_t *tbl, uin static int zap_table_load(zap_t *zap, zap_table_phys_t *tbl, uint64_t idx, uint64_t *valp) { - uint64_t blk, off; - int err; - dmu_buf_t *db; - dnode_t *dn; int bs = FZAP_BLOCK_SHIFT(zap); ASSERT(RW_LOCK_HELD(&zap->zap_rwlock)); - blk = idx >> (bs-3); - off = idx & ((1<<(bs-3))-1); + uint64_t blk = idx >> (bs-3); + uint64_t off = idx & ((1<<(bs-3))-1); /* * Note: this is equivalent to dmu_buf_hold(), but we use * _dnode_enter / _by_dnode because it's faster because we don't * have to hold the dnode. */ - dn = dmu_buf_dnode_enter(zap->zap_dbuf); - err = dmu_buf_hold_by_dnode(dn, + dnode_t *dn = dmu_buf_dnode_enter(zap->zap_dbuf); + dmu_buf_t *db; + int err = dmu_buf_hold_by_dnode(dn, (tbl->zt_blk + blk) << bs, FTAG, &db, DMU_READ_NO_PREFETCH); dmu_buf_dnode_exit(zap->zap_dbuf); - if (err) + if (err != 0) return (err); *valp = ((uint64_t *)db->db_data)[off]; dmu_buf_rele(db, FTAG); @@ -319,11 +308,10 @@ zap_table_load(zap_t *zap, zap_table_phys_t *tbl, uint static void zap_ptrtbl_transfer(const uint64_t *src, uint64_t *dst, int n) { - int i; - for (i = 0; i < n; i++) { + for (int i = 0; i < n; i++) { uint64_t lb = src[i]; - dst[2*i+0] = lb; - dst[2*i+1] = lb; + dst[2 * i + 0] = lb; + dst[2 * i + 1] = lb; } } @@ -345,19 +333,16 @@ zap_grow_ptrtbl(zap_t *zap, dmu_tx_t *tx) * stored in the header block). Give it its own entire * block, which will double the size of the ptrtbl. */ - uint64_t newblk; - dmu_buf_t *db_new; - int err; - ASSERT3U(zap_f_phys(zap)->zap_ptrtbl.zt_shift, ==, ZAP_EMBEDDED_PTRTBL_SHIFT(zap)); ASSERT0(zap_f_phys(zap)->zap_ptrtbl.zt_blk); - newblk = zap_allocate_blocks(zap, 1); - err = dmu_buf_hold(zap->zap_objset, zap->zap_object, + uint64_t newblk = zap_allocate_blocks(zap, 1); + dmu_buf_t *db_new; + int err = dmu_buf_hold(zap->zap_objset, zap->zap_object, newblk << FZAP_BLOCK_SHIFT(zap), FTAG, &db_new, DMU_READ_NO_PREFETCH); - if (err) + if (err != 0) return (err); dmu_buf_will_dirty(db_new, tx); zap_ptrtbl_transfer(&ZAP_EMBEDDED_PTRTBL_ENT(zap, 0), @@ -392,9 +377,8 @@ zap_increment_num_entries(zap_t *zap, int delta, dmu_t static uint64_t zap_allocate_blocks(zap_t *zap, int nblocks) { - uint64_t newblk; ASSERT(RW_WRITE_HELD(&zap->zap_rwlock)); - newblk = zap_f_phys(zap)->zap_freeblk; + uint64_t newblk = zap_f_phys(zap)->zap_freeblk; zap_f_phys(zap)->zap_freeblk += nblocks; return (newblk); } @@ -411,7 +395,6 @@ zap_leaf_evict_sync(void *dbu) static zap_leaf_t * zap_create_leaf(zap_t *zap, dmu_tx_t *tx) { - void *winner; zap_leaf_t *l = kmem_zalloc(sizeof (zap_leaf_t), KM_SLEEP); ASSERT(RW_WRITE_HELD(&zap->zap_rwlock)); @@ -421,12 +404,11 @@ zap_create_leaf(zap_t *zap, dmu_tx_t *tx) l->l_blkid = zap_allocate_blocks(zap, 1); l->l_dbuf = NULL; - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, l->l_blkid << FZAP_BLOCK_SHIFT(zap), NULL, &l->l_dbuf, DMU_READ_NO_PREFETCH)); dmu_buf_init_user(&l->l_dbu, zap_leaf_evict_sync, NULL, &l->l_dbuf); - winner = dmu_buf_set_user(l->l_dbuf, &l->l_dbu); - ASSERT(winner == NULL); + VERIFY3P(NULL, ==, dmu_buf_set_user(l->l_dbuf, &l->l_dbu)); dmu_buf_will_dirty(l->l_dbuf, tx); zap_leaf_init(l, zap->zap_normflags != 0); @@ -460,11 +442,9 @@ zap_put_leaf(zap_leaf_t *l) static zap_leaf_t * zap_open_leaf(uint64_t blkid, dmu_buf_t *db) { - zap_leaf_t *l, *winner; - ASSERT(blkid != 0); - l = kmem_zalloc(sizeof (zap_leaf_t), KM_SLEEP); + zap_leaf_t *l = kmem_zalloc(sizeof (zap_leaf_t), KM_SLEEP); rw_init(&l->l_rwlock, 0, 0, 0); rw_enter(&l->l_rwlock, RW_WRITER); l->l_blkid = blkid; @@ -472,7 +452,7 @@ zap_open_leaf(uint64_t blkid, dmu_buf_t *db) l->l_dbuf = db; dmu_buf_init_user(&l->l_dbu, zap_leaf_evict_sync, NULL, &l->l_dbuf); - winner = dmu_buf_set_user(db, &l->l_dbu); + zap_leaf_t *winner = dmu_buf_set_user(db, &l->l_dbu); rw_exit(&l->l_rwlock); if (winner != NULL) { @@ -510,17 +490,15 @@ zap_get_leaf_byblk(zap_t *zap, uint64_t blkid, dmu_tx_ zap_leaf_t **lp) { dmu_buf_t *db; - zap_leaf_t *l; - int bs = FZAP_BLOCK_SHIFT(zap); - int err; ASSERT(RW_LOCK_HELD(&zap->zap_rwlock)); + int bs = FZAP_BLOCK_SHIFT(zap); dnode_t *dn = dmu_buf_dnode_enter(zap->zap_dbuf); - err = dmu_buf_hold_by_dnode(dn, + int err = dmu_buf_hold_by_dnode(dn, blkid << bs, NULL, &db, DMU_READ_NO_PREFETCH); dmu_buf_dnode_exit(zap->zap_dbuf); - if (err) + if (err != 0) return (err); ASSERT3U(db->db_object, ==, zap->zap_object); @@ -528,7 +506,7 @@ zap_get_leaf_byblk(zap_t *zap, uint64_t blkid, dmu_tx_ ASSERT3U(db->db_size, ==, 1 << bs); ASSERT(blkid != 0); - l = dmu_buf_get_user(db); + zap_leaf_t *l = dmu_buf_get_user(db); if (l == NULL) l = zap_open_leaf(blkid, db); @@ -583,8 +561,7 @@ zap_set_idx_to_blk(zap_t *zap, uint64_t idx, uint64_t static int zap_deref_leaf(zap_t *zap, uint64_t h, dmu_tx_t *tx, krw_t lt, zap_leaf_t **lp) { - uint64_t idx, blk; - int err; + uint64_t blk; ASSERT(zap->zap_dbuf == NULL || zap_f_phys(zap) == zap->zap_dbuf->db_data); @@ -596,8 +573,8 @@ zap_deref_leaf(zap_t *zap, uint64_t h, dmu_tx_t *tx, k return (SET_ERROR(EIO)); } - idx = ZAP_HASH_IDX(h, zap_f_phys(zap)->zap_ptrtbl.zt_shift); - err = zap_idx_to_blk(zap, idx, &blk); + uint64_t idx = ZAP_HASH_IDX(h, zap_f_phys(zap)->zap_ptrtbl.zt_shift); + int err = zap_idx_to_blk(zap, idx, &blk); if (err != 0) return (err); err = zap_get_leaf_byblk(zap, blk, tx, lt, lp); @@ -614,9 +591,7 @@ zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, { zap_t *zap = zn->zn_zap; uint64_t hash = zn->zn_hash; - zap_leaf_t *nl; - int prefix_diff, i, err; - uint64_t sibling; + int err; int old_prefix_len = zap_leaf_phys(l)->l_hdr.lh_prefix_len; ASSERT3U(old_prefix_len, <=, zap_f_phys(zap)->zap_ptrtbl.zt_shift); @@ -636,19 +611,19 @@ zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, err = zap_lockdir(os, object, tx, RW_WRITER, FALSE, FALSE, tag, &zn->zn_zap); zap = zn->zn_zap; - if (err) + if (err != 0) return (err); ASSERT(!zap->zap_ismicro); while (old_prefix_len == zap_f_phys(zap)->zap_ptrtbl.zt_shift) { err = zap_grow_ptrtbl(zap, tx); - if (err) + if (err != 0) return (err); } err = zap_deref_leaf(zap, hash, tx, RW_WRITER, &l); - if (err) + if (err != 0) return (err); if (zap_leaf_phys(l)->l_hdr.lh_prefix_len != old_prefix_len) { @@ -662,25 +637,26 @@ zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, ASSERT3U(ZAP_HASH_IDX(hash, old_prefix_len), ==, zap_leaf_phys(l)->l_hdr.lh_prefix); - prefix_diff = zap_f_phys(zap)->zap_ptrtbl.zt_shift - + int prefix_diff = zap_f_phys(zap)->zap_ptrtbl.zt_shift - (old_prefix_len + 1); - sibling = (ZAP_HASH_IDX(hash, old_prefix_len + 1) | 1) << prefix_diff; + uint64_t sibling = + (ZAP_HASH_IDX(hash, old_prefix_len + 1) | 1) << prefix_diff; /* check for i/o errors before doing zap_leaf_split */ - for (i = 0; i < (1ULL<l_blkid); } - nl = zap_create_leaf(zap, tx); + zap_leaf_t *nl = zap_create_leaf(zap, tx); zap_leaf_split(l, nl, zap->zap_normflags != 0); /* set sibling pointers */ - for (i = 0; i < (1ULL << prefix_diff); i++) { - err = zap_set_idx_to_blk(zap, sibling+i, nl->l_blkid, tx); + for (int i = 0; i < (1ULL << prefix_diff); i++) { + err = zap_set_idx_to_blk(zap, sibling + i, nl->l_blkid, tx); ASSERT0(err); /* we checked for i/o errors above */ } @@ -708,8 +684,6 @@ zap_put_leaf_maybe_grow_ptrtbl(zap_name_t *zn, zap_lea zap_put_leaf(l); if (leaffull || zap_f_phys(zap)->zap_ptrtbl.zt_nextblk) { - int err; - /* * We are in the middle of growing the pointer table, or * this leaf will soon make us grow it. @@ -719,10 +693,10 @@ zap_put_leaf_maybe_grow_ptrtbl(zap_name_t *zn, zap_lea uint64_t zapobj = zap->zap_object; zap_unlockdir(zap, tag); - err = zap_lockdir(os, zapobj, tx, + int err = zap_lockdir(os, zapobj, tx, RW_WRITER, FALSE, FALSE, tag, &zn->zn_zap); zap = zn->zn_zap; - if (err) + if (err != 0) return; } @@ -763,9 +737,8 @@ fzap_checksize(uint64_t integer_size, uint64_t num_int static int fzap_check(zap_name_t *zn, uint64_t integer_size, uint64_t num_integers) { - int err; - - if ((err = fzap_checkname(zn)) != 0) + int err = fzap_checkname(zn); + if (err != 0) return (err); return (fzap_checksize(integer_size, num_integers)); } @@ -779,10 +752,10 @@ fzap_lookup(zap_name_t *zn, char *realname, int rn_len, boolean_t *ncp) { zap_leaf_t *l; - int err; zap_entry_handle_t zeh; - if ((err = fzap_checkname(zn)) != 0) + int err = fzap_checkname(zn); + if (err != 0) return (err); err = zap_deref_leaf(zn->zn_zap, zn->zn_hash, NULL, RW_READER, &l); @@ -870,7 +843,8 @@ fzap_update(zap_name_t *zn, void *tag, dmu_tx_t *tx) { zap_leaf_t *l; - int err, create; + int err; + boolean_t create; zap_entry_handle_t zeh; zap_t *zap = zn->zn_zap; @@ -923,9 +897,9 @@ fzap_length(zap_name_t *zn, if (err != 0) goto out; - if (integer_size) + if (integer_size != 0) *integer_size = zeh.zeh_integer_size; - if (num_integers) + if (num_integers != 0) *num_integers = zeh.zeh_num_integers; out: zap_put_leaf(l); @@ -954,15 +928,14 @@ fzap_remove(zap_name_t *zn, dmu_tx_t *tx) void fzap_prefetch(zap_name_t *zn) { - uint64_t idx, blk; + uint64_t blk; zap_t *zap = zn->zn_zap; - int bs; - idx = ZAP_HASH_IDX(zn->zn_hash, + uint64_t idx = ZAP_HASH_IDX(zn->zn_hash, zap_f_phys(zap)->zap_ptrtbl.zt_shift); if (zap_idx_to_blk(zap, idx, &blk) != 0) return; - bs = FZAP_BLOCK_SHIFT(zap); + int bs = FZAP_BLOCK_SHIFT(zap); dmu_prefetch(zap->zap_objset, zap->zap_object, 0, blk << bs, 1 << bs, ZIO_PRIORITY_SYNC_READ); } @@ -975,9 +948,8 @@ uint64_t zap_create_link(objset_t *os, dmu_object_type_t ot, uint64_t parent_obj, const char *name, dmu_tx_t *tx) { - uint64_t new_obj; - - VERIFY((new_obj = zap_create(os, ot, DMU_OT_NONE, 0, tx)) > 0); + uint64_t new_obj = zap_create(os, ot, DMU_OT_NONE, 0, tx); + VERIFY(new_obj != 0); VERIFY0(zap_add(os, parent_obj, name, sizeof (uint64_t), 1, &new_obj, tx)); @@ -989,13 +961,12 @@ zap_value_search(objset_t *os, uint64_t zapobj, uint64 char *name) { zap_cursor_t zc; - zap_attribute_t *za; int err; if (mask == 0) mask = -1ULL; - za = kmem_alloc(sizeof (zap_attribute_t), KM_SLEEP); + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, zapobj); (err = zap_cursor_retrieve(&zc, za)) == 0; zap_cursor_advance(&zc)) { @@ -1005,7 +976,7 @@ zap_value_search(objset_t *os, uint64_t zapobj, uint64 } } zap_cursor_fini(&zc); - kmem_free(za, sizeof (zap_attribute_t)); + kmem_free(za, sizeof (*za)); return (err); } @@ -1013,23 +984,23 @@ int zap_join(objset_t *os, uint64_t fromobj, uint64_t intoobj, dmu_tx_t *tx) { zap_cursor_t zc; - zap_attribute_t za; - int err; + int err = 0; - err = 0; + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, fromobj); - zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_retrieve(&zc, za) == 0; (void) zap_cursor_advance(&zc)) { - if (za.za_integer_length != 8 || za.za_num_integers != 1) { + if (za->za_integer_length != 8 || za->za_num_integers != 1) { err = SET_ERROR(EINVAL); break; } - err = zap_add(os, intoobj, za.za_name, - 8, 1, &za.za_first_integer, tx); - if (err) + err = zap_add(os, intoobj, za->za_name, + 8, 1, &za->za_first_integer, tx); + if (err != 0) break; } zap_cursor_fini(&zc); + kmem_free(za, sizeof (*za)); return (err); } @@ -1038,23 +1009,23 @@ zap_join_key(objset_t *os, uint64_t fromobj, uint64_t uint64_t value, dmu_tx_t *tx) { zap_cursor_t zc; - zap_attribute_t za; - int err; + int err = 0; - err = 0; + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, fromobj); - zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_retrieve(&zc, za) == 0; (void) zap_cursor_advance(&zc)) { - if (za.za_integer_length != 8 || za.za_num_integers != 1) { + if (za->za_integer_length != 8 || za->za_num_integers != 1) { err = SET_ERROR(EINVAL); break; } - err = zap_add(os, intoobj, za.za_name, + err = zap_add(os, intoobj, za->za_name, 8, 1, &value, tx); - if (err) + if (err != 0) break; } zap_cursor_fini(&zc); + kmem_free(za, sizeof (*za)); return (err); } @@ -1063,29 +1034,29 @@ zap_join_increment(objset_t *os, uint64_t fromobj, uin dmu_tx_t *tx) { zap_cursor_t zc; - zap_attribute_t za; - int err; + int err = 0; - err = 0; + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, fromobj); - zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_retrieve(&zc, za) == 0; (void) zap_cursor_advance(&zc)) { uint64_t delta = 0; - if (za.za_integer_length != 8 || za.za_num_integers != 1) { + if (za->za_integer_length != 8 || za->za_num_integers != 1) { err = SET_ERROR(EINVAL); break; } - err = zap_lookup(os, intoobj, za.za_name, 8, 1, &delta); + err = zap_lookup(os, intoobj, za->za_name, 8, 1, &delta); if (err != 0 && err != ENOENT) break; - delta += za.za_first_integer; - err = zap_update(os, intoobj, za.za_name, 8, 1, &delta, tx); - if (err) + delta += za->za_first_integer; + err = zap_update(os, intoobj, za->za_name, 8, 1, &delta, tx); + if (err != 0) break; } zap_cursor_fini(&zc); + kmem_free(za, sizeof (*za)); return (err); } @@ -1150,12 +1121,11 @@ zap_increment(objset_t *os, uint64_t obj, const char * dmu_tx_t *tx) { uint64_t value = 0; - int err; if (delta == 0) return (0); - err = zap_lookup(os, obj, name, 8, 1, &value); + int err = zap_lookup(os, obj, name, 8, 1, &value); if (err != 0 && err != ENOENT) return (err); value += delta; @@ -1253,7 +1223,6 @@ again: static void zap_stats_ptrtbl(zap_t *zap, uint64_t *tbl, int len, zap_stats_t *zs) { - int i, err; uint64_t lastblk = 0; /* @@ -1261,14 +1230,14 @@ zap_stats_ptrtbl(zap_t *zap, uint64_t *tbl, int len, z * can hold, then it'll be accounted for more than once, since * we won't have lastblk. */ - for (i = 0; i < len; i++) { + for (int i = 0; i < len; i++) { zap_leaf_t *l; if (tbl[i] == lastblk) continue; lastblk = tbl[i]; - err = zap_get_leaf_byblk(zap, tbl[i], NULL, RW_READER, &l); + int err = zap_get_leaf_byblk(zap, tbl[i], NULL, RW_READER, &l); if (err == 0) { zap_leaf_stats(zap, l, zs); zap_put_leaf(l); @@ -1308,14 +1277,12 @@ fzap_get_stats(zap_t *zap, zap_stats_t *zs) zap_stats_ptrtbl(zap, &ZAP_EMBEDDED_PTRTBL_ENT(zap, 0), 1 << ZAP_EMBEDDED_PTRTBL_SHIFT(zap), zs); } else { - int b; - dmu_prefetch(zap->zap_objset, zap->zap_object, 0, zap_f_phys(zap)->zap_ptrtbl.zt_blk << bs, zap_f_phys(zap)->zap_ptrtbl.zt_numblks << bs, ZIO_PRIORITY_SYNC_READ); - for (b = 0; b < zap_f_phys(zap)->zap_ptrtbl.zt_numblks; + for (int b = 0; b < zap_f_phys(zap)->zap_ptrtbl.zt_numblks; b++) { dmu_buf_t *db; int err; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap_leaf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zap_leaf.c Wed Aug 1 02:59:56 2018 (r337026) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zap_leaf.c Wed Aug 1 03:03:15 2018 (r337027) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, 2015 by Delphix. All rights reserved. + * Copyright (c) 2013, 2016 by Delphix. All rights reserved. * Copyright 2017 Nexenta Systems, Inc. */ @@ -107,7 +107,6 @@ ldv(int len, const void *addr) void zap_leaf_byteswap(zap_leaf_phys_t *buf, int size) { - int i; zap_leaf_t l; dmu_buf_t l_dbuf; @@ -123,10 +122,10 @@ zap_leaf_byteswap(zap_leaf_phys_t *buf, int size) buf->l_hdr.lh_prefix_len = BSWAP_16(buf->l_hdr.lh_prefix_len); buf->l_hdr.lh_freelist = BSWAP_16(buf->l_hdr.lh_freelist); - for (i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(&l); i++) + for (int i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(&l); i++) buf->l_hash[i] = BSWAP_16(buf->l_hash[i]); - for (i = 0; i < ZAP_LEAF_NUMCHUNKS(&l); i++) { + for (int i = 0; i < ZAP_LEAF_NUMCHUNKS(&l); i++) { zap_leaf_chunk_t *lc = &ZAP_LEAF_CHUNK(&l, i); struct zap_leaf_entry *le; @@ -162,14 +161,12 @@ zap_leaf_byteswap(zap_leaf_phys_t *buf, int size) void zap_leaf_init(zap_leaf_t *l, boolean_t sort) { - int i; - l->l_bs = highbit64(l->l_dbuf->db_size) - 1; zap_memset(&zap_leaf_phys(l)->l_hdr, 0, sizeof (struct zap_leaf_header)); zap_memset(zap_leaf_phys(l)->l_hash, CHAIN_END, 2*ZAP_LEAF_HASH_NUMENTRIES(l)); - for (i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { + for (int i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { ZAP_LEAF_CHUNK(l, i).l_free.lf_type = ZAP_CHUNK_FREE; ZAP_LEAF_CHUNK(l, i).l_free.lf_next = i+1; } @@ -188,11 +185,9 @@ zap_leaf_init(zap_leaf_t *l, boolean_t sort) static uint16_t zap_leaf_chunk_alloc(zap_leaf_t *l) { - int chunk; - ASSERT(zap_leaf_phys(l)->l_hdr.lh_nfree > 0); - chunk = zap_leaf_phys(l)->l_hdr.lh_freelist; + int chunk = zap_leaf_phys(l)->l_hdr.lh_freelist; ASSERT3U(chunk, <, ZAP_LEAF_NUMCHUNKS(l)); ASSERT3U(ZAP_LEAF_CHUNK(l, chunk).l_free.lf_type, ==, ZAP_CHUNK_FREE); @@ -232,7 +227,7 @@ zap_leaf_array_create(zap_leaf_t *l, const char *buf, uint16_t *chunkp = &chunk_head; int byten = 0; uint64_t value = 0; - int shift = (integer_size-1)*8; + int shift = (integer_size - 1) * 8; int len = num_integers; ASSERT3U(num_integers * integer_size, <, MAX_ARRAY_BYTES); @@ -240,10 +235,9 @@ zap_leaf_array_create(zap_leaf_t *l, const char *buf, while (len > 0) { uint16_t chunk = zap_leaf_chunk_alloc(l); struct zap_leaf_array *la = &ZAP_LEAF_CHUNK(l, chunk).l_array; - int i; la->la_type = ZAP_CHUNK_ARRAY; - for (i = 0; i < ZAP_LEAF_ARRAY_BYTES; i++) { + for (int i = 0; i < ZAP_LEAF_ARRAY_BYTES; i++) { if (byten == 0) value = ldv(integer_size, buf); la->la_array[i] = value >> shift; @@ -321,10 +315,9 @@ zap_leaf_array_read(zap_leaf_t *l, uint16_t chunk, while (len > 0) { struct zap_leaf_array *la = &ZAP_LEAF_CHUNK(l, chunk).l_array; - int i; ASSERT3U(chunk, <, ZAP_LEAF_NUMCHUNKS(l)); - for (i = 0; i < ZAP_LEAF_ARRAY_BYTES && len > 0; i++) { + for (int i = 0; i < ZAP_LEAF_ARRAY_BYTES && len > 0; i++) { value = (value << 8) | la->la_array[i]; byten++; if (byten == array_int_len) { @@ -347,16 +340,13 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, int bseen = 0; if (zap_getflags(zn->zn_zap) & ZAP_FLAG_UINT64_KEY) { - uint64_t *thiskey; - boolean_t match; - + uint64_t *thiskey = + kmem_alloc(array_numints * sizeof (*thiskey), KM_SLEEP); ASSERT(zn->zn_key_intlen == sizeof (*thiskey)); - thiskey = kmem_alloc(array_numints * sizeof (*thiskey), - KM_SLEEP); zap_leaf_array_read(l, chunk, sizeof (*thiskey), array_numints, sizeof (*thiskey), array_numints, thiskey); - match = bcmp(thiskey, zn->zn_key_orig, + boolean_t match = bcmp(thiskey, zn->zn_key_orig, array_numints * sizeof (*thiskey)) == 0; kmem_free(thiskey, array_numints * sizeof (*thiskey)); return (match); @@ -365,11 +355,10 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, ASSERT(zn->zn_key_intlen == 1); if (zn->zn_matchtype & MT_NORMALIZE) { char *thisname = kmem_alloc(array_numints, KM_SLEEP); - boolean_t match; zap_leaf_array_read(l, chunk, sizeof (char), array_numints, sizeof (char), array_numints, thisname); - match = zap_match(zn, thisname); + boolean_t match = zap_match(zn, thisname); kmem_free(thisname, array_numints); return (match); } @@ -400,12 +389,11 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, int zap_leaf_lookup(zap_leaf_t *l, zap_name_t *zn, zap_entry_handle_t *zeh) { - uint16_t *chunkp; struct zap_leaf_entry *le; ASSERT3U(zap_leaf_phys(l)->l_hdr.lh_magic, ==, ZAP_LEAF_MAGIC); - for (chunkp = LEAF_HASH_ENTPTR(l, zn->zn_hash); + for (uint16_t *chunkp = LEAF_HASH_ENTPTR(l, zn->zn_hash); *chunkp != CHAIN_END; chunkp = &le->le_next) { uint16_t chunk = *chunkp; le = ZAP_LEAF_ENTRY(l, chunk); @@ -446,17 +434,15 @@ int zap_leaf_lookup_closest(zap_leaf_t *l, uint64_t h, uint32_t cd, zap_entry_handle_t *zeh) { - uint16_t chunk; uint64_t besth = -1ULL; uint32_t bestcd = -1U; uint16_t bestlh = ZAP_LEAF_HASH_NUMENTRIES(l)-1; - uint16_t lh; struct zap_leaf_entry *le; ASSERT3U(zap_leaf_phys(l)->l_hdr.lh_magic, ==, ZAP_LEAF_MAGIC); - for (lh = LEAF_HASH(l, h); lh <= bestlh; lh++) { - for (chunk = zap_leaf_phys(l)->l_hash[lh]; + for (uint16_t lh = LEAF_HASH(l, h); lh <= bestlh; lh++) { + for (uint16_t chunk = zap_leaf_phys(l)->l_hash[lh]; chunk != CHAIN_END; chunk = le->le_next) { le = ZAP_LEAF_ENTRY(l, chunk); @@ -529,11 +515,10 @@ int zap_entry_update(zap_entry_handle_t *zeh, uint8_t integer_size, uint64_t num_integers, const void *buf) { - int delta_chunks; zap_leaf_t *l = zeh->zeh_leaf; struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, *zeh->zeh_chunkp); - delta_chunks = ZAP_LEAF_ARRAY_NCHUNKS(num_integers * integer_size) - + int delta_chunks = ZAP_LEAF_ARRAY_NCHUNKS(num_integers * integer_size) - ZAP_LEAF_ARRAY_NCHUNKS(le->le_value_numints * le->le_value_intlen); if ((int)zap_leaf_phys(l)->l_hdr.lh_nfree < delta_chunks) @@ -550,14 +535,12 @@ zap_entry_update(zap_entry_handle_t *zeh, void zap_entry_remove(zap_entry_handle_t *zeh) { - uint16_t entry_chunk; - struct zap_leaf_entry *le; zap_leaf_t *l = zeh->zeh_leaf; ASSERT3P(zeh->zeh_chunkp, !=, &zeh->zeh_fakechunk); - entry_chunk = *zeh->zeh_chunkp; - le = ZAP_LEAF_ENTRY(l, entry_chunk); + uint16_t entry_chunk = *zeh->zeh_chunkp; + struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, entry_chunk); ASSERT3U(le->le_type, ==, ZAP_CHUNK_ENTRY); zap_leaf_array_free(l, &le->le_name_chunk); @@ -575,15 +558,12 @@ zap_entry_create(zap_leaf_t *l, zap_name_t *zn, uint32 zap_entry_handle_t *zeh) { uint16_t chunk; - uint16_t *chunkp; struct zap_leaf_entry *le; - uint64_t valuelen; - int numchunks; uint64_t h = zn->zn_hash; - valuelen = integer_size * num_integers; + uint64_t valuelen = integer_size * num_integers; - numchunks = 1 + ZAP_LEAF_ARRAY_NCHUNKS(zn->zn_key_orig_numints * + int numchunks = 1 + ZAP_LEAF_ARRAY_NCHUNKS(zn->zn_key_orig_numints * zn->zn_key_intlen) + ZAP_LEAF_ARRAY_NCHUNKS(valuelen); if (numchunks > ZAP_LEAF_NUMCHUNKS(l)) return (E2BIG); @@ -645,7 +625,7 @@ zap_entry_create(zap_leaf_t *l, zap_name_t *zn, uint32 /* link it into the hash chain */ /* XXX if we did the search above, we could just use that */ - chunkp = zap_leaf_rehash_entry(l, chunk); + uint16_t *chunkp = zap_leaf_rehash_entry(l, chunk); zap_leaf_phys(l)->l_hdr.lh_nentries++; @@ -673,14 +653,13 @@ boolean_t zap_entry_normalization_conflict(zap_entry_handle_t *zeh, zap_name_t *zn, const char *name, zap_t *zap) { - uint64_t chunk; struct zap_leaf_entry *le; boolean_t allocdzn = B_FALSE; if (zap->zap_normflags == 0) return (B_FALSE); - for (chunk = *LEAF_HASH_ENTPTR(zeh->zeh_leaf, zeh->zeh_hash); + for (uint16_t chunk = *LEAF_HASH_ENTPTR(zeh->zeh_leaf, zeh->zeh_hash); chunk != CHAIN_END; chunk = le->le_next) { le = ZAP_LEAF_ENTRY(zeh->zeh_leaf, chunk); if (le->le_hash != zeh->zeh_hash) @@ -763,14 +742,11 @@ zap_leaf_transfer_array(zap_leaf_t *l, uint16_t chunk, static void zap_leaf_transfer_entry(zap_leaf_t *l, int entry, zap_leaf_t *nl) { - struct zap_leaf_entry *le, *nle; - uint16_t chunk; - - le = ZAP_LEAF_ENTRY(l, entry); + struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, entry); ASSERT3U(le->le_type, ==, ZAP_CHUNK_ENTRY); - chunk = zap_leaf_chunk_alloc(nl); - nle = ZAP_LEAF_ENTRY(nl, chunk); + uint16_t chunk = zap_leaf_chunk_alloc(nl); + struct zap_leaf_entry *nle = ZAP_LEAF_ENTRY(nl, chunk); *nle = *le; /* structure assignment */ (void) zap_leaf_rehash_entry(nl, chunk); @@ -791,7 +767,6 @@ zap_leaf_transfer_entry(zap_leaf_t *l, int entry, zap_ void zap_leaf_split(zap_leaf_t *l, zap_leaf_t *nl, boolean_t sort) { - int i; int bit = 64 - 1 - zap_leaf_phys(l)->l_hdr.lh_prefix_len; /* set new prefix and prefix_len */ @@ -818,7 +793,7 @@ zap_leaf_split(zap_leaf_t *l, zap_leaf_t *nl, boolean_ * but this accesses memory more sequentially, and when we're * called, the block is usually pretty full. */ - for (i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { + for (int i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, i); if (le->le_type != ZAP_CHUNK_ENTRY) continue; @@ -833,9 +808,7 @@ zap_leaf_split(zap_leaf_t *l, zap_leaf_t *nl, boolean_ void zap_leaf_stats(zap_t *zap, zap_leaf_t *l, zap_stats_t *zs) { - int i, n; - - n = zap_f_phys(zap)->zap_ptrtbl.zt_shift - + int n = zap_f_phys(zap)->zap_ptrtbl.zt_shift - zap_leaf_phys(l)->l_hdr.lh_prefix_len; n = MIN(n, ZAP_HISTOGRAM_SIZE-1); zs->zs_leafs_with_2n_pointers[n]++; @@ -851,7 +824,7 @@ zap_leaf_stats(zap_t *zap, zap_leaf_t *l, zap_stats_t n = MIN(n, ZAP_HISTOGRAM_SIZE-1); zs->zs_blocks_n_tenths_full[n]++; - for (i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(l); i++) { + for (int i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(l); i++) { int nentries = 0; int chunk = zap_leaf_phys(l)->l_hash[i]; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Wed Aug 1 02:59:56 2018 (r337026) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zap_micro.c Wed Aug 1 03:03:15 2018 (r337027) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2016 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2017 Nexenta Systems, Inc. @@ -89,22 +89,20 @@ zap_hash(zap_name_t *zn) ASSERT(zfs_crc64_table[128] == ZFS_CRC64_POLY); if (zap_getflags(zap) & ZAP_FLAG_UINT64_KEY) { *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Wed Aug 1 03:07:34 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5078106DF71; Wed, 1 Aug 2018 03:07:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56C4870371; Wed, 1 Aug 2018 03:07:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 380E91CAE8; Wed, 1 Aug 2018 03:07:34 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7137YQt037613; Wed, 1 Aug 2018 03:07:34 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7137XWg037608; Wed, 1 Aug 2018 03:07:33 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808010307.w7137XWg037608@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 03:07:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337028 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337028 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 03:07:35 -0000 Author: mav Date: Wed Aug 1 03:07:33 2018 New Revision: 337028 URL: https://svnweb.freebsd.org/changeset/base/337028 Log: MFV r337027: 9328 zap code can take advantage of c99 9329 panic in zap_leaf_lookup() due to concurrent zapification illumos/illumos-gate@bf26014c5541b6119f34e0d95294b7f2eb105ac2 Reviewed by: Steve Gonczi Reviewed by: George Wilson Reviewed by: Pavel Zakharov Reviewed by: Brad Lewis Approved by: Dan McDonald Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c Wed Aug 1 03:03:15 2018 (r337027) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c Wed Aug 1 03:07:33 2018 (r337028) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, 2015 by Delphix. All rights reserved. + * Copyright (c) 2013, 2017 by Delphix. All rights reserved. * Copyright 2014 HybridCluster. All rights reserved. */ @@ -204,12 +204,18 @@ dmu_object_zapify(objset_t *mos, uint64_t object, dmu_ } ASSERT3U(dn->dn_type, ==, old_type); ASSERT0(dn->dn_maxblkid); + + /* + * We must initialize the ZAP data before changing the type, + * so that concurrent calls to *_is_zapified() can determine if + * the object has been completely zapified by checking the type. + */ + mzap_create_impl(mos, object, 0, 0, tx); + dn->dn_next_type[tx->tx_txg & TXG_MASK] = dn->dn_type = DMU_OTN_ZAP_METADATA; dnode_setdirty(dn, tx); dnode_rele(dn, FTAG); - - mzap_create_impl(mos, object, 0, 0, tx); spa_feature_incr(dmu_objset_spa(mos), SPA_FEATURE_EXTENSIBLE_DATASET, tx); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c Wed Aug 1 03:03:15 2018 (r337027) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c Wed Aug 1 03:07:33 2018 (r337028) @@ -58,10 +58,8 @@ static uint64_t zap_allocate_blocks(zap_t *zap, int nb void fzap_byteswap(void *vbuf, size_t size) { - uint64_t block_type; + uint64_t block_type = *(uint64_t *)vbuf; - block_type = *(uint64_t *)vbuf; - if (block_type == ZBT_LEAF || block_type == BSWAP_64(ZBT_LEAF)) zap_leaf_byteswap(vbuf, size); else { @@ -73,11 +71,6 @@ fzap_byteswap(void *vbuf, size_t size) void fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t flags) { - dmu_buf_t *db; - zap_leaf_t *l; - int i; - zap_phys_t *zp; - ASSERT(RW_WRITE_HELD(&zap->zap_rwlock)); zap->zap_ismicro = FALSE; @@ -87,7 +80,7 @@ fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t fla mutex_init(&zap->zap_f.zap_num_entries_mtx, 0, 0, 0); zap->zap_f.zap_block_shift = highbit64(zap->zap_dbuf->db_size) - 1; - zp = zap_f_phys(zap); + zap_phys_t *zp = zap_f_phys(zap); /* * explicitly zero it since it might be coming from an * initialized microzap @@ -106,17 +99,18 @@ fzap_upgrade(zap_t *zap, dmu_tx_t *tx, zap_flags_t fla zp->zap_flags = flags; /* block 1 will be the first leaf */ - for (i = 0; i < (1<zap_ptrtbl.zt_shift); i++) + for (int i = 0; i < (1<zap_ptrtbl.zt_shift); i++) ZAP_EMBEDDED_PTRTBL_ENT(zap, i) = 1; /* * set up block 1 - the first leaf */ - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + dmu_buf_t *db; + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, 1<l_dbuf = db; zap_leaf_init(l, zp->zap_normflags != 0); @@ -146,9 +140,7 @@ zap_table_grow(zap_t *zap, zap_table_phys_t *tbl, void (*transfer_func)(const uint64_t *src, uint64_t *dst, int n), dmu_tx_t *tx) { - uint64_t b, newblk; - dmu_buf_t *db_old, *db_new; - int err; + uint64_t newblk; int bs = FZAP_BLOCK_SHIFT(zap); int hepb = 1<<(bs-4); /* hepb = half the number of entries in a block */ @@ -172,21 +164,23 @@ zap_table_grow(zap_t *zap, zap_table_phys_t *tbl, * Copy the ptrtbl from the old to new location. */ - b = tbl->zt_blks_copied; - err = dmu_buf_hold(zap->zap_objset, zap->zap_object, + uint64_t b = tbl->zt_blks_copied; + dmu_buf_t *db_old; + int err = dmu_buf_hold(zap->zap_objset, zap->zap_object, (tbl->zt_blk + b) << bs, FTAG, &db_old, DMU_READ_NO_PREFETCH); - if (err) + if (err != 0) return (err); /* first half of entries in old[b] go to new[2*b+0] */ - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + dmu_buf_t *db_new; + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, (newblk + 2*b+0) << bs, FTAG, &db_new, DMU_READ_NO_PREFETCH)); dmu_buf_will_dirty(db_new, tx); transfer_func(db_old->db_data, db_new->db_data, hepb); dmu_buf_rele(db_new, FTAG); /* second half of entries in old[b] go to new[2*b+1] */ - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, (newblk + 2*b+1) << bs, FTAG, &db_new, DMU_READ_NO_PREFETCH)); dmu_buf_will_dirty(db_new, tx); transfer_func((uint64_t *)db_old->db_data + hepb, @@ -221,22 +215,20 @@ static int zap_table_store(zap_t *zap, zap_table_phys_t *tbl, uint64_t idx, uint64_t val, dmu_tx_t *tx) { - int err; - uint64_t blk, off; int bs = FZAP_BLOCK_SHIFT(zap); - dmu_buf_t *db; ASSERT(RW_LOCK_HELD(&zap->zap_rwlock)); ASSERT(tbl->zt_blk != 0); dprintf("storing %llx at index %llx\n", val, idx); - blk = idx >> (bs-3); - off = idx & ((1<<(bs-3))-1); + uint64_t blk = idx >> (bs-3); + uint64_t off = idx & ((1<<(bs-3))-1); - err = dmu_buf_hold(zap->zap_objset, zap->zap_object, + dmu_buf_t *db; + int err = dmu_buf_hold(zap->zap_objset, zap->zap_object, (tbl->zt_blk + blk) << bs, FTAG, &db, DMU_READ_NO_PREFETCH); - if (err) + if (err != 0) return (err); dmu_buf_will_dirty(db, tx); @@ -249,7 +241,7 @@ zap_table_store(zap_t *zap, zap_table_phys_t *tbl, uin err = dmu_buf_hold(zap->zap_objset, zap->zap_object, (tbl->zt_nextblk + blk2) << bs, FTAG, &db2, DMU_READ_NO_PREFETCH); - if (err) { + if (err != 0) { dmu_buf_rele(db, FTAG); return (err); } @@ -268,27 +260,24 @@ zap_table_store(zap_t *zap, zap_table_phys_t *tbl, uin static int zap_table_load(zap_t *zap, zap_table_phys_t *tbl, uint64_t idx, uint64_t *valp) { - uint64_t blk, off; - int err; - dmu_buf_t *db; - dnode_t *dn; int bs = FZAP_BLOCK_SHIFT(zap); ASSERT(RW_LOCK_HELD(&zap->zap_rwlock)); - blk = idx >> (bs-3); - off = idx & ((1<<(bs-3))-1); + uint64_t blk = idx >> (bs-3); + uint64_t off = idx & ((1<<(bs-3))-1); /* * Note: this is equivalent to dmu_buf_hold(), but we use * _dnode_enter / _by_dnode because it's faster because we don't * have to hold the dnode. */ - dn = dmu_buf_dnode_enter(zap->zap_dbuf); - err = dmu_buf_hold_by_dnode(dn, + dnode_t *dn = dmu_buf_dnode_enter(zap->zap_dbuf); + dmu_buf_t *db; + int err = dmu_buf_hold_by_dnode(dn, (tbl->zt_blk + blk) << bs, FTAG, &db, DMU_READ_NO_PREFETCH); dmu_buf_dnode_exit(zap->zap_dbuf); - if (err) + if (err != 0) return (err); *valp = ((uint64_t *)db->db_data)[off]; dmu_buf_rele(db, FTAG); @@ -319,11 +308,10 @@ zap_table_load(zap_t *zap, zap_table_phys_t *tbl, uint static void zap_ptrtbl_transfer(const uint64_t *src, uint64_t *dst, int n) { - int i; - for (i = 0; i < n; i++) { + for (int i = 0; i < n; i++) { uint64_t lb = src[i]; - dst[2*i+0] = lb; - dst[2*i+1] = lb; + dst[2 * i + 0] = lb; + dst[2 * i + 1] = lb; } } @@ -345,19 +333,16 @@ zap_grow_ptrtbl(zap_t *zap, dmu_tx_t *tx) * stored in the header block). Give it its own entire * block, which will double the size of the ptrtbl. */ - uint64_t newblk; - dmu_buf_t *db_new; - int err; - ASSERT3U(zap_f_phys(zap)->zap_ptrtbl.zt_shift, ==, ZAP_EMBEDDED_PTRTBL_SHIFT(zap)); ASSERT0(zap_f_phys(zap)->zap_ptrtbl.zt_blk); - newblk = zap_allocate_blocks(zap, 1); - err = dmu_buf_hold(zap->zap_objset, zap->zap_object, + uint64_t newblk = zap_allocate_blocks(zap, 1); + dmu_buf_t *db_new; + int err = dmu_buf_hold(zap->zap_objset, zap->zap_object, newblk << FZAP_BLOCK_SHIFT(zap), FTAG, &db_new, DMU_READ_NO_PREFETCH); - if (err) + if (err != 0) return (err); dmu_buf_will_dirty(db_new, tx); zap_ptrtbl_transfer(&ZAP_EMBEDDED_PTRTBL_ENT(zap, 0), @@ -392,9 +377,8 @@ zap_increment_num_entries(zap_t *zap, int delta, dmu_t static uint64_t zap_allocate_blocks(zap_t *zap, int nblocks) { - uint64_t newblk; ASSERT(RW_WRITE_HELD(&zap->zap_rwlock)); - newblk = zap_f_phys(zap)->zap_freeblk; + uint64_t newblk = zap_f_phys(zap)->zap_freeblk; zap_f_phys(zap)->zap_freeblk += nblocks; return (newblk); } @@ -411,7 +395,6 @@ zap_leaf_evict_sync(void *dbu) static zap_leaf_t * zap_create_leaf(zap_t *zap, dmu_tx_t *tx) { - void *winner; zap_leaf_t *l = kmem_zalloc(sizeof (zap_leaf_t), KM_SLEEP); ASSERT(RW_WRITE_HELD(&zap->zap_rwlock)); @@ -421,12 +404,11 @@ zap_create_leaf(zap_t *zap, dmu_tx_t *tx) l->l_blkid = zap_allocate_blocks(zap, 1); l->l_dbuf = NULL; - VERIFY(0 == dmu_buf_hold(zap->zap_objset, zap->zap_object, + VERIFY0(dmu_buf_hold(zap->zap_objset, zap->zap_object, l->l_blkid << FZAP_BLOCK_SHIFT(zap), NULL, &l->l_dbuf, DMU_READ_NO_PREFETCH)); dmu_buf_init_user(&l->l_dbu, zap_leaf_evict_sync, NULL, &l->l_dbuf); - winner = dmu_buf_set_user(l->l_dbuf, &l->l_dbu); - ASSERT(winner == NULL); + VERIFY3P(NULL, ==, dmu_buf_set_user(l->l_dbuf, &l->l_dbu)); dmu_buf_will_dirty(l->l_dbuf, tx); zap_leaf_init(l, zap->zap_normflags != 0); @@ -460,11 +442,9 @@ zap_put_leaf(zap_leaf_t *l) static zap_leaf_t * zap_open_leaf(uint64_t blkid, dmu_buf_t *db) { - zap_leaf_t *l, *winner; - ASSERT(blkid != 0); - l = kmem_zalloc(sizeof (zap_leaf_t), KM_SLEEP); + zap_leaf_t *l = kmem_zalloc(sizeof (zap_leaf_t), KM_SLEEP); rw_init(&l->l_rwlock, 0, 0, 0); rw_enter(&l->l_rwlock, RW_WRITER); l->l_blkid = blkid; @@ -472,7 +452,7 @@ zap_open_leaf(uint64_t blkid, dmu_buf_t *db) l->l_dbuf = db; dmu_buf_init_user(&l->l_dbu, zap_leaf_evict_sync, NULL, &l->l_dbuf); - winner = dmu_buf_set_user(db, &l->l_dbu); + zap_leaf_t *winner = dmu_buf_set_user(db, &l->l_dbu); rw_exit(&l->l_rwlock); if (winner != NULL) { @@ -510,17 +490,15 @@ zap_get_leaf_byblk(zap_t *zap, uint64_t blkid, dmu_tx_ zap_leaf_t **lp) { dmu_buf_t *db; - zap_leaf_t *l; - int bs = FZAP_BLOCK_SHIFT(zap); - int err; ASSERT(RW_LOCK_HELD(&zap->zap_rwlock)); + int bs = FZAP_BLOCK_SHIFT(zap); dnode_t *dn = dmu_buf_dnode_enter(zap->zap_dbuf); - err = dmu_buf_hold_by_dnode(dn, + int err = dmu_buf_hold_by_dnode(dn, blkid << bs, NULL, &db, DMU_READ_NO_PREFETCH); dmu_buf_dnode_exit(zap->zap_dbuf); - if (err) + if (err != 0) return (err); ASSERT3U(db->db_object, ==, zap->zap_object); @@ -528,7 +506,7 @@ zap_get_leaf_byblk(zap_t *zap, uint64_t blkid, dmu_tx_ ASSERT3U(db->db_size, ==, 1 << bs); ASSERT(blkid != 0); - l = dmu_buf_get_user(db); + zap_leaf_t *l = dmu_buf_get_user(db); if (l == NULL) l = zap_open_leaf(blkid, db); @@ -583,8 +561,7 @@ zap_set_idx_to_blk(zap_t *zap, uint64_t idx, uint64_t static int zap_deref_leaf(zap_t *zap, uint64_t h, dmu_tx_t *tx, krw_t lt, zap_leaf_t **lp) { - uint64_t idx, blk; - int err; + uint64_t blk; ASSERT(zap->zap_dbuf == NULL || zap_f_phys(zap) == zap->zap_dbuf->db_data); @@ -596,8 +573,8 @@ zap_deref_leaf(zap_t *zap, uint64_t h, dmu_tx_t *tx, k return (SET_ERROR(EIO)); } - idx = ZAP_HASH_IDX(h, zap_f_phys(zap)->zap_ptrtbl.zt_shift); - err = zap_idx_to_blk(zap, idx, &blk); + uint64_t idx = ZAP_HASH_IDX(h, zap_f_phys(zap)->zap_ptrtbl.zt_shift); + int err = zap_idx_to_blk(zap, idx, &blk); if (err != 0) return (err); err = zap_get_leaf_byblk(zap, blk, tx, lt, lp); @@ -614,9 +591,7 @@ zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, { zap_t *zap = zn->zn_zap; uint64_t hash = zn->zn_hash; - zap_leaf_t *nl; - int prefix_diff, i, err; - uint64_t sibling; + int err; int old_prefix_len = zap_leaf_phys(l)->l_hdr.lh_prefix_len; ASSERT3U(old_prefix_len, <=, zap_f_phys(zap)->zap_ptrtbl.zt_shift); @@ -636,19 +611,19 @@ zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, err = zap_lockdir(os, object, tx, RW_WRITER, FALSE, FALSE, tag, &zn->zn_zap); zap = zn->zn_zap; - if (err) + if (err != 0) return (err); ASSERT(!zap->zap_ismicro); while (old_prefix_len == zap_f_phys(zap)->zap_ptrtbl.zt_shift) { err = zap_grow_ptrtbl(zap, tx); - if (err) + if (err != 0) return (err); } err = zap_deref_leaf(zap, hash, tx, RW_WRITER, &l); - if (err) + if (err != 0) return (err); if (zap_leaf_phys(l)->l_hdr.lh_prefix_len != old_prefix_len) { @@ -662,25 +637,26 @@ zap_expand_leaf(zap_name_t *zn, zap_leaf_t *l, ASSERT3U(ZAP_HASH_IDX(hash, old_prefix_len), ==, zap_leaf_phys(l)->l_hdr.lh_prefix); - prefix_diff = zap_f_phys(zap)->zap_ptrtbl.zt_shift - + int prefix_diff = zap_f_phys(zap)->zap_ptrtbl.zt_shift - (old_prefix_len + 1); - sibling = (ZAP_HASH_IDX(hash, old_prefix_len + 1) | 1) << prefix_diff; + uint64_t sibling = + (ZAP_HASH_IDX(hash, old_prefix_len + 1) | 1) << prefix_diff; /* check for i/o errors before doing zap_leaf_split */ - for (i = 0; i < (1ULL<l_blkid); } - nl = zap_create_leaf(zap, tx); + zap_leaf_t *nl = zap_create_leaf(zap, tx); zap_leaf_split(l, nl, zap->zap_normflags != 0); /* set sibling pointers */ - for (i = 0; i < (1ULL << prefix_diff); i++) { - err = zap_set_idx_to_blk(zap, sibling+i, nl->l_blkid, tx); + for (int i = 0; i < (1ULL << prefix_diff); i++) { + err = zap_set_idx_to_blk(zap, sibling + i, nl->l_blkid, tx); ASSERT0(err); /* we checked for i/o errors above */ } @@ -708,8 +684,6 @@ zap_put_leaf_maybe_grow_ptrtbl(zap_name_t *zn, zap_lea zap_put_leaf(l); if (leaffull || zap_f_phys(zap)->zap_ptrtbl.zt_nextblk) { - int err; - /* * We are in the middle of growing the pointer table, or * this leaf will soon make us grow it. @@ -719,10 +693,10 @@ zap_put_leaf_maybe_grow_ptrtbl(zap_name_t *zn, zap_lea uint64_t zapobj = zap->zap_object; zap_unlockdir(zap, tag); - err = zap_lockdir(os, zapobj, tx, + int err = zap_lockdir(os, zapobj, tx, RW_WRITER, FALSE, FALSE, tag, &zn->zn_zap); zap = zn->zn_zap; - if (err) + if (err != 0) return; } @@ -763,9 +737,8 @@ fzap_checksize(uint64_t integer_size, uint64_t num_int static int fzap_check(zap_name_t *zn, uint64_t integer_size, uint64_t num_integers) { - int err; - - if ((err = fzap_checkname(zn)) != 0) + int err = fzap_checkname(zn); + if (err != 0) return (err); return (fzap_checksize(integer_size, num_integers)); } @@ -779,10 +752,10 @@ fzap_lookup(zap_name_t *zn, char *realname, int rn_len, boolean_t *ncp) { zap_leaf_t *l; - int err; zap_entry_handle_t zeh; - if ((err = fzap_checkname(zn)) != 0) + int err = fzap_checkname(zn); + if (err != 0) return (err); err = zap_deref_leaf(zn->zn_zap, zn->zn_hash, NULL, RW_READER, &l); @@ -870,7 +843,8 @@ fzap_update(zap_name_t *zn, void *tag, dmu_tx_t *tx) { zap_leaf_t *l; - int err, create; + int err; + boolean_t create; zap_entry_handle_t zeh; zap_t *zap = zn->zn_zap; @@ -923,9 +897,9 @@ fzap_length(zap_name_t *zn, if (err != 0) goto out; - if (integer_size) + if (integer_size != 0) *integer_size = zeh.zeh_integer_size; - if (num_integers) + if (num_integers != 0) *num_integers = zeh.zeh_num_integers; out: zap_put_leaf(l); @@ -954,15 +928,14 @@ fzap_remove(zap_name_t *zn, dmu_tx_t *tx) void fzap_prefetch(zap_name_t *zn) { - uint64_t idx, blk; + uint64_t blk; zap_t *zap = zn->zn_zap; - int bs; - idx = ZAP_HASH_IDX(zn->zn_hash, + uint64_t idx = ZAP_HASH_IDX(zn->zn_hash, zap_f_phys(zap)->zap_ptrtbl.zt_shift); if (zap_idx_to_blk(zap, idx, &blk) != 0) return; - bs = FZAP_BLOCK_SHIFT(zap); + int bs = FZAP_BLOCK_SHIFT(zap); dmu_prefetch(zap->zap_objset, zap->zap_object, 0, blk << bs, 1 << bs, ZIO_PRIORITY_SYNC_READ); } @@ -975,9 +948,8 @@ uint64_t zap_create_link(objset_t *os, dmu_object_type_t ot, uint64_t parent_obj, const char *name, dmu_tx_t *tx) { - uint64_t new_obj; - - VERIFY((new_obj = zap_create(os, ot, DMU_OT_NONE, 0, tx)) > 0); + uint64_t new_obj = zap_create(os, ot, DMU_OT_NONE, 0, tx); + VERIFY(new_obj != 0); VERIFY0(zap_add(os, parent_obj, name, sizeof (uint64_t), 1, &new_obj, tx)); @@ -989,13 +961,12 @@ zap_value_search(objset_t *os, uint64_t zapobj, uint64 char *name) { zap_cursor_t zc; - zap_attribute_t *za; int err; if (mask == 0) mask = -1ULL; - za = kmem_alloc(sizeof (zap_attribute_t), KM_SLEEP); + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, zapobj); (err = zap_cursor_retrieve(&zc, za)) == 0; zap_cursor_advance(&zc)) { @@ -1005,7 +976,7 @@ zap_value_search(objset_t *os, uint64_t zapobj, uint64 } } zap_cursor_fini(&zc); - kmem_free(za, sizeof (zap_attribute_t)); + kmem_free(za, sizeof (*za)); return (err); } @@ -1013,23 +984,23 @@ int zap_join(objset_t *os, uint64_t fromobj, uint64_t intoobj, dmu_tx_t *tx) { zap_cursor_t zc; - zap_attribute_t za; - int err; + int err = 0; - err = 0; + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, fromobj); - zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_retrieve(&zc, za) == 0; (void) zap_cursor_advance(&zc)) { - if (za.za_integer_length != 8 || za.za_num_integers != 1) { + if (za->za_integer_length != 8 || za->za_num_integers != 1) { err = SET_ERROR(EINVAL); break; } - err = zap_add(os, intoobj, za.za_name, - 8, 1, &za.za_first_integer, tx); - if (err) + err = zap_add(os, intoobj, za->za_name, + 8, 1, &za->za_first_integer, tx); + if (err != 0) break; } zap_cursor_fini(&zc); + kmem_free(za, sizeof (*za)); return (err); } @@ -1038,23 +1009,23 @@ zap_join_key(objset_t *os, uint64_t fromobj, uint64_t uint64_t value, dmu_tx_t *tx) { zap_cursor_t zc; - zap_attribute_t za; - int err; + int err = 0; - err = 0; + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, fromobj); - zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_retrieve(&zc, za) == 0; (void) zap_cursor_advance(&zc)) { - if (za.za_integer_length != 8 || za.za_num_integers != 1) { + if (za->za_integer_length != 8 || za->za_num_integers != 1) { err = SET_ERROR(EINVAL); break; } - err = zap_add(os, intoobj, za.za_name, + err = zap_add(os, intoobj, za->za_name, 8, 1, &value, tx); if (err != 0) break; } zap_cursor_fini(&zc); + kmem_free(za, sizeof (*za)); return (err); } @@ -1063,29 +1034,29 @@ zap_join_increment(objset_t *os, uint64_t fromobj, uin dmu_tx_t *tx) { zap_cursor_t zc; - zap_attribute_t za; - int err; + int err = 0; - err = 0; + zap_attribute_t *za = kmem_alloc(sizeof (*za), KM_SLEEP); for (zap_cursor_init(&zc, os, fromobj); - zap_cursor_retrieve(&zc, &za) == 0; + zap_cursor_retrieve(&zc, za) == 0; (void) zap_cursor_advance(&zc)) { uint64_t delta = 0; - if (za.za_integer_length != 8 || za.za_num_integers != 1) { + if (za->za_integer_length != 8 || za->za_num_integers != 1) { err = SET_ERROR(EINVAL); break; } - err = zap_lookup(os, intoobj, za.za_name, 8, 1, &delta); + err = zap_lookup(os, intoobj, za->za_name, 8, 1, &delta); if (err != 0 && err != ENOENT) break; - delta += za.za_first_integer; - err = zap_update(os, intoobj, za.za_name, 8, 1, &delta, tx); - if (err) + delta += za->za_first_integer; + err = zap_update(os, intoobj, za->za_name, 8, 1, &delta, tx); + if (err != 0) break; } zap_cursor_fini(&zc); + kmem_free(za, sizeof (*za)); return (err); } @@ -1150,12 +1121,11 @@ zap_increment(objset_t *os, uint64_t obj, const char * dmu_tx_t *tx) { uint64_t value = 0; - int err; if (delta == 0) return (0); - err = zap_lookup(os, obj, name, 8, 1, &value); + int err = zap_lookup(os, obj, name, 8, 1, &value); if (err != 0 && err != ENOENT) return (err); value += delta; @@ -1253,7 +1223,6 @@ again: static void zap_stats_ptrtbl(zap_t *zap, uint64_t *tbl, int len, zap_stats_t *zs) { - int i, err; uint64_t lastblk = 0; /* @@ -1261,14 +1230,14 @@ zap_stats_ptrtbl(zap_t *zap, uint64_t *tbl, int len, z * can hold, then it'll be accounted for more than once, since * we won't have lastblk. */ - for (i = 0; i < len; i++) { + for (int i = 0; i < len; i++) { zap_leaf_t *l; if (tbl[i] == lastblk) continue; lastblk = tbl[i]; - err = zap_get_leaf_byblk(zap, tbl[i], NULL, RW_READER, &l); + int err = zap_get_leaf_byblk(zap, tbl[i], NULL, RW_READER, &l); if (err == 0) { zap_leaf_stats(zap, l, zs); zap_put_leaf(l); @@ -1333,14 +1302,12 @@ fzap_get_stats(zap_t *zap, zap_stats_t *zs) zap_stats_ptrtbl(zap, &ZAP_EMBEDDED_PTRTBL_ENT(zap, 0), 1 << ZAP_EMBEDDED_PTRTBL_SHIFT(zap), zs); } else { - int b; - dmu_prefetch(zap->zap_objset, zap->zap_object, 0, zap_f_phys(zap)->zap_ptrtbl.zt_blk << bs, zap_f_phys(zap)->zap_ptrtbl.zt_numblks << bs, ZIO_PRIORITY_SYNC_READ); - for (b = 0; b < zap_f_phys(zap)->zap_ptrtbl.zt_numblks; + for (int b = 0; b < zap_f_phys(zap)->zap_ptrtbl.zt_numblks; b++) { dmu_buf_t *db; int err; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c Wed Aug 1 03:03:15 2018 (r337027) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c Wed Aug 1 03:07:33 2018 (r337028) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2013, 2015 by Delphix. All rights reserved. + * Copyright (c) 2013, 2016 by Delphix. All rights reserved. * Copyright 2017 Nexenta Systems, Inc. */ @@ -107,7 +107,6 @@ ldv(int len, const void *addr) void zap_leaf_byteswap(zap_leaf_phys_t *buf, int size) { - int i; zap_leaf_t l; dmu_buf_t l_dbuf; @@ -123,10 +122,10 @@ zap_leaf_byteswap(zap_leaf_phys_t *buf, int size) buf->l_hdr.lh_prefix_len = BSWAP_16(buf->l_hdr.lh_prefix_len); buf->l_hdr.lh_freelist = BSWAP_16(buf->l_hdr.lh_freelist); - for (i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(&l); i++) + for (int i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(&l); i++) buf->l_hash[i] = BSWAP_16(buf->l_hash[i]); - for (i = 0; i < ZAP_LEAF_NUMCHUNKS(&l); i++) { + for (int i = 0; i < ZAP_LEAF_NUMCHUNKS(&l); i++) { zap_leaf_chunk_t *lc = &ZAP_LEAF_CHUNK(&l, i); struct zap_leaf_entry *le; @@ -162,14 +161,12 @@ zap_leaf_byteswap(zap_leaf_phys_t *buf, int size) void zap_leaf_init(zap_leaf_t *l, boolean_t sort) { - int i; - l->l_bs = highbit64(l->l_dbuf->db_size) - 1; zap_memset(&zap_leaf_phys(l)->l_hdr, 0, sizeof (struct zap_leaf_header)); zap_memset(zap_leaf_phys(l)->l_hash, CHAIN_END, 2*ZAP_LEAF_HASH_NUMENTRIES(l)); - for (i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { + for (int i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { ZAP_LEAF_CHUNK(l, i).l_free.lf_type = ZAP_CHUNK_FREE; ZAP_LEAF_CHUNK(l, i).l_free.lf_next = i+1; } @@ -188,11 +185,9 @@ zap_leaf_init(zap_leaf_t *l, boolean_t sort) static uint16_t zap_leaf_chunk_alloc(zap_leaf_t *l) { - int chunk; - ASSERT(zap_leaf_phys(l)->l_hdr.lh_nfree > 0); - chunk = zap_leaf_phys(l)->l_hdr.lh_freelist; + int chunk = zap_leaf_phys(l)->l_hdr.lh_freelist; ASSERT3U(chunk, <, ZAP_LEAF_NUMCHUNKS(l)); ASSERT3U(ZAP_LEAF_CHUNK(l, chunk).l_free.lf_type, ==, ZAP_CHUNK_FREE); @@ -232,7 +227,7 @@ zap_leaf_array_create(zap_leaf_t *l, const char *buf, uint16_t *chunkp = &chunk_head; int byten = 0; uint64_t value = 0; - int shift = (integer_size-1)*8; + int shift = (integer_size - 1) * 8; int len = num_integers; ASSERT3U(num_integers * integer_size, <, MAX_ARRAY_BYTES); @@ -240,10 +235,9 @@ zap_leaf_array_create(zap_leaf_t *l, const char *buf, while (len > 0) { uint16_t chunk = zap_leaf_chunk_alloc(l); struct zap_leaf_array *la = &ZAP_LEAF_CHUNK(l, chunk).l_array; - int i; la->la_type = ZAP_CHUNK_ARRAY; - for (i = 0; i < ZAP_LEAF_ARRAY_BYTES; i++) { + for (int i = 0; i < ZAP_LEAF_ARRAY_BYTES; i++) { if (byten == 0) value = ldv(integer_size, buf); la->la_array[i] = value >> shift; @@ -321,10 +315,9 @@ zap_leaf_array_read(zap_leaf_t *l, uint16_t chunk, while (len > 0) { struct zap_leaf_array *la = &ZAP_LEAF_CHUNK(l, chunk).l_array; - int i; ASSERT3U(chunk, <, ZAP_LEAF_NUMCHUNKS(l)); - for (i = 0; i < ZAP_LEAF_ARRAY_BYTES && len > 0; i++) { + for (int i = 0; i < ZAP_LEAF_ARRAY_BYTES && len > 0; i++) { value = (value << 8) | la->la_array[i]; byten++; if (byten == array_int_len) { @@ -347,16 +340,13 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, int bseen = 0; if (zap_getflags(zn->zn_zap) & ZAP_FLAG_UINT64_KEY) { - uint64_t *thiskey; - boolean_t match; - + uint64_t *thiskey = + kmem_alloc(array_numints * sizeof (*thiskey), KM_SLEEP); ASSERT(zn->zn_key_intlen == sizeof (*thiskey)); - thiskey = kmem_alloc(array_numints * sizeof (*thiskey), - KM_SLEEP); zap_leaf_array_read(l, chunk, sizeof (*thiskey), array_numints, sizeof (*thiskey), array_numints, thiskey); - match = bcmp(thiskey, zn->zn_key_orig, + boolean_t match = bcmp(thiskey, zn->zn_key_orig, array_numints * sizeof (*thiskey)) == 0; kmem_free(thiskey, array_numints * sizeof (*thiskey)); return (match); @@ -365,11 +355,10 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, ASSERT(zn->zn_key_intlen == 1); if (zn->zn_matchtype & MT_NORMALIZE) { char *thisname = kmem_alloc(array_numints, KM_SLEEP); - boolean_t match; zap_leaf_array_read(l, chunk, sizeof (char), array_numints, sizeof (char), array_numints, thisname); - match = zap_match(zn, thisname); + boolean_t match = zap_match(zn, thisname); kmem_free(thisname, array_numints); return (match); } @@ -400,12 +389,11 @@ zap_leaf_array_match(zap_leaf_t *l, zap_name_t *zn, int zap_leaf_lookup(zap_leaf_t *l, zap_name_t *zn, zap_entry_handle_t *zeh) { - uint16_t *chunkp; struct zap_leaf_entry *le; ASSERT3U(zap_leaf_phys(l)->l_hdr.lh_magic, ==, ZAP_LEAF_MAGIC); - for (chunkp = LEAF_HASH_ENTPTR(l, zn->zn_hash); + for (uint16_t *chunkp = LEAF_HASH_ENTPTR(l, zn->zn_hash); *chunkp != CHAIN_END; chunkp = &le->le_next) { uint16_t chunk = *chunkp; le = ZAP_LEAF_ENTRY(l, chunk); @@ -446,17 +434,15 @@ int zap_leaf_lookup_closest(zap_leaf_t *l, uint64_t h, uint32_t cd, zap_entry_handle_t *zeh) { - uint16_t chunk; uint64_t besth = -1ULL; uint32_t bestcd = -1U; uint16_t bestlh = ZAP_LEAF_HASH_NUMENTRIES(l)-1; - uint16_t lh; struct zap_leaf_entry *le; ASSERT3U(zap_leaf_phys(l)->l_hdr.lh_magic, ==, ZAP_LEAF_MAGIC); - for (lh = LEAF_HASH(l, h); lh <= bestlh; lh++) { - for (chunk = zap_leaf_phys(l)->l_hash[lh]; + for (uint16_t lh = LEAF_HASH(l, h); lh <= bestlh; lh++) { + for (uint16_t chunk = zap_leaf_phys(l)->l_hash[lh]; chunk != CHAIN_END; chunk = le->le_next) { le = ZAP_LEAF_ENTRY(l, chunk); @@ -529,11 +515,10 @@ int zap_entry_update(zap_entry_handle_t *zeh, uint8_t integer_size, uint64_t num_integers, const void *buf) { - int delta_chunks; zap_leaf_t *l = zeh->zeh_leaf; struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, *zeh->zeh_chunkp); - delta_chunks = ZAP_LEAF_ARRAY_NCHUNKS(num_integers * integer_size) - + int delta_chunks = ZAP_LEAF_ARRAY_NCHUNKS(num_integers * integer_size) - ZAP_LEAF_ARRAY_NCHUNKS(le->le_value_numints * le->le_value_intlen); if ((int)zap_leaf_phys(l)->l_hdr.lh_nfree < delta_chunks) @@ -550,14 +535,12 @@ zap_entry_update(zap_entry_handle_t *zeh, void zap_entry_remove(zap_entry_handle_t *zeh) { - uint16_t entry_chunk; - struct zap_leaf_entry *le; zap_leaf_t *l = zeh->zeh_leaf; ASSERT3P(zeh->zeh_chunkp, !=, &zeh->zeh_fakechunk); - entry_chunk = *zeh->zeh_chunkp; - le = ZAP_LEAF_ENTRY(l, entry_chunk); + uint16_t entry_chunk = *zeh->zeh_chunkp; + struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, entry_chunk); ASSERT3U(le->le_type, ==, ZAP_CHUNK_ENTRY); zap_leaf_array_free(l, &le->le_name_chunk); @@ -575,15 +558,12 @@ zap_entry_create(zap_leaf_t *l, zap_name_t *zn, uint32 zap_entry_handle_t *zeh) { uint16_t chunk; - uint16_t *chunkp; struct zap_leaf_entry *le; - uint64_t valuelen; - int numchunks; uint64_t h = zn->zn_hash; - valuelen = integer_size * num_integers; + uint64_t valuelen = integer_size * num_integers; - numchunks = 1 + ZAP_LEAF_ARRAY_NCHUNKS(zn->zn_key_orig_numints * + int numchunks = 1 + ZAP_LEAF_ARRAY_NCHUNKS(zn->zn_key_orig_numints * zn->zn_key_intlen) + ZAP_LEAF_ARRAY_NCHUNKS(valuelen); if (numchunks > ZAP_LEAF_NUMCHUNKS(l)) return (E2BIG); @@ -645,7 +625,7 @@ zap_entry_create(zap_leaf_t *l, zap_name_t *zn, uint32 /* link it into the hash chain */ /* XXX if we did the search above, we could just use that */ - chunkp = zap_leaf_rehash_entry(l, chunk); + uint16_t *chunkp = zap_leaf_rehash_entry(l, chunk); zap_leaf_phys(l)->l_hdr.lh_nentries++; @@ -673,14 +653,13 @@ boolean_t zap_entry_normalization_conflict(zap_entry_handle_t *zeh, zap_name_t *zn, const char *name, zap_t *zap) { - uint64_t chunk; struct zap_leaf_entry *le; boolean_t allocdzn = B_FALSE; if (zap->zap_normflags == 0) return (B_FALSE); - for (chunk = *LEAF_HASH_ENTPTR(zeh->zeh_leaf, zeh->zeh_hash); + for (uint16_t chunk = *LEAF_HASH_ENTPTR(zeh->zeh_leaf, zeh->zeh_hash); chunk != CHAIN_END; chunk = le->le_next) { le = ZAP_LEAF_ENTRY(zeh->zeh_leaf, chunk); if (le->le_hash != zeh->zeh_hash) @@ -763,14 +742,11 @@ zap_leaf_transfer_array(zap_leaf_t *l, uint16_t chunk, static void zap_leaf_transfer_entry(zap_leaf_t *l, int entry, zap_leaf_t *nl) { - struct zap_leaf_entry *le, *nle; - uint16_t chunk; - - le = ZAP_LEAF_ENTRY(l, entry); + struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, entry); ASSERT3U(le->le_type, ==, ZAP_CHUNK_ENTRY); - chunk = zap_leaf_chunk_alloc(nl); - nle = ZAP_LEAF_ENTRY(nl, chunk); + uint16_t chunk = zap_leaf_chunk_alloc(nl); + struct zap_leaf_entry *nle = ZAP_LEAF_ENTRY(nl, chunk); *nle = *le; /* structure assignment */ (void) zap_leaf_rehash_entry(nl, chunk); @@ -791,7 +767,6 @@ zap_leaf_transfer_entry(zap_leaf_t *l, int entry, zap_ void zap_leaf_split(zap_leaf_t *l, zap_leaf_t *nl, boolean_t sort) { - int i; int bit = 64 - 1 - zap_leaf_phys(l)->l_hdr.lh_prefix_len; /* set new prefix and prefix_len */ @@ -818,7 +793,7 @@ zap_leaf_split(zap_leaf_t *l, zap_leaf_t *nl, boolean_ * but this accesses memory more sequentially, and when we're * called, the block is usually pretty full. */ - for (i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { + for (int i = 0; i < ZAP_LEAF_NUMCHUNKS(l); i++) { struct zap_leaf_entry *le = ZAP_LEAF_ENTRY(l, i); if (le->le_type != ZAP_CHUNK_ENTRY) continue; @@ -833,9 +808,7 @@ zap_leaf_split(zap_leaf_t *l, zap_leaf_t *nl, boolean_ void zap_leaf_stats(zap_t *zap, zap_leaf_t *l, zap_stats_t *zs) { - int i, n; - - n = zap_f_phys(zap)->zap_ptrtbl.zt_shift - + int n = zap_f_phys(zap)->zap_ptrtbl.zt_shift - zap_leaf_phys(l)->l_hdr.lh_prefix_len; n = MIN(n, ZAP_HISTOGRAM_SIZE-1); zs->zs_leafs_with_2n_pointers[n]++; @@ -851,7 +824,7 @@ zap_leaf_stats(zap_t *zap, zap_leaf_t *l, zap_stats_t n = MIN(n, ZAP_HISTOGRAM_SIZE-1); zs->zs_blocks_n_tenths_full[n]++; - for (i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(l); i++) { + for (int i = 0; i < ZAP_LEAF_HASH_NUMENTRIES(l); i++) { int nentries = 0; int chunk = zap_leaf_phys(l)->l_hash[i]; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c Wed Aug 1 03:03:15 2018 (r337027) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c Wed Aug 1 03:07:33 2018 (r337028) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2011, 2016 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. * Copyright (c) 2014 Integros [integros.com] * Copyright 2017 Nexenta Systems, Inc. @@ -89,22 +89,20 @@ zap_hash(zap_name_t *zn) ASSERT(zfs_crc64_table[128] == ZFS_CRC64_POLY); if (zap_getflags(zap) & ZAP_FLAG_UINT64_KEY) { - int i; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Wed Aug 1 03:19:31 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C0ED1045224; Wed, 1 Aug 2018 03:19:31 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B5FB07082B; Wed, 1 Aug 2018 03:19:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9143D1CC89; Wed, 1 Aug 2018 03:19:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w713JUxJ042947; Wed, 1 Aug 2018 03:19:30 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w713JUkI042946; Wed, 1 Aug 2018 03:19:30 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808010319.w713JUkI042946@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 03:19:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337029 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337029 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 03:19:31 -0000 Author: mav Date: Wed Aug 1 03:19:30 2018 New Revision: 337029 URL: https://svnweb.freebsd.org/changeset/base/337029 Log: 9426 metaslab size can exceed offset addressable by spacemap metaslab size can exceed offset addressable by spacemap. The vdev can address up to 2^63 * SPA_MAXBLOCKSIZE (512). A metaslab can address up to 2^47 * 2^vdev_ashift. Therefore we may need to increase the number of metaslabs so that the maximum metaslab size is capped at the amount that can be addressed by the spacemap. This should happen in vdev_metaslab_set_size(). illumos/illumos-gate@b4bf0cf0458759c67920a031021a9d96cd683cfe Reviewed by: Paul Dagnelie Reviewed by: Matt Ahrens Approved by: Dan McDonald Author: Don Brady Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Wed Aug 1 03:07:33 2018 (r337028) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Wed Aug 1 03:19:30 2018 (r337029) @@ -72,15 +72,21 @@ static vdev_ops_t *vdev_ops_table[] = { /* maximum scrub/resilver I/O queue per leaf vdev */ int zfs_scrub_limit = 10; -/* maximum number of metaslabs per top-level vdev */ +/* target number of metaslabs per top-level vdev */ int vdev_max_ms_count = 200; -/* minimum amount of metaslabs per top-level vdev */ +/* minimum number of metaslabs per top-level vdev */ int vdev_min_ms_count = 16; -/* see comment in vdev_metaslab_set_size() */ +/* practical upper limit of total metaslabs per top-level vdev */ +int vdev_ms_count_limit = 1ULL << 17; + +/* lower limit for metaslab size (512M) */ int vdev_default_ms_shift = 29; +/* upper limit for metaslab size (256G) */ +int vdev_max_ms_shift = 38; + boolean_t vdev_validate_skip = B_FALSE; /* @@ -2028,34 +2034,53 @@ void vdev_metaslab_set_size(vdev_t *vd) { uint64_t asize = vd->vdev_asize; - uint64_t ms_shift = 0; + uint64_t ms_count = asize >> vdev_default_ms_shift; + uint64_t ms_shift; /* - * For vdevs that are bigger than 8G the metaslab size varies in - * a way that the number of metaslabs increases in powers of two, - * linearly in terms of vdev_asize, starting from 16 metaslabs. - * So for vdev_asize of 8G we get 16 metaslabs, for 16G, we get 32, - * and so on, until we hit the maximum metaslab count limit - * [vdev_max_ms_count] from which point the metaslab count stays - * the same. + * There are two dimensions to the metaslab sizing calculation: + * the size of the metaslab and the count of metaslabs per vdev. + * In general, we aim for vdev_max_ms_count (200) metaslabs. The + * range of the dimensions are as follows: + * + * 2^29 <= ms_size <= 2^38 + * 16 <= ms_count <= 131,072 + * + * On the lower end of vdev sizes, we aim for metaslabs sizes of + * at least 512MB (2^29) to minimize fragmentation effects when + * testing with smaller devices. However, the count constraint + * of at least 16 metaslabs will override this minimum size goal. + * + * On the upper end of vdev sizes, we aim for a maximum metaslab + * size of 256GB. However, we will cap the total count to 2^17 + * metaslabs to keep our memory footprint in check. + * + * The net effect of applying above constrains is summarized below. + * + * vdev size metaslab count + * -------------|----------------- + * < 8GB ~16 + * 8GB - 100GB one per 512MB + * 100GB - 50TB ~200 + * 50TB - 32PB one per 256GB + * > 32PB ~131,072 + * ------------------------------- */ - ms_shift = vdev_default_ms_shift; - if ((asize >> ms_shift) < vdev_min_ms_count) { - /* - * For devices that are less than 8G we want to have - * exactly 16 metaslabs. We don't want less as integer - * division rounds down, so less metaslabs mean more - * wasted space. We don't want more as these vdevs are - * small and in the likely event that we are running - * out of space, the SPA will have a hard time finding - * space due to fragmentation. - */ + if (ms_count < vdev_min_ms_count) ms_shift = highbit64(asize / vdev_min_ms_count); - ms_shift = MAX(ms_shift, SPA_MAXBLOCKSHIFT); - - } else if ((asize >> ms_shift) > vdev_max_ms_count) { + else if (ms_count > vdev_max_ms_count) ms_shift = highbit64(asize / vdev_max_ms_count); + else + ms_shift = vdev_default_ms_shift; + + if (ms_shift < SPA_MAXBLOCKSHIFT) { + ms_shift = SPA_MAXBLOCKSHIFT; + } else if (ms_shift > vdev_max_ms_shift) { + ms_shift = vdev_max_ms_shift; + /* cap the total count to constrain memory footprint */ + if ((asize >> ms_shift) > vdev_ms_count_limit) + ms_shift = highbit64(asize / vdev_ms_count_limit); } vd->vdev_ms_shift = ms_shift; From owner-svn-src-all@freebsd.org Wed Aug 1 03:21:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CF9A10452D2; Wed, 1 Aug 2018 03:21:18 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6D88709EA; Wed, 1 Aug 2018 03:21:17 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B7E411CCC1; Wed, 1 Aug 2018 03:21:17 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w713LHok044500; Wed, 1 Aug 2018 03:21:17 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w713LHHd044499; Wed, 1 Aug 2018 03:21:17 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808010321.w713LHHd044499@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 03:21:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337030 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337030 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 03:21:18 -0000 Author: mav Date: Wed Aug 1 03:21:17 2018 New Revision: 337030 URL: https://svnweb.freebsd.org/changeset/base/337030 Log: MFV r337029: 9426 metaslab size can exceed offset addressable by spacemap metaslab size can exceed offset addressable by spacemap. The vdev can address up to 2^63 * SPA_MAXBLOCKSIZE (512). A metaslab can address up to 2^47 * 2^vdev_ashift. Therefore we may need to increase the number of metaslabs so that the maximum metaslab size is capped at the amount that can be addressed by the spacemap. This should happen in vdev_metaslab_set_size(). illumos/illumos-gate@b4bf0cf0458759c67920a031021a9d96cd683cfe Reviewed by: Paul Dagnelie Reviewed by: Matt Ahrens Approved by: Dan McDonald Author: Don Brady Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Wed Aug 1 03:19:30 2018 (r337029) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Wed Aug 1 03:21:17 2018 (r337030) @@ -163,24 +163,30 @@ static vdev_ops_t *vdev_ops_table[] = { }; -/* maximum number of metaslabs per top-level vdev */ +/* target number of metaslabs per top-level vdev */ int vdev_max_ms_count = 200; SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, max_ms_count, CTLFLAG_RDTUN, &vdev_max_ms_count, 0, "Maximum number of metaslabs per top-level vdev"); -/* minimum amount of metaslabs per top-level vdev */ +/* minimum number of metaslabs per top-level vdev */ int vdev_min_ms_count = 16; SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, min_ms_count, CTLFLAG_RDTUN, &vdev_min_ms_count, 0, "Minimum number of metaslabs per top-level vdev"); -/* see comment in vdev_metaslab_set_size() */ +/* practical upper limit of total metaslabs per top-level vdev */ +int vdev_ms_count_limit = 1ULL << 17; + +/* lower limit for metaslab size (512M) */ int vdev_default_ms_shift = 29; SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, default_ms_shift, CTLFLAG_RDTUN, &vdev_default_ms_shift, 0, "Shift between vdev size and number of metaslabs"); +/* upper limit for metaslab size (256G) */ +int vdev_max_ms_shift = 38; + boolean_t vdev_validate_skip = B_FALSE; /* @@ -2167,34 +2173,53 @@ void vdev_metaslab_set_size(vdev_t *vd) { uint64_t asize = vd->vdev_asize; - uint64_t ms_shift = 0; + uint64_t ms_count = asize >> vdev_default_ms_shift; + uint64_t ms_shift; /* - * For vdevs that are bigger than 8G the metaslab size varies in - * a way that the number of metaslabs increases in powers of two, - * linearly in terms of vdev_asize, starting from 16 metaslabs. - * So for vdev_asize of 8G we get 16 metaslabs, for 16G, we get 32, - * and so on, until we hit the maximum metaslab count limit - * [vdev_max_ms_count] from which point the metaslab count stays - * the same. + * There are two dimensions to the metaslab sizing calculation: + * the size of the metaslab and the count of metaslabs per vdev. + * In general, we aim for vdev_max_ms_count (200) metaslabs. The + * range of the dimensions are as follows: + * + * 2^29 <= ms_size <= 2^38 + * 16 <= ms_count <= 131,072 + * + * On the lower end of vdev sizes, we aim for metaslabs sizes of + * at least 512MB (2^29) to minimize fragmentation effects when + * testing with smaller devices. However, the count constraint + * of at least 16 metaslabs will override this minimum size goal. + * + * On the upper end of vdev sizes, we aim for a maximum metaslab + * size of 256GB. However, we will cap the total count to 2^17 + * metaslabs to keep our memory footprint in check. + * + * The net effect of applying above constrains is summarized below. + * + * vdev size metaslab count + * -------------|----------------- + * < 8GB ~16 + * 8GB - 100GB one per 512MB + * 100GB - 50TB ~200 + * 50TB - 32PB one per 256GB + * > 32PB ~131,072 + * ------------------------------- */ - ms_shift = vdev_default_ms_shift; - if ((asize >> ms_shift) < vdev_min_ms_count) { - /* - * For devices that are less than 8G we want to have - * exactly 16 metaslabs. We don't want less as integer - * division rounds down, so less metaslabs mean more - * wasted space. We don't want more as these vdevs are - * small and in the likely event that we are running - * out of space, the SPA will have a hard time finding - * space due to fragmentation. - */ + if (ms_count < vdev_min_ms_count) ms_shift = highbit64(asize / vdev_min_ms_count); - ms_shift = MAX(ms_shift, SPA_MAXBLOCKSHIFT); - - } else if ((asize >> ms_shift) > vdev_max_ms_count) { + else if (ms_count > vdev_max_ms_count) ms_shift = highbit64(asize / vdev_max_ms_count); + else + ms_shift = vdev_default_ms_shift; + + if (ms_shift < SPA_MAXBLOCKSHIFT) { + ms_shift = SPA_MAXBLOCKSHIFT; + } else if (ms_shift > vdev_max_ms_shift) { + ms_shift = vdev_max_ms_shift; + /* cap the total count to constrain memory footprint */ + if ((asize >> ms_shift) > vdev_ms_count_limit) + ms_shift = highbit64(asize / vdev_ms_count_limit); } vd->vdev_ms_shift = ms_shift; From owner-svn-src-all@freebsd.org Wed Aug 1 03:46:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE6EC1045B52; Wed, 1 Aug 2018 03:46:07 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A478971763; Wed, 1 Aug 2018 03:46:07 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8544D1D168; Wed, 1 Aug 2018 03:46:07 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w713k7XO058344; Wed, 1 Aug 2018 03:46:07 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w713k7JS058343; Wed, 1 Aug 2018 03:46:07 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808010346.w713k7JS058343@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 1 Aug 2018 03:46:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337031 - head/sys/security/mac X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/security/mac X-SVN-Commit-Revision: 337031 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 03:46:08 -0000 Author: markj Date: Wed Aug 1 03:46:07 2018 New Revision: 337031 URL: https://svnweb.freebsd.org/changeset/base/337031 Log: Require that MAC label buffers be able to store a non-empty string. The buffer size may be used to initialize an sbuf in MAC_POLICY_EXTERNALIZE, and without this constraint it's possible to trigger an assertion failure in the sbuf code. With INVARIANTS disabled, the first attempt to write to the sbuf will fail. Reported by: pho Reviewed by: delphij MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D16527 Modified: head/sys/security/mac/mac_framework.c Modified: head/sys/security/mac/mac_framework.c ============================================================================== --- head/sys/security/mac/mac_framework.c Wed Aug 1 03:21:17 2018 (r337030) +++ head/sys/security/mac/mac_framework.c Wed Aug 1 03:46:07 2018 (r337031) @@ -586,7 +586,9 @@ int mac_check_structmac_consistent(struct mac *mac) { - if (mac->m_buflen > MAC_MAX_LABEL_BUF_LEN) + /* Require that labels have a non-zero length. */ + if (mac->m_buflen > MAC_MAX_LABEL_BUF_LEN || + mac->m_buflen <= sizeof("")) return (EINVAL); return (0); From owner-svn-src-all@freebsd.org Wed Aug 1 03:52:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B9DE1045E67; Wed, 1 Aug 2018 03:52:49 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2223071C0F; Wed, 1 Aug 2018 03:52:49 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 034E41D305; Wed, 1 Aug 2018 03:52:49 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w713qmR1063353; Wed, 1 Aug 2018 03:52:48 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w713qmdS063352; Wed, 1 Aug 2018 03:52:48 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808010352.w713qmdS063352@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 1 Aug 2018 03:52:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337032 - head/sys/sys X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/sys X-SVN-Commit-Revision: 337032 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 03:52:49 -0000 Author: markj Date: Wed Aug 1 03:52:48 2018 New Revision: 337032 URL: https://svnweb.freebsd.org/changeset/base/337032 Log: Remove MT_NTYPES. Its last reference was removed in r253361. Modified: head/sys/sys/mbuf.h Modified: head/sys/sys/mbuf.h ============================================================================== --- head/sys/sys/mbuf.h Wed Aug 1 03:46:07 2018 (r337031) +++ head/sys/sys/mbuf.h Wed Aug 1 03:52:48 2018 (r337032) @@ -569,7 +569,6 @@ struct mbuf { #define MT_CONTROL 14 /* extra-data protocol message */ #define MT_OOBDATA 15 /* expedited data */ -#define MT_NTYPES 16 /* number of mbuf types for mbtypes[] */ #define MT_NOINIT 255 /* Not a type but a flag to allocate a non-initialized mbuf */ From owner-svn-src-all@freebsd.org Wed Aug 1 08:24:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E2F31060156; Wed, 1 Aug 2018 08:24:37 +0000 (UTC) (envelope-from rpokala@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B4835888A5; Wed, 1 Aug 2018 08:24:36 +0000 (UTC) (envelope-from rpokala@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 94D291FFBF; Wed, 1 Aug 2018 08:24:36 +0000 (UTC) (envelope-from rpokala@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w718OaXd002679; Wed, 1 Aug 2018 08:24:36 GMT (envelope-from rpokala@FreeBSD.org) Received: (from rpokala@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w718OZLK002670; Wed, 1 Aug 2018 08:24:35 GMT (envelope-from rpokala@FreeBSD.org) Message-Id: <201808010824.w718OZLK002670@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rpokala set sender to rpokala@FreeBSD.org using -f From: Ravi Pokala Date: Wed, 1 Aug 2018 08:24:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337033 - in head: . share/man/man4 sys/conf sys/dev/jedec_dimm sys/dev/jedec_ts sys/modules/i2c sys/modules/i2c/jedec_ts X-SVN-Group: head X-SVN-Commit-Author: rpokala X-SVN-Commit-Paths: in head: . share/man/man4 sys/conf sys/dev/jedec_dimm sys/dev/jedec_ts sys/modules/i2c sys/modules/i2c/jedec_ts X-SVN-Commit-Revision: 337033 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 08:24:37 -0000 Author: rpokala Date: Wed Aug 1 08:24:34 2018 New Revision: 337033 URL: https://svnweb.freebsd.org/changeset/base/337033 Log: Remove jedec_ts(4) The jedec_ts(4) driver has been marked as deprecated in stable/11, and is now being removed from -HEAD. Add a notice in UPDATING, and update the few remaining references (regarding jedec_dimm(4)'s compatibility and history) to reflect the fact that jedec_ts(4) is now deleted. Reviewed by: avg Relnotes: yes Differential Revision: https://reviews.freebsd.org/D16537 Deleted: head/share/man/man4/jedec_ts.4 head/sys/dev/jedec_ts/ head/sys/modules/i2c/jedec_ts/ Modified: head/UPDATING head/share/man/man4/Makefile head/share/man/man4/jedec_dimm.4 head/sys/conf/NOTES head/sys/conf/files head/sys/dev/jedec_dimm/jedec_dimm.c head/sys/modules/i2c/Makefile Modified: head/UPDATING ============================================================================== --- head/UPDATING Wed Aug 1 03:52:48 2018 (r337032) +++ head/UPDATING Wed Aug 1 08:24:34 2018 (r337033) @@ -31,6 +31,12 @@ NOTE TO PEOPLE WHO THINK THAT FreeBSD 12.x IS SLOW: disable the most expensive debugging functionality run "ln -s 'abort:false,junk:false' /etc/malloc.conf".) +20180731: + The jedec_ts(4) driver has been removed. A superset of its functionality + is available in the jedec_dimm(4) driver, and the manpage for that + driver includes migration instructions. If you have "device jedec_ts" + in your kernel configuration file, it must be removed. + 20180730: amd64/GENERIC now has EFI runtime services, EFIRT, enabled by default. This should have no effect if the kernel is booted via BIOS/legacy boot. Modified: head/share/man/man4/Makefile ============================================================================== --- head/share/man/man4/Makefile Wed Aug 1 03:52:48 2018 (r337032) +++ head/share/man/man4/Makefile Wed Aug 1 08:24:34 2018 (r337033) @@ -241,7 +241,6 @@ MAN= aac.4 \ ixl.4 \ ixlv.4 \ jedec_dimm.4 \ - jedec_ts.4 \ jme.4 \ joy.4 \ kbdmux.4 \ Modified: head/share/man/man4/jedec_dimm.4 ============================================================================== --- head/share/man/man4/jedec_dimm.4 Wed Aug 1 03:52:48 2018 (r337032) +++ head/share/man/man4/jedec_dimm.4 Wed Aug 1 08:24:34 2018 (r337033) @@ -27,7 +27,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 24, 2018 +.Dd July 31, 2018 .Dt JEDEC_DIMM 4 .Os .Sh NAME @@ -164,7 +164,7 @@ dev.jedec_dimm.6.type: DDR4 .Ed .Sh COMPATIBILITY .Nm -implements a superset of the functionality of +implements a superset of the functionality of the now-deleted .Xr jedec_ts 4 . Hints for .Xr jedec_ts 4 @@ -199,7 +199,6 @@ sed -i ".old" -e 's/jedec_ts/jedec_dimm/' \\ .Sh SEE ALSO .Xr iicbus 4 , .Xr iicsmb 4 , -.Xr jedec_ts 4 , .Xr smbus 4 , .Xr sysctl 8 .Sh STANDARDS @@ -237,7 +236,7 @@ The .Nm driver and this manual page were written by .An Ravi Pokala Aq Mt rpokala@freebsd.org . -They are both based in part on the +They are both based in part on the now-deleted .Xr jedec_ts 4 driver and manual page, written by .An Andriy Gapon Aq Mt avg@FreeBSD.org . Modified: head/sys/conf/NOTES ============================================================================== --- head/sys/conf/NOTES Wed Aug 1 03:52:48 2018 (r337032) +++ head/sys/conf/NOTES Wed Aug 1 08:24:34 2018 (r337033) @@ -2478,10 +2478,8 @@ device smb # SMBus peripheral devices # # jedec_dimm Asset and temperature reporting for DDR3 and DDR4 DIMMs -# jedec_ts Temperature Sensor compliant with JEDEC Standard 21-C # device jedec_dimm -device jedec_ts # I2C Bus # Modified: head/sys/conf/files ============================================================================== --- head/sys/conf/files Wed Aug 1 03:52:48 2018 (r337032) +++ head/sys/conf/files Wed Aug 1 08:24:34 2018 (r337033) @@ -2308,7 +2308,6 @@ dev/ixgbe/ixgbe_dcb_82598.c optional ix inet | ixv ine dev/ixgbe/ixgbe_dcb_82599.c optional ix inet | ixv inet \ compile-with "${NORMAL_C} -I$S/dev/ixgbe" dev/jedec_dimm/jedec_dimm.c optional jedec_dimm smbus -dev/jedec_ts/jedec_ts.c optional jedec_ts smbus dev/jme/if_jme.c optional jme pci dev/joy/joy.c optional joy dev/joy/joy_isa.c optional joy isa Modified: head/sys/dev/jedec_dimm/jedec_dimm.c ============================================================================== --- head/sys/dev/jedec_dimm/jedec_dimm.c Wed Aug 1 03:52:48 2018 (r337032) +++ head/sys/dev/jedec_dimm/jedec_dimm.c Wed Aug 1 08:24:34 2018 (r337033) @@ -32,9 +32,9 @@ */ /* - * This driver is a super-set of jedec_ts(4), and most of the code for reading - * and reporting the temperature is either based on that driver, or copied - * from it verbatim. + * This driver is a super-set of the now-deleted jedec_ts(4), and most of the + * code for reading and reporting the temperature is either based on that driver, + * or copied from it verbatim. */ #include Modified: head/sys/modules/i2c/Makefile ============================================================================== --- head/sys/modules/i2c/Makefile Wed Aug 1 03:52:48 2018 (r337032) +++ head/sys/modules/i2c/Makefile Wed Aug 1 08:24:34 2018 (r337033) @@ -16,7 +16,6 @@ SUBDIR = \ isl \ isl12xx \ jedec_dimm \ - jedec_ts \ nxprtc \ rtc8583 \ s35390a \ From owner-svn-src-all@freebsd.org Wed Aug 1 10:34:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63D0A1050DE1; Wed, 1 Aug 2018 10:34:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B3FA8CC8F; Wed, 1 Aug 2018 10:34:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C295D21524; Wed, 1 Aug 2018 10:34:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71AYssi069348; Wed, 1 Aug 2018 10:34:54 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71AYrVX069341; Wed, 1 Aug 2018 10:34:53 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808011034.w71AYrVX069341@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 1 Aug 2018 10:34:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337034 - in stable/10: share/man/man9 sys/kern sys/sys X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/10: share/man/man9 sys/kern sys/sys X-SVN-Commit-Revision: 337034 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 10:34:55 -0000 Author: hselasky Date: Wed Aug 1 10:34:53 2018 New Revision: 337034 URL: https://svnweb.freebsd.org/changeset/base/337034 Log: MFC r330349 and r330362: Allow pause_sbt() to catch signals during sleep by passing C_CATCH flag. Define pause_sig() function macro helper similarly to other kernel functions which catch signals. Update outdated function description. Document pause_sig(9) and update prototypes for existing pause(9) and pause_sbt(9) functions. Discussed with: kib@ Suggested by: cem@ Sponsored by: Mellanox Technologies Modified: stable/10/share/man/man9/Makefile stable/10/share/man/man9/sleep.9 stable/10/sys/kern/kern_synch.c stable/10/sys/sys/callout.h stable/10/sys/sys/systm.h Directory Properties: stable/10/ (props changed) Modified: stable/10/share/man/man9/Makefile ============================================================================== --- stable/10/share/man/man9/Makefile Wed Aug 1 08:24:34 2018 (r337033) +++ stable/10/share/man/man9/Makefile Wed Aug 1 10:34:53 2018 (r337034) @@ -1249,6 +1249,7 @@ MLINKS+=sleep.9 msleep.9 \ sleep.9 msleep_spin.9 \ sleep.9 msleep_spin_sbt.9 \ sleep.9 pause.9 \ + sleep.9 pause_sig.9 \ sleep.9 pause_sbt.9 \ sleep.9 tsleep.9 \ sleep.9 tsleep_sbt.9 \ Modified: stable/10/share/man/man9/sleep.9 ============================================================================== --- stable/10/share/man/man9/sleep.9 Wed Aug 1 08:24:34 2018 (r337033) +++ stable/10/share/man/man9/sleep.9 Wed Aug 1 10:34:53 2018 (r337034) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd February 19, 2013 +.Dd August 1, 2018 .Dt SLEEP 9 .Os .Sh NAME @@ -34,6 +34,7 @@ .Nm msleep_spin , .Nm msleep_spin_sbt , .Nm pause , +.Nm pause_sig , .Nm pause_sbt , .Nm tsleep , .Nm tsleep_sbt , @@ -53,9 +54,11 @@ .Ft int .Fn msleep_spin_sbt "void *chan" "struct mtx *mtx" "const char *wmesg" \ "sbintime_t sbt" "sbintime_t pr" "int flags" -.Ft void +.Ft int .Fn pause "const char *wmesg" "int timo" -.Ft void +.Ft int +.Fn pause_sig "const char *wmesg" "int timo" +.Ft int .Fn pause_sbt "const char *wmesg" "sbintime_t sbt" "sbintime_t pr" \ "int flags" .Ft int @@ -73,6 +76,8 @@ The functions .Fn msleep , .Fn msleep_spin , .Fn pause , +.Fn pause_sig , +.Fn pause_sbt , .Fn wakeup , and .Fn wakeup_one @@ -82,8 +87,10 @@ external event, it is put to sleep by .Fn tsleep , .Fn msleep , .Fn msleep_spin , +.Fn pause , +.Fn pause_sig , or -.Fn pause . +.Fn pause_sbt . Threads may also wait using one of the locking primitive sleep routines .Xr mtx_sleep 9 , .Xr rw_sleep 9 , @@ -261,6 +268,11 @@ The thread can not be awakened early by signals or cal .Fn wakeup or .Fn wakeup_one . +The +.Fn pause_sig +function is a variant of +.Fn pause +which can be awakened early by signals. .Pp The .Fn wakeup_one @@ -380,6 +392,10 @@ The .Fn pause function appeared in .Fx 7.0 . +The +.Fn pause_sig +function appeared in +.Fx 12.0 . .Sh AUTHORS .An -nosplit This manual page was written by Modified: stable/10/sys/kern/kern_synch.c ============================================================================== --- stable/10/sys/kern/kern_synch.c Wed Aug 1 08:24:34 2018 (r337033) +++ stable/10/sys/kern/kern_synch.c Wed Aug 1 10:34:53 2018 (r337034) @@ -334,16 +334,16 @@ msleep_spin_sbt(void *ident, struct mtx *mtx, const ch } /* - * pause() delays the calling thread by the given number of system ticks. - * During cold bootup, pause() uses the DELAY() function instead of - * the tsleep() function to do the waiting. The "timo" argument must be - * greater than or equal to zero. A "timo" value of zero is equivalent - * to a "timo" value of one. + * pause_sbt() delays the calling thread by the given signed binary + * time. During cold bootup, pause_sbt() uses the DELAY() function + * instead of the _sleep() function to do the waiting. The "sbt" + * argument must be greater than or equal to zero. A "sbt" value of + * zero is equivalent to a "sbt" value of one tick. */ int pause_sbt(const char *wmesg, sbintime_t sbt, sbintime_t pr, int flags) { - KASSERT(sbt >= 0, ("pause: timeout must be >= 0")); + KASSERT(sbt >= 0, ("pause_sbt: timeout must be >= 0")); /* silently convert invalid timeouts */ if (sbt == 0) @@ -364,7 +364,8 @@ pause_sbt(const char *wmesg, sbintime_t sbt, sbintime_ DELAY(sbt); return (0); } - return (_sleep(&pause_wchan[curcpu], NULL, 0, wmesg, sbt, pr, flags)); + return (_sleep(&pause_wchan[curcpu], NULL, + (flags & C_CATCH) ? PCATCH : 0, wmesg, sbt, pr, flags)); } /* Modified: stable/10/sys/sys/callout.h ============================================================================== --- stable/10/sys/sys/callout.h Wed Aug 1 08:24:34 2018 (r337033) +++ stable/10/sys/sys/callout.h Wed Aug 1 10:34:53 2018 (r337034) @@ -58,6 +58,7 @@ #define C_HARDCLOCK 0x0100 /* align to hardclock() calls */ #define C_ABSOLUTE 0x0200 /* event time is absolute. */ #define C_PRECALC 0x0400 /* event time is pre-calculated. */ +#define C_CATCH 0x0800 /* catch signals, used by pause_sbt(9) */ struct callout_handle { struct callout *callout; Modified: stable/10/sys/sys/systm.h ============================================================================== --- stable/10/sys/sys/systm.h Wed Aug 1 08:24:34 2018 (r337033) +++ stable/10/sys/sys/systm.h Wed Aug 1 10:34:53 2018 (r337034) @@ -386,6 +386,8 @@ int pause_sbt(const char *wmesg, sbintime_t sbt, sbint int flags); #define pause(wmesg, timo) \ pause_sbt((wmesg), tick_sbt * (timo), 0, C_HARDCLOCK) +#define pause_sig(wmesg, timo) \ + pause_sbt((wmesg), tick_sbt * (timo), 0, C_HARDCLOCK | C_CATCH) #define tsleep(chan, pri, wmesg, timo) \ _sleep((chan), NULL, (pri), (wmesg), tick_sbt * (timo), \ 0, C_HARDCLOCK) From owner-svn-src-all@freebsd.org Wed Aug 1 10:35:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A19FA1050EEE; Wed, 1 Aug 2018 10:35:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 573638CD81; Wed, 1 Aug 2018 10:35:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3855321525; Wed, 1 Aug 2018 10:35:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71AZoYP069471; Wed, 1 Aug 2018 10:35:50 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71AZotf069470; Wed, 1 Aug 2018 10:35:50 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808011035.w71AZotf069470@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 1 Aug 2018 10:35:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337035 - stable/10/sys/kern X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/10/sys/kern X-SVN-Commit-Revision: 337035 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 10:35:50 -0000 Author: hselasky Date: Wed Aug 1 10:35:49 2018 New Revision: 337035 URL: https://svnweb.freebsd.org/changeset/base/337035 Log: MFC r330344: Correct the return code from pause() during cold startup from zero to EWOULDBLOCK. This also matches the description in pause(9). Discussed with: kib@ Sponsored by: Mellanox Technologies Modified: stable/10/sys/kern/kern_synch.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/kern/kern_synch.c ============================================================================== --- stable/10/sys/kern/kern_synch.c Wed Aug 1 10:34:53 2018 (r337034) +++ stable/10/sys/kern/kern_synch.c Wed Aug 1 10:35:49 2018 (r337035) @@ -362,7 +362,7 @@ pause_sbt(const char *wmesg, sbintime_t sbt, sbintime_ sbt = (sbt + SBT_1US - 1) / SBT_1US; if (sbt > 0) DELAY(sbt); - return (0); + return (EWOULDBLOCK); } return (_sleep(&pause_wchan[curcpu], NULL, (flags & C_CATCH) ? PCATCH : 0, wmesg, sbt, pr, flags)); From owner-svn-src-all@freebsd.org Wed Aug 1 11:40:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 631451057513; Wed, 1 Aug 2018 11:40:53 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19C898F3C8; Wed, 1 Aug 2018 11:40:53 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EF44421EE1; Wed, 1 Aug 2018 11:40:52 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71BeqsI000697; Wed, 1 Aug 2018 11:40:52 GMT (envelope-from tsoome@FreeBSD.org) Received: (from tsoome@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71BeqJJ000696; Wed, 1 Aug 2018 11:40:52 GMT (envelope-from tsoome@FreeBSD.org) Message-Id: <201808011140.w71BeqJJ000696@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tsoome set sender to tsoome@FreeBSD.org using -f From: Toomas Soome Date: Wed, 1 Aug 2018 11:40:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337037 - head/stand/libsa X-SVN-Group: head X-SVN-Commit-Author: tsoome X-SVN-Commit-Paths: head/stand/libsa X-SVN-Commit-Revision: 337037 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 11:40:53 -0000 Author: tsoome Date: Wed Aug 1 11:40:52 2018 New Revision: 337037 URL: https://svnweb.freebsd.org/changeset/base/337037 Log: libsa: pointer differs in signedness A small cleanup, fix the argument type and while there, replace (char *)0 with NULL. Modified: head/stand/libsa/net.c Modified: head/stand/libsa/net.c ============================================================================== --- head/stand/libsa/net.c Wed Aug 1 11:08:52 2018 (r337036) +++ head/stand/libsa/net.c Wed Aug 1 11:40:52 2018 (r337037) @@ -267,7 +267,7 @@ intoa(n_long addr) } static char * -number(char *s, int *n) +number(char *s, n_long *n) { for (*n = 0; isdigit(*s); s++) *n = (*n * 10) + *s - '0'; @@ -280,7 +280,7 @@ ip_convertaddr(char *p) #define IP_ANYADDR 0 n_long addr = 0, n; - if (p == (char *)0 || *p == '\0') + if (p == NULL || *p == '\0') return IP_ANYADDR; p = number(p, &n); addr |= (n << 24) & 0xff000000; From owner-svn-src-all@freebsd.org Wed Aug 1 11:08:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D4981053EF3; Wed, 1 Aug 2018 11:08:53 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C63368DEBE; Wed, 1 Aug 2018 11:08:52 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A7439219F4; Wed, 1 Aug 2018 11:08:52 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71B8q6U085482; Wed, 1 Aug 2018 11:08:52 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71B8qPB085480; Wed, 1 Aug 2018 11:08:52 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808011108.w71B8qPB085480@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 1 Aug 2018 11:08:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337036 - in stable/10/sys: arm/arm sys X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/10/sys: arm/arm sys X-SVN-Commit-Revision: 337036 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 11:08:53 -0000 Author: hselasky Date: Wed Aug 1 11:08:52 2018 New Revision: 337036 URL: https://svnweb.freebsd.org/changeset/base/337036 Log: MFC r321686 and r330361: Add inline functions to convert between sbintime_t and decimal time units. Use them in some existing code that is vulnerable to roundoff errors. Modified: stable/10/sys/arm/arm/mpcore_timer.c stable/10/sys/sys/time.h Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/arm/arm/mpcore_timer.c ============================================================================== --- stable/10/sys/arm/arm/mpcore_timer.c Wed Aug 1 10:35:49 2018 (r337035) +++ stable/10/sys/arm/arm/mpcore_timer.c Wed Aug 1 11:08:52 2018 (r337036) @@ -349,7 +349,7 @@ attach_et(struct arm_tmr_softc *sc) sc->et.et_flags = ET_FLAGS_PERIODIC | ET_FLAGS_ONESHOT | ET_FLAGS_PERCPU; sc->et.et_quality = 1000; sc->et.et_frequency = sc->clkfreq; - sc->et.et_min_period = 20 * SBT_1NS; + sc->et.et_min_period = nstosbt(20); sc->et.et_max_period = 2 * SBT_1S; sc->et.et_start = arm_tmr_start; sc->et.et_stop = arm_tmr_stop; Modified: stable/10/sys/sys/time.h ============================================================================== --- stable/10/sys/sys/time.h Wed Aug 1 10:35:49 2018 (r337035) +++ stable/10/sys/sys/time.h Wed Aug 1 11:08:52 2018 (r337036) @@ -128,7 +128,7 @@ bintime_shift(struct bintime *_bt, int _exp) #define SBT_1M (SBT_1S * 60) #define SBT_1MS (SBT_1S / 1000) #define SBT_1US (SBT_1S / 1000000) -#define SBT_1NS (SBT_1S / 1000000000) +#define SBT_1NS (SBT_1S / 1000000000) /* beware rounding, see nstosbt() */ #define SBT_MAX 0x7fffffffffffffffLL static __inline int @@ -155,6 +155,53 @@ sbttobt(sbintime_t _sbt) return (_bt); } +/* + * Decimal<->sbt conversions. Multiplying or dividing by SBT_1NS results in + * large roundoff errors which sbttons() and nstosbt() avoid. Millisecond and + * microsecond functions are also provided for completeness. + */ +static __inline int64_t +sbttons(sbintime_t _sbt) +{ + + return ((1000000000 * _sbt) >> 32); +} + +static __inline sbintime_t +nstosbt(int64_t _ns) +{ + + return ((_ns * (((uint64_t)1 << 63) / 500000000)) >> 32); +} + +static __inline int64_t +sbttous(sbintime_t _sbt) +{ + + return ((1000000 * _sbt) >> 32); +} + +static __inline sbintime_t +ustosbt(int64_t _us) +{ + + return ((_us * (((uint64_t)1 << 63) / 500000)) >> 32); +} + +static __inline int64_t +sbttoms(sbintime_t _sbt) +{ + + return ((1000 * _sbt) >> 32); +} + +static __inline sbintime_t +mstosbt(int64_t _ms) +{ + + return ((_ms * (((uint64_t)1 << 63) / 500)) >> 32); +} + /*- * Background information: * @@ -210,7 +257,7 @@ sbttots(sbintime_t _sbt) struct timespec _ts; _ts.tv_sec = _sbt >> 32; - _ts.tv_nsec = ((uint64_t)1000000000 * (uint32_t)_sbt) >> 32; + _ts.tv_nsec = sbttons((uint32_t)_sbt); return (_ts); } @@ -218,8 +265,7 @@ static __inline sbintime_t tstosbt(struct timespec _ts) { - return (((sbintime_t)_ts.tv_sec << 32) + - (_ts.tv_nsec * (((uint64_t)1 << 63) / 500000000) >> 32)); + return (((sbintime_t)_ts.tv_sec << 32) + nstosbt(_ts.tv_nsec)); } static __inline struct timeval @@ -228,7 +274,7 @@ sbttotv(sbintime_t _sbt) struct timeval _tv; _tv.tv_sec = _sbt >> 32; - _tv.tv_usec = ((uint64_t)1000000 * (uint32_t)_sbt) >> 32; + _tv.tv_usec = sbttous((uint32_t)_sbt); return (_tv); } @@ -236,8 +282,7 @@ static __inline sbintime_t tvtosbt(struct timeval _tv) { - return (((sbintime_t)_tv.tv_sec << 32) + - (_tv.tv_usec * (((uint64_t)1 << 63) / 500000) >> 32)); + return (((sbintime_t)_tv.tv_sec << 32) + ustosbt(_tv.tv_usec)); } #endif /* __BSD_VISIBLE */ From owner-svn-src-all@freebsd.org Wed Aug 1 12:06:40 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC63A105A1DA; Wed, 1 Aug 2018 12:06:40 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81DF870F5A; Wed, 1 Aug 2018 12:06:40 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 571E7223BD; Wed, 1 Aug 2018 12:06:40 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71C6e8F016143; Wed, 1 Aug 2018 12:06:40 GMT (envelope-from tsoome@FreeBSD.org) Received: (from tsoome@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71C6drD016141; Wed, 1 Aug 2018 12:06:39 GMT (envelope-from tsoome@FreeBSD.org) Message-Id: <201808011206.w71C6drD016141@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tsoome set sender to tsoome@FreeBSD.org using -f From: Toomas Soome Date: Wed, 1 Aug 2018 12:06:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337038 - head/stand/libsa X-SVN-Group: head X-SVN-Commit-Author: tsoome X-SVN-Commit-Paths: head/stand/libsa X-SVN-Commit-Revision: 337038 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 12:06:41 -0000 Author: tsoome Date: Wed Aug 1 12:06:39 2018 New Revision: 337038 URL: https://svnweb.freebsd.org/changeset/base/337038 Log: libsa: bootp is using pointers with different sign Just change bp_file to char and same for variable s. Modified: head/stand/libsa/bootp.c head/stand/libsa/bootp.h Modified: head/stand/libsa/bootp.c ============================================================================== --- head/stand/libsa/bootp.c Wed Aug 1 11:40:52 2018 (r337037) +++ head/stand/libsa/bootp.c Wed Aug 1 12:06:39 2018 (r337038) @@ -735,7 +735,7 @@ setenv_(u_char *cp, u_char *ep, struct dhcp_opt *opts bcopy(cp, buf, size); /* cannot overflow */ buf[size] = '\0'; for (endv = buf; endv; endv = vp) { - u_char *s = NULL; /* semicolon ? */ + char *s = NULL; /* semicolon ? */ /* skip leading whitespace */ while (*endv && strchr(" \t\n\r", *endv)) Modified: head/stand/libsa/bootp.h ============================================================================== --- head/stand/libsa/bootp.h Wed Aug 1 11:40:52 2018 (r337037) +++ head/stand/libsa/bootp.h Wed Aug 1 12:06:39 2018 (r337038) @@ -39,7 +39,7 @@ struct bootp { struct in_addr bp_giaddr; /* gateway IP address */ unsigned char bp_chaddr[16]; /* client hardware address */ unsigned char bp_sname[64]; /* server host name */ - unsigned char bp_file[128]; /* boot file name */ + char bp_file[128]; /* boot file name */ #ifdef SUPPORT_DHCP #define BOOTP_VENDSIZE 312 #else From owner-svn-src-all@freebsd.org Wed Aug 1 12:14:11 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10C03105A4AD; Wed, 1 Aug 2018 12:14:11 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B98C17140C; Wed, 1 Aug 2018 12:14:10 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9A9872254F; Wed, 1 Aug 2018 12:14:10 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71CEA2V021013; Wed, 1 Aug 2018 12:14:10 GMT (envelope-from tsoome@FreeBSD.org) Received: (from tsoome@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71CEA90021012; Wed, 1 Aug 2018 12:14:10 GMT (envelope-from tsoome@FreeBSD.org) Message-Id: <201808011214.w71CEA90021012@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tsoome set sender to tsoome@FreeBSD.org using -f From: Toomas Soome Date: Wed, 1 Aug 2018 12:14:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337039 - head/stand/libsa X-SVN-Group: head X-SVN-Commit-Author: tsoome X-SVN-Commit-Paths: head/stand/libsa X-SVN-Commit-Revision: 337039 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 12:14:11 -0000 Author: tsoome Date: Wed Aug 1 12:14:10 2018 New Revision: 337039 URL: https://svnweb.freebsd.org/changeset/base/337039 Log: libsa: assignment to char * from u_char * Cast to char * instead of u_char * Modified: head/stand/libsa/nfs.c Modified: head/stand/libsa/nfs.c ============================================================================== --- head/stand/libsa/nfs.c Wed Aug 1 12:06:39 2018 (r337038) +++ head/stand/libsa/nfs.c Wed Aug 1 12:14:10 2018 (r337039) @@ -837,7 +837,7 @@ nfs_readdir(struct open_file *f, struct dirent *d) fp->off = cookie = ((uint64_t)ntohl(rent->nameplus[pos]) << 32) | ntohl(rent->nameplus[pos + 1]); pos += 2; - buf = (u_char *)&rent->nameplus[pos]; + buf = (char *)&rent->nameplus[pos]; return (0); err: From owner-svn-src-all@freebsd.org Wed Aug 1 12:18:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4E09105A647; Wed, 1 Aug 2018 12:18:53 +0000 (UTC) (envelope-from 0mp@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6BF4271699; Wed, 1 Aug 2018 12:18:53 +0000 (UTC) (envelope-from 0mp@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4CF5522552; Wed, 1 Aug 2018 12:18:53 +0000 (UTC) (envelope-from 0mp@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71CIrf1021240; Wed, 1 Aug 2018 12:18:53 GMT (envelope-from 0mp@FreeBSD.org) Received: (from 0mp@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71CIrOl021239; Wed, 1 Aug 2018 12:18:53 GMT (envelope-from 0mp@FreeBSD.org) Message-Id: <201808011218.w71CIrOl021239@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: 0mp set sender to 0mp@FreeBSD.org using -f From: Mateusz Piotrowski <0mp@FreeBSD.org> Date: Wed, 1 Aug 2018 12:18:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337040 - head/usr.sbin/daemon X-SVN-Group: head X-SVN-Commit-Author: 0mp X-SVN-Commit-Paths: head/usr.sbin/daemon X-SVN-Commit-Revision: 337040 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 12:18:53 -0000 Author: 0mp (ports committer) Date: Wed Aug 1 12:18:52 2018 New Revision: 337040 URL: https://svnweb.freebsd.org/changeset/base/337040 Log: Add a missing "Ar" macro to the description of the -R flag in the manpage. Approved by: bjk (doc committer), krion (mentor) Differential Revision: https://reviews.freebsd.org/D16520 Modified: head/usr.sbin/daemon/daemon.8 Modified: head/usr.sbin/daemon/daemon.8 ============================================================================== --- head/usr.sbin/daemon/daemon.8 Wed Aug 1 12:14:10 2018 (r337039) +++ head/usr.sbin/daemon/daemon.8 Wed Aug 1 12:18:52 2018 (r337040) @@ -66,7 +66,7 @@ Redirect standard input, standard output and standard .It Fl S Enable syslog output. This is implicitly applied if other syslog parameters are provided. -The default values are daemon, notice, and daemon for facility, priority, and +The default values are daemon, notice, and daemon for facility, priority, and tag, respectively. .It Fl o Ar output_file Append output from the daemonized process to @@ -117,7 +117,7 @@ option is used or not. .It Fl r Supervise and restart the program after a one-second delay if it has been terminated. -.It Fl R restart_delay_seconds +.It Fl R Ar restart_delay_seconds Supervise and restart the program after the specified delay if it has been terminated. .It Fl t Ar title From owner-svn-src-all@freebsd.org Wed Aug 1 12:49:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0CA2105B41E; Wed, 1 Aug 2018 12:49:52 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5581D725F6; Wed, 1 Aug 2018 12:49:52 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 33A2E22A51; Wed, 1 Aug 2018 12:49:52 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71Cnqcb036477; Wed, 1 Aug 2018 12:49:52 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71CnpWg036475; Wed, 1 Aug 2018 12:49:51 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808011249.w71CnpWg036475@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 1 Aug 2018 12:49:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337041 - in stable/10/sys: amd64/include i386/include X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/10/sys: amd64/include i386/include X-SVN-Commit-Revision: 337041 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 12:49:52 -0000 Author: hselasky Date: Wed Aug 1 12:49:51 2018 New Revision: 337041 URL: https://svnweb.freebsd.org/changeset/base/337041 Log: MFC r311169, r312975, r313080, r315718 and r316031: Add support for atomic_(f)cmpset to x86. Modified: stable/10/sys/amd64/include/atomic.h stable/10/sys/i386/include/atomic.h Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/amd64/include/atomic.h ============================================================================== --- stable/10/sys/amd64/include/atomic.h Wed Aug 1 12:18:52 2018 (r337040) +++ stable/10/sys/amd64/include/atomic.h Wed Aug 1 12:49:51 2018 (r337041) @@ -78,8 +78,15 @@ void atomic_##NAME##_##TYPE(volatile u_##TYPE *p, u_##TYPE v); \ void atomic_##NAME##_barr_##TYPE(volatile u_##TYPE *p, u_##TYPE v) +int atomic_cmpset_char(volatile u_char *dst, u_char expect, u_char src); +int atomic_cmpset_short(volatile u_short *dst, u_short expect, u_short src); int atomic_cmpset_int(volatile u_int *dst, u_int expect, u_int src); int atomic_cmpset_long(volatile u_long *dst, u_long expect, u_long src); +int atomic_fcmpset_char(volatile u_char *dst, u_char *expect, u_char src); +int atomic_fcmpset_short(volatile u_short *dst, u_short *expect, + u_short src); +int atomic_fcmpset_int(volatile u_int *dst, u_int *expect, u_int src); +int atomic_fcmpset_long(volatile u_long *dst, u_long *expect, u_long src); u_int atomic_fetchadd_int(volatile u_int *p, u_int v); u_long atomic_fetchadd_long(volatile u_long *p, u_long v); int atomic_testandset_int(volatile u_int *p, u_int v); @@ -130,49 +137,62 @@ atomic_##NAME##_barr_##TYPE(volatile u_##TYPE *p, u_## struct __hack /* - * Atomic compare and set, used by the mutex functions + * Atomic compare and set, used by the mutex functions. * - * if (*dst == expect) *dst = src (all 32 bit words) + * cmpset: + * if (*dst == expect) + * *dst = src * - * Returns 0 on failure, non-zero on success + * fcmpset: + * if (*dst == *expect) + * *dst = src + * else + * *expect = *dst + * + * Returns 0 on failure, non-zero on success. */ - -static __inline int -atomic_cmpset_int(volatile u_int *dst, u_int expect, u_int src) -{ - u_char res; - - __asm __volatile( - " " MPLOCKED " " - " cmpxchgl %3,%1 ; " - " sete %0 ; " - "# atomic_cmpset_int" - : "=q" (res), /* 0 */ - "+m" (*dst), /* 1 */ - "+a" (expect) /* 2 */ - : "r" (src) /* 3 */ - : "memory", "cc"); - return (res); +#define ATOMIC_CMPSET(TYPE) \ +static __inline int \ +atomic_cmpset_##TYPE(volatile u_##TYPE *dst, u_##TYPE expect, u_##TYPE src) \ +{ \ + u_char res; \ + \ + __asm __volatile( \ + " " MPLOCKED " " \ + " cmpxchg %3,%1 ; " \ + " sete %0 ; " \ + "# atomic_cmpset_" #TYPE " " \ + : "=q" (res), /* 0 */ \ + "+m" (*dst), /* 1 */ \ + "+a" (expect) /* 2 */ \ + : "r" (src) /* 3 */ \ + : "memory", "cc"); \ + return (res); \ +} \ + \ +static __inline int \ +atomic_fcmpset_##TYPE(volatile u_##TYPE *dst, u_##TYPE *expect, u_##TYPE src) \ +{ \ + u_char res; \ + \ + __asm __volatile( \ + " " MPLOCKED " " \ + " cmpxchg %3,%1 ; " \ + " sete %0 ; " \ + "# atomic_fcmpset_" #TYPE " " \ + : "=q" (res), /* 0 */ \ + "+m" (*dst), /* 1 */ \ + "+a" (*expect) /* 2 */ \ + : "r" (src) /* 3 */ \ + : "memory", "cc"); \ + return (res); \ } -static __inline int -atomic_cmpset_long(volatile u_long *dst, u_long expect, u_long src) -{ - u_char res; +ATOMIC_CMPSET(char); +ATOMIC_CMPSET(short); +ATOMIC_CMPSET(int); +ATOMIC_CMPSET(long); - __asm __volatile( - " " MPLOCKED " " - " cmpxchgq %3,%1 ; " - " sete %0 ; " - "# atomic_cmpset_long" - : "=q" (res), /* 0 */ - "+m" (*dst), /* 1 */ - "+a" (expect) /* 2 */ - : "r" (src) /* 3 */ - : "memory", "cc"); - return (res); -} - /* * Atomically add the value of v to the integer pointed to by p and return * the previous value of *p. @@ -409,6 +429,10 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_add_rel_char atomic_add_barr_char #define atomic_subtract_acq_char atomic_subtract_barr_char #define atomic_subtract_rel_char atomic_subtract_barr_char +#define atomic_cmpset_acq_char atomic_cmpset_char +#define atomic_cmpset_rel_char atomic_cmpset_char +#define atomic_fcmpset_acq_char atomic_fcmpset_char +#define atomic_fcmpset_rel_char atomic_fcmpset_char #define atomic_set_acq_short atomic_set_barr_short #define atomic_set_rel_short atomic_set_barr_short @@ -418,6 +442,10 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_add_rel_short atomic_add_barr_short #define atomic_subtract_acq_short atomic_subtract_barr_short #define atomic_subtract_rel_short atomic_subtract_barr_short +#define atomic_cmpset_acq_short atomic_cmpset_short +#define atomic_cmpset_rel_short atomic_cmpset_short +#define atomic_fcmpset_acq_short atomic_fcmpset_short +#define atomic_fcmpset_rel_short atomic_fcmpset_short #define atomic_set_acq_int atomic_set_barr_int #define atomic_set_rel_int atomic_set_barr_int @@ -429,6 +457,8 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_int atomic_subtract_barr_int #define atomic_cmpset_acq_int atomic_cmpset_int #define atomic_cmpset_rel_int atomic_cmpset_int +#define atomic_fcmpset_acq_int atomic_fcmpset_int +#define atomic_fcmpset_rel_int atomic_fcmpset_int #define atomic_set_acq_long atomic_set_barr_long #define atomic_set_rel_long atomic_set_barr_long @@ -440,6 +470,8 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_long atomic_subtract_barr_long #define atomic_cmpset_acq_long atomic_cmpset_long #define atomic_cmpset_rel_long atomic_cmpset_long +#define atomic_fcmpset_acq_long atomic_fcmpset_long +#define atomic_fcmpset_rel_long atomic_fcmpset_long #define atomic_readandclear_int(p) atomic_swap_int(p, 0) #define atomic_readandclear_long(p) atomic_swap_long(p, 0) @@ -459,6 +491,12 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_8 atomic_subtract_rel_char #define atomic_load_acq_8 atomic_load_acq_char #define atomic_store_rel_8 atomic_store_rel_char +#define atomic_cmpset_8 atomic_cmpset_char +#define atomic_cmpset_acq_8 atomic_cmpset_acq_char +#define atomic_cmpset_rel_8 atomic_cmpset_rel_char +#define atomic_fcmpset_8 atomic_fcmpset_char +#define atomic_fcmpset_acq_8 atomic_fcmpset_acq_char +#define atomic_fcmpset_rel_8 atomic_fcmpset_rel_char /* Operations on 16-bit words. */ #define atomic_set_16 atomic_set_short @@ -475,6 +513,12 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_16 atomic_subtract_rel_short #define atomic_load_acq_16 atomic_load_acq_short #define atomic_store_rel_16 atomic_store_rel_short +#define atomic_cmpset_16 atomic_cmpset_short +#define atomic_cmpset_acq_16 atomic_cmpset_acq_short +#define atomic_cmpset_rel_16 atomic_cmpset_rel_short +#define atomic_fcmpset_16 atomic_fcmpset_short +#define atomic_fcmpset_acq_16 atomic_fcmpset_acq_short +#define atomic_fcmpset_rel_16 atomic_fcmpset_rel_short /* Operations on 32-bit double words. */ #define atomic_set_32 atomic_set_int @@ -494,6 +538,9 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_cmpset_32 atomic_cmpset_int #define atomic_cmpset_acq_32 atomic_cmpset_acq_int #define atomic_cmpset_rel_32 atomic_cmpset_rel_int +#define atomic_fcmpset_32 atomic_fcmpset_int +#define atomic_fcmpset_acq_32 atomic_fcmpset_acq_int +#define atomic_fcmpset_rel_32 atomic_fcmpset_rel_int #define atomic_swap_32 atomic_swap_int #define atomic_readandclear_32 atomic_readandclear_int #define atomic_fetchadd_32 atomic_fetchadd_int @@ -518,6 +565,9 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_cmpset_64 atomic_cmpset_long #define atomic_cmpset_acq_64 atomic_cmpset_acq_long #define atomic_cmpset_rel_64 atomic_cmpset_rel_long +#define atomic_fcmpset_64 atomic_fcmpset_long +#define atomic_fcmpset_acq_64 atomic_fcmpset_acq_long +#define atomic_fcmpset_rel_64 atomic_fcmpset_rel_long #define atomic_swap_64 atomic_swap_long #define atomic_readandclear_64 atomic_readandclear_long #define atomic_fetchadd_64 atomic_fetchadd_long @@ -542,6 +592,9 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_cmpset_ptr atomic_cmpset_long #define atomic_cmpset_acq_ptr atomic_cmpset_acq_long #define atomic_cmpset_rel_ptr atomic_cmpset_rel_long +#define atomic_fcmpset_ptr atomic_fcmpset_long +#define atomic_fcmpset_acq_ptr atomic_fcmpset_acq_long +#define atomic_fcmpset_rel_ptr atomic_fcmpset_rel_long #define atomic_swap_ptr atomic_swap_long #define atomic_readandclear_ptr atomic_readandclear_long Modified: stable/10/sys/i386/include/atomic.h ============================================================================== --- stable/10/sys/i386/include/atomic.h Wed Aug 1 12:18:52 2018 (r337040) +++ stable/10/sys/i386/include/atomic.h Wed Aug 1 12:49:51 2018 (r337041) @@ -83,7 +83,13 @@ void atomic_##NAME##_##TYPE(volatile u_##TYPE *p, u_##TYPE v); \ void atomic_##NAME##_barr_##TYPE(volatile u_##TYPE *p, u_##TYPE v) +int atomic_cmpset_char(volatile u_char *dst, u_char expect, u_char src); +int atomic_cmpset_short(volatile u_short *dst, u_short expect, u_short src); int atomic_cmpset_int(volatile u_int *dst, u_int expect, u_int src); +int atomic_fcmpset_char(volatile u_char *dst, u_char *expect, u_char src); +int atomic_fcmpset_short(volatile u_short *dst, u_short *expect, + u_short src); +int atomic_fcmpset_int(volatile u_int *dst, u_int *expect, u_int src); u_int atomic_fetchadd_int(volatile u_int *p, u_int v); int atomic_testandset_int(volatile u_int *p, u_int v); int atomic_testandclear_int(volatile u_int *p, u_int v); @@ -137,60 +143,61 @@ atomic_##NAME##_barr_##TYPE(volatile u_##TYPE *p, u_## struct __hack /* - * Atomic compare and set, used by the mutex functions + * Atomic compare and set, used by the mutex functions. * - * if (*dst == expect) *dst = src (all 32 bit words) + * cmpset: + * if (*dst == expect) + * *dst = src * - * Returns 0 on failure, non-zero on success + * fcmpset: + * if (*dst == *expect) + * *dst = src + * else + * *expect = *dst + * + * Returns 0 on failure, non-zero on success. */ - -#ifdef CPU_DISABLE_CMPXCHG - -static __inline int -atomic_cmpset_int(volatile u_int *dst, u_int expect, u_int src) -{ - u_char res; - - __asm __volatile( - " pushfl ; " - " cli ; " - " cmpl %3,%1 ; " - " jne 1f ; " - " movl %2,%1 ; " - "1: " - " sete %0 ; " - " popfl ; " - "# atomic_cmpset_int" - : "=q" (res), /* 0 */ - "+m" (*dst) /* 1 */ - : "r" (src), /* 2 */ - "r" (expect) /* 3 */ - : "memory"); - return (res); +#define ATOMIC_CMPSET(TYPE, CONS) \ +static __inline int \ +atomic_cmpset_##TYPE(volatile u_##TYPE *dst, u_##TYPE expect, u_##TYPE src) \ +{ \ + u_char res; \ + \ + __asm __volatile( \ + " " MPLOCKED " " \ + " cmpxchg %3,%1 ; " \ + " sete %0 ; " \ + "# atomic_cmpset_" #TYPE " " \ + : "=q" (res), /* 0 */ \ + "+m" (*dst), /* 1 */ \ + "+a" (expect) /* 2 */ \ + : CONS (src) /* 3 */ \ + : "memory", "cc"); \ + return (res); \ +} \ + \ +static __inline int \ +atomic_fcmpset_##TYPE(volatile u_##TYPE *dst, u_##TYPE *expect, u_##TYPE src) \ +{ \ + u_char res; \ + \ + __asm __volatile( \ + " " MPLOCKED " " \ + " cmpxchg %3,%1 ; " \ + " sete %0 ; " \ + "# atomic_fcmpset_" #TYPE " " \ + : "=q" (res), /* 0 */ \ + "+m" (*dst), /* 1 */ \ + "+a" (*expect) /* 2 */ \ + : CONS (src) /* 3 */ \ + : "memory", "cc"); \ + return (res); \ } -#else /* !CPU_DISABLE_CMPXCHG */ +ATOMIC_CMPSET(char, "q"); +ATOMIC_CMPSET(short, "r"); +ATOMIC_CMPSET(int, "r"); -static __inline int -atomic_cmpset_int(volatile u_int *dst, u_int expect, u_int src) -{ - u_char res; - - __asm __volatile( - " " MPLOCKED " " - " cmpxchgl %3,%1 ; " - " sete %0 ; " - "# atomic_cmpset_int" - : "=q" (res), /* 0 */ - "+m" (*dst), /* 1 */ - "+a" (expect) /* 2 */ - : "r" (src) /* 3 */ - : "memory", "cc"); - return (res); -} - -#endif /* CPU_DISABLE_CMPXCHG */ - /* * Atomically add the value of v to the integer pointed to by p and return * the previous value of *p. @@ -623,6 +630,10 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_add_rel_char atomic_add_barr_char #define atomic_subtract_acq_char atomic_subtract_barr_char #define atomic_subtract_rel_char atomic_subtract_barr_char +#define atomic_cmpset_acq_char atomic_cmpset_char +#define atomic_cmpset_rel_char atomic_cmpset_char +#define atomic_fcmpset_acq_char atomic_fcmpset_char +#define atomic_fcmpset_rel_char atomic_fcmpset_char #define atomic_set_acq_short atomic_set_barr_short #define atomic_set_rel_short atomic_set_barr_short @@ -632,6 +643,10 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_add_rel_short atomic_add_barr_short #define atomic_subtract_acq_short atomic_subtract_barr_short #define atomic_subtract_rel_short atomic_subtract_barr_short +#define atomic_cmpset_acq_short atomic_cmpset_short +#define atomic_cmpset_rel_short atomic_cmpset_short +#define atomic_fcmpset_acq_short atomic_fcmpset_short +#define atomic_fcmpset_rel_short atomic_fcmpset_short #define atomic_set_acq_int atomic_set_barr_int #define atomic_set_rel_int atomic_set_barr_int @@ -643,6 +658,8 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_int atomic_subtract_barr_int #define atomic_cmpset_acq_int atomic_cmpset_int #define atomic_cmpset_rel_int atomic_cmpset_int +#define atomic_fcmpset_acq_int atomic_fcmpset_int +#define atomic_fcmpset_rel_int atomic_fcmpset_int #define atomic_set_acq_long atomic_set_barr_long #define atomic_set_rel_long atomic_set_barr_long @@ -654,6 +671,8 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_long atomic_subtract_barr_long #define atomic_cmpset_acq_long atomic_cmpset_long #define atomic_cmpset_rel_long atomic_cmpset_long +#define atomic_fcmpset_acq_long atomic_fcmpset_long +#define atomic_fcmpset_rel_long atomic_fcmpset_long #define atomic_readandclear_int(p) atomic_swap_int(p, 0) #define atomic_readandclear_long(p) atomic_swap_long(p, 0) @@ -673,6 +692,12 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_8 atomic_subtract_rel_char #define atomic_load_acq_8 atomic_load_acq_char #define atomic_store_rel_8 atomic_store_rel_char +#define atomic_cmpset_8 atomic_cmpset_char +#define atomic_cmpset_acq_8 atomic_cmpset_acq_char +#define atomic_cmpset_rel_8 atomic_cmpset_rel_char +#define atomic_fcmpset_8 atomic_fcmpset_char +#define atomic_fcmpset_acq_8 atomic_fcmpset_acq_char +#define atomic_fcmpset_rel_8 atomic_fcmpset_rel_char /* Operations on 16-bit words. */ #define atomic_set_16 atomic_set_short @@ -689,6 +714,12 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_subtract_rel_16 atomic_subtract_rel_short #define atomic_load_acq_16 atomic_load_acq_short #define atomic_store_rel_16 atomic_store_rel_short +#define atomic_cmpset_16 atomic_cmpset_short +#define atomic_cmpset_acq_16 atomic_cmpset_acq_short +#define atomic_cmpset_rel_16 atomic_cmpset_rel_short +#define atomic_fcmpset_16 atomic_fcmpset_short +#define atomic_fcmpset_acq_16 atomic_fcmpset_acq_short +#define atomic_fcmpset_rel_16 atomic_fcmpset_rel_short /* Operations on 32-bit double words. */ #define atomic_set_32 atomic_set_int @@ -708,6 +739,9 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); #define atomic_cmpset_32 atomic_cmpset_int #define atomic_cmpset_acq_32 atomic_cmpset_acq_int #define atomic_cmpset_rel_32 atomic_cmpset_rel_int +#define atomic_fcmpset_32 atomic_fcmpset_int +#define atomic_fcmpset_acq_32 atomic_fcmpset_acq_int +#define atomic_fcmpset_rel_32 atomic_fcmpset_rel_int #define atomic_swap_32 atomic_swap_int #define atomic_readandclear_32 atomic_readandclear_int #define atomic_fetchadd_32 atomic_fetchadd_int @@ -750,6 +784,14 @@ u_long atomic_swap_long(volatile u_long *p, u_long v); (u_int)(new)) #define atomic_cmpset_rel_ptr(dst, old, new) \ atomic_cmpset_rel_int((volatile u_int *)(dst), (u_int)(old), \ + (u_int)(new)) +#define atomic_fcmpset_ptr(dst, old, new) \ + atomic_fcmpset_int((volatile u_int *)(dst), (u_int *)(old), (u_int)(new)) +#define atomic_fcmpset_acq_ptr(dst, old, new) \ + atomic_fcmpset_acq_int((volatile u_int *)(dst), (u_int *)(old), \ + (u_int)(new)) +#define atomic_fcmpset_rel_ptr(dst, old, new) \ + atomic_fcmpset_rel_int((volatile u_int *)(dst), (u_int *)(old), \ (u_int)(new)) #define atomic_swap_ptr(p, v) \ atomic_swap_int((volatile u_int *)(p), (u_int)(v)) From owner-svn-src-all@freebsd.org Wed Aug 1 13:13:44 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F348105BE58; Wed, 1 Aug 2018 13:13:44 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 157837359B; Wed, 1 Aug 2018 13:13:44 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EB03122F37; Wed, 1 Aug 2018 13:13:43 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71DDhkt051427; Wed, 1 Aug 2018 13:13:43 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71DDhHO051426; Wed, 1 Aug 2018 13:13:43 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808011313.w71DDhHO051426@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 1 Aug 2018 13:13:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337042 - stable/10/sys/sys X-SVN-Group: stable-10 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/10/sys/sys X-SVN-Commit-Revision: 337042 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 13:13:44 -0000 Author: hselasky Date: Wed Aug 1 13:13:43 2018 New Revision: 337042 URL: https://svnweb.freebsd.org/changeset/base/337042 Log: MFC r322596: Add SI_SUB_TASKQ after SI_SUB_INTR and move taskqueue initialization there for EARLY_AP_STARTUP This fixes a regression accidentally introduced in r322588, due to an interaction with EARLY_AP_STARTUP. Reviewed by: bdrewery@, jhb@ Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D12053 Modified: stable/10/sys/sys/kernel.h stable/10/sys/sys/taskqueue.h Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/sys/kernel.h ============================================================================== --- stable/10/sys/sys/kernel.h Wed Aug 1 12:49:51 2018 (r337041) +++ stable/10/sys/sys/kernel.h Wed Aug 1 13:13:43 2018 (r337042) @@ -127,8 +127,9 @@ enum sysinit_sub_id { SI_SUB_SCHED_IDLE = 0x2600000, /* required idle procs */ SI_SUB_MBUF = 0x2700000, /* mbuf subsystem */ SI_SUB_INTR = 0x2800000, /* interrupt threads */ - SI_SUB_SOFTINTR = 0x2800001, /* start soft interrupt thread */ + SI_SUB_TASKQ = 0x2880000, /* task queues */ SI_SUB_ACL = 0x2900000, /* start for filesystem ACLs */ + SI_SUB_SOFTINTR = 0x2A00000, /* start soft interrupt thread */ SI_SUB_DEVFS = 0x2F00000, /* devfs ready for devices */ SI_SUB_INIT_IF = 0x3000000, /* prep for net interfaces */ SI_SUB_NETGRAPH = 0x3010000, /* Let Netgraph initialize */ Modified: stable/10/sys/sys/taskqueue.h ============================================================================== --- stable/10/sys/sys/taskqueue.h Wed Aug 1 12:49:51 2018 (r337041) +++ stable/10/sys/sys/taskqueue.h Wed Aug 1 13:13:43 2018 (r337042) @@ -141,7 +141,7 @@ taskqueue_define_##name(void *arg) \ init; \ } \ \ -SYSINIT(taskqueue_##name, SI_SUB_CONFIGURE, SI_ORDER_SECOND, \ +SYSINIT(taskqueue_##name, SI_SUB_TASKQ, SI_ORDER_SECOND, \ taskqueue_define_##name, NULL); \ \ struct __hack @@ -166,7 +166,7 @@ taskqueue_define_##name(void *arg) \ init; \ } \ \ -SYSINIT(taskqueue_##name, SI_SUB_CONFIGURE, SI_ORDER_SECOND, \ +SYSINIT(taskqueue_##name, SI_SUB_TASKQ, SI_ORDER_SECOND, \ taskqueue_define_##name, NULL); \ \ struct __hack From owner-svn-src-all@freebsd.org Wed Aug 1 14:50:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37E12105F71D; Wed, 1 Aug 2018 14:50:42 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3DFF77970; Wed, 1 Aug 2018 14:50:41 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B4E3823F0F; Wed, 1 Aug 2018 14:50:41 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71EofVk098438; Wed, 1 Aug 2018 14:50:41 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71EofqW098435; Wed, 1 Aug 2018 14:50:41 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201808011450.w71EofqW098435@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Wed, 1 Aug 2018 14:50:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337043 - head/sys/dev/sound/pci/hda X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/dev/sound/pci/hda X-SVN-Commit-Revision: 337043 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 14:50:42 -0000 Author: jhibbits Date: Wed Aug 1 14:50:41 2018 New Revision: 337043 URL: https://svnweb.freebsd.org/changeset/base/337043 Log: snd_hda: Synchronize DMA buffers for the control path Make sure both sides of the DMA buffer memory accesses for the CORB and RIRB (control buffers) in snd_hda (device and CPU) can see coherent memory. This is needed on weakly ordered architectures including PowerPC and ARM. Patch originally by mmel, with small changes. This does not cover the data path of snd_hda. We don't have sync operations for in-progress DMA buffers, to sync ranges of a map. Reviewed By: mmel Differential Revision: https://reviews.freebsd.org/D16517 Modified: head/sys/dev/sound/pci/hda/hdaa.c head/sys/dev/sound/pci/hda/hdac.c Modified: head/sys/dev/sound/pci/hda/hdaa.c ============================================================================== --- head/sys/dev/sound/pci/hda/hdaa.c Wed Aug 1 13:13:43 2018 (r337042) +++ head/sys/dev/sound/pci/hda/hdaa.c Wed Aug 1 14:50:41 2018 (r337043) @@ -1750,7 +1750,8 @@ hdaa_channel_init(kobj_t obj, void *data, struct snd_d hdaa_unlock(devinfo); if (sndbuf_alloc(ch->b, bus_get_dma_tag(devinfo->dev), - hda_get_dma_nocache(devinfo->dev) ? BUS_DMA_NOCACHE : 0, + hda_get_dma_nocache(devinfo->dev) ? BUS_DMA_NOCACHE : + BUS_DMA_COHERENT, pdevinfo->chan_size) != 0) return (NULL); Modified: head/sys/dev/sound/pci/hda/hdac.c ============================================================================== --- head/sys/dev/sound/pci/hda/hdac.c Wed Aug 1 13:13:43 2018 (r337042) +++ head/sys/dev/sound/pci/hda/hdac.c Wed Aug 1 14:50:41 2018 (r337043) @@ -580,7 +580,8 @@ hdac_dma_alloc(struct hdac_softc *sc, struct hdac_dma */ result = bus_dmamem_alloc(dma->dma_tag, (void **)&dma->dma_vaddr, BUS_DMA_NOWAIT | BUS_DMA_ZERO | - ((sc->flags & HDAC_F_DMA_NOCACHE) ? BUS_DMA_NOCACHE : 0), + ((sc->flags & HDAC_F_DMA_NOCACHE) ? BUS_DMA_NOCACHE : + BUS_DMA_COHERENT), &dma->dma_map); if (result != 0) { device_printf(sc->dev, "%s: bus_dmamem_alloc failed (%d)\n", @@ -627,11 +628,9 @@ static void hdac_dma_free(struct hdac_softc *sc, struct hdac_dma *dma) { if (dma->dma_paddr != 0) { -#if 0 /* Flush caches */ bus_dmamap_sync(dma->dma_tag, dma->dma_map, BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); -#endif bus_dmamap_unload(dma->dma_tag, dma->dma_map); dma->dma_paddr = 0; } @@ -854,7 +853,6 @@ hdac_rirb_init(struct hdac_softc *sc) HDAC_WRITE_1(&sc->mem, HDAC_RIRBCTL, HDAC_RIRBCTL_RINTCTL); #endif -#if 0 /* * Make sure that the Host CPU cache doesn't contain any dirty * cache lines that falls in the rirb. If I understood correctly, it @@ -863,7 +861,6 @@ hdac_rirb_init(struct hdac_softc *sc) */ bus_dmamap_sync(sc->rirb_dma.dma_tag, sc->rirb_dma.dma_map, BUS_DMASYNC_PREREAD); -#endif } /**************************************************************************** @@ -907,10 +904,8 @@ hdac_rirb_flush(struct hdac_softc *sc) rirb_base = (struct hdac_rirb *)sc->rirb_dma.dma_vaddr; rirbwp = HDAC_READ_1(&sc->mem, HDAC_RIRBWP); -#if 0 bus_dmamap_sync(sc->rirb_dma.dma_tag, sc->rirb_dma.dma_map, BUS_DMASYNC_POSTREAD); -#endif ret = 0; while (sc->rirb_rp != rirbwp) { @@ -934,6 +929,9 @@ hdac_rirb_flush(struct hdac_softc *sc) } ret++; } + + bus_dmamap_sync(sc->rirb_dma.dma_tag, sc->rirb_dma.dma_map, + BUS_DMASYNC_PREREAD); return (ret); } @@ -983,15 +981,11 @@ hdac_send_command(struct hdac_softc *sc, nid_t cad, ui sc->corb_wp++; sc->corb_wp %= sc->corb_size; corb = (uint32_t *)sc->corb_dma.dma_vaddr; -#if 0 bus_dmamap_sync(sc->corb_dma.dma_tag, sc->corb_dma.dma_map, BUS_DMASYNC_PREWRITE); -#endif corb[sc->corb_wp] = htole32(verb); -#if 0 bus_dmamap_sync(sc->corb_dma.dma_tag, sc->corb_dma.dma_map, BUS_DMASYNC_POSTWRITE); -#endif HDAC_WRITE_2(&sc->mem, HDAC_CORBWP, sc->corb_wp); timeout = 10000; @@ -1937,6 +1931,9 @@ hdac_stream_start(device_t dev, device_t child, addr += blksz; } + bus_dmamap_sync(sc->streams[ss].bdl.dma_tag, + sc->streams[ss].bdl.dma_map, BUS_DMASYNC_PREWRITE); + off = ss << 5; HDAC_WRITE_4(&sc->mem, off + HDAC_SDCBL, blksz * blkcnt); HDAC_WRITE_2(&sc->mem, off + HDAC_SDLVI, blkcnt - 1); @@ -1984,6 +1981,9 @@ hdac_stream_stop(device_t dev, device_t child, int dir ss = hdac_find_stream(sc, dir, stream); KASSERT(ss >= 0, ("Stop for not allocated stream (%d/%d)\n", dir, stream)); + + bus_dmamap_sync(sc->streams[ss].bdl.dma_tag, + sc->streams[ss].bdl.dma_map, BUS_DMASYNC_POSTWRITE); off = ss << 5; ctl = HDAC_READ_1(&sc->mem, off + HDAC_SDCTL0); From owner-svn-src-all@freebsd.org Wed Aug 1 15:50:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A79D310616BF; Wed, 1 Aug 2018 15:50:43 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CC2F7A7CC; Wed, 1 Aug 2018 15:50:43 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5C6FC24815; Wed, 1 Aug 2018 15:50:43 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71Foh8T028639; Wed, 1 Aug 2018 15:50:43 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71Foh3g028638; Wed, 1 Aug 2018 15:50:43 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808011550.w71Foh3g028638@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 1 Aug 2018 15:50:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337044 - head/sys/amd64/amd64 X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/amd64/amd64 X-SVN-Commit-Revision: 337044 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 15:50:43 -0000 Author: markj Date: Wed Aug 1 15:50:42 2018 New Revision: 337044 URL: https://svnweb.freebsd.org/changeset/base/337044 Log: Make sure that ENTRY() and END() refer to the same symbol. X-MFC with: r336876 Modified: head/sys/amd64/amd64/support.S Modified: head/sys/amd64/amd64/support.S ============================================================================== --- head/sys/amd64/amd64/support.S Wed Aug 1 14:50:41 2018 (r337043) +++ head/sys/amd64/amd64/support.S Wed Aug 1 15:50:42 2018 (r337044) @@ -330,7 +330,7 @@ copyout_fault: movq $EFAULT,%rax POP_FRAME_POINTER ret -END(copyout) +END(copyout_smap) /* * copyin(from_user, to_kernel, len) @@ -573,7 +573,7 @@ ENTRY(fueword_nosmap) movq %r11,(%rsi) POP_FRAME_POINTER ret -END(fueword64_nosmap) +END(fueword_nosmap) ENTRY(fueword_smap) PUSH_FRAME_POINTER @@ -592,7 +592,7 @@ ENTRY(fueword_smap) movq %r11,(%rsi) POP_FRAME_POINTER ret -END(fueword64_smap) +END(fueword_smap) ENTRY(fueword32_nosmap) PUSH_FRAME_POINTER From owner-svn-src-all@freebsd.org Wed Aug 1 15:55:15 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9BD01061C5A; Wed, 1 Aug 2018 15:55:15 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A4C47ADD3; Wed, 1 Aug 2018 15:55:15 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 51B56249AC; Wed, 1 Aug 2018 15:55:15 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71FtF4h033502; Wed, 1 Aug 2018 15:55:15 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71FtFKp033501; Wed, 1 Aug 2018 15:55:15 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201808011555.w71FtFKp033501@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Wed, 1 Aug 2018 15:55:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337045 - head/contrib/elftoolchain/libelf X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/contrib/elftoolchain/libelf X-SVN-Commit-Revision: 337045 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 15:55:16 -0000 Author: emaste Date: Wed Aug 1 15:55:14 2018 New Revision: 337045 URL: https://svnweb.freebsd.org/changeset/base/337045 Log: libelf: reload section headers after update with ELF_C_WRITE These issues (PR 218860 and PR 218861) are still undergoing additional discussion in the upstream context and there may be additional changes to come. PR: 218860 Submitted by: Eric McCorkle MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D10486 Modified: head/contrib/elftoolchain/libelf/elf_update.c Modified: head/contrib/elftoolchain/libelf/elf_update.c ============================================================================== --- head/contrib/elftoolchain/libelf/elf_update.c Wed Aug 1 15:50:42 2018 (r337044) +++ head/contrib/elftoolchain/libelf/elf_update.c Wed Aug 1 15:55:14 2018 (r337045) @@ -1212,5 +1212,6 @@ elf_update(Elf *e, Elf_Cmd c) done: _libelf_release_extents(&extents); + e->e_flags &= ~LIBELF_F_SHDRS_LOADED; return (rc); } From owner-svn-src-all@freebsd.org Wed Aug 1 16:37:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADBE51062BA6; Wed, 1 Aug 2018 16:37:56 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 442AF7C62F; Wed, 1 Aug 2018 16:37:56 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-it0-x233.google.com with SMTP id v71-v6so10185333itb.3; Wed, 01 Aug 2018 09:37:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=feVFXpWFcS9XTvVPs2qpxnuJndgsPZBzljnBsSyahUY=; b=ueJ9ZbP6BxjN4Uj+vuLXXs7sE+Mw1k9BqvTzqLDHsc3DjZcKM7A6JCuDzQP2hIjK13 jub4X9riMlYoFwxboQ+iErEZKpQv86O/dMB4vyZBNq+FW1dCkWpsqMv3wqYS5tX8fkdb I+qt/o2p6sIA8lOnzlBdOipgLN1osKUduRj0057UHJE8/Z1F7/99kg6bdScW/kcq6q4T EE2mG/G9bfqrjM3/KL0Hbq/Ntgiuw/oPhyvPCG5EMIvxA7794b0WNr024mclTswKdH9y gNj/JbT9gVC2Rwp7a9izEBvZyP4zEfFbbTdzR0n9fIILyKe7IE9dPUgfZT9mXoMPGAr/ i1rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=feVFXpWFcS9XTvVPs2qpxnuJndgsPZBzljnBsSyahUY=; b=rsgZ5S7yxMn0e4G5dj4bH16rZoXqf/TKVwr03y1trdXCmoaLggB29I3U+g1/RL/smp c8Z/qs8RwGfG8Ob29XzdmhsJErwi83+A9RYN5npA/VYH+R8w8v5sM2iY5ZA02Ef/AeAv mzLmW9QFrrM+es5tiJMgbBl3jer1j+C76FULA5n3VQ8g4dI4qx9r6ABotHYDLXsRg//U V3PBlOjhSriQdIVBepNqjlR4oPCpJyZN2PEP9LT/V7fdEofcln9G2vrCTDymEFQlHAln 9j8KUxVEmpqfnGQNBekMotVQi81kcAYfuHOc2tXnlFFeQ8Vygn28Q7yXyGISn6D/y2sv mlWw== X-Gm-Message-State: AOUpUlFogwb1DapAs77mE+1WlMO8y9trN+QBLg2WtRXEvO+Kwtv+xZrV FELNhGMUq/1/VsvL4lnuH07m+OLRsZJPL0A2FIiXeA== X-Google-Smtp-Source: AAOMgpd1zuBJ/2U/y1He6pI5lHIF9+LIYGvOe0o+1az4wjQnC65uPinC9JUZwOe+KpQO8Ct4t0gqxusukCAn0QZJRNg= X-Received: by 2002:a24:1fc7:: with SMTP id d190-v6mr3956450itd.52.1533141475515; Wed, 01 Aug 2018 09:37:55 -0700 (PDT) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 2002:a6b:4a08:0:0:0:0:0 with HTTP; Wed, 1 Aug 2018 09:37:35 -0700 (PDT) In-Reply-To: <201808011555.w71FtFKp033501@repo.freebsd.org> References: <201808011555.w71FtFKp033501@repo.freebsd.org> From: Ed Maste Date: Wed, 1 Aug 2018 12:37:35 -0400 X-Google-Sender-Auth: GBY37DclNfwGry_xpSNyTVOlaJE Message-ID: Subject: Re: svn commit: r337045 - head/contrib/elftoolchain/libelf To: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 16:37:56 -0000 On 1 August 2018 at 11:55, Ed Maste wrote: > Author: emaste > Date: Wed Aug 1 15:55:14 2018 > New Revision: 337045 > URL: https://svnweb.freebsd.org/changeset/base/337045 > > Log: > libelf: reload section headers after update with ELF_C_WRITE > > These issues (PR 218860 and PR 218861) are still undergoing additional > discussion in the upstream context and there may be additional changes > to come. > > PR: 218860 > Submitted by: Eric McCorkle Reviewed by: cem From owner-svn-src-all@freebsd.org Wed Aug 1 17:40:20 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3354106478C; Wed, 1 Aug 2018 17:40:19 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 761107F4BD; Wed, 1 Aug 2018 17:40:19 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 571B7259F3; Wed, 1 Aug 2018 17:40:19 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71HeJhd086197; Wed, 1 Aug 2018 17:40:19 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71HeIZu086191; Wed, 1 Aug 2018 17:40:18 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201808011740.w71HeIZu086191@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Wed, 1 Aug 2018 17:40:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337046 - in stable/11/sys: amd64/linux amd64/linux32 compat/freebsd32 i386/linux kern sys X-SVN-Group: stable-11 X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: in stable/11/sys: amd64/linux amd64/linux32 compat/freebsd32 i386/linux kern sys X-SVN-Commit-Revision: 337046 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 17:40:20 -0000 Author: jhb Date: Wed Aug 1 17:40:17 2018 New Revision: 337046 URL: https://svnweb.freebsd.org/changeset/base/337046 Log: MFC 332782: Simplify the code to allocate stack for auxv, argv[], and environment vectors. Remove auxarg_size as it was only used once right after a confusing assignment in each of the variants of exec_copyout_strings(). Modified: stable/11/sys/amd64/linux/linux_sysvec.c stable/11/sys/amd64/linux32/linux32_sysvec.c stable/11/sys/compat/freebsd32/freebsd32_misc.c stable/11/sys/i386/linux/linux_sysvec.c stable/11/sys/kern/kern_exec.c stable/11/sys/sys/imgact.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/linux/linux_sysvec.c ============================================================================== --- stable/11/sys/amd64/linux/linux_sysvec.c Wed Aug 1 15:55:14 2018 (r337045) +++ stable/11/sys/amd64/linux/linux_sysvec.c Wed Aug 1 17:40:17 2018 (r337046) @@ -359,31 +359,21 @@ linux_copyout_strings(struct image_params *imgp) roundup(sizeof(canary), sizeof(char *)); copyout(canary, (void *)imgp->canary, sizeof(canary)); - /* If we have a valid auxargs ptr, prepare some room on the stack. */ + vectp = (char **)destp; if (imgp->auxargs) { /* - * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for - * lower compatibility. + * Allocate room on the stack for the ELF auxargs + * array. It has LINUX_AT_COUNT entries. */ - imgp->auxarg_size = (imgp->auxarg_size) ? imgp->auxarg_size : - (LINUX_AT_COUNT * 2); - - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets,and imgp->auxarg_size is room - * for argument of Runtime loader. - */ - vectp = (char **)(destp - (imgp->args->argc + - imgp->args->envc + 2 + imgp->auxarg_size) * sizeof(char *)); - - } else { - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets - */ - vectp = (char **)(destp - (imgp->args->argc + - imgp->args->envc + 2) * sizeof(char *)); + vectp -= howmany(LINUX_AT_COUNT * sizeof(Elf64_Auxinfo), + sizeof(*vectp)); } + + /* + * Allocate room for the argv[] and env vectors including the + * terminating NULL pointers. + */ + vectp -= imgp->args->argc + 1 + imgp->args->envc + 1; /* vectp also becomes our initial stack base. */ stack_base = (register_t *)vectp; Modified: stable/11/sys/amd64/linux32/linux32_sysvec.c ============================================================================== --- stable/11/sys/amd64/linux32/linux32_sysvec.c Wed Aug 1 15:55:14 2018 (r337045) +++ stable/11/sys/amd64/linux32/linux32_sysvec.c Wed Aug 1 17:40:17 2018 (r337046) @@ -854,31 +854,21 @@ linux_copyout_strings(struct image_params *imgp) roundup(sizeof(canary), sizeof(char *)); copyout(canary, (void *)imgp->canary, sizeof(canary)); - /* If we have a valid auxargs ptr, prepare some room on the stack. */ + vectp = (uint32_t *)destp; if (imgp->auxargs) { /* - * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for - * lower compatibility. + * Allocate room on the stack for the ELF auxargs + * array. It has LINUX_AT_COUNT entries. */ - imgp->auxarg_size = (imgp->auxarg_size) ? imgp->auxarg_size : - (LINUX_AT_COUNT * 2); - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets,and imgp->auxarg_size is room - * for argument of Runtime loader. - */ - vectp = (u_int32_t *) (destp - (imgp->args->argc + - imgp->args->envc + 2 + imgp->auxarg_size) * - sizeof(u_int32_t)); - - } else { - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets - */ - vectp = (u_int32_t *)(destp - (imgp->args->argc + - imgp->args->envc + 2) * sizeof(u_int32_t)); + vectp -= howmany(LINUX_AT_COUNT * sizeof(Elf32_Auxinfo), + sizeof(*vectp)); } + + /* + * Allocate room for the argv[] and env vectors including the + * terminating NULL pointers. + */ + vectp -= imgp->args->argc + 1 + imgp->args->envc + 1; /* vectp also becomes our initial stack base. */ stack_base = vectp; Modified: stable/11/sys/compat/freebsd32/freebsd32_misc.c ============================================================================== --- stable/11/sys/compat/freebsd32/freebsd32_misc.c Wed Aug 1 15:55:14 2018 (r337045) +++ stable/11/sys/compat/freebsd32/freebsd32_misc.c Wed Aug 1 17:40:17 2018 (r337046) @@ -2885,33 +2885,21 @@ freebsd32_copyout_strings(struct image_params *imgp) destp -= ARG_MAX - imgp->args->stringspace; destp = rounddown2(destp, sizeof(uint32_t)); - /* - * If we have a valid auxargs ptr, prepare some room - * on the stack. - */ + vectp = (uint32_t *)destp; if (imgp->auxargs) { /* - * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for - * lower compatibility. + * Allocate room on the stack for the ELF auxargs + * array. It has up to AT_COUNT entries. */ - imgp->auxarg_size = (imgp->auxarg_size) ? imgp->auxarg_size - : (AT_COUNT * 2); - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets,and imgp->auxarg_size is room - * for argument of Runtime loader. - */ - vectp = (u_int32_t *) (destp - (imgp->args->argc + - imgp->args->envc + 2 + imgp->auxarg_size + execpath_len) * - sizeof(u_int32_t)); - } else { - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets - */ - vectp = (u_int32_t *)(destp - (imgp->args->argc + - imgp->args->envc + 2) * sizeof(u_int32_t)); + vectp -= howmany(AT_COUNT * sizeof(Elf32_Auxinfo), + sizeof(*vectp)); } + + /* + * Allocate room for the argv[] and env vectors including the + * terminating NULL pointers. + */ + vectp -= imgp->args->argc + 1 + imgp->args->envc + 1; /* * vectp also becomes our initial stack base Modified: stable/11/sys/i386/linux/linux_sysvec.c ============================================================================== --- stable/11/sys/i386/linux/linux_sysvec.c Wed Aug 1 15:55:14 2018 (r337045) +++ stable/11/sys/i386/linux/linux_sysvec.c Wed Aug 1 17:40:17 2018 (r337046) @@ -336,29 +336,21 @@ linux_copyout_strings(struct image_params *imgp) roundup(sizeof(canary), sizeof(char *)); copyout(canary, (void *)imgp->canary, sizeof(canary)); - /* If we have a valid auxargs ptr, prepare some room on the stack. */ + vectp = (char **)destp; if (imgp->auxargs) { /* - * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for - * lower compatibility. + * Allocate room on the stack for the ELF auxargs + * array. It has LINUX_AT_COUNT entries. */ - imgp->auxarg_size = (imgp->auxarg_size) ? imgp->auxarg_size : - (LINUX_AT_COUNT * 2); - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets,and imgp->auxarg_size is room - * for argument of Runtime loader. - */ - vectp = (char **)(destp - (imgp->args->argc + - imgp->args->envc + 2 + imgp->auxarg_size) * sizeof(char *)); - } else { - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets - */ - vectp = (char **)(destp - (imgp->args->argc + imgp->args->envc + 2) * - sizeof(char *)); + vectp -= howmany(LINUX_AT_COUNT * sizeof(Elf32_Auxinfo), + sizeof(*vectp)); } + + /* + * Allocate room for the argv[] and env vectors including the + * terminating NULL pointers. + */ + vectp -= imgp->args->argc + 1 + imgp->args->envc + 1; /* vectp also becomes our initial stack base. */ stack_base = (register_t *)vectp; Modified: stable/11/sys/kern/kern_exec.c ============================================================================== --- stable/11/sys/kern/kern_exec.c Wed Aug 1 15:55:14 2018 (r337045) +++ stable/11/sys/kern/kern_exec.c Wed Aug 1 17:40:17 2018 (r337046) @@ -1533,33 +1533,21 @@ exec_copyout_strings(struct image_params *imgp) destp -= ARG_MAX - imgp->args->stringspace; destp = rounddown2(destp, sizeof(void *)); - /* - * If we have a valid auxargs ptr, prepare some room - * on the stack. - */ + vectp = (char **)destp; if (imgp->auxargs) { /* - * 'AT_COUNT*2' is size for the ELF Auxargs data. This is for - * lower compatibility. + * Allocate room on the stack for the ELF auxargs + * array. It has up to AT_COUNT entries. */ - imgp->auxarg_size = (imgp->auxarg_size) ? imgp->auxarg_size : - (AT_COUNT * 2); - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets,and imgp->auxarg_size is room - * for argument of Runtime loader. - */ - vectp = (char **)(destp - (imgp->args->argc + - imgp->args->envc + 2 + imgp->auxarg_size) - * sizeof(char *)); - } else { - /* - * The '+ 2' is for the null pointers at the end of each of - * the arg and env vector sets - */ - vectp = (char **)(destp - (imgp->args->argc + imgp->args->envc - + 2) * sizeof(char *)); + vectp -= howmany(AT_COUNT * sizeof(Elf_Auxinfo), + sizeof(*vectp)); } + + /* + * Allocate room for the argv[] and env vectors including the + * terminating NULL pointers. + */ + vectp -= imgp->args->argc + 1 + imgp->args->envc + 1; /* * vectp also becomes our initial stack base Modified: stable/11/sys/sys/imgact.h ============================================================================== --- stable/11/sys/sys/imgact.h Wed Aug 1 15:55:14 2018 (r337045) +++ stable/11/sys/sys/imgact.h Wed Aug 1 17:40:17 2018 (r337046) @@ -73,7 +73,6 @@ struct image_params { void *auxargs; /* ELF Auxinfo structure pointer */ struct sf_buf *firstpage; /* first page that we mapped */ unsigned long ps_strings; /* PS_STRINGS for BSD/OS binaries */ - size_t auxarg_size; struct image_args *args; /* system call arguments */ struct sysentvec *sysent; /* system entry vector */ char *execpath; From owner-svn-src-all@freebsd.org Wed Aug 1 18:28:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 777161065A54; Wed, 1 Aug 2018 18:28:18 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28909812B0; Wed, 1 Aug 2018 18:28:18 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0A56B2623F; Wed, 1 Aug 2018 18:28:18 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71ISHr8011889; Wed, 1 Aug 2018 18:28:17 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71ISHVj011888; Wed, 1 Aug 2018 18:28:17 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808011828.w71ISHVj011888@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 18:28:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337047 - vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zfs vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zfs vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 337047 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:28:18 -0000 Author: mav Date: Wed Aug 1 18:28:17 2018 New Revision: 337047 URL: https://svnweb.freebsd.org/changeset/base/337047 Log: 8115 parallel zfs mount Mounting of filesystems in "filesystem/local" is done using `zfs mount -a`, which mounts each filesystems serially. The bottleneck for each mount is the I/O done to load metadata for each filesystem. As such, mounting filesystems using a parallel algorithm should be a big win, and bring down the runtime of "filesystem/local"'s start method. illumos/illumos-gate@591e0e133f9980083db5d64ac33a30bcc3382ff7 Reviewed by: Matthew Ahrens Reviewed by: Pavel Zakharov Reviewed by: Brad Lewis Reviewed by: George Wilson Reviewed by: Paul Dagnelie Reviewed by: Prashanth Sreenivasa Approved by: Matt Ahrens Author: Sebastien Roy Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h Changes in other areas also in this revision: Added: vendor/illumos/dist/lib/libzfs/common/libzfs_taskq.c (contents, props changed) vendor/illumos/dist/lib/libzfs/common/libzfs_taskq.h (contents, props changed) Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h Wed Aug 1 17:40:17 2018 (r337046) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h Wed Aug 1 18:28:17 2018 (r337047) @@ -89,7 +89,7 @@ typedef struct dsl_pool { struct dsl_dir *dp_leak_dir; struct dsl_dataset *dp_origin_snap; uint64_t dp_root_dir_obj; - struct taskq *dp_vnrele_taskq; + taskq_t *dp_vnrele_taskq; /* No lock needed - sync context only */ blkptr_t dp_meta_rootbp; From owner-svn-src-all@freebsd.org Wed Aug 1 18:28:20 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B90BB1065A58; Wed, 1 Aug 2018 18:28:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6EBFE812B2; Wed, 1 Aug 2018 18:28:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 514E726240; Wed, 1 Aug 2018 18:28:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71ISJa9011901; Wed, 1 Aug 2018 18:28:19 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71ISIZa011894; Wed, 1 Aug 2018 18:28:18 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808011828.w71ISIZa011894@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 18:28:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337047 - vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zfs vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs/sys vendor/illumos/dist/cmd/zfs vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 337047 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:28:20 -0000 Author: mav Date: Wed Aug 1 18:28:17 2018 New Revision: 337047 URL: https://svnweb.freebsd.org/changeset/base/337047 Log: 8115 parallel zfs mount Mounting of filesystems in "filesystem/local" is done using `zfs mount -a`, which mounts each filesystems serially. The bottleneck for each mount is the I/O done to load metadata for each filesystem. As such, mounting filesystems using a parallel algorithm should be a big win, and bring down the runtime of "filesystem/local"'s start method. illumos/illumos-gate@591e0e133f9980083db5d64ac33a30bcc3382ff7 Reviewed by: Matthew Ahrens Reviewed by: Pavel Zakharov Reviewed by: Brad Lewis Reviewed by: George Wilson Reviewed by: Paul Dagnelie Reviewed by: Prashanth Sreenivasa Approved by: Matt Ahrens Author: Sebastien Roy Added: vendor/illumos/dist/lib/libzfs/common/libzfs_taskq.c (contents, props changed) vendor/illumos/dist/lib/libzfs/common/libzfs_taskq.h (contents, props changed) Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/lib/libzfs/common/libzfs.h vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dsl_pool.h Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Aug 1 17:40:17 2018 (r337046) +++ vendor/illumos/dist/cmd/zfs/zfs_main.c Wed Aug 1 18:28:17 2018 (r337047) @@ -60,6 +60,7 @@ #include #include #include +#include #include #include @@ -5732,8 +5733,13 @@ zfs_do_holds(int argc, char **argv) #define CHECK_SPINNER 30 #define SPINNER_TIME 3 /* seconds */ -#define MOUNT_TIME 5 /* seconds */ +#define MOUNT_TIME 1 /* seconds */ +typedef struct get_all_state { + boolean_t ga_verbose; + get_all_cb_t *ga_cbp; +} get_all_state_t; + static int get_one_dataset(zfs_handle_t *zhp, void *data) { @@ -5741,10 +5747,10 @@ get_one_dataset(zfs_handle_t *zhp, void *data) static int spinval = 0; static int spincheck = 0; static time_t last_spin_time = (time_t)0; - get_all_cb_t *cbp = data; + get_all_state_t *state = data; zfs_type_t type = zfs_get_type(zhp); - if (cbp->cb_verbose) { + if (state->ga_verbose) { if (--spincheck < 0) { time_t now = time(NULL); if (last_spin_time + SPINNER_TIME < now) { @@ -5770,26 +5776,24 @@ get_one_dataset(zfs_handle_t *zhp, void *data) zfs_close(zhp); return (0); } - libzfs_add_handle(cbp, zhp); - assert(cbp->cb_used <= cbp->cb_alloc); + libzfs_add_handle(state->ga_cbp, zhp); + assert(state->ga_cbp->cb_used <= state->ga_cbp->cb_alloc); return (0); } static void -get_all_datasets(zfs_handle_t ***dslist, size_t *count, boolean_t verbose) +get_all_datasets(get_all_cb_t *cbp, boolean_t verbose) { - get_all_cb_t cb = { 0 }; - cb.cb_verbose = verbose; - cb.cb_getone = get_one_dataset; + get_all_state_t state = { + .ga_verbose = verbose, + .ga_cbp = cbp + }; if (verbose) set_progress_header(gettext("Reading ZFS config")); - (void) zfs_iter_root(g_zfs, get_one_dataset, &cb); + (void) zfs_iter_root(g_zfs, get_one_dataset, &state); - *dslist = cb.cb_handles; - *count = cb.cb_used; - if (verbose) finish_progress(gettext("done.")); } @@ -5799,9 +5803,20 @@ get_all_datasets(zfs_handle_t ***dslist, size_t *count * similar, we have a common function with an extra parameter to determine which * mode we are using. */ -#define OP_SHARE 0x1 -#define OP_MOUNT 0x2 +typedef enum { OP_SHARE, OP_MOUNT } share_mount_op_t; +typedef struct share_mount_state { + share_mount_op_t sm_op; + boolean_t sm_verbose; + int sm_flags; + char *sm_options; + char *sm_proto; /* only valid for OP_SHARE */ + mutex_t sm_lock; /* protects the remaining fields */ + uint_t sm_total; /* number of filesystems to process */ + uint_t sm_done; /* number of filesystems processed */ + int sm_status; /* -1 if any of the share/mount operations failed */ +} share_mount_state_t; + /* * Share or mount a dataset. */ @@ -6042,6 +6057,29 @@ report_mount_progress(int current, int total) update_progress(info); } +/* + * zfs_foreach_mountpoint() callback that mounts or shares one filesystem and + * updates the progress meter. + */ +static int +share_mount_one_cb(zfs_handle_t *zhp, void *arg) +{ + share_mount_state_t *sms = arg; + int ret; + + ret = share_mount_one(zhp, sms->sm_op, sms->sm_flags, sms->sm_proto, + B_FALSE, sms->sm_options); + + mutex_enter(&sms->sm_lock); + if (ret != 0) + sms->sm_status = ret; + sms->sm_done++; + if (sms->sm_verbose) + report_mount_progress(sms->sm_done, sms->sm_total); + mutex_exit(&sms->sm_lock); + return (ret); +} + static void append_options(char *mntopts, char *newopts) { @@ -6114,8 +6152,6 @@ share_mount(int op, int argc, char **argv) /* check number of arguments */ if (do_all) { - zfs_handle_t **dslist = NULL; - size_t i, count = 0; char *protocol = NULL; if (op == OP_SHARE && argc > 0) { @@ -6136,33 +6172,44 @@ share_mount(int op, int argc, char **argv) } start_progress_timer(); - get_all_datasets(&dslist, &count, verbose); + get_all_cb_t cb = { 0 }; + get_all_datasets(&cb, verbose); - if (count == 0) + if (cb.cb_used == 0) return (0); - qsort(dslist, count, sizeof (void *), libzfs_dataset_cmp); - sa_init_selective_arg_t sharearg; - sharearg.zhandle_arr = dslist; - sharearg.zhandle_len = count; - if ((ret = zfs_init_libshare_arg(zfs_get_handle(dslist[0]), - SA_INIT_SHARE_API_SELECTIVE, &sharearg)) != SA_OK) { - (void) fprintf(stderr, - gettext("Could not initialize libshare, %d"), ret); - return (ret); + if (op == OP_SHARE) { + sa_init_selective_arg_t sharearg; + sharearg.zhandle_arr = cb.cb_handles; + sharearg.zhandle_len = cb.cb_used; + if ((ret = zfs_init_libshare_arg(g_zfs, + SA_INIT_SHARE_API_SELECTIVE, &sharearg)) != SA_OK) { + (void) fprintf(stderr, gettext( + "Could not initialize libshare, %d"), ret); + return (ret); + } } - for (i = 0; i < count; i++) { - if (verbose) - report_mount_progress(i, count); + share_mount_state_t share_mount_state = { 0 }; + share_mount_state.sm_op = op; + share_mount_state.sm_verbose = verbose; + share_mount_state.sm_flags = flags; + share_mount_state.sm_options = options; + share_mount_state.sm_proto = protocol; + share_mount_state.sm_total = cb.cb_used; + (void) mutex_init(&share_mount_state.sm_lock, + LOCK_NORMAL | LOCK_ERRORCHECK, NULL); + /* + * libshare isn't mt-safe, so only do the operation in parallel + * if we're mounting. + */ + zfs_foreach_mountpoint(g_zfs, cb.cb_handles, cb.cb_used, + share_mount_one_cb, &share_mount_state, op == OP_MOUNT); + ret = share_mount_state.sm_status; - if (share_mount_one(dslist[i], op, flags, protocol, - B_FALSE, options) != 0) - ret = 1; - zfs_close(dslist[i]); - } - - free(dslist); + for (int i = 0; i < cb.cb_used; i++) + zfs_close(cb.cb_handles[i]); + free(cb.cb_handles); } else if (argc == 0) { struct mnttab entry; Modified: vendor/illumos/dist/lib/libzfs/common/libzfs.h ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Aug 1 17:40:17 2018 (r337046) +++ vendor/illumos/dist/lib/libzfs/common/libzfs.h Wed Aug 1 18:28:17 2018 (r337047) @@ -576,12 +576,11 @@ typedef struct get_all_cb { zfs_handle_t **cb_handles; size_t cb_alloc; size_t cb_used; - boolean_t cb_verbose; - int (*cb_getone)(zfs_handle_t *, void *); } get_all_cb_t; +void zfs_foreach_mountpoint(libzfs_handle_t *, zfs_handle_t **, size_t, + zfs_iter_f, void *, boolean_t); void libzfs_add_handle(get_all_cb_t *, zfs_handle_t *); -int libzfs_dataset_cmp(const void *, const void *); /* * Functions to create and destroy datasets. Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Aug 1 17:40:17 2018 (r337046) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Wed Aug 1 18:28:17 2018 (r337047) @@ -54,6 +54,7 @@ #include #include #include +#include #include #include @@ -785,6 +786,8 @@ libzfs_mnttab_cache_compare(const void *arg1, const vo void libzfs_mnttab_init(libzfs_handle_t *hdl) { + (void) mutex_init(&hdl->libzfs_mnttab_cache_lock, + LOCK_NORMAL | LOCK_ERRORCHECK, NULL); assert(avl_numnodes(&hdl->libzfs_mnttab_cache) == 0); avl_create(&hdl->libzfs_mnttab_cache, libzfs_mnttab_cache_compare, sizeof (mnttab_node_t), offsetof(mnttab_node_t, mtn_node)); @@ -825,6 +828,7 @@ libzfs_mnttab_fini(libzfs_handle_t *hdl) free(mtn); } avl_destroy(&hdl->libzfs_mnttab_cache); + (void) mutex_destroy(&hdl->libzfs_mnttab_cache_lock); } void @@ -839,6 +843,7 @@ libzfs_mnttab_find(libzfs_handle_t *hdl, const char *f { mnttab_node_t find; mnttab_node_t *mtn; + int ret = ENOENT; if (!hdl->libzfs_mnttab_enable) { struct mnttab srch = { 0 }; @@ -854,6 +859,7 @@ libzfs_mnttab_find(libzfs_handle_t *hdl, const char *f return (ENOENT); } + mutex_enter(&hdl->libzfs_mnttab_cache_lock); if (avl_numnodes(&hdl->libzfs_mnttab_cache) == 0) libzfs_mnttab_update(hdl); @@ -861,9 +867,10 @@ libzfs_mnttab_find(libzfs_handle_t *hdl, const char *f mtn = avl_find(&hdl->libzfs_mnttab_cache, &find, NULL); if (mtn) { *entry = mtn->mtn_mt; - return (0); + ret = 0; } - return (ENOENT); + mutex_exit(&hdl->libzfs_mnttab_cache_lock); + return (ret); } void @@ -872,14 +879,16 @@ libzfs_mnttab_add(libzfs_handle_t *hdl, const char *sp { mnttab_node_t *mtn; - if (avl_numnodes(&hdl->libzfs_mnttab_cache) == 0) - return; - mtn = zfs_alloc(hdl, sizeof (mnttab_node_t)); - mtn->mtn_mt.mnt_special = zfs_strdup(hdl, special); - mtn->mtn_mt.mnt_mountp = zfs_strdup(hdl, mountp); - mtn->mtn_mt.mnt_fstype = zfs_strdup(hdl, MNTTYPE_ZFS); - mtn->mtn_mt.mnt_mntopts = zfs_strdup(hdl, mntopts); - avl_add(&hdl->libzfs_mnttab_cache, mtn); + mutex_enter(&hdl->libzfs_mnttab_cache_lock); + if (avl_numnodes(&hdl->libzfs_mnttab_cache) != 0) { + mtn = zfs_alloc(hdl, sizeof (mnttab_node_t)); + mtn->mtn_mt.mnt_special = zfs_strdup(hdl, special); + mtn->mtn_mt.mnt_mountp = zfs_strdup(hdl, mountp); + mtn->mtn_mt.mnt_fstype = zfs_strdup(hdl, MNTTYPE_ZFS); + mtn->mtn_mt.mnt_mntopts = zfs_strdup(hdl, mntopts); + avl_add(&hdl->libzfs_mnttab_cache, mtn); + } + mutex_exit(&hdl->libzfs_mnttab_cache_lock); } void @@ -888,6 +897,7 @@ libzfs_mnttab_remove(libzfs_handle_t *hdl, const char mnttab_node_t find; mnttab_node_t *ret; + mutex_enter(&hdl->libzfs_mnttab_cache_lock); find.mtn_mt.mnt_special = (char *)fsname; if ((ret = avl_find(&hdl->libzfs_mnttab_cache, (void *)&find, NULL)) != NULL) { @@ -898,6 +908,7 @@ libzfs_mnttab_remove(libzfs_handle_t *hdl, const char free(ret->mtn_mt.mnt_mntopts); free(ret); } + mutex_exit(&hdl->libzfs_mnttab_cache_lock); } int Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h Wed Aug 1 17:40:17 2018 (r337046) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_impl.h Wed Aug 1 18:28:17 2018 (r337047) @@ -22,7 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright (c) 2011 Pawel Jakub Dawidek. All rights reserved. - * Copyright (c) 2011, 2016 by Delphix. All rights reserved. + * Copyright (c) 2011, 2017 by Delphix. All rights reserved. */ #ifndef _LIBZFS_IMPL_H @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -73,6 +74,13 @@ struct libzfs_handle { int libzfs_storeerr; /* stuff error messages into buffer */ void *libzfs_sharehdl; /* libshare handle */ boolean_t libzfs_mnttab_enable; + /* + * We need a lock to handle the case where parallel mount + * threads are populating the mnttab cache simultaneously. The + * lock only protects the integrity of the avl tree, and does + * not protect the contents of the mnttab entries themselves. + */ + mutex_t libzfs_mnttab_cache_lock; avl_tree_t libzfs_mnttab_cache; int libzfs_pool_iter; topo_hdl_t *libzfs_topo_hdl; Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Wed Aug 1 17:40:17 2018 (r337046) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_mount.c Wed Aug 1 18:28:17 2018 (r337047) @@ -22,7 +22,7 @@ /* * Copyright 2015 Nexenta Systems, Inc. All rights reserved. * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2014, 2016 by Delphix. All rights reserved. + * Copyright (c) 2014, 2017 by Delphix. All rights reserved. * Copyright 2016 Igor Kozhukhov * Copyright 2017 Joyent, Inc. * Copyright 2017 RackTop Systems. @@ -34,25 +34,25 @@ * they are used by mount and unmount and when changing a filesystem's * mountpoint. * - * zfs_is_mounted() - * zfs_mount() - * zfs_unmount() - * zfs_unmountall() + * zfs_is_mounted() + * zfs_mount() + * zfs_unmount() + * zfs_unmountall() * * This file also contains the functions used to manage sharing filesystems via * NFS and iSCSI: * - * zfs_is_shared() - * zfs_share() - * zfs_unshare() + * zfs_is_shared() + * zfs_share() + * zfs_unshare() * - * zfs_is_shared_nfs() - * zfs_is_shared_smb() - * zfs_share_proto() - * zfs_shareall(); - * zfs_unshare_nfs() - * zfs_unshare_smb() - * zfs_unshareall_nfs() + * zfs_is_shared_nfs() + * zfs_is_shared_smb() + * zfs_share_proto() + * zfs_shareall(); + * zfs_unshare_nfs() + * zfs_unshare_smb() + * zfs_unshareall_nfs() * zfs_unshareall_smb() * zfs_unshareall() * zfs_unshareall_bypath() @@ -60,8 +60,8 @@ * The following functions are available for pool consumers, and will * mount/unmount and share/unshare all datasets within pool: * - * zpool_enable_datasets() - * zpool_disable_datasets() + * zpool_enable_datasets() + * zpool_disable_datasets() */ #include @@ -83,11 +83,15 @@ #include #include "libzfs_impl.h" +#include "libzfs_taskq.h" #include #include #define MAXISALEN 257 /* based on sysinfo(2) man page */ +static int mount_tq_nthr = 512; /* taskq threads for multi-threaded mounting */ + +static void zfs_mount_task(void *); static int zfs_share_proto(zfs_handle_t *, zfs_share_proto_t *); zfs_share_type_t zfs_is_shared_proto(zfs_handle_t *, char **, zfs_share_proto_t); @@ -1077,25 +1081,32 @@ remove_mountpoint(zfs_handle_t *zhp) } } +/* + * Add the given zfs handle to the cb_handles array, dynamically reallocating + * the array if it is out of space. + */ void libzfs_add_handle(get_all_cb_t *cbp, zfs_handle_t *zhp) { if (cbp->cb_alloc == cbp->cb_used) { size_t newsz; - void *ptr; + zfs_handle_t **newhandles; - newsz = cbp->cb_alloc ? cbp->cb_alloc * 2 : 64; - ptr = zfs_realloc(zhp->zfs_hdl, - cbp->cb_handles, cbp->cb_alloc * sizeof (void *), - newsz * sizeof (void *)); - cbp->cb_handles = ptr; + newsz = cbp->cb_alloc != 0 ? cbp->cb_alloc * 2 : 64; + newhandles = zfs_realloc(zhp->zfs_hdl, + cbp->cb_handles, cbp->cb_alloc * sizeof (zfs_handle_t *), + newsz * sizeof (zfs_handle_t *)); + cbp->cb_handles = newhandles; cbp->cb_alloc = newsz; } cbp->cb_handles[cbp->cb_used++] = zhp; } +/* + * Recursive helper function used during file system enumeration + */ static int -mount_cb(zfs_handle_t *zhp, void *data) +zfs_iter_cb(zfs_handle_t *zhp, void *data) { get_all_cb_t *cbp = data; @@ -1121,104 +1132,350 @@ mount_cb(zfs_handle_t *zhp, void *data) } libzfs_add_handle(cbp, zhp); - if (zfs_iter_filesystems(zhp, mount_cb, cbp) != 0) { + if (zfs_iter_filesystems(zhp, zfs_iter_cb, cbp) != 0) { zfs_close(zhp); return (-1); } return (0); } +/* + * Sort comparator that compares two mountpoint paths. We sort these paths so + * that subdirectories immediately follow their parents. This means that we + * effectively treat the '/' character as the lowest value non-nul char. An + * example sorted list using this comparator would look like: + * + * /foo + * /foo/bar + * /foo/bar/baz + * /foo/baz + * /foo.bar + * + * The mounting code depends on this ordering to deterministically iterate + * over filesystems in order to spawn parallel mount tasks. + */ int -libzfs_dataset_cmp(const void *a, const void *b) +mountpoint_cmp(const void *arga, const void *argb) { - zfs_handle_t **za = (zfs_handle_t **)a; - zfs_handle_t **zb = (zfs_handle_t **)b; + zfs_handle_t *const *zap = arga; + zfs_handle_t *za = *zap; + zfs_handle_t *const *zbp = argb; + zfs_handle_t *zb = *zbp; char mounta[MAXPATHLEN]; char mountb[MAXPATHLEN]; + const char *a = mounta; + const char *b = mountb; boolean_t gota, gotb; - if ((gota = (zfs_get_type(*za) == ZFS_TYPE_FILESYSTEM)) != 0) - verify(zfs_prop_get(*za, ZFS_PROP_MOUNTPOINT, mounta, + gota = (zfs_get_type(za) == ZFS_TYPE_FILESYSTEM); + if (gota) { + verify(zfs_prop_get(za, ZFS_PROP_MOUNTPOINT, mounta, sizeof (mounta), NULL, NULL, 0, B_FALSE) == 0); - if ((gotb = (zfs_get_type(*zb) == ZFS_TYPE_FILESYSTEM)) != 0) - verify(zfs_prop_get(*zb, ZFS_PROP_MOUNTPOINT, mountb, + } + gotb = (zfs_get_type(zb) == ZFS_TYPE_FILESYSTEM); + if (gotb) { + verify(zfs_prop_get(zb, ZFS_PROP_MOUNTPOINT, mountb, sizeof (mountb), NULL, NULL, 0, B_FALSE) == 0); + } - if (gota && gotb) - return (strcmp(mounta, mountb)); + if (gota && gotb) { + while (*a != '\0' && (*a == *b)) { + a++; + b++; + } + if (*a == *b) + return (0); + if (*a == '\0') + return (-1); + if (*b == '\0') + return (1); + if (*a == '/') + return (-1); + if (*b == '/') + return (1); + return (*a < *b ? -1 : *a > *b); + } if (gota) return (-1); if (gotb) return (1); - return (strcmp(zfs_get_name(a), zfs_get_name(b))); + /* + * If neither filesystem has a mountpoint, revert to sorting by + * dataset name. + */ + return (strcmp(zfs_get_name(za), zfs_get_name(zb))); } /* + * Return true if path2 is a child of path1. + */ +static boolean_t +libzfs_path_contains(const char *path1, const char *path2) +{ + return (strstr(path2, path1) == path2 && path2[strlen(path1)] == '/'); +} + +/* + * Given a mountpoint specified by idx in the handles array, find the first + * non-descendent of that mountpoint and return its index. Descendant paths + * start with the parent's path. This function relies on the ordering + * enforced by mountpoint_cmp(). + */ +static int +non_descendant_idx(zfs_handle_t **handles, size_t num_handles, int idx) +{ + char parent[ZFS_MAXPROPLEN]; + char child[ZFS_MAXPROPLEN]; + int i; + + verify(zfs_prop_get(handles[idx], ZFS_PROP_MOUNTPOINT, parent, + sizeof (parent), NULL, NULL, 0, B_FALSE) == 0); + + for (i = idx + 1; i < num_handles; i++) { + verify(zfs_prop_get(handles[i], ZFS_PROP_MOUNTPOINT, child, + sizeof (child), NULL, NULL, 0, B_FALSE) == 0); + if (!libzfs_path_contains(parent, child)) + break; + } + return (i); +} + +typedef struct mnt_param { + libzfs_handle_t *mnt_hdl; + zfs_taskq_t *mnt_tq; + zfs_handle_t **mnt_zhps; /* filesystems to mount */ + size_t mnt_num_handles; + int mnt_idx; /* Index of selected entry to mount */ + zfs_iter_f mnt_func; + void *mnt_data; +} mnt_param_t; + +/* + * Allocate and populate the parameter struct for mount function, and + * schedule mounting of the entry selected by idx. + */ +static void +zfs_dispatch_mount(libzfs_handle_t *hdl, zfs_handle_t **handles, + size_t num_handles, int idx, zfs_iter_f func, void *data, zfs_taskq_t *tq) +{ + mnt_param_t *mnt_param = zfs_alloc(hdl, sizeof (mnt_param_t)); + + mnt_param->mnt_hdl = hdl; + mnt_param->mnt_tq = tq; + mnt_param->mnt_zhps = handles; + mnt_param->mnt_num_handles = num_handles; + mnt_param->mnt_idx = idx; + mnt_param->mnt_func = func; + mnt_param->mnt_data = data; + + (void) zfs_taskq_dispatch(tq, zfs_mount_task, (void*)mnt_param, + ZFS_TQ_SLEEP); +} + +/* + * This is the structure used to keep state of mounting or sharing operations + * during a call to zpool_enable_datasets(). + */ +typedef struct mount_state { + /* + * ms_mntstatus is set to -1 if any mount fails. While multiple threads + * could update this variable concurrently, no synchronization is + * needed as it's only ever set to -1. + */ + int ms_mntstatus; + int ms_mntflags; + const char *ms_mntopts; +} mount_state_t; + +static int +zfs_mount_one(zfs_handle_t *zhp, void *arg) +{ + mount_state_t *ms = arg; + int ret = 0; + + if (zfs_mount(zhp, ms->ms_mntopts, ms->ms_mntflags) != 0) + ret = ms->ms_mntstatus = -1; + return (ret); +} + +static int +zfs_share_one(zfs_handle_t *zhp, void *arg) +{ + mount_state_t *ms = arg; + int ret = 0; + + if (zfs_share(zhp) != 0) + ret = ms->ms_mntstatus = -1; + return (ret); +} + +/* + * Task queue function to mount one file system. On completion, it finds and + * schedules its children to be mounted. This depends on the sorting done in + * zfs_foreach_mountpoint(). Note that the degenerate case (chain of entries + * each descending from the previous) will have no parallelism since we always + * have to wait for the parent to finish mounting before we can schedule + * its children. + */ +static void +zfs_mount_task(void *arg) +{ + mnt_param_t *mp = arg; + int idx = mp->mnt_idx; + zfs_handle_t **handles = mp->mnt_zhps; + size_t num_handles = mp->mnt_num_handles; + char mountpoint[ZFS_MAXPROPLEN]; + + verify(zfs_prop_get(handles[idx], ZFS_PROP_MOUNTPOINT, mountpoint, + sizeof (mountpoint), NULL, NULL, 0, B_FALSE) == 0); + + if (mp->mnt_func(handles[idx], mp->mnt_data) != 0) + return; + + /* + * We dispatch tasks to mount filesystems with mountpoints underneath + * this one. We do this by dispatching the next filesystem with a + * descendant mountpoint of the one we just mounted, then skip all of + * its descendants, dispatch the next descendant mountpoint, and so on. + * The non_descendant_idx() function skips over filesystems that are + * descendants of the filesystem we just dispatched. + */ + for (int i = idx + 1; i < num_handles; + i = non_descendant_idx(handles, num_handles, i)) { + char child[ZFS_MAXPROPLEN]; + verify(zfs_prop_get(handles[i], ZFS_PROP_MOUNTPOINT, + child, sizeof (child), NULL, NULL, 0, B_FALSE) == 0); + + if (!libzfs_path_contains(mountpoint, child)) + break; /* not a descendant, return */ + zfs_dispatch_mount(mp->mnt_hdl, handles, num_handles, i, + mp->mnt_func, mp->mnt_data, mp->mnt_tq); + } + free(mp); +} + +/* + * Issue the func callback for each ZFS handle contained in the handles + * array. This function is used to mount all datasets, and so this function + * guarantees that filesystems for parent mountpoints are called before their + * children. As such, before issuing any callbacks, we first sort the array + * of handles by mountpoint. + * + * Callbacks are issued in one of two ways: + * + * 1. Sequentially: If the parallel argument is B_FALSE or the ZFS_SERIAL_MOUNT + * environment variable is set, then we issue callbacks sequentially. + * + * 2. In parallel: If the parallel argument is B_TRUE and the ZFS_SERIAL_MOUNT + * environment variable is not set, then we use a taskq to dispatch threads + * to mount filesystems is parallel. This function dispatches tasks to mount + * the filesystems at the top-level mountpoints, and these tasks in turn + * are responsible for recursively mounting filesystems in their children + * mountpoints. + */ +void +zfs_foreach_mountpoint(libzfs_handle_t *hdl, zfs_handle_t **handles, + size_t num_handles, zfs_iter_f func, void *data, boolean_t parallel) +{ + /* + * The ZFS_SERIAL_MOUNT environment variable is an undocumented + * variable that can be used as a convenience to do a/b comparison + * of serial vs. parallel mounting. + */ + boolean_t serial_mount = !parallel || + (getenv("ZFS_SERIAL_MOUNT") != NULL); + + /* + * Sort the datasets by mountpoint. See mountpoint_cmp for details + * of how these are sorted. + */ + qsort(handles, num_handles, sizeof (zfs_handle_t *), mountpoint_cmp); + + if (serial_mount) { + for (int i = 0; i < num_handles; i++) { + func(handles[i], data); + } + return; + } + + /* + * Issue the callback function for each dataset using a parallel + * algorithm that uses a taskq to manage threads. + */ + zfs_taskq_t *tq = zfs_taskq_create("mount_taskq", mount_tq_nthr, 0, + mount_tq_nthr, mount_tq_nthr, ZFS_TASKQ_PREPOPULATE); + + /* + * There may be multiple "top level" mountpoints outside of the pool's + * root mountpoint, e.g.: /foo /bar. Dispatch a mount task for each of + * these. + */ + for (int i = 0; i < num_handles; + i = non_descendant_idx(handles, num_handles, i)) { + zfs_dispatch_mount(hdl, handles, num_handles, i, func, data, + tq); + } + + zfs_taskq_wait(tq); /* wait for all scheduled mounts to complete */ + zfs_taskq_destroy(tq); +} + +/* * Mount and share all datasets within the given pool. This assumes that no - * datasets within the pool are currently mounted. Because users can create - * complicated nested hierarchies of mountpoints, we first gather all the - * datasets and mountpoints within the pool, and sort them by mountpoint. Once - * we have the list of all filesystems, we iterate over them in order and mount - * and/or share each one. + * datasets within the pool are currently mounted. */ #pragma weak zpool_mount_datasets = zpool_enable_datasets int zpool_enable_datasets(zpool_handle_t *zhp, const char *mntopts, int flags) { get_all_cb_t cb = { 0 }; - libzfs_handle_t *hdl = zhp->zpool_hdl; + mount_state_t ms = { 0 }; zfs_handle_t *zfsp; - int i, ret = -1; - int *good; + sa_init_selective_arg_t sharearg; + int ret = 0; - /* - * Gather all non-snap datasets within the pool. - */ - if ((zfsp = zfs_open(hdl, zhp->zpool_name, ZFS_TYPE_DATASET)) == NULL) + if ((zfsp = zfs_open(zhp->zpool_hdl, zhp->zpool_name, + ZFS_TYPE_DATASET)) == NULL) goto out; - libzfs_add_handle(&cb, zfsp); - if (zfs_iter_filesystems(zfsp, mount_cb, &cb) != 0) - goto out; - /* - * Sort the datasets by mountpoint. - */ - qsort(cb.cb_handles, cb.cb_used, sizeof (void *), - libzfs_dataset_cmp); /* - * And mount all the datasets, keeping track of which ones - * succeeded or failed. + * Gather all non-snapshot datasets within the pool. Start by adding + * the root filesystem for this pool to the list, and then iterate + * over all child filesystems. */ - if ((good = zfs_alloc(zhp->zpool_hdl, - cb.cb_used * sizeof (int))) == NULL) + libzfs_add_handle(&cb, zfsp); + if (zfs_iter_filesystems(zfsp, zfs_iter_cb, &cb) != 0) goto out; - ret = 0; - for (i = 0; i < cb.cb_used; i++) { - if (zfs_mount(cb.cb_handles[i], mntopts, flags) != 0) - ret = -1; - else - good[i] = 1; - } + ms.ms_mntopts = mntopts; + ms.ms_mntflags = flags; + zfs_foreach_mountpoint(zhp->zpool_hdl, cb.cb_handles, cb.cb_used, + zfs_mount_one, &ms, B_TRUE); + if (ms.ms_mntstatus != 0) + ret = ms.ms_mntstatus; /* - * Then share all the ones that need to be shared. This needs - * to be a separate pass in order to avoid excessive reloading - * of the configuration. Good should never be NULL since - * zfs_alloc is supposed to exit if memory isn't available. + * Share all filesystems that need to be shared. This needs to be + * a separate pass because libshare is not mt-safe, and so we need + * to share serially. */ - for (i = 0; i < cb.cb_used; i++) { - if (good[i] && zfs_share(cb.cb_handles[i]) != 0) - ret = -1; - } + sharearg.zhandle_arr = cb.cb_handles; + sharearg.zhandle_len = cb.cb_used; + if ((ret = zfs_init_libshare_arg(zhp->zpool_hdl, + SA_INIT_SHARE_API_SELECTIVE, &sharearg)) != 0) + goto out; - free(good); + ms.ms_mntstatus = 0; + zfs_foreach_mountpoint(zhp->zpool_hdl, cb.cb_handles, cb.cb_used, + zfs_share_one, &ms, B_FALSE); + if (ms.ms_mntstatus != 0) + ret = ms.ms_mntstatus; out: - for (i = 0; i < cb.cb_used; i++) + for (int i = 0; i < cb.cb_used; i++) zfs_close(cb.cb_handles[i]); free(cb.cb_handles); Added: vendor/illumos/dist/lib/libzfs/common/libzfs_taskq.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_taskq.c Wed Aug 1 18:28:17 2018 (r337047) @@ -0,0 +1,297 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2010 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ +/* + * Copyright 2011 Nexenta Systems, Inc. All rights reserved. + * Copyright 2012 Garrett D'Amore . All rights reserved. + * Copyright (c) 2014, 2018 by Delphix. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "libzfs_taskq.h" + +#define ZFS_TASKQ_ACTIVE 0x00010000 +#define ZFS_TASKQ_NAMELEN 31 + +typedef struct zfs_taskq_ent { + struct zfs_taskq_ent *ztqent_next; + struct zfs_taskq_ent *ztqent_prev; + ztask_func_t *ztqent_func; + void *ztqent_arg; + uintptr_t ztqent_flags; +} zfs_taskq_ent_t; + +struct zfs_taskq { + char ztq_name[ZFS_TASKQ_NAMELEN + 1]; + mutex_t ztq_lock; + rwlock_t ztq_threadlock; + cond_t ztq_dispatch_cv; + cond_t ztq_wait_cv; + thread_t *ztq_threadlist; + int ztq_flags; + int ztq_active; + int ztq_nthreads; + int ztq_nalloc; + int ztq_minalloc; + int ztq_maxalloc; + cond_t ztq_maxalloc_cv; + int ztq_maxalloc_wait; + zfs_taskq_ent_t *ztq_freelist; + zfs_taskq_ent_t ztq_task; +}; + +static zfs_taskq_ent_t * +ztask_alloc(zfs_taskq_t *ztq, int ztqflags) +{ + zfs_taskq_ent_t *t; + timestruc_t ts; + int err; + +again: if ((t = ztq->ztq_freelist) != NULL && + ztq->ztq_nalloc >= ztq->ztq_minalloc) { + ztq->ztq_freelist = t->ztqent_next; + } else { + if (ztq->ztq_nalloc >= ztq->ztq_maxalloc) { + if (!(ztqflags & UMEM_NOFAIL)) + return (NULL); + + /* + * We don't want to exceed ztq_maxalloc, but we can't + * wait for other tasks to complete (and thus free up + * task structures) without risking deadlock with + * the caller. So, we just delay for one second + * to throttle the allocation rate. If we have tasks + * complete before one second timeout expires then + * zfs_taskq_ent_free will signal us and we will + * immediately retry the allocation. + */ + ztq->ztq_maxalloc_wait++; + + ts.tv_sec = 1; + ts.tv_nsec = 0; + err = cond_reltimedwait(&ztq->ztq_maxalloc_cv, + &ztq->ztq_lock, &ts); + + ztq->ztq_maxalloc_wait--; + if (err == 0) + goto again; /* signaled */ + } + mutex_exit(&ztq->ztq_lock); + + t = umem_alloc(sizeof (zfs_taskq_ent_t), ztqflags); + + mutex_enter(&ztq->ztq_lock); + if (t != NULL) + ztq->ztq_nalloc++; + } + return (t); +} + +static void +ztask_free(zfs_taskq_t *ztq, zfs_taskq_ent_t *t) +{ + if (ztq->ztq_nalloc <= ztq->ztq_minalloc) { + t->ztqent_next = ztq->ztq_freelist; + ztq->ztq_freelist = t; + } else { + ztq->ztq_nalloc--; + mutex_exit(&ztq->ztq_lock); + umem_free(t, sizeof (zfs_taskq_ent_t)); + mutex_enter(&ztq->ztq_lock); *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Wed Aug 1 18:35:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE1781065DA7; Wed, 1 Aug 2018 18:35:18 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0FC981A4A; Wed, 1 Aug 2018 18:35:18 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 81DC0263F7; Wed, 1 Aug 2018 18:35:18 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71IZI5N017039; Wed, 1 Aug 2018 18:35:18 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71IZHfH017033; Wed, 1 Aug 2018 18:35:17 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808011835.w71IZHfH017033@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Wed, 1 Aug 2018 18:35:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337048 - in head: lib/libkvm sys/mips/include X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head: lib/libkvm sys/mips/include X-SVN-Commit-Revision: 337048 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:35:19 -0000 Author: kib Date: Wed Aug 1 18:35:17 2018 New Revision: 337048 URL: https://svnweb.freebsd.org/changeset/base/337048 Log: Make cache coherency attributes definitions available in machine/vm.h on MIPS. Move definitions from cpuregs.h into the cca.h, and include cca.h into vm.h. This is required to make MIPS MD memattr definitions usable in userspace. Sponsored by: The FreeBSD Foundation, Mellanox Technologies MFC after: 1 week Differential revision: https://reviews.freebsd.org/D15583 Added: head/sys/mips/include/cca.h - copied, changed from r337047, head/sys/mips/include/cpuregs.h Modified: head/lib/libkvm/kvm_minidump_mips.c head/sys/mips/include/cpuregs.h head/sys/mips/include/vm.h Modified: head/lib/libkvm/kvm_minidump_mips.c ============================================================================== --- head/lib/libkvm/kvm_minidump_mips.c Wed Aug 1 18:28:17 2018 (r337047) +++ head/lib/libkvm/kvm_minidump_mips.c Wed Aug 1 18:35:17 2018 (r337048) @@ -45,6 +45,8 @@ __FBSDID("$FreeBSD$"); #include #include +#include "../../sys/mips/include/cca.h" +#define _KVM_MINIDUMP #include "../../sys/mips/include/cpuregs.h" #include "../../sys/mips/include/minidump.h" Copied and modified: head/sys/mips/include/cca.h (from r337047, head/sys/mips/include/cpuregs.h) ============================================================================== --- head/sys/mips/include/cpuregs.h Wed Aug 1 18:28:17 2018 (r337047, copy source) +++ head/sys/mips/include/cca.h Wed Aug 1 18:35:17 2018 (r337048) @@ -57,55 +57,10 @@ * $FreeBSD$ */ -#ifndef _MIPS_CPUREGS_H_ -#define _MIPS_CPUREGS_H_ +#ifndef _MIPS_CCA_H_ +#define _MIPS_CCA_H_ /* - * Address space. - * 32-bit mips CPUS partition their 32-bit address space into four segments: - * - * kuseg 0x00000000 - 0x7fffffff User virtual mem, mapped - * kseg0 0x80000000 - 0x9fffffff Physical memory, cached, unmapped - * kseg1 0xa0000000 - 0xbfffffff Physical memory, uncached, unmapped - * kseg2 0xc0000000 - 0xffffffff kernel-virtual, mapped - * - * Caching of mapped addresses is controlled by bits in the TLB entry. - */ - -#define MIPS_KSEG0_LARGEST_PHYS (0x20000000) -#define MIPS_KSEG0_PHYS_MASK (0x1fffffff) -#define MIPS_XKPHYS_LARGEST_PHYS (0x10000000000) /* 40 bit PA */ -#define MIPS_XKPHYS_PHYS_MASK (0x0ffffffffff) - -#ifndef LOCORE -#define MIPS_KUSEG_START 0x00000000 -#define MIPS_KSEG0_START ((intptr_t)(int32_t)0x80000000) -#define MIPS_KSEG0_END ((intptr_t)(int32_t)0x9fffffff) -#define MIPS_KSEG1_START ((intptr_t)(int32_t)0xa0000000) -#define MIPS_KSEG1_END ((intptr_t)(int32_t)0xbfffffff) -#define MIPS_KSSEG_START ((intptr_t)(int32_t)0xc0000000) -#define MIPS_KSSEG_END ((intptr_t)(int32_t)0xdfffffff) -#define MIPS_KSEG3_START ((intptr_t)(int32_t)0xe0000000) -#define MIPS_KSEG3_END ((intptr_t)(int32_t)0xffffffff) -#define MIPS_KSEG2_START MIPS_KSSEG_START -#define MIPS_KSEG2_END MIPS_KSSEG_END -#endif - -#define MIPS_PHYS_TO_KSEG0(x) ((uintptr_t)(x) | MIPS_KSEG0_START) -#define MIPS_PHYS_TO_KSEG1(x) ((uintptr_t)(x) | MIPS_KSEG1_START) -#define MIPS_KSEG0_TO_PHYS(x) ((uintptr_t)(x) & MIPS_KSEG0_PHYS_MASK) -#define MIPS_KSEG1_TO_PHYS(x) ((uintptr_t)(x) & MIPS_KSEG0_PHYS_MASK) - -#define MIPS_IS_KSEG0_ADDR(x) \ - (((vm_offset_t)(x) >= MIPS_KSEG0_START) && \ - ((vm_offset_t)(x) <= MIPS_KSEG0_END)) -#define MIPS_IS_KSEG1_ADDR(x) \ - (((vm_offset_t)(x) >= MIPS_KSEG1_START) && \ - ((vm_offset_t)(x) <= MIPS_KSEG1_END)) -#define MIPS_IS_VALID_PTR(x) (MIPS_IS_KSEG0_ADDR(x) || \ - MIPS_IS_KSEG1_ADDR(x)) - -/* * Cache Coherency Attributes: * UC: Uncached. * UA: Uncached accelerated. @@ -195,537 +150,4 @@ #endif #endif -#define MIPS_PHYS_TO_XKPHYS(cca,x) \ - ((0x2ULL << 62) | ((unsigned long long)(cca) << 59) | (x)) -#define MIPS_PHYS_TO_XKPHYS_CACHED(x) \ - ((0x2ULL << 62) | ((unsigned long long)(MIPS_CCA_CACHED) << 59) | (x)) -#define MIPS_PHYS_TO_XKPHYS_UNCACHED(x) \ - ((0x2ULL << 62) | ((unsigned long long)(MIPS_CCA_UNCACHED) << 59) | (x)) - -#define MIPS_XKPHYS_TO_PHYS(x) ((uintptr_t)(x) & MIPS_XKPHYS_PHYS_MASK) - -#define MIPS_XKPHYS_START 0x8000000000000000 -#define MIPS_XKPHYS_END 0xbfffffffffffffff -#define MIPS_XUSEG_START 0x0000000000000000 -#define MIPS_XUSEG_END 0x0000010000000000 -#define MIPS_XKSEG_START 0xc000000000000000 -#define MIPS_XKSEG_END 0xc00000ff80000000 -#define MIPS_XKSEG_COMPAT32_START 0xffffffff80000000 -#define MIPS_XKSEG_COMPAT32_END 0xffffffffffffffff -#define MIPS_XKSEG_TO_COMPAT32(va) ((va) & 0xffffffff) - -#ifdef __mips_n64 -#define MIPS_DIRECT_MAPPABLE(pa) 1 -#define MIPS_PHYS_TO_DIRECT(pa) MIPS_PHYS_TO_XKPHYS_CACHED(pa) -#define MIPS_PHYS_TO_DIRECT_UNCACHED(pa) MIPS_PHYS_TO_XKPHYS_UNCACHED(pa) -#define MIPS_DIRECT_TO_PHYS(va) MIPS_XKPHYS_TO_PHYS(va) -#else -#define MIPS_DIRECT_MAPPABLE(pa) ((pa) < MIPS_KSEG0_LARGEST_PHYS) -#define MIPS_PHYS_TO_DIRECT(pa) MIPS_PHYS_TO_KSEG0(pa) -#define MIPS_PHYS_TO_DIRECT_UNCACHED(pa) MIPS_PHYS_TO_KSEG1(pa) -#define MIPS_DIRECT_TO_PHYS(va) MIPS_KSEG0_TO_PHYS(va) #endif - -/* CPU dependent mtc0 hazard hook */ -#if defined(CPU_CNMIPS) || defined(CPU_RMI) -#define COP0_SYNC -#elif defined(CPU_NLM) -#define COP0_SYNC .word 0xc0 /* ehb */ -#elif defined(CPU_SB1) -#define COP0_SYNC ssnop; ssnop; ssnop; ssnop; ssnop; ssnop; ssnop; ssnop; ssnop -#elif defined(CPU_MIPS24K) || defined(CPU_MIPS34K) || \ - defined(CPU_MIPS74K) || defined(CPU_MIPS1004K) || \ - defined(CPU_MIPS1074K) || defined(CPU_INTERAPTIV) || \ - defined(CPU_PROAPTIV) -/* - * According to MIPS32tm Architecture for Programmers, Vol.II, rev. 2.00: - * "As EHB becomes standard in MIPS implementations, the previous SSNOPs can be - * removed, leaving only the EHB". - * Also, all MIPS32 Release 2 implementations have the EHB instruction, which - * resolves all execution hazards. The same goes for MIPS32 Release 3. - */ -#define COP0_SYNC .word 0xc0 /* ehb */ -#else -/* - * Pick a reasonable default based on the "typical" spacing described in the - * "CP0 Hazards" chapter of MIPS Architecture Book Vol III. - */ -#define COP0_SYNC ssnop; ssnop; ssnop; ssnop; .word 0xc0; -#endif -#define COP0_HAZARD_FPUENABLE nop; nop; nop; nop; - -/* - * The bits in the cause register. - * - * Bits common to r3000 and r4000: - * - * MIPS_CR_BR_DELAY Exception happened in branch delay slot. - * MIPS_CR_COP_ERR Coprocessor error. - * MIPS_CR_IP Interrupt pending bits defined below. - * (same meaning as in CAUSE register). - * MIPS_CR_EXC_CODE The exception type (see exception codes below). - * - * Differences: - * r3k has 4 bits of execption type, r4k has 5 bits. - */ -#define MIPS_CR_BR_DELAY 0x80000000 -#define MIPS_CR_COP_ERR 0x30000000 -#define MIPS_CR_EXC_CODE 0x0000007C /* five bits */ -#define MIPS_CR_IP 0x0000FF00 -#define MIPS_CR_EXC_CODE_SHIFT 2 -#define MIPS_CR_COP_ERR_SHIFT 28 - -/* - * The bits in the status register. All bits are active when set to 1. - * - * R3000 status register fields: - * MIPS_SR_COP_USABILITY Control the usability of the four coprocessors. - * MIPS_SR_TS TLB shutdown. - * - * MIPS_SR_INT_IE Master (current) interrupt enable bit. - * - * Differences: - * r3k has cache control is via frobbing SR register bits, whereas the - * r4k cache control is via explicit instructions. - * r3k has a 3-entry stack of kernel/user bits, whereas the - * r4k has kernel/supervisor/user. - */ -#define MIPS_SR_COP_USABILITY 0xf0000000 -#define MIPS_SR_COP_0_BIT 0x10000000 -#define MIPS_SR_COP_1_BIT 0x20000000 -#define MIPS_SR_COP_2_BIT 0x40000000 - - /* r4k and r3k differences, see below */ - -#define MIPS_SR_MX 0x01000000 /* MIPS64 */ -#define MIPS_SR_PX 0x00800000 /* MIPS64 */ -#define MIPS_SR_BEV 0x00400000 /* Use boot exception vector */ -#define MIPS_SR_TS 0x00200000 -#define MIPS_SR_DE 0x00010000 - -#define MIPS_SR_INT_IE 0x00000001 -/*#define MIPS_SR_MBZ 0x0f8000c0*/ /* Never used, true for r3k */ -#define MIPS_SR_INT_MASK 0x0000ff00 - -/* - * R4000 status register bit definitons, - * where different from r2000/r3000. - */ -#define MIPS_SR_XX 0x80000000 -#define MIPS_SR_RP 0x08000000 -#define MIPS_SR_FR 0x04000000 -#define MIPS_SR_RE 0x02000000 - -#define MIPS_SR_DIAG_DL 0x01000000 /* QED 52xx */ -#define MIPS_SR_DIAG_IL 0x00800000 /* QED 52xx */ -#define MIPS_SR_SR 0x00100000 -#define MIPS_SR_NMI 0x00080000 /* MIPS32/64 */ -#define MIPS_SR_DIAG_CH 0x00040000 -#define MIPS_SR_DIAG_CE 0x00020000 -#define MIPS_SR_DIAG_PE 0x00010000 -#define MIPS_SR_EIE 0x00010000 /* TX79/R5900 */ -#define MIPS_SR_KX 0x00000080 -#define MIPS_SR_SX 0x00000040 -#define MIPS_SR_UX 0x00000020 -#define MIPS_SR_KSU_MASK 0x00000018 -#define MIPS_SR_KSU_USER 0x00000010 -#define MIPS_SR_KSU_SUPER 0x00000008 -#define MIPS_SR_KSU_KERNEL 0x00000000 -#define MIPS_SR_ERL 0x00000004 -#define MIPS_SR_EXL 0x00000002 - -/* - * The interrupt masks. - * If a bit in the mask is 1 then the interrupt is enabled (or pending). - */ -#define MIPS_INT_MASK 0xff00 -#define MIPS_INT_MASK_5 0x8000 -#define MIPS_INT_MASK_4 0x4000 -#define MIPS_INT_MASK_3 0x2000 -#define MIPS_INT_MASK_2 0x1000 -#define MIPS_INT_MASK_1 0x0800 -#define MIPS_INT_MASK_0 0x0400 -#define MIPS_HARD_INT_MASK 0xfc00 -#define MIPS_SOFT_INT_MASK_1 0x0200 -#define MIPS_SOFT_INT_MASK_0 0x0100 - -/* - * The bits in the MIPS3 config register. - * - * bit 0..5: R/W, Bit 6..31: R/O - */ - -/* kseg0 coherency algorithm - see MIPS3_TLB_ATTR values */ -#define MIPS_CONFIG_K0_MASK 0x00000007 - -/* - * R/W Update on Store Conditional - * 0: Store Conditional uses coherency algorithm specified by TLB - * 1: Store Conditional uses cacheable coherent update on write - */ -#define MIPS_CONFIG_CU 0x00000008 - -#define MIPS_CONFIG_DB 0x00000010 /* Primary D-cache line size */ -#define MIPS_CONFIG_IB 0x00000020 /* Primary I-cache line size */ -#define MIPS_CONFIG_CACHE_L1_LSIZE(config, bit) \ - (((config) & (bit)) ? 32 : 16) - -#define MIPS_CONFIG_DC_MASK 0x000001c0 /* Primary D-cache size */ -#define MIPS_CONFIG_DC_SHIFT 6 -#define MIPS_CONFIG_IC_MASK 0x00000e00 /* Primary I-cache size */ -#define MIPS_CONFIG_IC_SHIFT 9 -#define MIPS_CONFIG_C_DEFBASE 0x1000 /* default base 2^12 */ - -/* Cache size mode indication: available only on Vr41xx CPUs */ -#define MIPS_CONFIG_CS 0x00001000 -#define MIPS_CONFIG_C_4100BASE 0x0400 /* base is 2^10 if CS=1 */ -#define MIPS_CONFIG_CACHE_SIZE(config, mask, base, shift) \ - ((base) << (((config) & (mask)) >> (shift))) - -/* External cache enable: Controls L2 for R5000/Rm527x and L3 for Rm7000 */ -#define MIPS_CONFIG_SE 0x00001000 - -/* Block ordering: 0: sequential, 1: sub-block */ -#define MIPS_CONFIG_EB 0x00002000 - -/* ECC mode - 0: ECC mode, 1: parity mode */ -#define MIPS_CONFIG_EM 0x00004000 - -/* BigEndianMem - 0: kernel and memory are little endian, 1: big endian */ -#define MIPS_CONFIG_BE 0x00008000 - -/* Dirty Shared coherency state - 0: enabled, 1: disabled */ -#define MIPS_CONFIG_SM 0x00010000 - -/* Secondary Cache - 0: present, 1: not present */ -#define MIPS_CONFIG_SC 0x00020000 - -/* System Port width - 0: 64-bit, 1: 32-bit (QED RM523x), 2,3: reserved */ -#define MIPS_CONFIG_EW_MASK 0x000c0000 -#define MIPS_CONFIG_EW_SHIFT 18 - -/* Secondary Cache port width - 0: 128-bit data path to S-cache, 1: reserved */ -#define MIPS_CONFIG_SW 0x00100000 - -/* Split Secondary Cache Mode - 0: I/D mixed, 1: I/D separated by SCAddr(17) */ -#define MIPS_CONFIG_SS 0x00200000 - -/* Secondary Cache line size */ -#define MIPS_CONFIG_SB_MASK 0x00c00000 -#define MIPS_CONFIG_SB_SHIFT 22 -#define MIPS_CONFIG_CACHE_L2_LSIZE(config) \ - (0x10 << (((config) & MIPS_CONFIG_SB_MASK) >> MIPS_CONFIG_SB_SHIFT)) - -/* Write back data rate */ -#define MIPS_CONFIG_EP_MASK 0x0f000000 -#define MIPS_CONFIG_EP_SHIFT 24 - -/* System clock ratio - this value is CPU dependent */ -#define MIPS_CONFIG_EC_MASK 0x70000000 -#define MIPS_CONFIG_EC_SHIFT 28 - -/* Master-Checker Mode - 1: enabled */ -#define MIPS_CONFIG_CM 0x80000000 - -/* - * The bits in the MIPS4 config register. - */ - -/* - * Location of exception vectors. - * - * Common vectors: reset and UTLB miss. - */ -#define MIPS_RESET_EXC_VEC ((intptr_t)(int32_t)0xBFC00000) -#define MIPS_UTLB_MISS_EXC_VEC ((intptr_t)(int32_t)0x80000000) - -/* - * MIPS-III exception vectors - */ -#define MIPS_XTLB_MISS_EXC_VEC ((intptr_t)(int32_t)0x80000080) -#define MIPS_CACHE_ERR_EXC_VEC ((intptr_t)(int32_t)0x80000100) -#define MIPS_GEN_EXC_VEC ((intptr_t)(int32_t)0x80000180) - -/* - * MIPS32/MIPS64 (and some MIPS3) dedicated interrupt vector. - */ -#define MIPS_INTR_EXC_VEC 0x80000200 - -/* - * Coprocessor 0 registers: - * - * v--- width for mips I,III,32,64 - * (3=32bit, 6=64bit, i=impl dep) - * 0 MIPS_COP_0_TLB_INDEX 3333 TLB Index. - * 1 MIPS_COP_0_TLB_RANDOM 3333 TLB Random. - * 2 MIPS_COP_0_TLB_LO0 .636 r4k TLB entry low. - * 3 MIPS_COP_0_TLB_LO1 .636 r4k TLB entry low, extended. - * 4 MIPS_COP_0_TLB_CONTEXT 3636 TLB Context. - * 4/2 MIPS_COP_0_USERLOCAL ..36 UserLocal. - * 5 MIPS_COP_0_TLB_PG_MASK .333 TLB Page Mask register. - * 6 MIPS_COP_0_TLB_WIRED .333 Wired TLB number. - * 7 MIPS_COP_0_HWRENA ..33 rdHWR Enable. - * 8 MIPS_COP_0_BAD_VADDR 3636 Bad virtual address. - * 9 MIPS_COP_0_COUNT .333 Count register. - * 10 MIPS_COP_0_TLB_HI 3636 TLB entry high. - * 11 MIPS_COP_0_COMPARE .333 Compare (against Count). - * 12 MIPS_COP_0_STATUS 3333 Status register. - * 12/1 MIPS_COP_0_INTCTL ..33 Interrupt setup (MIPS32/64 r2). - * 13 MIPS_COP_0_CAUSE 3333 Exception cause register. - * 14 MIPS_COP_0_EXC_PC 3636 Exception PC. - * 15 MIPS_COP_0_PRID 3333 Processor revision identifier. - * 16 MIPS_COP_0_CONFIG 3333 Configuration register. - * 16/1 MIPS_COP_0_CONFIG1 ..33 Configuration register 1. - * 16/2 MIPS_COP_0_CONFIG2 ..33 Configuration register 2. - * 16/3 MIPS_COP_0_CONFIG3 ..33 Configuration register 3. - * 16/4 MIPS_COP_0_CONFIG4 ..33 Configuration register 4. - * 17 MIPS_COP_0_LLADDR .336 Load Linked Address. - * 18 MIPS_COP_0_WATCH_LO .336 WatchLo register. - * 19 MIPS_COP_0_WATCH_HI .333 WatchHi register. - * 20 MIPS_COP_0_TLB_XCONTEXT .6.6 TLB XContext register. - * 23 MIPS_COP_0_DEBUG .... Debug JTAG register. - * 24 MIPS_COP_0_DEPC .... DEPC JTAG register. - * 25 MIPS_COP_0_PERFCNT ..36 Performance Counter register. - * 26 MIPS_COP_0_ECC .3ii ECC / Error Control register. - * 27 MIPS_COP_0_CACHE_ERR .3ii Cache Error register. - * 28/0 MIPS_COP_0_TAG_LO .3ii Cache TagLo register (instr). - * 28/1 MIPS_COP_0_DATA_LO ..ii Cache DataLo register (instr). - * 28/2 MIPS_COP_0_TAG_LO ..ii Cache TagLo register (data). - * 28/3 MIPS_COP_0_DATA_LO ..ii Cache DataLo register (data). - * 29/0 MIPS_COP_0_TAG_HI .3ii Cache TagHi register (instr). - * 29/1 MIPS_COP_0_DATA_HI ..ii Cache DataHi register (instr). - * 29/2 MIPS_COP_0_TAG_HI ..ii Cache TagHi register (data). - * 29/3 MIPS_COP_0_DATA_HI ..ii Cache DataHi register (data). - * 30 MIPS_COP_0_ERROR_PC .636 Error EPC register. - * 31 MIPS_COP_0_DESAVE .... DESAVE JTAG register. - */ - -/* Deal with inclusion from an assembly file. */ -#if defined(_LOCORE) || defined(LOCORE) -#define _(n) $n -#else -#define _(n) n -#endif - - -#define MIPS_COP_0_TLB_INDEX _(0) -#define MIPS_COP_0_TLB_RANDOM _(1) - /* Name and meaning of TLB bits for $2 differ on r3k and r4k. */ - -#define MIPS_COP_0_TLB_CONTEXT _(4) - /* $5 and $6 new with MIPS-III */ -#define MIPS_COP_0_BAD_VADDR _(8) -#define MIPS_COP_0_TLB_HI _(10) -#define MIPS_COP_0_STATUS _(12) -#define MIPS_COP_0_CAUSE _(13) -#define MIPS_COP_0_EXC_PC _(14) -#define MIPS_COP_0_PRID _(15) - -/* MIPS-III */ -#define MIPS_COP_0_TLB_LO0 _(2) -#define MIPS_COP_0_TLB_LO1 _(3) - -#define MIPS_COP_0_TLB_PG_MASK _(5) -#define MIPS_COP_0_TLB_WIRED _(6) - -#define MIPS_COP_0_COUNT _(9) -#define MIPS_COP_0_COMPARE _(11) -#ifdef CPU_XBURST -#define MIPS_COP_0_XBURST_C12 _(12) -#endif -#define MIPS_COP_0_CONFIG _(16) -#define MIPS_COP_0_LLADDR _(17) -#define MIPS_COP_0_WATCH_LO _(18) -#define MIPS_COP_0_WATCH_HI _(19) -#define MIPS_COP_0_TLB_XCONTEXT _(20) -#ifdef CPU_XBURST -#define MIPS_COP_0_XBURST_MBOX _(20) -#endif - -#define MIPS_COP_0_ECC _(26) -#define MIPS_COP_0_CACHE_ERR _(27) -#define MIPS_COP_0_TAG_LO _(28) -#define MIPS_COP_0_TAG_HI _(29) -#define MIPS_COP_0_ERROR_PC _(30) - -/* MIPS32/64 */ -#define MIPS_COP_0_USERLOCAL _(4) /* sel 2 is userlevel register */ -#define MIPS_COP_0_HWRENA _(7) -#define MIPS_COP_0_INTCTL _(12) -#define MIPS_COP_0_DEBUG _(23) -#define MIPS_COP_0_DEPC _(24) -#define MIPS_COP_0_PERFCNT _(25) -#define MIPS_COP_0_DATA_LO _(28) -#define MIPS_COP_0_DATA_HI _(29) -#define MIPS_COP_0_DESAVE _(31) - -/* MIPS32 Config register definitions */ -#define MIPS_MMU_NONE 0x00 /* No MMU present */ -#define MIPS_MMU_TLB 0x01 /* Standard TLB */ -#define MIPS_MMU_BAT 0x02 /* Standard BAT */ -#define MIPS_MMU_FIXED 0x03 /* Standard fixed mapping */ - -/* - * IntCtl Register Fields - */ -#define MIPS_INTCTL_IPTI_MASK 0xE0000000 /* bits 31..29 timer intr # */ -#define MIPS_INTCTL_IPTI_SHIFT 29 -#define MIPS_INTCTL_IPPCI_MASK 0x1C000000 /* bits 26..29 perf counter intr # */ -#define MIPS_INTCTL_IPPCI_SHIFT 26 -#define MIPS_INTCTL_VS_MASK 0x000001F0 /* bits 5..9 vector spacing */ -#define MIPS_INTCTL_VS_SHIFT 4 - -/* - * Config Register Fields - * (See "MIPS Architecture for Programmers Volume III", MD00091, Table 9.39) - */ -#define MIPS_CONFIG0_M 0x80000000 /* Flag: Config1 is present. */ -#define MIPS_CONFIG0_MT_MASK 0x00000380 /* bits 9..7 MMU Type */ -#define MIPS_CONFIG0_MT_SHIFT 7 -#define MIPS_CONFIG0_BE 0x00008000 /* data is big-endian */ -#define MIPS_CONFIG0_VI 0x00000008 /* inst cache is virtual */ - -/* - * Config1 Register Fields - * (See "MIPS Architecture for Programmers Volume III", MD00091, Table 9-1) - */ -#define MIPS_CONFIG1_M 0x80000000 /* Flag: Config2 is present. */ -#define MIPS_CONFIG1_TLBSZ_MASK 0x7E000000 /* bits 30..25 # tlb entries minus one */ -#define MIPS_CONFIG1_TLBSZ_SHIFT 25 - -#define MIPS_CONFIG1_IS_MASK 0x01C00000 /* bits 24..22 icache sets per way */ -#define MIPS_CONFIG1_IS_SHIFT 22 -#define MIPS_CONFIG1_IL_MASK 0x00380000 /* bits 21..19 icache line size */ -#define MIPS_CONFIG1_IL_SHIFT 19 -#define MIPS_CONFIG1_IA_MASK 0x00070000 /* bits 18..16 icache associativity */ -#define MIPS_CONFIG1_IA_SHIFT 16 -#define MIPS_CONFIG1_DS_MASK 0x0000E000 /* bits 15..13 dcache sets per way */ -#define MIPS_CONFIG1_DS_SHIFT 13 -#define MIPS_CONFIG1_DL_MASK 0x00001C00 /* bits 12..10 dcache line size */ -#define MIPS_CONFIG1_DL_SHIFT 10 -#define MIPS_CONFIG1_DA_MASK 0x00000380 /* bits 9.. 7 dcache associativity */ -#define MIPS_CONFIG1_DA_SHIFT 7 -#define MIPS_CONFIG1_LOWBITS 0x0000007F -#define MIPS_CONFIG1_C2 0x00000040 /* Coprocessor 2 implemented */ -#define MIPS_CONFIG1_MD 0x00000020 /* MDMX ASE implemented (MIPS64) */ -#define MIPS_CONFIG1_PC 0x00000010 /* Performance counters implemented */ -#define MIPS_CONFIG1_WR 0x00000008 /* Watch registers implemented */ -#define MIPS_CONFIG1_CA 0x00000004 /* MIPS16e ISA implemented */ -#define MIPS_CONFIG1_EP 0x00000002 /* EJTAG implemented */ -#define MIPS_CONFIG1_FP 0x00000001 /* FPU implemented */ - -#define MIPS_CONFIG2_SA_SHIFT 0 /* Secondary cache associativity */ -#define MIPS_CONFIG2_SA_MASK 0xf -#define MIPS_CONFIG2_SL_SHIFT 4 /* Secondary cache line size */ -#define MIPS_CONFIG2_SL_MASK 0xf -#define MIPS_CONFIG2_SS_SHIFT 8 /* Secondary cache sets per way */ -#define MIPS_CONFIG2_SS_MASK 0xf - -#define MIPS_CONFIG3_CMGCR_MASK (1 << 29) /* Coherence manager present */ - -/* - * Config2 Register Fields - * (See "MIPS Architecture for Programmers Volume III", MD00091, Table 9.40) - */ -#define MIPS_CONFIG2_M 0x80000000 /* Flag: Config3 is present. */ - -/* - * Config3 Register Fields - * (See "MIPS Architecture for Programmers Volume III", MD00091, Table 9.41) - */ -#define MIPS_CONFIG3_M 0x80000000 /* Flag: Config4 is present */ -#define MIPS_CONFIG3_ULR 0x00002000 /* UserLocal reg implemented */ - -#define MIPS_CONFIG4_MMUSIZEEXT 0x000000FF /* bits 7.. 0 MMU Size Extension */ -#define MIPS_CONFIG4_MMUEXTDEF 0x0000C000 /* bits 15.14 MMU Extension Definition */ -#define MIPS_CONFIG4_MMUEXTDEF_MMUSIZEEXT 0x00004000 /* This values denotes CONFIG4 bits */ - -/* - * Values for the code field in a break instruction. - */ -#define MIPS_BREAK_INSTR 0x0000000d -#define MIPS_BREAK_VAL_MASK 0x03ff0000 -#define MIPS_BREAK_VAL_SHIFT 16 -#define MIPS_BREAK_KDB_VAL 512 -#define MIPS_BREAK_SSTEP_VAL 513 -#define MIPS_BREAK_BRKPT_VAL 514 -#define MIPS_BREAK_SOVER_VAL 515 -#define MIPS_BREAK_DDB_VAL 516 -#define MIPS_BREAK_KDB (MIPS_BREAK_INSTR | \ - (MIPS_BREAK_KDB_VAL << MIPS_BREAK_VAL_SHIFT)) -#define MIPS_BREAK_SSTEP (MIPS_BREAK_INSTR | \ - (MIPS_BREAK_SSTEP_VAL << MIPS_BREAK_VAL_SHIFT)) -#define MIPS_BREAK_BRKPT (MIPS_BREAK_INSTR | \ - (MIPS_BREAK_BRKPT_VAL << MIPS_BREAK_VAL_SHIFT)) -#define MIPS_BREAK_SOVER (MIPS_BREAK_INSTR | \ - (MIPS_BREAK_SOVER_VAL << MIPS_BREAK_VAL_SHIFT)) -#define MIPS_BREAK_DDB (MIPS_BREAK_INSTR | \ - (MIPS_BREAK_DDB_VAL << MIPS_BREAK_VAL_SHIFT)) - -/* - * Mininum and maximum cache sizes. - */ -#define MIPS_MIN_CACHE_SIZE (16 * 1024) -#define MIPS_MAX_CACHE_SIZE (256 * 1024) -#define MIPS_MAX_PCACHE_SIZE (32 * 1024) /* max. primary cache size */ - -/* - * The floating point version and status registers. - */ -#define MIPS_FPU_ID $0 -#define MIPS_FPU_CSR $31 - -/* - * The floating point coprocessor status register bits. - */ -#define MIPS_FPU_ROUNDING_BITS 0x00000003 -#define MIPS_FPU_ROUND_RN 0x00000000 -#define MIPS_FPU_ROUND_RZ 0x00000001 -#define MIPS_FPU_ROUND_RP 0x00000002 -#define MIPS_FPU_ROUND_RM 0x00000003 -#define MIPS_FPU_STICKY_BITS 0x0000007c -#define MIPS_FPU_STICKY_INEXACT 0x00000004 -#define MIPS_FPU_STICKY_UNDERFLOW 0x00000008 -#define MIPS_FPU_STICKY_OVERFLOW 0x00000010 -#define MIPS_FPU_STICKY_DIV0 0x00000020 -#define MIPS_FPU_STICKY_INVALID 0x00000040 -#define MIPS_FPU_ENABLE_BITS 0x00000f80 -#define MIPS_FPU_ENABLE_INEXACT 0x00000080 -#define MIPS_FPU_ENABLE_UNDERFLOW 0x00000100 -#define MIPS_FPU_ENABLE_OVERFLOW 0x00000200 -#define MIPS_FPU_ENABLE_DIV0 0x00000400 -#define MIPS_FPU_ENABLE_INVALID 0x00000800 -#define MIPS_FPU_EXCEPTION_BITS 0x0003f000 -#define MIPS_FPU_EXCEPTION_INEXACT 0x00001000 -#define MIPS_FPU_EXCEPTION_UNDERFLOW 0x00002000 -#define MIPS_FPU_EXCEPTION_OVERFLOW 0x00004000 -#define MIPS_FPU_EXCEPTION_DIV0 0x00008000 -#define MIPS_FPU_EXCEPTION_INVALID 0x00010000 -#define MIPS_FPU_EXCEPTION_UNIMPL 0x00020000 -#define MIPS_FPU_COND_BIT 0x00800000 -#define MIPS_FPU_FLUSH_BIT 0x01000000 /* r4k, MBZ on r3k */ -#define MIPS_FPC_MBZ_BITS 0xfe7c0000 - - -/* - * Constants to determine if have a floating point instruction. - */ -#define MIPS_OPCODE_SHIFT 26 -#define MIPS_OPCODE_C1 0x11 - -/* Coherence manager constants */ -#define MIPS_CMGCRB_BASE 11 -#define MIPS_CMGCRF_BASE (~((1 << MIPS_CMGCRB_BASE) - 1)) - -/* - * Bits defined for for the HWREna (CP0 register 7, select 0). - */ -#define MIPS_HWRENA_CPUNUM (1<<0) /* CPU number program is running on */ -#define MIPS_HWRENA_SYNCI_STEP (1<<1) /* Address step sized used with SYNCI */ -#define MIPS_HWRENA_CC (1<<2) /* Hi Res cycle counter */ -#define MIPS_HWRENA_CCRES (1<<3) /* Cycle counter resolution */ -#define MIPS_HWRENA_UL (1<<29) /* UserLocal Register */ -#define MIPS_HWRENA_IMPL30 (1<<30) /* Implementation-dependent 30 */ -#define MIPS_HWRENA_IMPL31 (1<<31) /* Implementation-dependent 31 */ - -#endif /* _MIPS_CPUREGS_H_ */ Modified: head/sys/mips/include/cpuregs.h ============================================================================== --- head/sys/mips/include/cpuregs.h Wed Aug 1 18:28:17 2018 (r337047) +++ head/sys/mips/include/cpuregs.h Wed Aug 1 18:35:17 2018 (r337048) @@ -60,6 +60,10 @@ #ifndef _MIPS_CPUREGS_H_ #define _MIPS_CPUREGS_H_ +#ifndef _KVM_MINIDUMP +#include +#endif + /* * Address space. * 32-bit mips CPUS partition their 32-bit address space into four segments: @@ -104,96 +108,6 @@ ((vm_offset_t)(x) <= MIPS_KSEG1_END)) #define MIPS_IS_VALID_PTR(x) (MIPS_IS_KSEG0_ADDR(x) || \ MIPS_IS_KSEG1_ADDR(x)) - -/* - * Cache Coherency Attributes: - * UC: Uncached. - * UA: Uncached accelerated. - * C: Cacheable, coherency unspecified. - * CNC: Cacheable non-coherent. - * CC: Cacheable coherent. - * CCS: Cacheable coherent, shared read. - * CCE: Cacheable coherent, exclusive read. - * CCEW: Cacheable coherent, exclusive write. - * CCUOW: Cacheable coherent, update on write. - * - * Note that some bits vary in meaning across implementations (and that the - * listing here is no doubt incomplete) and that the optimal cached mode varies - * between implementations. 0x02 is required to be UC and 0x03 is required to - * be a least C. - * - * We define the following logical bits: - * UNCACHED: - * The optimal uncached mode for the target CPU type. This must - * be suitable for use in accessing memory-mapped devices. - * CACHED: The optional cached mode for the target CPU type. - */ - -#define MIPS_CCA_UC 0x02 /* Uncached. */ -#define MIPS_CCA_C 0x03 /* Cacheable, coherency unspecified. */ - -#if defined(CPU_R4000) || defined(CPU_R10000) -#define MIPS_CCA_CNC 0x03 -#define MIPS_CCA_CCE 0x04 -#define MIPS_CCA_CCEW 0x05 - -#ifdef CPU_R4000 -#define MIPS_CCA_CCUOW 0x06 -#endif - -#ifdef CPU_R10000 -#define MIPS_CCA_UA 0x07 -#endif - -#define MIPS_CCA_CACHED MIPS_CCA_CCEW -#endif /* defined(CPU_R4000) || defined(CPU_R10000) */ - -#if defined(CPU_SB1) -#define MIPS_CCA_CC 0x05 /* Cacheable Coherent. */ -#endif - -#if defined(CPU_MIPS74K) -#define MIPS_CCA_UNCACHED 0x02 -#define MIPS_CCA_CACHED 0x03 -#endif - -/* - * 1004K and 1074K cores, as well as interAptiv and proAptiv cores, support - * Cacheable Coherent CCAs 0x04 and 0x05, as well as Cacheable non-Coherent - * CCA 0x03 and Uncached Accelerated CCA 0x07 - */ -#if defined(CPU_MIPS1004K) || defined(CPU_MIPS1074K) || \ - defined(CPU_INTERAPTIV) || defined(CPU_PROAPTIV) -#define MIPS_CCA_CNC 0x03 -#define MIPS_CCA_CCE 0x04 -#define MIPS_CCA_CCS 0x05 -#define MIPS_CCA_UA 0x07 - -/* We use shared read CCA for CACHED CCA */ -#define MIPS_CCA_CACHED MIPS_CCA_CCS -#endif - -#if defined(CPU_XBURST) -#define MIPS_CCA_UA 0x01 -#define MIPS_CCA_WC MIPS_CCA_UA -#endif - -#ifndef MIPS_CCA_UNCACHED -#define MIPS_CCA_UNCACHED MIPS_CCA_UC -#endif - -/* - * If we don't know which cached mode to use and there is a cache coherent - * mode, use it. If there is not a cache coherent mode, use the required - * cacheable mode. - */ -#ifndef MIPS_CCA_CACHED -#ifdef MIPS_CCA_CC -#define MIPS_CCA_CACHED MIPS_CCA_CC -#else -#define MIPS_CCA_CACHED MIPS_CCA_C -#endif -#endif #define MIPS_PHYS_TO_XKPHYS(cca,x) \ ((0x2ULL << 62) | ((unsigned long long)(cca) << 59) | (x)) Modified: head/sys/mips/include/vm.h ============================================================================== --- head/sys/mips/include/vm.h Wed Aug 1 18:28:17 2018 (r337047) +++ head/sys/mips/include/vm.h Wed Aug 1 18:35:17 2018 (r337048) @@ -32,6 +32,7 @@ #define _MACHINE_VM_H_ #include +#include /* Memory attributes. */ #define VM_MEMATTR_UNCACHEABLE ((vm_memattr_t)MIPS_CCA_UNCACHED) From owner-svn-src-all@freebsd.org Wed Aug 1 18:41:44 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C12F21065FBD; Wed, 1 Aug 2018 18:41:44 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7737781DA8; Wed, 1 Aug 2018 18:41:44 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5834D26581; Wed, 1 Aug 2018 18:41:44 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71IfiUu018848; Wed, 1 Aug 2018 18:41:44 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71IfiO8018847; Wed, 1 Aug 2018 18:41:44 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808011841.w71IfiO8018847@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 1 Aug 2018 18:41:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337049 - head/usr.sbin/newsyslog/tests X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/usr.sbin/newsyslog/tests X-SVN-Commit-Revision: 337049 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:41:44 -0000 Author: markj Date: Wed Aug 1 18:41:43 2018 New Revision: 337049 URL: https://svnweb.freebsd.org/changeset/base/337049 Log: Adjust the number of tests after r336913. Modified: head/usr.sbin/newsyslog/tests/legacy_test.sh Modified: head/usr.sbin/newsyslog/tests/legacy_test.sh ============================================================================== --- head/usr.sbin/newsyslog/tests/legacy_test.sh Wed Aug 1 18:35:17 2018 (r337048) +++ head/usr.sbin/newsyslog/tests/legacy_test.sh Wed Aug 1 18:41:43 2018 (r337049) @@ -451,7 +451,7 @@ tests_rfc5424() { tmpdir_clean } -echo 1..128 +echo 1..180 mkdir -p ${TMPDIR} cd ${TMPDIR} From owner-svn-src-all@freebsd.org Wed Aug 1 18:45:34 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B50DD10660D8; Wed, 1 Aug 2018 18:45:34 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 68E2782131; Wed, 1 Aug 2018 18:45:34 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 302AB265C6; Wed, 1 Aug 2018 18:45:34 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71IjYU0022244; Wed, 1 Aug 2018 18:45:34 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71IjYvM022243; Wed, 1 Aug 2018 18:45:34 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808011845.w71IjYvM022243@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 1 Aug 2018 18:45:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337050 - head/usr.sbin/newsyslog X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/usr.sbin/newsyslog X-SVN-Commit-Revision: 337050 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:45:35 -0000 Author: markj Date: Wed Aug 1 18:45:33 2018 New Revision: 337050 URL: https://svnweb.freebsd.org/changeset/base/337050 Log: Don't hard-code the number of compression utility arguments. The zstd invocation constructed by newsyslog contains one more parameter than invocations for the other supported compression utilities. However, the maximum number of arguments was hard-coded, leading to an out-of-bounds array access when using zstd compression. Modified: head/usr.sbin/newsyslog/newsyslog.c Modified: head/usr.sbin/newsyslog/newsyslog.c ============================================================================== --- head/usr.sbin/newsyslog/newsyslog.c Wed Aug 1 18:41:43 2018 (r337049) +++ head/usr.sbin/newsyslog/newsyslog.c Wed Aug 1 18:45:33 2018 (r337050) @@ -153,23 +153,24 @@ struct compress_types { const char *suffix; /* Compression suffix */ const char *path; /* Path to compression program */ char **args; /* Compression program arguments */ + int nargs; /* Program argument count */ }; static char f_arg[] = "-f"; static char q_arg[] = "-q"; static char rm_arg[] = "--rm"; -static char *gz_args[] ={ NULL, f_arg, NULL, NULL }; -#define bzip2_args gz_args +static char *gz_args[] = { NULL, f_arg, NULL, NULL }; +#define bz2_args gz_args #define xz_args gz_args static char *zstd_args[] = { NULL, q_arg, rm_arg, NULL, NULL }; -#define ARGS_NUM 4 +#define ARGS_NUM 5 static const struct compress_types compress_type[COMPRESS_TYPES] = { - { "", "", "", NULL}, /* none */ - { "Z", COMPRESS_SUFFIX_GZ, _PATH_GZIP, gz_args}, /* gzip */ - { "J", COMPRESS_SUFFIX_BZ2, _PATH_BZIP2, bzip2_args}, /* bzip2 */ - { "X", COMPRESS_SUFFIX_XZ, _PATH_XZ, xz_args }, /* xz */ - { "Y", COMPRESS_SUFFIX_ZST, _PATH_ZSTD, zstd_args } /* zst */ + { "", "", "", NULL, 0}, + { "Z", COMPRESS_SUFFIX_GZ, _PATH_GZIP, gz_args, nitems(gz_args) }, + { "J", COMPRESS_SUFFIX_BZ2, _PATH_BZIP2, bz2_args, nitems(bz2_args) }, + { "X", COMPRESS_SUFFIX_XZ, _PATH_XZ, xz_args, nitems(xz_args) }, + { "Y", COMPRESS_SUFFIX_ZST, _PATH_ZSTD, zstd_args, nitems(zstd_args) } }; struct conf_entry { @@ -2027,23 +2028,24 @@ do_zipwork(struct zipwork_entry *zwork) char zresult[MAXPATHLEN]; char command[BUFSIZ]; char **args; - int c, i; + int c, i, nargs; assert(zwork != NULL); pgm_path = NULL; strlcpy(zresult, zwork->zw_fname, sizeof(zresult)); - args = calloc(ARGS_NUM, sizeof(*args)); - if (args == NULL) - err(1, "calloc()"); if (zwork->zw_conf != NULL && zwork->zw_conf->compress > COMPRESS_NONE) for (c = 1; c < COMPRESS_TYPES; c++) { if (zwork->zw_conf->compress == c) { + nargs = compress_type[c].nargs; + args = calloc(nargs, sizeof(*args)); + if (args == NULL) + err(1, "calloc()"); pgm_path = compress_type[c].path; (void) strlcat(zresult, compress_type[c].suffix, sizeof(zresult)); /* the first argument is always NULL, skip it */ - for (i = 1; i < ARGS_NUM; i++) { + for (i = 1; i < nargs; i++) { if (compress_type[c].args[i] == NULL) break; args[i] = compress_type[c].args[i]; From owner-svn-src-all@freebsd.org Wed Aug 1 18:45:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 463CF10660FE; Wed, 1 Aug 2018 18:45:55 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EECC782239; Wed, 1 Aug 2018 18:45:54 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B57E7265C7; Wed, 1 Aug 2018 18:45:54 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71Ijsrl022318; Wed, 1 Aug 2018 18:45:54 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71Ijp9c022303; Wed, 1 Aug 2018 18:45:51 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808011845.w71Ijp9c022303@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Wed, 1 Aug 2018 18:45:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337051 - in head/sys: amd64/amd64 arm/arm arm64/arm64 i386/i386 i386/include mips/mips powerpc/powerpc riscv/riscv sparc64/sparc64 vm X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys: amd64/amd64 arm/arm arm64/arm64 i386/i386 i386/include mips/mips powerpc/powerpc riscv/riscv sparc64/sparc64 vm X-SVN-Commit-Revision: 337051 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:45:55 -0000 Author: kib Date: Wed Aug 1 18:45:51 2018 New Revision: 337051 URL: https://svnweb.freebsd.org/changeset/base/337051 Log: Add pmap_is_valid_memattr(9). Discussed with: alc Sponsored by: The FreeBSD Foundation, Mellanox Technologies MFC after: 1 week Differential revision: https://reviews.freebsd.org/D15583 Modified: head/sys/amd64/amd64/pmap.c head/sys/arm/arm/pmap-v4.c head/sys/arm/arm/pmap-v6.c head/sys/arm64/arm64/pmap.c head/sys/i386/i386/copyout.c head/sys/i386/i386/pmap.c head/sys/i386/i386/vm_machdep.c head/sys/i386/include/pmap.h head/sys/mips/mips/pmap.c head/sys/powerpc/powerpc/pmap_dispatch.c head/sys/riscv/riscv/pmap.c head/sys/sparc64/sparc64/pmap.c head/sys/vm/pmap.h Modified: head/sys/amd64/amd64/pmap.c ============================================================================== --- head/sys/amd64/amd64/pmap.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/amd64/amd64/pmap.c Wed Aug 1 18:45:51 2018 (r337051) @@ -1489,6 +1489,14 @@ pmap_swap_pat(pmap_t pmap, pt_entry_t entry) return (entry); } +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + return (mode >= 0 && mode < PAT_INDEX_SIZE && + pat_index[(int)mode] >= 0); +} + /* * Determine the appropriate bits to set in a PTE or PDE for a specified * caching mode. @@ -1498,7 +1506,7 @@ pmap_cache_bits(pmap_t pmap, int mode, boolean_t is_pd { int cache_bits, pat_flag, pat_idx; - if (mode < 0 || mode >= PAT_INDEX_SIZE || pat_index[mode] < 0) + if (!pmap_is_valid_memattr(pmap, mode)) panic("Unknown caching mode %d\n", mode); switch (pmap->pm_type) { Modified: head/sys/arm/arm/pmap-v4.c ============================================================================== --- head/sys/arm/arm/pmap-v4.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/arm/arm/pmap-v4.c Wed Aug 1 18:45:51 2018 (r337051) @@ -4859,4 +4859,9 @@ pmap_page_set_memattr(vm_page_t m, vm_memattr_t ma) panic("Can't change memattr on page with existing mappings"); } +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + return (mode == VM_MEMATTR_DEFAULT || mode == VM_MEMATTR_UNCACHEABLE); +} Modified: head/sys/arm/arm/pmap-v6.c ============================================================================== --- head/sys/arm/arm/pmap-v6.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/arm/arm/pmap-v6.c Wed Aug 1 18:45:51 2018 (r337051) @@ -393,12 +393,21 @@ CTASSERT(VM_MEMATTR_NOCACHE == 1); CTASSERT(VM_MEMATTR_DEVICE == 2); CTASSERT(VM_MEMATTR_SO == 3); CTASSERT(VM_MEMATTR_WRITE_THROUGH == 4); +#define VM_MEMATTR_END (VM_MEMATTR_WRITE_THROUGH + 1) +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + return (mode >= 0 && mode < VM_MEMATTR_END); +} + static inline uint32_t vm_memattr_to_pte2(vm_memattr_t ma) { - KASSERT((u_int)ma < 5, ("%s: bad vm_memattr_t %d", __func__, ma)); + KASSERT((u_int)ma < VM_MEMATTR_END, + ("%s: bad vm_memattr_t %d", __func__, ma)); return (pte2_attr_tab[(u_int)ma]); } Modified: head/sys/arm64/arm64/pmap.c ============================================================================== --- head/sys/arm64/arm64/pmap.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/arm64/arm64/pmap.c Wed Aug 1 18:45:51 2018 (r337051) @@ -5361,3 +5361,10 @@ pmap_unmap_io_transient(vm_page_t page[], vm_offset_t } } } + +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + return (mode >= VM_MEMATTR_DEVICE && mode <= VM_MEMATTR_WRITE_THROUGH); +} Modified: head/sys/i386/i386/copyout.c ============================================================================== --- head/sys/i386/i386/copyout.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/i386/i386/copyout.c Wed Aug 1 18:45:51 2018 (r337051) @@ -130,7 +130,8 @@ cp_slow0(vm_offset_t uva, size_t len, bool write, } for (i = 0, pte = vtopte(kaddr); i < plen; i++, pte++) { *pte = PG_V | PG_RW | PG_A | PG_M | VM_PAGE_TO_PHYS(m[i]) | - pmap_cache_bits(pmap_page_get_memattr(m[i]), FALSE); + pmap_cache_bits(kernel_pmap, pmap_page_get_memattr(m[i]), + FALSE); invlpg(kaddr + ptoa(i)); } kaddr += uva - trunc_page(uva); Modified: head/sys/i386/i386/pmap.c ============================================================================== --- head/sys/i386/i386/pmap.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/i386/i386/pmap.c Wed Aug 1 18:45:51 2018 (r337051) @@ -1059,16 +1059,24 @@ SYSCTL_ULONG(_vm_pmap_pde, OID_AUTO, promotions, CTLFL * Low level helper routines..... ***************************************************/ +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + return (mode >= 0 && mode < PAT_INDEX_SIZE && + pat_index[(int)mode] >= 0); +} + /* * Determine the appropriate bits to set in a PTE or PDE for a specified * caching mode. */ int -pmap_cache_bits(int mode, boolean_t is_pde) +pmap_cache_bits(pmap_t pmap, int mode, boolean_t is_pde) { int cache_bits, pat_flag, pat_idx; - if (mode < 0 || mode >= PAT_INDEX_SIZE || pat_index[mode] < 0) + if (!pmap_is_valid_memattr(pmap, mode)) panic("Unknown caching mode %d\n", mode); /* The PAT bit is different for PTE's and PDE's. */ @@ -1744,7 +1752,8 @@ pmap_kenter_attr(vm_offset_t va, vm_paddr_t pa, int mo pt_entry_t *pte; pte = vtopte(va); - pte_store(pte, pa | PG_RW | PG_V | pmap_cache_bits(mode, 0)); + pte_store(pte, pa | PG_RW | PG_V | pmap_cache_bits(kernel_pmap, + mode, 0)); } /* @@ -1839,7 +1848,8 @@ pmap_qenter(vm_offset_t sva, vm_page_t *ma, int count) endpte = pte + count; while (pte < endpte) { m = *ma++; - pa = VM_PAGE_TO_PHYS(m) | pmap_cache_bits(m->md.pat_mode, 0); + pa = VM_PAGE_TO_PHYS(m) | pmap_cache_bits(kernel_pmap, + m->md.pat_mode, 0); if ((*pte & (PG_FRAME | PG_PTE_CACHE)) != pa) { oldpte |= *pte; #if defined(PAE) || defined(PAE_TABLES) @@ -3654,7 +3664,7 @@ pmap_enter(pmap_t pmap, vm_offset_t va, vm_page_t m, v newpte |= PG_W; if (pmap != kernel_pmap) newpte |= PG_U; - newpte |= pmap_cache_bits(m->md.pat_mode, psind > 0); + newpte |= pmap_cache_bits(pmap, m->md.pat_mode, psind > 0); if ((m->oflags & VPO_UNMANAGED) == 0) newpte |= PG_MANAGED; @@ -3877,7 +3887,7 @@ pmap_enter_4mpage(pmap_t pmap, vm_offset_t va, vm_page pd_entry_t newpde; PMAP_LOCK_ASSERT(pmap, MA_OWNED); - newpde = VM_PAGE_TO_PHYS(m) | pmap_cache_bits(m->md.pat_mode, 1) | + newpde = VM_PAGE_TO_PHYS(m) | pmap_cache_bits(pmap, m->md.pat_mode, 1) | PG_PS | PG_V; if ((m->oflags & VPO_UNMANAGED) == 0) newpde |= PG_MANAGED; @@ -4135,7 +4145,7 @@ pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va, v */ pmap->pm_stats.resident_count++; - pa = VM_PAGE_TO_PHYS(m) | pmap_cache_bits(m->md.pat_mode, 0); + pa = VM_PAGE_TO_PHYS(m) | pmap_cache_bits(pmap, m->md.pat_mode, 0); #if defined(PAE) || defined(PAE_TABLES) if ((prot & VM_PROT_EXECUTE) == 0) pa |= pg_nx; @@ -4223,8 +4233,8 @@ pmap_object_init_pt(pmap_t pmap, vm_offset_t addr, vm_ * "pa" will not affect the termination of this loop. */ PMAP_LOCK(pmap); - for (pa = ptepa | pmap_cache_bits(pat_mode, 1); pa < ptepa + - size; pa += NBPDR) { + for (pa = ptepa | pmap_cache_bits(pmap, pat_mode, 1); + pa < ptepa + size; pa += NBPDR) { pde = pmap_pde(pmap, addr); if (*pde == 0) { pde_store(pde, pa | PG_PS | PG_M | PG_A | @@ -4489,7 +4499,7 @@ pmap_zero_page(vm_page_t m) if (*cmap_pte2) panic("pmap_zero_page: CMAP2 busy"); *cmap_pte2 = PG_V | PG_RW | VM_PAGE_TO_PHYS(m) | PG_A | PG_M | - pmap_cache_bits(m->md.pat_mode, 0); + pmap_cache_bits(kernel_pmap, m->md.pat_mode, 0); invlcaddr(pc->pc_cmap_addr2); pagezero(pc->pc_cmap_addr2); *cmap_pte2 = 0; @@ -4520,7 +4530,7 @@ pmap_zero_page_area(vm_page_t m, int off, int size) if (*cmap_pte2) panic("pmap_zero_page_area: CMAP2 busy"); *cmap_pte2 = PG_V | PG_RW | VM_PAGE_TO_PHYS(m) | PG_A | PG_M | - pmap_cache_bits(m->md.pat_mode, 0); + pmap_cache_bits(kernel_pmap, m->md.pat_mode, 0); invlcaddr(pc->pc_cmap_addr2); if (off == 0 && size == PAGE_SIZE) pagezero(pc->pc_cmap_addr2); @@ -4550,10 +4560,10 @@ pmap_copy_page(vm_page_t src, vm_page_t dst) if (*cmap_pte2) panic("pmap_copy_page: CMAP2 busy"); *cmap_pte1 = PG_V | VM_PAGE_TO_PHYS(src) | PG_A | - pmap_cache_bits(src->md.pat_mode, 0); + pmap_cache_bits(kernel_pmap, src->md.pat_mode, 0); invlcaddr(pc->pc_cmap_addr1); *cmap_pte2 = PG_V | PG_RW | VM_PAGE_TO_PHYS(dst) | PG_A | PG_M | - pmap_cache_bits(dst->md.pat_mode, 0); + pmap_cache_bits(kernel_pmap, dst->md.pat_mode, 0); invlcaddr(pc->pc_cmap_addr2); bcopy(pc->pc_cmap_addr1, pc->pc_cmap_addr2, PAGE_SIZE); *cmap_pte1 = 0; @@ -4592,10 +4602,10 @@ pmap_copy_pages(vm_page_t ma[], vm_offset_t a_offset, b_pg_offset = b_offset & PAGE_MASK; cnt = min(cnt, PAGE_SIZE - b_pg_offset); *cmap_pte1 = PG_V | VM_PAGE_TO_PHYS(a_pg) | PG_A | - pmap_cache_bits(a_pg->md.pat_mode, 0); + pmap_cache_bits(kernel_pmap, a_pg->md.pat_mode, 0); invlcaddr(pc->pc_cmap_addr1); *cmap_pte2 = PG_V | PG_RW | VM_PAGE_TO_PHYS(b_pg) | PG_A | - PG_M | pmap_cache_bits(b_pg->md.pat_mode, 0); + PG_M | pmap_cache_bits(kernel_pmap, b_pg->md.pat_mode, 0); invlcaddr(pc->pc_cmap_addr2); a_cp = pc->pc_cmap_addr1 + a_pg_offset; b_cp = pc->pc_cmap_addr2 + b_pg_offset; @@ -5567,7 +5577,8 @@ pmap_flush_page(vm_page_t m) if (*cmap_pte2) panic("pmap_flush_page: CMAP2 busy"); *cmap_pte2 = PG_V | PG_RW | VM_PAGE_TO_PHYS(m) | - PG_A | PG_M | pmap_cache_bits(m->md.pat_mode, 0); + PG_A | PG_M | pmap_cache_bits(kernel_pmap, m->md.pat_mode, + 0); invlcaddr(pc->pc_cmap_addr2); sva = (vm_offset_t)pc->pc_cmap_addr2; eva = sva + PAGE_SIZE; @@ -5628,8 +5639,8 @@ pmap_change_attr(vm_offset_t va, vm_size_t size, int m if (base < VM_MIN_KERNEL_ADDRESS) return (EINVAL); - cache_bits_pde = pmap_cache_bits(mode, 1); - cache_bits_pte = pmap_cache_bits(mode, 0); + cache_bits_pde = pmap_cache_bits(kernel_pmap, mode, 1); + cache_bits_pte = pmap_cache_bits(kernel_pmap, mode, 0); changed = FALSE; /* @@ -5835,7 +5846,7 @@ pmap_quick_enter_page(vm_page_t m) KASSERT(*pte == 0, ("pmap_quick_enter_page: PTE busy")); *pte = PG_V | PG_RW | VM_PAGE_TO_PHYS(m) | PG_A | PG_M | - pmap_cache_bits(pmap_page_get_memattr(m), 0); + pmap_cache_bits(kernel_pmap, pmap_page_get_memattr(m), 0); invlpg(qaddr); return (qaddr); @@ -5886,7 +5897,7 @@ pmap_trm_import(void *unused __unused, vmem_size_t siz VM_ALLOC_NORMAL | VM_ALLOC_WIRED | VM_ALLOC_WAITOK); pte_store(&trm_pte[atop(af - prev_addr)], VM_PAGE_TO_PHYS(m) | PG_M | PG_A | PG_RW | PG_V | pgeflag | - pmap_cache_bits(VM_MEMATTR_DEFAULT, FALSE)); + pmap_cache_bits(kernel_pmap, VM_MEMATTR_DEFAULT, FALSE)); } *addrp = prev_addr; return (0); @@ -5907,7 +5918,7 @@ void pmap_init_trm(void) if ((pd_m->flags & PG_ZERO) == 0) pmap_zero_page(pd_m); PTD[TRPTDI] = VM_PAGE_TO_PHYS(pd_m) | PG_M | PG_A | PG_RW | PG_V | - pmap_cache_bits(VM_MEMATTR_DEFAULT, TRUE); + pmap_cache_bits(kernel_pmap, VM_MEMATTR_DEFAULT, TRUE); } void * Modified: head/sys/i386/i386/vm_machdep.c ============================================================================== --- head/sys/i386/i386/vm_machdep.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/i386/i386/vm_machdep.c Wed Aug 1 18:45:51 2018 (r337051) @@ -584,7 +584,7 @@ sf_buf_map(struct sf_buf *sf, int flags) ptep = vtopte(sf->kva); opte = *ptep; *ptep = VM_PAGE_TO_PHYS(sf->m) | PG_RW | PG_V | - pmap_cache_bits(sf->m->md.pat_mode, 0); + pmap_cache_bits(kernel_pmap, sf->m->md.pat_mode, 0); /* * Avoid unnecessary TLB invalidations: If the sf_buf's old Modified: head/sys/i386/include/pmap.h ============================================================================== --- head/sys/i386/include/pmap.h Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/i386/include/pmap.h Wed Aug 1 18:45:51 2018 (r337051) @@ -374,7 +374,7 @@ extern vm_offset_t virtual_end; * vtopte(). */ void pmap_bootstrap(vm_paddr_t); -int pmap_cache_bits(int mode, boolean_t is_pde); +int pmap_cache_bits(pmap_t, int mode, boolean_t is_pde); int pmap_change_attr(vm_offset_t, vm_size_t, int); void pmap_init_pat(void); void pmap_kenter(vm_offset_t va, vm_paddr_t pa); Modified: head/sys/mips/mips/pmap.c ============================================================================== --- head/sys/mips/mips/pmap.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/mips/mips/pmap.c Wed Aug 1 18:45:51 2018 (r337051) @@ -3645,3 +3645,19 @@ pmap_change_attr(vm_offset_t sva, vm_size_t size, vm_m mips_dcache_wbinv_range(ova, size); return 0; } + +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + switch (mode) { + case VM_MEMATTR_UNCACHEABLE: + case VM_MEMATTR_WRITE_BACK: +#ifdef MIPS_CCA_WC + case VM_MEMATTR_WRITE_COMBINING: +#endif + return (TRUE); + default: + return (FALSE); + } +} Modified: head/sys/powerpc/powerpc/pmap_dispatch.c ============================================================================== --- head/sys/powerpc/powerpc/pmap_dispatch.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/powerpc/powerpc/pmap_dispatch.c Wed Aug 1 18:45:51 2018 (r337051) @@ -620,3 +620,21 @@ pmap_mmu_install(char *name, int prio) } int unmapped_buf_allowed; + +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + switch (mode) { + case VM_MEMATTR_DEFAULT: + case VM_MEMATTR_UNCACHEABLE: + case VM_MEMATTR_CACHEABLE: + case VM_MEMATTR_WRITE_COMBINING: + case VM_MEMATTR_WRITE_BACK: + case VM_MEMATTR_WRITE_THROUGH: + case VM_MEMATTR_PREFETCHABLE: + return (TRUE); + default: + return (FALSE); + } +} Modified: head/sys/riscv/riscv/pmap.c ============================================================================== --- head/sys/riscv/riscv/pmap.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/riscv/riscv/pmap.c Wed Aug 1 18:45:51 2018 (r337051) @@ -3336,3 +3336,10 @@ pmap_unmap_io_transient(vm_page_t page[], vm_offset_t } } } + +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + return (mode >= VM_MEMATTR_DEVICE && mode <= VM_MEMATTR_WRITE_BACK); +} Modified: head/sys/sparc64/sparc64/pmap.c ============================================================================== --- head/sys/sparc64/sparc64/pmap.c Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/sparc64/sparc64/pmap.c Wed Aug 1 18:45:51 2018 (r337051) @@ -2318,3 +2318,10 @@ pmap_align_superpage(vm_object_t object, vm_ooffset_t { } + +boolean_t +pmap_is_valid_memattr(pmap_t pmap __unused, vm_memattr_t mode) +{ + + return (mode == VM_MEMATTR_DEFAULT); +} Modified: head/sys/vm/pmap.h ============================================================================== --- head/sys/vm/pmap.h Wed Aug 1 18:45:33 2018 (r337050) +++ head/sys/vm/pmap.h Wed Aug 1 18:45:51 2018 (r337051) @@ -142,6 +142,7 @@ void pmap_init(void); boolean_t pmap_is_modified(vm_page_t m); boolean_t pmap_is_prefaultable(pmap_t pmap, vm_offset_t va); boolean_t pmap_is_referenced(vm_page_t m); +boolean_t pmap_is_valid_memattr(pmap_t, vm_memattr_t); vm_offset_t pmap_map(vm_offset_t *, vm_paddr_t, vm_paddr_t, int); int pmap_mincore(pmap_t pmap, vm_offset_t addr, vm_paddr_t *locked_pa); From owner-svn-src-all@freebsd.org Wed Aug 1 18:49:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE21510661E8; Wed, 1 Aug 2018 18:49:40 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97D3F8242B; Wed, 1 Aug 2018 18:49:40 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 75792265CA; Wed, 1 Aug 2018 18:49:40 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71IneGH022526; Wed, 1 Aug 2018 18:49:40 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71Ineog022525; Wed, 1 Aug 2018 18:49:40 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808011849.w71Ineog022525@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 1 Aug 2018 18:49:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337052 - head/usr.sbin/newsyslog X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/usr.sbin/newsyslog X-SVN-Commit-Revision: 337052 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:49:41 -0000 Author: markj Date: Wed Aug 1 18:49:40 2018 New Revision: 337052 URL: https://svnweb.freebsd.org/changeset/base/337052 Log: Remove ARGS_NUM, accidentally left over after r337050. Modified: head/usr.sbin/newsyslog/newsyslog.c Modified: head/usr.sbin/newsyslog/newsyslog.c ============================================================================== --- head/usr.sbin/newsyslog/newsyslog.c Wed Aug 1 18:45:51 2018 (r337051) +++ head/usr.sbin/newsyslog/newsyslog.c Wed Aug 1 18:49:40 2018 (r337052) @@ -164,7 +164,6 @@ static char *gz_args[] = { NULL, f_arg, NULL, NULL }; #define xz_args gz_args static char *zstd_args[] = { NULL, q_arg, rm_arg, NULL, NULL }; -#define ARGS_NUM 5 static const struct compress_types compress_type[COMPRESS_TYPES] = { { "", "", "", NULL, 0}, { "Z", COMPRESS_SUFFIX_GZ, _PATH_GZIP, gz_args, nitems(gz_args) }, From owner-svn-src-all@freebsd.org Wed Aug 1 18:58:26 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0A671066480; Wed, 1 Aug 2018 18:58:26 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 556BF82929; Wed, 1 Aug 2018 18:58:26 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 325A32675F; Wed, 1 Aug 2018 18:58:26 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71IwQol027470; Wed, 1 Aug 2018 18:58:26 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71IwPtm027465; Wed, 1 Aug 2018 18:58:25 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808011858.w71IwPtm027465@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Wed, 1 Aug 2018 18:58:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337053 - in head: share/man/man4 sys/compat/freebsd32 sys/dev/pci sys/sys X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head: share/man/man4 sys/compat/freebsd32 sys/dev/pci sys/sys X-SVN-Commit-Revision: 337053 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 18:58:26 -0000 Author: kib Date: Wed Aug 1 18:58:24 2018 New Revision: 337053 URL: https://svnweb.freebsd.org/changeset/base/337053 Log: Add ioctl to conveniently mmap a PCI device BAR into userspace. Add the ioctl PCIOCBARMMAP on /dev/pci to conveniently create userspace mapping of a PCI device BAR. This is enormously superior to read the BAR value with PCIOCREAD and then try to mmap /dev/mem, and should allow to automatically activate the mapped BARs when needed in future. Current implementation creates new sg pager for each user mmap request. If the pointer (and reference) to a managed device pager is stored in pci_map, we would be able to revoke all mappings on the BAR deactivation or relocation. This is related to the unimplemented BAR activation on mmap, and is postponed for the future. Discussed with: imp, jhb Sponsored by: The FreeBSD Foundation, Mellanox Technologies MFC after: 2 weeks Differential revision: https://reviews.freebsd.org/D15583 Modified: head/share/man/man4/pci.4 head/sys/compat/freebsd32/freebsd32_ioctl.c head/sys/compat/freebsd32/freebsd32_ioctl.h head/sys/dev/pci/pci_user.c head/sys/sys/pciio.h Modified: head/share/man/man4/pci.4 ============================================================================== --- head/share/man/man4/pci.4 Wed Aug 1 18:49:40 2018 (r337052) +++ head/share/man/man4/pci.4 Wed Aug 1 18:58:24 2018 (r337053) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd December 20, 2017 +.Dd June 14, 2018 .Dt PCI 4 .Os .Sh NAME @@ -333,6 +333,72 @@ The limitations on data width described for reading registers, above, also apply to writing .Tn PCI configuration registers. +.It PCIOCBARMMAP +This +.Xr ioctl 2 +command allows userspace processes to +.Xr mmap 2 +the memory-mapped PCI BAR into its address space. +The input parameters and results are passed in the +.Va pci_bar_mmap +structure, which has the following fields: +.Bl -tag -width Vt struct pcise pbm_sel +.It Vt uint64_t pbm_map_base +Reports the established mapping base to the caller. +If +.Va PCIIO_BAR_MMAP_FIXED +flag was specified, then this field must be filled before the call +with the desired address for the mapping. +.It Vt uint64_t pbm_map_length +Reports the mapped length of the BAR, in bytes. +Its .Vt uint64_t value is always multiple of machine pages. +.It Vt int64_t pbm_bar_length +Reports length of the bar as exposed by the device. +.It Vt int pbm_bar_off +Reports offset from the mapped base to the start of the +first register in the bar. +.It Vt struct pcisel pbm_sel +Should be filled before the call. +Describes the device to operate on. +.It Vt int pbm_reg +The BAR index to mmap. +.It Vt int pbm_flags +Flags which augments the operation. +See below. +.It Vt int pbm_memattr +The caching attribute for the mapping. +Typical values are +.Dv VM_MEMATTR_UNCACHEABLE +for control registers BARs, and +.Dv VM_MEMATTR_WRITE_COMBINING +for frame buffers. +Regular memory-like BAR should be mapped with +.Dv VM_MEMATTR_DEFAULT +attribute. +.El +.Pp +Currently defined flags are: +.Bl -tag -width PCIIO_BAR_MMAP_ACTIVATE +.It PCIIO_BAR_MMAP_FIXED +The resulted mappings should be established at the address +specified by the +.Va pbm_map_base +member, otherwise fail. +.It PCIIO_BAR_MMAP_EXCL +Must be used together with +.Vd PCIIO_BAR_MMAP_FIXED +If the specified base contains already established mappings, the +operation fails instead of implicitly unmapping them. +.It PCIIO_BAR_MMAP_RW +The requested mapping allows both reading and writing. +Without the flag, read-only mapping is established. +Note that it is common for the device registers to have side-effects +even on reads. +.It PCIIO_BAR_MMAP_ACTIVATE +(Unimplemented) If the BAR is not activated, activate it in the course +of mapping. +Currently attempt to mmap an inactive BAR results in error. +.El .El .Sh LOADER TUNABLES Tunables can be set at the Modified: head/sys/compat/freebsd32/freebsd32_ioctl.c ============================================================================== --- head/sys/compat/freebsd32/freebsd32_ioctl.c Wed Aug 1 18:49:40 2018 (r337052) +++ head/sys/compat/freebsd32/freebsd32_ioctl.c Wed Aug 1 18:58:24 2018 (r337053) @@ -53,6 +53,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include CTASSERT(sizeof(struct ioc_read_toc_entry32) == 8); @@ -249,6 +250,40 @@ cleanup: } static int +freebsd32_ioctl_barmmap(struct thread *td, + struct freebsd32_ioctl_args *uap, struct file *fp) +{ + struct pci_bar_mmap32 pbm32; + struct pci_bar_mmap pbm; + int error; + + error = copyin(uap->data, &pbm32, sizeof(pbm32)); + if (error != 0) + return (error); + PTRIN_CP(pbm32, pbm, pbm_map_base); + CP(pbm32, pbm, pbm_sel); + CP(pbm32, pbm, pbm_reg); + CP(pbm32, pbm, pbm_flags); + CP(pbm32, pbm, pbm_memattr); + pbm.pbm_bar_length = PAIR32TO64(uint64_t, pbm32.pbm_bar_length); + error = fo_ioctl(fp, PCIOCBARMMAP, (caddr_t)&pbm, td->td_ucred, td); + if (error == 0) { + PTROUT_CP(pbm, pbm32, pbm_map_base); + CP(pbm, pbm32, pbm_map_length); +#if BYTE_ORDER == LITTLE_ENDIAN + pbm32.pbm_bar_length1 = pbm.pbm_bar_length; + pbm32.pbm_bar_length2 = pbm.pbm_bar_length >> 32; +#else + pbm32.pbm_bar_length1 = pbm.pbm_bar_length >> 32; + pbm32.pbm_bar_length2 = pbm.pbm_bar_length; +#endif + CP(pbm, pbm32, pbm_bar_off); + error = copyout(&pbm32, uap->data, sizeof(pbm32)); + } + return (error); +} + +static int freebsd32_ioctl_sg(struct thread *td, struct freebsd32_ioctl_args *uap, struct file *fp) { @@ -355,6 +390,10 @@ freebsd32_ioctl(struct thread *td, struct freebsd32_io error = freebsd32_ioctl_sg(td, uap, fp); break; + case PCIOCBARMMAP_32: + error = freebsd32_ioctl_barmmap(td, uap, fp); + break; + default: fdrop(fp, td); ap.fd = uap->fd; @@ -364,5 +403,5 @@ freebsd32_ioctl(struct thread *td, struct freebsd32_io } fdrop(fp, td); - return error; + return (error); } Modified: head/sys/compat/freebsd32/freebsd32_ioctl.h ============================================================================== --- head/sys/compat/freebsd32/freebsd32_ioctl.h Wed Aug 1 18:49:40 2018 (r337052) +++ head/sys/compat/freebsd32/freebsd32_ioctl.h Wed Aug 1 18:58:24 2018 (r337053) @@ -95,11 +95,23 @@ struct pci_conf_io32 { u_int32_t status; /* request status */ }; +struct pci_bar_mmap32 { + uint32_t pbm_map_base; + uint32_t pbm_map_length; + uint32_t pbm_bar_length1, pbm_bar_length2; + int pbm_bar_off; + struct pcisel pbm_sel; + int pbm_reg; + int pbm_flags; + int pbm_memattr; +}; + #define CDIOREADTOCENTRYS_32 _IOWR('c', 5, struct ioc_read_toc_entry32) #define FIODGNAME_32 _IOW('f', 120, struct fiodgname_arg32) #define MEMRANGE_GET32 _IOWR('m', 50, struct mem_range_op32) #define MEMRANGE_SET32 _IOW('m', 51, struct mem_range_op32) #define PCIOCGETCONF_32 _IOWR('p', 5, struct pci_conf_io32) #define SG_IO_32 _IOWR(SGIOC, 0x85, struct sg_io_hdr32) +#define PCIOCBARMMAP_32 _IOWR('p', 8, struct pci_bar_mmap32) #endif /* _COMPAT_FREEBSD32_IOCTL_H_ */ Modified: head/sys/dev/pci/pci_user.c ============================================================================== --- head/sys/dev/pci/pci_user.c Wed Aug 1 18:49:40 2018 (r337052) +++ head/sys/dev/pci/pci_user.c Wed Aug 1 18:58:24 2018 (r337053) @@ -31,6 +31,7 @@ __FBSDID("$FreeBSD$"); #include "opt_bus.h" /* XXX trim includes */ +#include #include #include #include @@ -39,13 +40,19 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include -#include +#include +#include #include #include #include +#include +#include +#include +#include #include #include @@ -697,6 +704,77 @@ pci_conf_for_copyout(const struct pci_conf *pcp, union } static int +pci_bar_mmap(device_t pcidev, struct pci_bar_mmap *pbm) +{ + vm_map_t map; + vm_object_t obj; + struct thread *td; + struct sglist *sg; + struct pci_map *pm; + vm_paddr_t pbase; + vm_size_t plen; + vm_offset_t addr; + vm_prot_t prot; + int error, flags; + + td = curthread; + map = &td->td_proc->p_vmspace->vm_map; + if ((pbm->pbm_flags & ~(PCIIO_BAR_MMAP_FIXED | PCIIO_BAR_MMAP_EXCL | + PCIIO_BAR_MMAP_RW | PCIIO_BAR_MMAP_ACTIVATE)) != 0 || + pbm->pbm_memattr != (vm_memattr_t)pbm->pbm_memattr || + !pmap_is_valid_memattr(map->pmap, pbm->pbm_memattr)) + return (EINVAL); + + /* Fetch the BAR physical base and length. */ + pm = pci_find_bar(pcidev, pbm->pbm_reg); + if (pm == NULL) + return (EINVAL); + if (!pci_bar_enabled(pcidev, pm)) + return (EBUSY); /* XXXKIB enable if _ACTIVATE */ + if (!PCI_BAR_MEM(pm->pm_value)) + return (EIO); + pbase = trunc_page(pm->pm_value); + plen = round_page(pm->pm_value + ((pci_addr_t)1 << pm->pm_size)) - + pbase; + prot = VM_PROT_READ | (((pbm->pbm_flags & PCIIO_BAR_MMAP_RW) != 0) ? + VM_PROT_WRITE : 0); + + /* Create vm structures and mmap. */ + sg = sglist_alloc(1, M_WAITOK); + error = sglist_append_phys(sg, pbase, plen); + if (error != 0) + goto out; + obj = vm_pager_allocate(OBJT_SG, sg, plen, prot, 0, td->td_ucred); + if (obj == NULL) { + error = EIO; + goto out; + } + obj->memattr = pbm->pbm_memattr; + flags = MAP_SHARED; + addr = 0; + if ((pbm->pbm_flags & PCIIO_BAR_MMAP_FIXED) != 0) { + addr = (uintptr_t)pbm->pbm_map_base; + flags |= MAP_FIXED; + } + if ((pbm->pbm_flags & PCIIO_BAR_MMAP_EXCL) != 0) + flags |= MAP_CHECK_EXCL; + error = vm_mmap_object(map, &addr, plen, prot, prot, flags, obj, 0, + FALSE, td); + if (error != 0) { + vm_object_deallocate(obj); + goto out; + } + pbm->pbm_map_base = (void *)addr; + pbm->pbm_map_length = plen; + pbm->pbm_bar_off = pm->pm_value - pbase; + pbm->pbm_bar_length = (pci_addr_t)1 << pm->pm_size; + +out: + sglist_free(sg); + return (error); +} + +static int pci_ioctl(struct cdev *dev, u_long cmd, caddr_t data, int flag, struct thread *td) { device_t pcidev; @@ -709,6 +787,7 @@ pci_ioctl(struct cdev *dev, u_long cmd, caddr_t data, struct pci_list_vpd_io *lvio; struct pci_match_conf *pattern_buf; struct pci_map *pm; + struct pci_bar_mmap *pbm; size_t confsz, iolen; int error, ionum, i, num_patterns; union pci_conf_union pcu; @@ -730,6 +809,7 @@ pci_ioctl(struct cdev *dev, u_long cmd, caddr_t data, #endif case PCIOCGETBAR: case PCIOCLISTVPD: + case PCIOCBARMMAP: break; default: return (EPERM); @@ -1053,6 +1133,18 @@ getconfexit: } error = pci_list_vpd(pcidev, lvio); break; + + case PCIOCBARMMAP: + pbm = (struct pci_bar_mmap *)data; + if ((flag & FWRITE) == 0 && + (pbm->pbm_flags & PCIIO_BAR_MMAP_RW) != 0) + return (EPERM); + pcidev = pci_find_dbsf(pbm->pbm_sel.pc_domain, + pbm->pbm_sel.pc_bus, pbm->pbm_sel.pc_dev, + pbm->pbm_sel.pc_func); + error = pcidev == NULL ? ENODEV : pci_bar_mmap(pcidev, pbm); + break; + default: error = ENOTTY; break; Modified: head/sys/sys/pciio.h ============================================================================== --- head/sys/sys/pciio.h Wed Aug 1 18:49:40 2018 (r337052) +++ head/sys/sys/pciio.h Wed Aug 1 18:58:24 2018 (r337053) @@ -138,11 +138,30 @@ struct pci_list_vpd_io { struct pci_vpd_element *plvi_data; }; +struct pci_bar_mmap { + void *pbm_map_base; /* (sometimes IN)/OUT mmaped base */ + size_t pbm_map_length; /* mapped length of the BAR, multiple + of pages */ + uint64_t pbm_bar_length; /* actual length of the BAR */ + int pbm_bar_off; /* offset from the mapped base to the + start of BAR */ + struct pcisel pbm_sel; /* device to operate on */ + int pbm_reg; /* starting address of BAR */ + int pbm_flags; + int pbm_memattr; +}; + +#define PCIIO_BAR_MMAP_FIXED 0x01 +#define PCIIO_BAR_MMAP_EXCL 0x02 +#define PCIIO_BAR_MMAP_RW 0x04 +#define PCIIO_BAR_MMAP_ACTIVATE 0x08 + #define PCIOCGETCONF _IOWR('p', 5, struct pci_conf_io) #define PCIOCREAD _IOWR('p', 2, struct pci_io) #define PCIOCWRITE _IOWR('p', 3, struct pci_io) #define PCIOCATTACHED _IOWR('p', 4, struct pci_io) #define PCIOCGETBAR _IOWR('p', 6, struct pci_bar_io) #define PCIOCLISTVPD _IOWR('p', 7, struct pci_list_vpd_io) +#define PCIOCBARMMAP _IOWR('p', 8, struct pci_bar_mmap) #endif /* !_SYS_PCIIO_H_ */ From owner-svn-src-all@freebsd.org Wed Aug 1 19:02:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA4CD1066720; Wed, 1 Aug 2018 19:02:06 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FA0D82E7D; Wed, 1 Aug 2018 19:02:06 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7218A26902; Wed, 1 Aug 2018 19:02:06 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71J26eE030137; Wed, 1 Aug 2018 19:02:06 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71J26PK030020; Wed, 1 Aug 2018 19:02:06 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808011902.w71J26PK030020@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Wed, 1 Aug 2018 19:02:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337054 - head/usr.sbin/pciconf X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/usr.sbin/pciconf X-SVN-Commit-Revision: 337054 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:02:07 -0000 Author: kib Date: Wed Aug 1 19:02:05 2018 New Revision: 337054 URL: https://svnweb.freebsd.org/changeset/base/337054 Log: Add -D option to pciconf(8) to mmap and dump content of the device BAR. Discussed with: imp, jhb Sponsored by: The FreeBSD Foundation, Mellanox Technologies MFC after: 2 weeks Differential revision: https://reviews.freebsd.org/D15583 Modified: head/usr.sbin/pciconf/pciconf.8 head/usr.sbin/pciconf/pciconf.c Modified: head/usr.sbin/pciconf/pciconf.8 ============================================================================== --- head/usr.sbin/pciconf/pciconf.8 Wed Aug 1 18:58:24 2018 (r337053) +++ head/usr.sbin/pciconf/pciconf.8 Wed Aug 1 19:02:05 2018 (r337054) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd November 23, 2015 +.Dd June 14, 2018 .Dt PCICONF 8 .Os .Sh NAME @@ -40,6 +40,8 @@ .Fl r Oo Fl b | h Oc Ar device addr Ns Op : Ns Ar addr2 .Nm .Fl w Oo Fl b | h Oc Ar device addr value +.Nm +.Fl D Oo Fl b | h | x Oc Ar device addr Op start Ns Op : Ns Ar count .Sh DESCRIPTION The .Nm @@ -305,17 +307,38 @@ into a configuration space register at byte offset .Ar addr of device .Ar selector . -For both operations, the flags -.Fl b +.Pp +The +.Fl D +option request a dump of the specified BAR. +Dump is performed to the standard output, raw register values +are written. +Use +.Xr hexdump 1 +to convert them to human-readable dump, +or redirect into a file to save the snapshot of the device state. +Optionally, the +.Ar start and -.Fl h +.Ar count +of the registers dumped can be specified, in multiple of the operation width, +see next paragraph. +.Pp +For read, write, and dump operations, the flags +.Fl b , +.Fl h , +and +.Fl x select the width of the operation; .Fl b indicates a byte operation, and .Fl h indicates a halfword (two-byte) operation. +.Fl x +indicates a quadword (four-byte) operation. The default is to read or write a longword (four bytes). +The quadword mode is only valid for BAR dump. .Sh ENVIRONMENT PCI vendor and device information is read from .Pa /usr/local/share/pciids/pci.ids . @@ -368,3 +391,11 @@ to provide the device with a driver KLD, and reading o registers may cause a failure in badly designed .Tn PCI chips. +.Pp +There is currently no way to specify the caching mode for the mapping +established by the +.Fl D +option, +.Nm +always uses uncached access. +This is fine for control register BARs. Modified: head/usr.sbin/pciconf/pciconf.c ============================================================================== --- head/usr.sbin/pciconf/pciconf.c Wed Aug 1 18:58:24 2018 (r337053) +++ head/usr.sbin/pciconf/pciconf.c Wed Aug 1 19:02:05 2018 (r337054) @@ -34,7 +34,14 @@ static const char rcsid[] = #include #include +#include +#include +#include +#include + +#include + #include #include #include @@ -44,11 +51,7 @@ static const char rcsid[] = #include #include #include -#include -#include -#include - #include "pathnames.h" #include "pciconf.h" @@ -82,32 +85,37 @@ static int load_vendors(void); static void readit(const char *, const char *, int); static void writeit(const char *, const char *, const char *, int); static void chkattached(const char *); +static void dump_bar(const char *name, const char *reg, const char *bar_start, + const char *bar_count, int width, int verbose); static int exitstatus = 0; static void usage(void) { - fprintf(stderr, "%s\n%s\n%s\n%s\n", - "usage: pciconf -l [-BbcevV] [device]", - " pciconf -a device", - " pciconf -r [-b | -h] device addr[:addr2]", - " pciconf -w [-b | -h] device addr value"); - exit (1); + + fprintf(stderr, "%s", + "usage: pciconf -l [-BbcevV] [device]\n" + " pciconf -a device\n" + " pciconf -r [-b | -h] device addr[:addr2]\n" + " pciconf -w [-b | -h] device addr value\n" + " pciconf -D [-b | -h | -x] device bar [start [count]]" + "\n"); + exit(1); } int main(int argc, char **argv) { - int c; - int listmode, readmode, writemode, attachedmode; + int c, width; + int listmode, readmode, writemode, attachedmode, dumpbarmode; int bars, bridge, caps, errors, verbose, vpd; - int byte, isshort; - listmode = readmode = writemode = attachedmode = 0; - bars = bridge = caps = errors = verbose = vpd = byte = isshort = 0; + listmode = readmode = writemode = attachedmode = dumpbarmode = 0; + bars = bridge = caps = errors = verbose = vpd= 0; + width = 4; - while ((c = getopt(argc, argv, "aBbcehlrwVv")) != -1) { + while ((c = getopt(argc, argv, "aBbcDehlrwVv")) != -1) { switch(c) { case 'a': attachedmode = 1; @@ -119,19 +127,23 @@ main(int argc, char **argv) case 'b': bars = 1; - byte = 1; + width = 1; break; case 'c': caps = 1; break; + case 'D': + dumpbarmode = 1; + break; + case 'e': errors = 1; break; case 'h': - isshort = 1; + width = 2; break; case 'l': @@ -154,6 +166,10 @@ main(int argc, char **argv) vpd = 1; break; + case 'x': + width = 8; + break; + default: usage(); } @@ -162,7 +178,9 @@ main(int argc, char **argv) if ((listmode && optind >= argc + 1) || (writemode && optind + 3 != argc) || (readmode && optind + 2 != argc) - || (attachedmode && optind + 1 != argc)) + || (attachedmode && optind + 1 != argc) + || (dumpbarmode && (optind + 2 > argc || optind + 4 < argc)) + || (width == 8 && !dumpbarmode)) usage(); if (listmode) { @@ -171,16 +189,20 @@ main(int argc, char **argv) } else if (attachedmode) { chkattached(argv[optind]); } else if (readmode) { - readit(argv[optind], argv[optind + 1], - byte ? 1 : isshort ? 2 : 4); + readit(argv[optind], argv[optind + 1], width); } else if (writemode) { writeit(argv[optind], argv[optind + 1], argv[optind + 2], - byte ? 1 : isshort ? 2 : 4); + width); + } else if (dumpbarmode) { + dump_bar(argv[optind], argv[optind + 1], + optind + 2 < argc ? argv[optind + 2] : NULL, + optind + 3 < argc ? argv[optind + 3] : NULL, + width, verbose); } else { usage(); } - return exitstatus; + return (exitstatus); } static void @@ -1025,5 +1047,119 @@ chkattached(const char *name) exitstatus = pi.pi_data ? 0 : 2; /* exit(2), if NOT attached */ printf("%s: %s%s\n", name, pi.pi_data == 0 ? "not " : "", "attached"); + close(fd); +} + +static void +dump_bar(const char *name, const char *reg, const char *bar_start, + const char *bar_count, int width, int verbose) +{ + struct pci_bar_mmap pbm; + uint32_t *dd; + uint16_t *dh; + uint8_t *db; + uint64_t *dx, a, start, count; + char *el; + size_t res; + int fd; + + start = 0; + if (bar_start != NULL) { + start = strtoul(bar_start, &el, 0); + if (*el != '\0') + errx(1, "Invalid bar start specification %s", + bar_start); + } + count = 0; + if (bar_count != NULL) { + count = strtoul(bar_count, &el, 0); + if (*el != '\0') + errx(1, "Invalid count specification %s", + bar_count); + } + + pbm.pbm_sel = getsel(name); + pbm.pbm_reg = strtoul(reg, &el, 0); + if (*reg == '\0' || *el != '\0') + errx(1, "Invalid bar specification %s", reg); + pbm.pbm_flags = 0; + pbm.pbm_memattr = VM_MEMATTR_UNCACHEABLE; /* XXX */ + + fd = open(_PATH_DEVPCI, O_RDONLY, 0); + if (fd < 0) + err(1, "%s", _PATH_DEVPCI); + + if (ioctl(fd, PCIOCBARMMAP, &pbm) < 0) + err(1, "ioctl(PCIOCBARMMAP)"); + + if (count == 0) + count = pbm.pbm_bar_length / width; + if (start + count < start || (start + count) * width < (uint64_t)width) + errx(1, "(start + count) x width overflow"); + if ((start + count) * width > pbm.pbm_bar_length) { + if (start * width > pbm.pbm_bar_length) + count = 0; + else + count = (pbm.pbm_bar_length - start * width) / width; + } + if (verbose) { + fprintf(stderr, + "Dumping pci%d:%d:%d:%d BAR %x mapped base %p " + "off %#x length %#jx from %#jx count %#jx in %d-bytes\n", + pbm.pbm_sel.pc_domain, pbm.pbm_sel.pc_bus, + pbm.pbm_sel.pc_dev, pbm.pbm_sel.pc_func, + pbm.pbm_reg, pbm.pbm_map_base, pbm.pbm_bar_off, + pbm.pbm_bar_length, start, count, width); + } + switch (width) { + case 1: + db = (uint8_t *)(uintptr_t)((uintptr_t)pbm.pbm_map_base + + pbm.pbm_bar_off + start * width); + for (a = 0; a < count; a += width, db++) { + res = fwrite(db, width, 1, stdout); + if (res != 1) { + errx(1, "error writing to stdout"); + break; + } + } + break; + case 2: + dh = (uint16_t *)(uintptr_t)((uintptr_t)pbm.pbm_map_base + + pbm.pbm_bar_off + start * width); + for (a = 0; a < count; a += width, dh++) { + res = fwrite(dh, width, 1, stdout); + if (res != 1) { + errx(1, "error writing to stdout"); + break; + } + } + break; + case 4: + dd = (uint32_t *)(uintptr_t)((uintptr_t)pbm.pbm_map_base + + pbm.pbm_bar_off + start * width); + for (a = 0; a < count; a += width) { + res = fwrite(dd, width, 1, stdout); + if (res != 1) { + errx(1, "error writing to stdout"); + break; + } + } + break; + case 8: + dx = (uint64_t *)(uintptr_t)((uintptr_t)pbm.pbm_map_base + + pbm.pbm_bar_off + start * width); + for (a = 0; a < count; a += width, dx++) { + res = fwrite(dx, width, 1, stdout); + if (res != 1) { + errx(1, "error writing to stdout"); + break; + } + } + break; + default: + errx(1, "invalid access width"); + } + + munmap((void *)pbm.pbm_map_base, pbm.pbm_map_length); close(fd); } From owner-svn-src-all@freebsd.org Wed Aug 1 19:06:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 216BB1066899; Wed, 1 Aug 2018 19:06:56 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9A5183115; Wed, 1 Aug 2018 19:06:55 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AA48F2693A; Wed, 1 Aug 2018 19:06:55 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71J6t3I032797; Wed, 1 Aug 2018 19:06:55 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71J6tBe032796; Wed, 1 Aug 2018 19:06:55 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808011906.w71J6tBe032796@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Wed, 1 Aug 2018 19:06:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337055 - head/sys/ufs/ffs X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/ufs/ffs X-SVN-Commit-Revision: 337055 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:06:56 -0000 Author: kib Date: Wed Aug 1 19:06:55 2018 New Revision: 337055 URL: https://svnweb.freebsd.org/changeset/base/337055 Log: Avoid assertion in /dev/ufssuspend when the suspend ioctl is (incorrectly) called while another suspension is already active. PR: 230220 Reported by: dexuan Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 1 week Modified: head/sys/ufs/ffs/ffs_suspend.c Modified: head/sys/ufs/ffs/ffs_suspend.c ============================================================================== --- head/sys/ufs/ffs/ffs_suspend.c Wed Aug 1 19:02:05 2018 (r337054) +++ head/sys/ufs/ffs/ffs_suspend.c Wed Aug 1 19:06:55 2018 (r337055) @@ -218,6 +218,33 @@ ffs_susp_suspend(struct mount *mp) } static void +ffs_susp_unsuspend(struct mount *mp) +{ + struct ufsmount *ump; + + sx_assert(&ffs_susp_lock, SA_XLOCKED); + + /* + * XXX: The status is kept per-process; the vfs_write_resume() routine + * asserts that the resuming thread is the same one that called + * vfs_write_suspend(). The cdevpriv data, however, is attached + * to the file descriptor, e.g. is inherited during fork. Thus, + * it's possible that the resuming process will be different from + * the one that started the suspension. + * + * Work around by fooling the check in vfs_write_resume(). + */ + mp->mnt_susp_owner = curthread; + + vfs_write_resume(mp, 0); + ump = VFSTOUFS(mp); + UFS_LOCK(ump); + ump->um_flags &= ~UM_WRITESUSPENDED; + UFS_UNLOCK(ump); + vfs_unbusy(mp); +} + +static void ffs_susp_dtor(void *data) { struct fs *fs; @@ -243,24 +270,7 @@ ffs_susp_dtor(void *data) if (error != 0) panic("failed to unsuspend writes on %s", fs->fs_fsmnt); - /* - * XXX: The status is kept per-process; the vfs_write_resume() routine - * asserts that the resuming thread is the same one that called - * vfs_write_suspend(). The cdevpriv data, however, is attached - * to the file descriptor, e.g. is inherited during fork. Thus, - * it's possible that the resuming process will be different from - * the one that started the suspension. - * - * Work around by fooling the check in vfs_write_resume(). - */ - mp->mnt_susp_owner = curthread; - - vfs_write_resume(mp, 0); - vfs_unbusy(mp); - UFS_LOCK(ump); - ump->um_flags &= ~UM_WRITESUSPENDED; - UFS_UNLOCK(ump); - + ffs_susp_unsuspend(mp); sx_xunlock(&ffs_susp_lock); } @@ -300,7 +310,8 @@ ffs_susp_ioctl(struct cdev *dev, u_long cmd, caddr_t a break; } error = devfs_set_cdevpriv(mp, ffs_susp_dtor); - KASSERT(error == 0, ("devfs_set_cdevpriv failed")); + if (error != 0) + ffs_susp_unsuspend(mp); break; case UFSRESUME: error = devfs_get_cdevpriv((void **)&mp); From owner-svn-src-all@freebsd.org Wed Aug 1 19:10:47 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CBB710669E9; Wed, 1 Aug 2018 19:10:47 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B35C58333E; Wed, 1 Aug 2018 19:10:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 95BCC2694D; Wed, 1 Aug 2018 19:10:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71JAkqb033734; Wed, 1 Aug 2018 19:10:46 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71JAkFt033733; Wed, 1 Aug 2018 19:10:46 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808011910.w71JAkFt033733@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Wed, 1 Aug 2018 19:10:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337056 - head/sys/compat/linuxkpi/common/include/asm X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: head/sys/compat/linuxkpi/common/include/asm X-SVN-Commit-Revision: 337056 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:10:47 -0000 Author: hselasky Date: Wed Aug 1 19:10:46 2018 New Revision: 337056 URL: https://svnweb.freebsd.org/changeset/base/337056 Log: Don't refer to non-existing atomic functions, even though not compiled, in the LinuxKPI. Found by: rpolka @ MFC after: 1 week Sponsored by: Mellanox Technologies Modified: head/sys/compat/linuxkpi/common/include/asm/atomic.h Modified: head/sys/compat/linuxkpi/common/include/asm/atomic.h ============================================================================== --- head/sys/compat/linuxkpi/common/include/asm/atomic.h Wed Aug 1 19:06:55 2018 (r337055) +++ head/sys/compat/linuxkpi/common/include/asm/atomic.h Wed Aug 1 19:10:46 2018 (r337056) @@ -2,7 +2,7 @@ * Copyright (c) 2010 Isilon Systems, Inc. * Copyright (c) 2010 iX Systems, Inc. * Copyright (c) 2010 Panasas, Inc. - * Copyright (c) 2013-2017 Mellanox Technologies, Ltd. + * Copyright (c) 2013-2018 Mellanox Technologies, Ltd. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -159,6 +159,21 @@ atomic_cmpxchg(atomic_t *v, int old, int new) return (ret); } +#if defined(__i386__) || defined(__amd64__) +#define LINUXKPI_ATOMIC_8(...) __VA_ARGS__ +#define LINUXKPI_ATOMIC_16(...) __VA_ARGS__ +#else +#define LINUXKPI_ATOMIC_8(...) +#define LINUXKPI_ATOMIC_16(...) +#endif + +#if defined(__amd64__) || (defined(__ARM_ARCH) && (__ARM_ARCH >= 6)) || \ + defined(__aarch64__) || defined(__powerpc64__) || defined(__riscv) +#define LINUXKPI_ATOMIC_64(...) __VA_ARGS__ +#else +#define LINUXKPI_ATOMIC_64(...) +#endif + #define cmpxchg(ptr, old, new) ({ \ union { \ __typeof(*(ptr)) val; \ @@ -168,30 +183,39 @@ atomic_cmpxchg(atomic_t *v, int old, int new) u64 u64[0]; \ } __ret = { .val = (old) }, __new = { .val = (new) }; \ \ - CTASSERT(sizeof(__ret.val) == 1 || sizeof(__ret.val) == 2 || \ - sizeof(__ret.val) == 4 || sizeof(__ret.val) == 8); \ + CTASSERT( \ + LINUXKPI_ATOMIC_8(sizeof(__ret.val) == 1 ||) \ + LINUXKPI_ATOMIC_16(sizeof(__ret.val) == 2 ||) \ + LINUXKPI_ATOMIC_64(sizeof(__ret.val) == 8 ||) \ + sizeof(__ret.val) == 4); \ \ switch (sizeof(__ret.val)) { \ + LINUXKPI_ATOMIC_8( \ case 1: \ while (!atomic_fcmpset_8((volatile u8 *)(ptr), \ __ret.u8, __new.u8[0]) && __ret.val == (old)) \ ; \ break; \ + ) \ + LINUXKPI_ATOMIC_16( \ case 2: \ while (!atomic_fcmpset_16((volatile u16 *)(ptr), \ __ret.u16, __new.u16[0]) && __ret.val == (old)) \ ; \ break; \ + ) \ case 4: \ while (!atomic_fcmpset_32((volatile u32 *)(ptr), \ __ret.u32, __new.u32[0]) && __ret.val == (old)) \ ; \ break; \ + LINUXKPI_ATOMIC_64( \ case 8: \ while (!atomic_fcmpset_64((volatile u64 *)(ptr), \ __ret.u64, __new.u64[0]) && __ret.val == (old)) \ ; \ break; \ + ) \ } \ __ret.val; \ }) @@ -207,30 +231,39 @@ atomic_cmpxchg(atomic_t *v, int old, int new) u64 u64[0]; \ } __ret, __new = { .val = (new) }; \ \ - CTASSERT(sizeof(__ret.val) == 1 || sizeof(__ret.val) == 2 || \ - sizeof(__ret.val) == 4 || sizeof(__ret.val) == 8); \ + CTASSERT( \ + LINUXKPI_ATOMIC_8(sizeof(__ret.val) == 1 ||) \ + LINUXKPI_ATOMIC_16(sizeof(__ret.val) == 2 ||) \ + LINUXKPI_ATOMIC_64(sizeof(__ret.val) == 8 ||) \ + sizeof(__ret.val) == 4); \ \ switch (sizeof(__ret.val)) { \ + LINUXKPI_ATOMIC_8( \ case 1: \ __ret.val = READ_ONCE(*ptr); \ while (!atomic_fcmpset_8((volatile u8 *)(ptr), \ __ret.u8, __new.u8[0])) \ ; \ break; \ + ) \ + LINUXKPI_ATOMIC_16( \ case 2: \ __ret.val = READ_ONCE(*ptr); \ while (!atomic_fcmpset_16((volatile u16 *)(ptr), \ __ret.u16, __new.u16[0])) \ ; \ break; \ + ) \ case 4: \ __ret.u32[0] = atomic_swap_32((volatile u32 *)(ptr), \ __new.u32[0]); \ break; \ + LINUXKPI_ATOMIC_64( \ case 8: \ __ret.u64[0] = atomic_swap_64((volatile u64 *)(ptr), \ __new.u64[0]); \ break; \ + ) \ } \ __ret.val; \ }) From owner-svn-src-all@freebsd.org Wed Aug 1 19:12:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED6791066BC1; Wed, 1 Aug 2018 19:12:51 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91BB0836FB; Wed, 1 Aug 2018 19:12:51 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.128.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id A472C26048E; Wed, 1 Aug 2018 21:12:50 +0200 (CEST) Subject: Re: svn commit: r337056 - head/sys/compat/linuxkpi/common/include/asm To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201808011910.w71JAkFt033733@repo.freebsd.org> From: Hans Petter Selasky Message-ID: <598e7e81-6015-5c4b-b8a4-17353c153cbd@selasky.org> Date: Wed, 1 Aug 2018 21:12:30 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <201808011910.w71JAkFt033733@repo.freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:12:52 -0000 On 08/01/18 21:10, Hans Petter Selasky wrote: > Small typo: > Found by: rpolka @ ^^ rpokala @ --HPS From owner-svn-src-all@freebsd.org Wed Aug 1 19:17:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFE941066CFC; Wed, 1 Aug 2018 19:17:56 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 65E5B83A72; Wed, 1 Aug 2018 19:17:56 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 46F5B26ACC; Wed, 1 Aug 2018 19:17:56 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71JHuxC037888; Wed, 1 Aug 2018 19:17:56 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71JHuIs037887; Wed, 1 Aug 2018 19:17:56 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201808011917.w71JHuIs037887@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Wed, 1 Aug 2018 19:17:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337057 - head/share/mk X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/share/mk X-SVN-Commit-Revision: 337057 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:17:56 -0000 Author: emaste Date: Wed Aug 1 19:17:55 2018 New Revision: 337057 URL: https://svnweb.freebsd.org/changeset/base/337057 Log: Use lld as the bootstrap linker on armv7 by default As of r336972 lld is capable linking the armv7 kernel and userland, so enable it by default. PR: 229050 Reviewed by: kevans Tested by: kevans Relnotes: Yes Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D16528 Modified: head/share/mk/src.opts.mk Modified: head/share/mk/src.opts.mk ============================================================================== --- head/share/mk/src.opts.mk Wed Aug 1 19:10:46 2018 (r337056) +++ head/share/mk/src.opts.mk Wed Aug 1 19:17:55 2018 (r337057) @@ -311,7 +311,7 @@ __DEFAULT_NO_OPTIONS+=LLVM_LIBUNWIND .endif .if ${__T} == "aarch64" || ${__T} == "amd64" __DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP LLD_IS_LD -.elif ${__T} == "i386" +.elif ${__T} == "armv7" || ${__T} == "i386" __DEFAULT_YES_OPTIONS+=LLD_BOOTSTRAP __DEFAULT_NO_OPTIONS+=LLD_IS_LD .else From owner-svn-src-all@freebsd.org Wed Aug 1 19:25:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 25B931067253; Wed, 1 Aug 2018 19:25:14 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BEBFC84025; Wed, 1 Aug 2018 19:25:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9A37826C6A; Wed, 1 Aug 2018 19:25:13 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71JPDAv043025; Wed, 1 Aug 2018 19:25:13 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71JPDd8043022; Wed, 1 Aug 2018 19:25:13 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201808011925.w71JPDd8043022@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Wed, 1 Aug 2018 19:25:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337058 - in stable/10/sys/fs: nfs nfsserver X-SVN-Group: stable-10 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in stable/10/sys/fs: nfs nfsserver X-SVN-Commit-Revision: 337058 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:25:14 -0000 Author: rmacklem Date: Wed Aug 1 19:25:12 2018 New Revision: 337058 URL: https://svnweb.freebsd.org/changeset/base/337058 Log: MFC: r336357 Modify the reasons for not issuing a delegation in the NFSv4.1 server. The ESXi NFSv4.1 client will generate warning messages when the reason for not issuing a delegation is two. Two refers to a resource limit and I do not see why it would be considered invalid. However it probably was not the best choice of reason for not issuing a delegation. This patch changes the reasons used to ones that the ESXi client doesn't complain about. This change does not affect the FreeBSD client and does not appear to affect behaviour of the Linux NFSv4.1 client. RFC5661 defines these "reasons" but does not give any guidance w.r.t. which ones are more appropriate to return to a client. Modified: stable/10/sys/fs/nfs/nfsproto.h stable/10/sys/fs/nfsserver/nfs_nfsdserv.c stable/10/sys/fs/nfsserver/nfs_nfsdstate.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/fs/nfs/nfsproto.h ============================================================================== --- stable/10/sys/fs/nfs/nfsproto.h Wed Aug 1 19:17:55 2018 (r337057) +++ stable/10/sys/fs/nfs/nfsproto.h Wed Aug 1 19:25:12 2018 (r337058) @@ -552,6 +552,7 @@ #define NFSV4OPEN_WDRESOURCE 0x00080000 #define NFSV4OPEN_WDCONTENTION 0x00100000 #define NFSV4OPEN_WDNOTWANTED 0x00200000 +#define NFSV4OPEN_WDSUPPFTYPE 0x00400000 /* * NFS V4 File Handle types Modified: stable/10/sys/fs/nfsserver/nfs_nfsdserv.c ============================================================================== --- stable/10/sys/fs/nfsserver/nfs_nfsdserv.c Wed Aug 1 19:17:55 2018 (r337057) +++ stable/10/sys/fs/nfsserver/nfs_nfsdserv.c Wed Aug 1 19:25:12 2018 (r337058) @@ -2929,7 +2929,13 @@ nfsrvd_open(struct nfsrv_descript *nd, __unused int is *tl = txdr_unsigned(NFSV4OPEN_DELEGATEWRITE); else if (retext != 0) { *tl = txdr_unsigned(NFSV4OPEN_DELEGATENONEEXT); - if ((rflags & NFSV4OPEN_WDCONTENTION) != 0) { + if ((rflags & NFSV4OPEN_WDNOTWANTED) != 0) { + NFSM_BUILD(tl, u_int32_t *, NFSX_UNSIGNED); + *tl = txdr_unsigned(NFSV4OPEN_NOTWANTED); + } else if ((rflags & NFSV4OPEN_WDSUPPFTYPE) != 0) { + NFSM_BUILD(tl, u_int32_t *, NFSX_UNSIGNED); + *tl = txdr_unsigned(NFSV4OPEN_NOTSUPPFTYPE); + } else if ((rflags & NFSV4OPEN_WDCONTENTION) != 0) { NFSM_BUILD(tl, u_int32_t *, 2 * NFSX_UNSIGNED); *tl++ = txdr_unsigned(NFSV4OPEN_CONTENTION); *tl = newnfs_false; Modified: stable/10/sys/fs/nfsserver/nfs_nfsdstate.c ============================================================================== --- stable/10/sys/fs/nfsserver/nfs_nfsdstate.c Wed Aug 1 19:17:55 2018 (r337057) +++ stable/10/sys/fs/nfsserver/nfs_nfsdstate.c Wed Aug 1 19:25:12 2018 (r337058) @@ -3022,7 +3022,13 @@ tryagain: /* * This is where we can choose to issue a delegation. */ - if (delegate == 0 || writedeleg == 0 || + if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) + *rflagsp |= NFSV4OPEN_WDNOTWANTED; + else if (nfsrv_issuedelegs == 0) + *rflagsp |= NFSV4OPEN_WDSUPPFTYPE; + else if (NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) + *rflagsp |= NFSV4OPEN_WDRESOURCE; + else if (delegate == 0 || writedeleg == 0 || NFSVNO_EXRDONLY(exp) || (readonly != 0 && nfsrv_writedelegifpos == 0) || !NFSVNO_DELEGOK(vp) || @@ -3030,11 +3036,6 @@ tryagain: (clp->lc_flags & (LCL_CALLBACKSON | LCL_CBDOWN)) != LCL_CALLBACKSON) *rflagsp |= NFSV4OPEN_WDCONTENTION; - else if (nfsrv_issuedelegs == 0 || - NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) - *rflagsp |= NFSV4OPEN_WDRESOURCE; - else if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) - *rflagsp |= NFSV4OPEN_WDNOTWANTED; else { new_deleg->ls_stateid.seqid = delegstateidp->seqid = 1; new_deleg->ls_stateid.other[0] = delegstateidp->other[0] @@ -3085,16 +3086,17 @@ tryagain: /* * This is where we can choose to issue a delegation. */ - if (delegate == 0 || (writedeleg == 0 && readonly == 0) || - !NFSVNO_DELEGOK(vp) || + if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) + *rflagsp |= NFSV4OPEN_WDNOTWANTED; + else if (nfsrv_issuedelegs == 0) + *rflagsp |= NFSV4OPEN_WDSUPPFTYPE; + else if (NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) + *rflagsp |= NFSV4OPEN_WDRESOURCE; + else if (delegate == 0 || (writedeleg == 0 && + readonly == 0) || !NFSVNO_DELEGOK(vp) || (clp->lc_flags & (LCL_CALLBACKSON | LCL_CBDOWN)) != LCL_CALLBACKSON) *rflagsp |= NFSV4OPEN_WDCONTENTION; - else if (nfsrv_issuedelegs == 0 || - NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) - *rflagsp |= NFSV4OPEN_WDRESOURCE; - else if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) - *rflagsp |= NFSV4OPEN_WDNOTWANTED; else { new_deleg->ls_stateid.seqid = delegstateidp->seqid = 1; new_deleg->ls_stateid.other[0] = delegstateidp->other[0] From owner-svn-src-all@freebsd.org Wed Aug 1 19:45:05 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 353BD106774E; Wed, 1 Aug 2018 19:45:05 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE831848FE; Wed, 1 Aug 2018 19:45:04 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AC4BB26FB2; Wed, 1 Aug 2018 19:45:04 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71Jj4e2053454; Wed, 1 Aug 2018 19:45:04 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71Jj48F053453; Wed, 1 Aug 2018 19:45:04 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808011945.w71Jj48F053453@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 1 Aug 2018 19:45:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337059 - head/tests/sys/kern X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/tests/sys/kern X-SVN-Commit-Revision: 337059 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 19:45:05 -0000 Author: markj Date: Wed Aug 1 19:45:04 2018 New Revision: 337059 URL: https://svnweb.freebsd.org/changeset/base/337059 Log: Fix some nits in the unix_passfd tests. - Remove return statements in functions with a void return type. - Allocate enough space for the SCM_CREDS and SCM_RIGHTS messages received in the rights_creds_payload test. MFC after: 1 week Sponsored by: The FreeBSD Foundation Modified: head/tests/sys/kern/unix_passfd_test.c Modified: head/tests/sys/kern/unix_passfd_test.c ============================================================================== --- head/tests/sys/kern/unix_passfd_test.c Wed Aug 1 19:25:12 2018 (r337058) +++ head/tests/sys/kern/unix_passfd_test.c Wed Aug 1 19:45:04 2018 (r337059) @@ -165,14 +165,15 @@ sendfd(int sockfd, int send_fd) { char ch = 0; - return (sendfd_payload(sockfd, send_fd, &ch, sizeof(ch))); + sendfd_payload(sockfd, send_fd, &ch, sizeof(ch)); } static void recvfd_payload(int sockfd, int *recv_fd, void *buf, size_t buflen) { struct cmsghdr *cmsghdr; - char message[CMSG_SPACE(SOCKCREDSIZE(CMGROUP_MAX)) + sizeof(int)]; + char message[CMSG_SPACE(SOCKCREDSIZE(CMGROUP_MAX)) + + CMSG_SPACE(sizeof(int))]; struct msghdr msghdr; struct iovec iovec; ssize_t len; @@ -214,7 +215,7 @@ recvfd(int sockfd, int *recv_fd) { char ch = 0; - return (recvfd_payload(sockfd, recv_fd, &ch, sizeof(ch))); + recvfd_payload(sockfd, recv_fd, &ch, sizeof(ch)); } /* From owner-svn-src-all@freebsd.org Wed Aug 1 20:08:21 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1ED51067E15; Wed, 1 Aug 2018 20:08:21 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B53C8569A; Wed, 1 Aug 2018 20:08:21 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2868327312; Wed, 1 Aug 2018 20:08:21 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71K8Ksg063941; Wed, 1 Aug 2018 20:08:20 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71K8KF5063940; Wed, 1 Aug 2018 20:08:20 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201808012008.w71K8KF5063940@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Wed, 1 Aug 2018 20:08:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337060 - head/stand/uboot/common X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: head/stand/uboot/common X-SVN-Commit-Revision: 337060 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 20:08:21 -0000 Author: kevans Date: Wed Aug 1 20:08:20 2018 New Revision: 337060 URL: https://svnweb.freebsd.org/changeset/base/337060 Log: ubldr: Bump heap size, 1MB -> 2MB 1MB was leaving very little margin in some of the worse-case scenarios with lualoader. 2MB is still low enough that we shouldn't have any problems with UBoot-supported boards. MFC after: 1 week Modified: head/stand/uboot/common/main.c Modified: head/stand/uboot/common/main.c ============================================================================== --- head/stand/uboot/common/main.c Wed Aug 1 19:45:04 2018 (r337059) +++ head/stand/uboot/common/main.c Wed Aug 1 20:08:20 2018 (r337060) @@ -42,7 +42,7 @@ __FBSDID("$FreeBSD$"); #endif #ifndef HEAP_SIZE -#define HEAP_SIZE (1 * 1024 * 1024) +#define HEAP_SIZE (2 * 1024 * 1024) #endif struct uboot_devdesc currdev; From owner-svn-src-all@freebsd.org Wed Aug 1 20:38:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CE521068A96; Wed, 1 Aug 2018 20:38:02 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C698086BC4; Wed, 1 Aug 2018 20:38:01 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A37FF27806; Wed, 1 Aug 2018 20:38:01 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71Kc1c7079966; Wed, 1 Aug 2018 20:38:01 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71Kc14u079965; Wed, 1 Aug 2018 20:38:01 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201808012038.w71Kc14u079965@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Wed, 1 Aug 2018 20:38:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337061 - stable/11/sys/sparc64/sparc64 X-SVN-Group: stable-11 X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: stable/11/sys/sparc64/sparc64 X-SVN-Commit-Revision: 337061 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 20:38:02 -0000 Author: jhb Date: Wed Aug 1 20:38:01 2018 New Revision: 337061 URL: https://svnweb.freebsd.org/changeset/base/337061 Log: MFC 333416: Report TRAP_BRKPT for breakpoint traps on sparc64. Modified: stable/11/sys/sparc64/sparc64/trap.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/sparc64/sparc64/trap.c ============================================================================== --- stable/11/sys/sparc64/sparc64/trap.c Wed Aug 1 20:08:20 2018 (r337060) +++ stable/11/sys/sparc64/sparc64/trap.c Wed Aug 1 20:38:01 2018 (r337061) @@ -257,7 +257,7 @@ trap(struct trapframe *tf) struct thread *td; struct proc *p; int error; - int sig; + int sig, ucode; register_t addr; ksiginfo_t ksi; @@ -277,6 +277,7 @@ trap(struct trapframe *tf) td->td_pticks = 0; td->td_frame = tf; addr = tf->tf_tpc; + ucode = (int)tf->tf_type; /* XXX not POSIX */ if (td->td_cowgen != p->p_cowgen) thread_cow_update(td); @@ -300,6 +301,10 @@ trap(struct trapframe *tf) case T_CORRECTED_ECC_ERROR: sig = trap_cecc(); break; + case T_BREAKPOINT: + sig = SIGTRAP; + ucode = TRAP_BRKPT; + break; default: if (tf->tf_type > T_MAX) panic("trap: bad trap type %#lx (user)", @@ -322,7 +327,7 @@ trap(struct trapframe *tf) kdb_enter(KDB_WHY_TRAPSIG, "trapsig"); ksiginfo_init_trap(&ksi); ksi.ksi_signo = sig; - ksi.ksi_code = (int)tf->tf_type; /* XXX not POSIX */ + ksi.ksi_code = ucode; ksi.ksi_addr = (void *)addr; ksi.ksi_trapno = (int)tf->tf_type; trapsignal(td, &ksi); From owner-svn-src-all@freebsd.org Wed Aug 1 21:51:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F214B106A3E7; Wed, 1 Aug 2018 21:51:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A795B89E85; Wed, 1 Aug 2018 21:51:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8494D526; Wed, 1 Aug 2018 21:51:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w71LpoOQ017620; Wed, 1 Aug 2018 21:51:50 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w71LpnfV017615; Wed, 1 Aug 2018 21:51:49 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808012151.w71LpnfV017615@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Wed, 1 Aug 2018 21:51:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337063 - in head/cddl/contrib/opensolaris: cmd/zfs lib/libzfs/common X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/cddl/contrib/opensolaris: cmd/zfs lib/libzfs/common X-SVN-Commit-Revision: 337063 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2018 21:51:51 -0000 Author: mav Date: Wed Aug 1 21:51:49 2018 New Revision: 337063 URL: https://svnweb.freebsd.org/changeset/base/337063 Log: MFV r316926: 7955 libshare needs to initialize only those datasets being modified by the consumer illumos/illumos-gate@8a981c3356b194b3b5c0ae9276a9cc31cd2f93a3 https://github.com/illumos/illumos-gate/commit/8a981c3356b194b3b5c0ae9276a9cc31cd2f93a3 https://www.illumos.org/issues/7955 Libshare currently initializes all available filesystems when doing any libshare operation. This requires iterating through all the filesystem multiple times, which is a huge performance problem for sharing and unsharing operations. Reviewed by: Steve Gonczi Reviewed by: Sebastien Roy Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: Pavel Zakharov Reviewed by: Yuri Pankov Approved by: Gordon Ross Author: Daniel Hoffman For FreeBSD this is practically a NOP, just a diff reduction. Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_changelist.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_impl.h head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_mount.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c Wed Aug 1 20:46:06 2018 (r337062) +++ head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c Wed Aug 1 21:51:49 2018 (r337063) @@ -72,6 +72,7 @@ #include #include #include +#include #endif #include "zfs_iter.h" @@ -6221,6 +6222,17 @@ share_mount(int op, int argc, char **argv) return (0); qsort(dslist, count, sizeof (void *), libzfs_dataset_cmp); +#ifdef illumos + sa_init_selective_arg_t sharearg; + sharearg.zhandle_arr = dslist; + sharearg.zhandle_len = count; + if ((ret = zfs_init_libshare_arg(zfs_get_handle(dslist[0]), + SA_INIT_SHARE_API_SELECTIVE, &sharearg)) != SA_OK) { + (void) fprintf(stderr, + gettext("Could not initialize libshare, %d"), ret); + return (ret); + } +#endif for (i = 0; i < count; i++) { if (verbose) Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h Wed Aug 1 20:46:06 2018 (r337062) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h Wed Aug 1 21:51:49 2018 (r337063) @@ -843,6 +843,17 @@ extern int zmount(const char *, const char *, int, cha #endif extern int zfs_remap_indirects(libzfs_handle_t *hdl, const char *); +/* Allow consumers to initialize libshare externally for optimal performance */ +extern int zfs_init_libshare_arg(libzfs_handle_t *, int, void *); +/* + * For most consumers, zfs_init_libshare_arg is sufficient on its own, and + * zfs_uninit_libshare is unnecessary. zfs_uninit_libshare should only be called + * if the caller has already initialized libshare for one set of zfs handles, + * and wishes to share or unshare filesystems outside of that set. In that case, + * the caller should uninitialize libshare, and then re-initialize it with the + * new handles being shared or unshared. + */ +extern void zfs_uninit_libshare(libzfs_handle_t *); #ifdef __cplusplus } #endif Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_changelist.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_changelist.c Wed Aug 1 20:46:06 2018 (r337062) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_changelist.c Wed Aug 1 21:51:49 2018 (r337063) @@ -26,7 +26,7 @@ * Portions Copyright 2007 Ramprakash Jelari * Copyright (c) 2011 Pawel Jakub Dawidek . * All rights reserved. - * Copyright (c) 2014, 2015 by Delphix. All rights reserved. + * Copyright (c) 2014, 2016 by Delphix. All rights reserved. * Copyright 2016 Igor Kozhukhov */ @@ -166,6 +166,11 @@ changelist_postfix(prop_changelist_t *clp) char shareopts[ZFS_MAXPROPLEN]; int errors = 0; libzfs_handle_t *hdl; +#ifdef illumos + size_t num_datasets = 0, i; + zfs_handle_t **zhandle_arr; + sa_init_selective_arg_t sharearg; +#endif /* * If we're changing the mountpoint, attempt to destroy the underlying @@ -192,8 +197,33 @@ changelist_postfix(prop_changelist_t *clp) hdl = cn->cn_handle->zfs_hdl; assert(hdl != NULL); zfs_uninit_libshare(hdl); - } +#ifdef illumos + /* + * For efficiencies sake, we initialize libshare for only a few + * shares (the ones affected here). Future initializations in + * this process should just use the cached initialization. + */ + for (cn = uu_list_last(clp->cl_list); cn != NULL; + cn = uu_list_prev(clp->cl_list, cn)) { + num_datasets++; + } + + zhandle_arr = zfs_alloc(hdl, + num_datasets * sizeof (zfs_handle_t *)); + for (i = 0, cn = uu_list_last(clp->cl_list); cn != NULL; + cn = uu_list_prev(clp->cl_list, cn)) { + zhandle_arr[i++] = cn->cn_handle; + zfs_refresh_properties(cn->cn_handle); + } + assert(i == num_datasets); + sharearg.zhandle_arr = zhandle_arr; + sharearg.zhandle_len = num_datasets; + errors = zfs_init_libshare_arg(hdl, SA_INIT_SHARE_API_SELECTIVE, + &sharearg); + free(zhandle_arr); +#endif + } /* * We walk the datasets in reverse, because we want to mount any parent * datasets before mounting the children. We walk all datasets even if @@ -218,7 +248,9 @@ changelist_postfix(prop_changelist_t *clp) continue; cn->cn_needpost = B_FALSE; +#ifndef illumos zfs_refresh_properties(cn->cn_handle); +#endif if (ZFS_IS_VOLUME(cn->cn_handle)) continue; Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_impl.h ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_impl.h Wed Aug 1 20:46:06 2018 (r337062) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_impl.h Wed Aug 1 21:51:49 2018 (r337063) @@ -207,7 +207,6 @@ void namespace_clear(libzfs_handle_t *); */ extern int zfs_init_libshare(libzfs_handle_t *, int); -extern void zfs_uninit_libshare(libzfs_handle_t *); extern int zfs_parse_options(char *, zfs_share_proto_t); extern int zfs_unshare_proto(zfs_handle_t *, Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_mount.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_mount.c Wed Aug 1 20:46:06 2018 (r337062) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_mount.c Wed Aug 1 21:51:49 2018 (r337063) @@ -579,6 +579,7 @@ zfs_is_shared_smb(zfs_handle_t *zhp, char **where) #ifdef illumos static sa_handle_t (*_sa_init)(int); +static sa_handle_t (*_sa_init_arg)(int, void *); static void (*_sa_fini)(sa_handle_t); static sa_share_t (*_sa_find_share)(sa_handle_t, char *); static int (*_sa_enable_share)(sa_share_t, char *); @@ -620,6 +621,8 @@ _zfs_init_libshare(void) if ((libshare = dlopen(path, RTLD_LAZY | RTLD_GLOBAL)) != NULL) { _sa_init = (sa_handle_t (*)(int))dlsym(libshare, "sa_init"); + _sa_init_arg = (sa_handle_t (*)(int, void *))dlsym(libshare, + "sa_init_arg"); _sa_fini = (void (*)(sa_handle_t))dlsym(libshare, "sa_fini"); _sa_find_share = (sa_share_t (*)(sa_handle_t, char *)) dlsym(libshare, "sa_find_share"); @@ -639,14 +642,15 @@ _zfs_init_libshare(void) char *, char *))dlsym(libshare, "sa_zfs_process_share"); _sa_update_sharetab_ts = (void (*)(sa_handle_t)) dlsym(libshare, "sa_update_sharetab_ts"); - if (_sa_init == NULL || _sa_fini == NULL || - _sa_find_share == NULL || _sa_enable_share == NULL || - _sa_disable_share == NULL || _sa_errorstr == NULL || - _sa_parse_legacy_options == NULL || + if (_sa_init == NULL || _sa_init_arg == NULL || + _sa_fini == NULL || _sa_find_share == NULL || + _sa_enable_share == NULL || _sa_disable_share == NULL || + _sa_errorstr == NULL || _sa_parse_legacy_options == NULL || _sa_needs_refresh == NULL || _sa_get_zfs_handle == NULL || _sa_zfs_process_share == NULL || _sa_update_sharetab_ts == NULL) { _sa_init = NULL; + _sa_init_arg = NULL; _sa_fini = NULL; _sa_disable_share = NULL; _sa_enable_share = NULL; @@ -670,8 +674,8 @@ _zfs_init_libshare(void) * service value is which part(s) of the API to initialize and is a * direct map to the libshare sa_init(service) interface. */ -int -zfs_init_libshare(libzfs_handle_t *zhandle, int service) +static int +zfs_init_libshare_impl(libzfs_handle_t *zhandle, int service, void *arg) { #ifdef illumos /* @@ -694,11 +698,11 @@ zfs_init_libshare(libzfs_handle_t *zhandle, int servic if (_sa_needs_refresh != NULL && _sa_needs_refresh(zhandle->libzfs_sharehdl)) { zfs_uninit_libshare(zhandle); - zhandle->libzfs_sharehdl = _sa_init(service); + zhandle->libzfs_sharehdl = _sa_init_arg(service, arg); } if (zhandle && zhandle->libzfs_sharehdl == NULL) - zhandle->libzfs_sharehdl = _sa_init(service); + zhandle->libzfs_sharehdl = _sa_init_arg(service, arg); if (zhandle->libzfs_sharehdl == NULL) return (SA_NO_MEMORY); @@ -706,7 +710,19 @@ zfs_init_libshare(libzfs_handle_t *zhandle, int servic return (SA_OK); } +int +zfs_init_libshare(libzfs_handle_t *zhandle, int service) +{ + return (zfs_init_libshare_impl(zhandle, service, NULL)); +} +int +zfs_init_libshare_arg(libzfs_handle_t *zhandle, int service, void *arg) +{ + return (zfs_init_libshare_impl(zhandle, service, arg)); +} + + /* * zfs_uninit_libshare(zhandle) * @@ -817,9 +833,9 @@ zfs_share_proto(zfs_handle_t *zhp, zfs_share_proto_t * ZFS_MAXPROPLEN, B_FALSE) != 0 || strcmp(shareopts, "off") == 0) continue; - #ifdef illumos - ret = zfs_init_libshare(hdl, SA_INIT_SHARE_API); + ret = zfs_init_libshare_arg(hdl, SA_INIT_ONE_SHARE_FROM_HANDLE, + zhp); if (ret != SA_OK) { (void) zfs_error_fmt(hdl, EZFS_SHARENFSFAILED, dgettext(TEXT_DOMAIN, "cannot share '%s': %s"), @@ -930,6 +946,7 @@ unshare_one(libzfs_handle_t *hdl, const char *name, co sa_share_t share; int err; char *mntpt; + /* * Mountpoint could get trashed if libshare calls getmntany * which it does during API initialization, so strdup the @@ -937,8 +954,14 @@ unshare_one(libzfs_handle_t *hdl, const char *name, co */ mntpt = zfs_strdup(hdl, mountpoint); - /* make sure libshare initialized */ - if ((err = zfs_init_libshare(hdl, SA_INIT_SHARE_API)) != SA_OK) { + /* + * make sure libshare initialized, initialize everything because we + * don't know what other unsharing may happen later. Functions up the + * stack are allowed to initialize instead a subset of shares at the + * time the set is known. + */ + if ((err = zfs_init_libshare_arg(hdl, SA_INIT_ONE_SHARE_FROM_NAME, + (void *)name)) != SA_OK) { free(mntpt); /* don't need the copy anymore */ return (zfs_error_fmt(hdl, proto_table[proto].p_unshare_err, dgettext(TEXT_DOMAIN, "cannot unshare '%s': %s"), @@ -1289,6 +1312,9 @@ zpool_disable_datasets(zpool_handle_t *zhp, boolean_t int i; int ret = -1; int flags = (force ? MS_FORCE : 0); +#ifdef illumos + sa_init_selective_arg_t sharearg; +#endif namelen = strlen(zhp->zpool_name); @@ -1363,6 +1389,14 @@ zpool_disable_datasets(zpool_handle_t *zhp, boolean_t * At this point, we have the entire list of filesystems, so sort it by * mountpoint. */ +#ifdef illumos + sharearg.zhandle_arr = datasets; + sharearg.zhandle_len = used; + ret = zfs_init_libshare_arg(hdl, SA_INIT_SHARE_API_SELECTIVE, + &sharearg); + if (ret != 0) + goto out; +#endif qsort(mountpoints, used, sizeof (char *), mountpoint_compare); /* From owner-svn-src-all@freebsd.org Thu Aug 2 03:14:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45A551052BA3; Thu, 2 Aug 2018 03:14:01 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF8109607A; Thu, 2 Aug 2018 03:14:00 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D03AC3FEF; Thu, 2 Aug 2018 03:14:00 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w723E0ZP084887; Thu, 2 Aug 2018 03:14:00 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w723E0pD084885; Thu, 2 Aug 2018 03:14:00 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201808020314.w723E0pD084885@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Thu, 2 Aug 2018 03:14:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337064 - in stable/11/sys/fs: nfs nfsserver X-SVN-Group: stable-11 X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: in stable/11/sys/fs: nfs nfsserver X-SVN-Commit-Revision: 337064 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 03:14:01 -0000 Author: rmacklem Date: Thu Aug 2 03:13:59 2018 New Revision: 337064 URL: https://svnweb.freebsd.org/changeset/base/337064 Log: MFC: r336357 Modify the reasons for not issuing a delegation in the NFSv4.1 server. The ESXi NFSv4.1 client will generate warning messages when the reason for not issuing a delegation is two. Two refers to a resource limit and I do not see why it would be considered invalid. However it probably was not the best choice of reason for not issuing a delegation. This patch changes the reasons used to ones that the ESXi client doesn't complain about. This change does not affect the FreeBSD client and does not appear to affect behaviour of the Linux NFSv4.1 client. RFC5661 defines these "reasons" but does not give any guidance w.r.t. which ones are more appropriate to return to a client. Modified: stable/11/sys/fs/nfs/nfsproto.h stable/11/sys/fs/nfsserver/nfs_nfsdserv.c stable/11/sys/fs/nfsserver/nfs_nfsdstate.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/fs/nfs/nfsproto.h ============================================================================== --- stable/11/sys/fs/nfs/nfsproto.h Wed Aug 1 21:51:49 2018 (r337063) +++ stable/11/sys/fs/nfs/nfsproto.h Thu Aug 2 03:13:59 2018 (r337064) @@ -554,6 +554,7 @@ #define NFSV4OPEN_WDRESOURCE 0x00080000 #define NFSV4OPEN_WDCONTENTION 0x00100000 #define NFSV4OPEN_WDNOTWANTED 0x00200000 +#define NFSV4OPEN_WDSUPPFTYPE 0x00400000 /* * NFS V4 File Handle types Modified: stable/11/sys/fs/nfsserver/nfs_nfsdserv.c ============================================================================== --- stable/11/sys/fs/nfsserver/nfs_nfsdserv.c Wed Aug 1 21:51:49 2018 (r337063) +++ stable/11/sys/fs/nfsserver/nfs_nfsdserv.c Thu Aug 2 03:13:59 2018 (r337064) @@ -2951,7 +2951,13 @@ nfsrvd_open(struct nfsrv_descript *nd, __unused int is *tl = txdr_unsigned(NFSV4OPEN_DELEGATEWRITE); else if (retext != 0) { *tl = txdr_unsigned(NFSV4OPEN_DELEGATENONEEXT); - if ((rflags & NFSV4OPEN_WDCONTENTION) != 0) { + if ((rflags & NFSV4OPEN_WDNOTWANTED) != 0) { + NFSM_BUILD(tl, u_int32_t *, NFSX_UNSIGNED); + *tl = txdr_unsigned(NFSV4OPEN_NOTWANTED); + } else if ((rflags & NFSV4OPEN_WDSUPPFTYPE) != 0) { + NFSM_BUILD(tl, u_int32_t *, NFSX_UNSIGNED); + *tl = txdr_unsigned(NFSV4OPEN_NOTSUPPFTYPE); + } else if ((rflags & NFSV4OPEN_WDCONTENTION) != 0) { NFSM_BUILD(tl, u_int32_t *, 2 * NFSX_UNSIGNED); *tl++ = txdr_unsigned(NFSV4OPEN_CONTENTION); *tl = newnfs_false; Modified: stable/11/sys/fs/nfsserver/nfs_nfsdstate.c ============================================================================== --- stable/11/sys/fs/nfsserver/nfs_nfsdstate.c Wed Aug 1 21:51:49 2018 (r337063) +++ stable/11/sys/fs/nfsserver/nfs_nfsdstate.c Thu Aug 2 03:13:59 2018 (r337064) @@ -3025,7 +3025,13 @@ tryagain: /* * This is where we can choose to issue a delegation. */ - if (delegate == 0 || writedeleg == 0 || + if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) + *rflagsp |= NFSV4OPEN_WDNOTWANTED; + else if (nfsrv_issuedelegs == 0) + *rflagsp |= NFSV4OPEN_WDSUPPFTYPE; + else if (NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) + *rflagsp |= NFSV4OPEN_WDRESOURCE; + else if (delegate == 0 || writedeleg == 0 || NFSVNO_EXRDONLY(exp) || (readonly != 0 && nfsrv_writedelegifpos == 0) || !NFSVNO_DELEGOK(vp) || @@ -3033,11 +3039,6 @@ tryagain: (clp->lc_flags & (LCL_CALLBACKSON | LCL_CBDOWN)) != LCL_CALLBACKSON) *rflagsp |= NFSV4OPEN_WDCONTENTION; - else if (nfsrv_issuedelegs == 0 || - NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) - *rflagsp |= NFSV4OPEN_WDRESOURCE; - else if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) - *rflagsp |= NFSV4OPEN_WDNOTWANTED; else { new_deleg->ls_stateid.seqid = delegstateidp->seqid = 1; new_deleg->ls_stateid.other[0] = delegstateidp->other[0] @@ -3088,16 +3089,17 @@ tryagain: /* * This is where we can choose to issue a delegation. */ - if (delegate == 0 || (writedeleg == 0 && readonly == 0) || - !NFSVNO_DELEGOK(vp) || + if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) + *rflagsp |= NFSV4OPEN_WDNOTWANTED; + else if (nfsrv_issuedelegs == 0) + *rflagsp |= NFSV4OPEN_WDSUPPFTYPE; + else if (NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) + *rflagsp |= NFSV4OPEN_WDRESOURCE; + else if (delegate == 0 || (writedeleg == 0 && + readonly == 0) || !NFSVNO_DELEGOK(vp) || (clp->lc_flags & (LCL_CALLBACKSON | LCL_CBDOWN)) != LCL_CALLBACKSON) *rflagsp |= NFSV4OPEN_WDCONTENTION; - else if (nfsrv_issuedelegs == 0 || - NFSRV_V4DELEGLIMIT(nfsrv_delegatecnt)) - *rflagsp |= NFSV4OPEN_WDRESOURCE; - else if ((new_stp->ls_flags & NFSLCK_WANTNODELEG) != 0) - *rflagsp |= NFSV4OPEN_WDNOTWANTED; else { new_deleg->ls_stateid.seqid = delegstateidp->seqid = 1; new_deleg->ls_stateid.other[0] = delegstateidp->other[0] From owner-svn-src-all@freebsd.org Thu Aug 2 06:22:11 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2DAF7105A04A; Thu, 2 Aug 2018 06:22:11 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6E5B75BA8; Thu, 2 Aug 2018 06:22:10 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B80DC5E33; Thu, 2 Aug 2018 06:22:10 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w726MATB081966; Thu, 2 Aug 2018 06:22:10 GMT (envelope-from tsoome@FreeBSD.org) Received: (from tsoome@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w726MAa4081963; Thu, 2 Aug 2018 06:22:10 GMT (envelope-from tsoome@FreeBSD.org) Message-Id: <201808020622.w726MAa4081963@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tsoome set sender to tsoome@FreeBSD.org using -f From: Toomas Soome Date: Thu, 2 Aug 2018 06:22:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337065 - head/stand/libsa X-SVN-Group: head X-SVN-Commit-Author: tsoome X-SVN-Commit-Paths: head/stand/libsa X-SVN-Commit-Revision: 337065 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 06:22:11 -0000 Author: tsoome Date: Thu Aug 2 06:22:10 2018 New Revision: 337065 URL: https://svnweb.freebsd.org/changeset/base/337065 Log: libsa: dereferencing type-punned pointer in cd9660 The warning is given by gcc build, but it is good to fix anyhow. use bcopy instead of direct assignment. Modified: head/stand/libsa/cd9660.c head/stand/libsa/cd9660read.c Modified: head/stand/libsa/cd9660.c ============================================================================== --- head/stand/libsa/cd9660.c Thu Aug 2 03:13:59 2018 (r337064) +++ head/stand/libsa/cd9660.c Thu Aug 2 06:22:10 2018 (r337065) @@ -304,7 +304,7 @@ cd9660_open(const char *path, struct open_file *f) if (isonum_723(vd->logical_block_size) != ISO_DEFAULT_BLOCK_SIZE) goto out; - rec = *(struct iso_directory_record *) vd->root_directory_record; + bcopy(vd->root_directory_record, &rec, sizeof(rec)); if (*path == '/') path++; /* eat leading '/' */ first = 1; Modified: head/stand/libsa/cd9660read.c ============================================================================== --- head/stand/libsa/cd9660read.c Thu Aug 2 03:13:59 2018 (r337064) +++ head/stand/libsa/cd9660read.c Thu Aug 2 06:22:10 2018 (r337065) @@ -241,7 +241,7 @@ cd9660_lookup(const char *path) break; } - rec = *(struct iso_directory_record *) vd->root_directory_record; + bcopy(vd->root_directory_record, &rec, sizeof(rec)); if (*path == '/') path++; /* eat leading '/' */ first = 1; From owner-svn-src-all@freebsd.org Thu Aug 2 07:43:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F6CB105C0EF; Thu, 2 Aug 2018 07:43:29 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45E5A78317; Thu, 2 Aug 2018 07:43:29 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 217376C05; Thu, 2 Aug 2018 07:43:29 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w727hTK8023395; Thu, 2 Aug 2018 07:43:29 GMT (envelope-from trasz@FreeBSD.org) Received: (from trasz@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w727hS6S023393; Thu, 2 Aug 2018 07:43:28 GMT (envelope-from trasz@FreeBSD.org) Message-Id: <201808020743.w727hS6S023393@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: trasz set sender to trasz@FreeBSD.org using -f From: Edward Tomasz Napierala Date: Thu, 2 Aug 2018 07:43:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337067 - head/libexec/rtld-elf X-SVN-Group: head X-SVN-Commit-Author: trasz X-SVN-Commit-Paths: head/libexec/rtld-elf X-SVN-Commit-Revision: 337067 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 07:43:29 -0000 Author: trasz Date: Thu Aug 2 07:43:28 2018 New Revision: 337067 URL: https://svnweb.freebsd.org/changeset/base/337067 Log: Make sure the rtld(1) error messages go to stderr, not stdout. While here fix capitalization of a few nearby strings, add the rtld's file name prefix so it's obvious where the message come from, and return zero when "-h" is used. Reviewed by: kib MFC after: 2 weeks Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D16530 Modified: head/libexec/rtld-elf/malloc.c head/libexec/rtld-elf/paths.h head/libexec/rtld-elf/rtld.c Modified: head/libexec/rtld-elf/malloc.c ============================================================================== --- head/libexec/rtld-elf/malloc.c Thu Aug 2 06:26:51 2018 (r337066) +++ head/libexec/rtld-elf/malloc.c Thu Aug 2 07:43:28 2018 (r337067) @@ -47,6 +47,7 @@ static char *rcsid = "$FreeBSD$"; #include #include +#include #include #include #include @@ -55,7 +56,9 @@ static char *rcsid = "$FreeBSD$"; #include #include #include +#include "rtld.h" #include "rtld_printf.h" +#include "paths.h" static void morecore(); static int findbucket(); @@ -472,9 +475,11 @@ int n; if (pagepool_end - pagepool_start > pagesz) { caddr_t addr = (caddr_t) (((long)pagepool_start + pagesz - 1) & ~(pagesz - 1)); - if (munmap(addr, pagepool_end - addr) != 0) - rtld_fdprintf(STDERR_FILENO, "morepages: munmap %p", - addr); + if (munmap(addr, pagepool_end - addr) != 0) { + rtld_fdprintf(STDERR_FILENO, _BASENAME_RTLD ": " + "morepages: cannot munmap %p: %s\n", + addr, rtld_strerror(errno)); + } } offset = (long)pagepool_start - ((long)pagepool_start & ~(pagesz - 1)); @@ -482,7 +487,9 @@ int n; if ((pagepool_start = mmap(0, n * pagesz, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, fd, 0)) == (caddr_t)-1) { - rtld_printf("Cannot map anonymous memory\n"); + rtld_fdprintf(STDERR_FILENO, _BASENAME_RTLD ": morepages: " + "cannot mmap anonymous memory: %s\n", + rtld_strerror(errno)); return 0; } pagepool_end = pagepool_start + n * pagesz; Modified: head/libexec/rtld-elf/paths.h ============================================================================== --- head/libexec/rtld-elf/paths.h Thu Aug 2 06:26:51 2018 (r337066) +++ head/libexec/rtld-elf/paths.h Thu Aug 2 07:43:28 2018 (r337067) @@ -34,7 +34,7 @@ #ifdef COMPAT_32BIT #define _PATH_ELF_HINTS "/var/run/ld-elf32.so.hints" #define _PATH_LIBMAP_CONF "/etc/libmap32.conf" -#define _PATH_RTLD "/libexec/ld-elf32.so.1" +#define _BASENAME_RTLD "ld-elf32.so.1" #define STANDARD_LIBRARY_PATH "/lib32:/usr/lib32" #define LD_ "LD_32_" #endif @@ -47,8 +47,12 @@ #define _PATH_LIBMAP_CONF "/etc/libmap.conf" #endif +#ifndef _BASENAME_RTLD +#define _BASENAME_RTLD "ld-elf.so.1" +#endif + #ifndef _PATH_RTLD -#define _PATH_RTLD "/libexec/ld-elf.so.1" +#define _PATH_RTLD "/libexec/" _BASENAME_RTLD #endif #ifndef STANDARD_LIBRARY_PATH Modified: head/libexec/rtld-elf/rtld.c ============================================================================== --- head/libexec/rtld-elf/rtld.c Thu Aug 2 06:26:51 2018 (r337066) +++ head/libexec/rtld-elf/rtld.c Thu Aug 2 07:43:28 2018 (r337067) @@ -408,8 +408,8 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entr phdr = (const Elf_Phdr *)aux_info[AT_PHDR]->a_un.a_ptr; if (phdr == obj_rtld.phdr) { if (!trust) { - rtld_printf("Tainted process refusing to run binary %s\n", - argv0); + _rtld_error("Tainted process refusing to run binary %s", + argv0); rtld_die(); } dbg("opening main program in direct exec mode"); @@ -420,7 +420,7 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entr if (!explicit_fd) fd = open_binary_fd(argv0, search_in_path); if (fstat(fd, &st) == -1) { - _rtld_error("failed to fstat FD %d (%s): %s", fd, + _rtld_error("Failed to fstat FD %d (%s): %s", fd, explicit_fd ? "user-provided descriptor" : argv0, rtld_strerror(errno)); rtld_die(); @@ -447,8 +447,8 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entr dir_enable = true; } if (!dir_enable) { - rtld_printf("No execute permission for binary %s\n", - argv0); + _rtld_error("No execute permission for binary %s", + argv0); rtld_die(); } @@ -477,7 +477,7 @@ _rtld(Elf_Addr *sp, func_ptr_type *exit_proc, Obj_Entr break; } } else { - rtld_printf("no binary\n"); + _rtld_error("No binary"); rtld_die(); } } @@ -962,6 +962,7 @@ rtld_die(void) if (msg == NULL) msg = "Fatal error"; + rtld_fdputstr(STDERR_FILENO, _BASENAME_RTLD ": "); rtld_fdputstr(STDERR_FILENO, msg); rtld_fdputchar(STDERR_FILENO, '\n'); _exit(1); @@ -2441,7 +2442,7 @@ do_load_object(int fd, const char *name, char *path, s return NULL; } if (fs.f_flags & MNT_NOEXEC) { - _rtld_error("Cannot execute objects on %s\n", fs.f_mntonname); + _rtld_error("Cannot execute objects on %s", fs.f_mntonname); return NULL; } } @@ -5317,12 +5318,12 @@ open_binary_fd(const char *argv0, bool search_in_path) if (search_in_path && strchr(argv0, '/') == NULL) { pathenv = getenv("PATH"); if (pathenv == NULL) { - rtld_printf("-p and no PATH environment variable\n"); + _rtld_error("-p and no PATH environment variable"); rtld_die(); } pathenv = strdup(pathenv); if (pathenv == NULL) { - rtld_printf("Cannot allocate memory\n"); + _rtld_error("Cannot allocate memory"); rtld_die(); } fd = -1; @@ -5348,8 +5349,7 @@ open_binary_fd(const char *argv0, bool search_in_path) } if (fd == -1) { - rtld_printf("Opening %s: %s\n", argv0, - rtld_strerror(errno)); + _rtld_error("Cannot open %s: %s", argv0, rtld_strerror(errno)); rtld_die(); } return (fd); @@ -5393,7 +5393,7 @@ parse_args(char* argv[], int argc, bool *use_pathp, in opt = arg[j]; if (opt == 'h') { print_usage(argv[0]); - rtld_die(); + _exit(0); } else if (opt == 'f') { /* * -f XX can be used to specify a descriptor for the @@ -5403,13 +5403,13 @@ parse_args(char* argv[], int argc, bool *use_pathp, in */ if (j != arglen - 1) { /* -f must be the last option in, e.g., -abcf */ - _rtld_error("invalid options: %s", arg); + _rtld_error("Invalid options: %s", arg); rtld_die(); } i++; fd = parse_integer(argv[i]); if (fd == -1) { - _rtld_error("invalid file descriptor: '%s'", + _rtld_error("Invalid file descriptor: '%s'", argv[i]); rtld_die(); } @@ -5418,7 +5418,7 @@ parse_args(char* argv[], int argc, bool *use_pathp, in } else if (opt == 'p') { *use_pathp = true; } else { - rtld_printf("invalid argument: '%s'\n", arg); + _rtld_error("Invalid argument: '%s'", arg); print_usage(argv[0]); rtld_die(); } From owner-svn-src-all@freebsd.org Thu Aug 2 08:04:22 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B8A6105CBF2; Thu, 2 Aug 2018 08:04:22 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3082379069; Thu, 2 Aug 2018 08:04:22 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 114236F3C; Thu, 2 Aug 2018 08:04:22 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7284L1N033603; Thu, 2 Aug 2018 08:04:21 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7284LIq033602; Thu, 2 Aug 2018 08:04:21 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020804.w7284LIq033602@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:04:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337068 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337068 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:04:22 -0000 Author: hselasky Date: Thu Aug 2 08:04:21 2018 New Revision: 337068 URL: https://svnweb.freebsd.org/changeset/base/337068 Log: MFC r335094 and r335123: Revert r335094 and properly fix OFED build after r335053. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c Thu Aug 2 07:43:28 2018 (r337067) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c Thu Aug 2 08:04:21 2018 (r337068) @@ -130,7 +130,7 @@ struct ib_umad_packet { static struct class *umad_class; -static const dev_t base_dev = MKDEV(IB_UMAD_MAJOR, IB_UMAD_MINOR_BASE); +#define base_dev MKDEV(IB_UMAD_MAJOR, IB_UMAD_MINOR_BASE) static DEFINE_SPINLOCK(port_lock); static DECLARE_BITMAP(dev_map, IB_UMAD_MAX_PORTS); From owner-svn-src-all@freebsd.org Thu Aug 2 08:05:21 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8B37105CC85; Thu, 2 Aug 2018 08:05:20 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DCD6791AF; Thu, 2 Aug 2018 08:05:20 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 807856F3D; Thu, 2 Aug 2018 08:05:20 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7285KdG033715; Thu, 2 Aug 2018 08:05:20 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7285KXt033714; Thu, 2 Aug 2018 08:05:20 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020805.w7285KXt033714@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:05:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337069 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337069 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:05:21 -0000 Author: hselasky Date: Thu Aug 2 08:05:20 2018 New Revision: 337069 URL: https://svnweb.freebsd.org/changeset/base/337069 Log: MFC r336363: Process address resolve requests at least one time per second in ibcore. When setting a large address resolve timeout it was observed that the address resolving would succeed at the timeout and not when the address was available. Make sure the address resolving requests are processed no slower than one time every second. While at it use "int" for jiffies instead of "unsigned long" to match FreeBSD ticks. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:04:21 2018 (r337068) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:05:20 2018 (r337069) @@ -63,7 +63,7 @@ struct addr_req { void *context; void (*callback)(int status, struct sockaddr *src_addr, struct rdma_dev_addr *addr, void *context); - unsigned long timeout; + int timeout; int status; }; @@ -190,13 +190,15 @@ int rdma_translate_ip(const struct sockaddr *addr, } EXPORT_SYMBOL(rdma_translate_ip); -static void set_timeout(unsigned long time) +static void set_timeout(int time) { int delay; /* under FreeBSD ticks are 32-bit */ delay = time - jiffies; if (delay <= 0) delay = 1; + else if (delay > hz) + delay = hz; mod_delayed_work(addr_wq, &work, delay); } From owner-svn-src-all@freebsd.org Thu Aug 2 08:06:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4409A105CD1C; Thu, 2 Aug 2018 08:06:18 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ECDB079329; Thu, 2 Aug 2018 08:06:17 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CF3A86F3E; Thu, 2 Aug 2018 08:06:17 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7286HaP033813; Thu, 2 Aug 2018 08:06:17 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7286HqW033812; Thu, 2 Aug 2018 08:06:17 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020806.w7286HqW033812@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:06:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337070 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337070 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:06:18 -0000 Author: hselasky Date: Thu Aug 2 08:06:17 2018 New Revision: 337070 URL: https://svnweb.freebsd.org/changeset/base/337070 Log: MFC r336364: Only update source address when resolving is successful in ibcore. When resolving an IP address in ibcore, only update the source address upon normal completion. The ibcore address resolve function does not care about the scope ID value of the IPv6 link-local addresses and expects this information has already been extracted into the bound_dev_if field. Because the same IPv6 link-local address can exist on multiple interfaces the ibcore address resolver gets confused and returns ENETUNREACH. Instead of updating both source address and bound_dev_if just keep the address set to any address until resolving completes. For the sake of code symmetry a similar change has been applied to the IPv4 address resolve path. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:05:20 2018 (r337069) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:06:17 2018 (r337070) @@ -257,7 +257,7 @@ static int addr4_resolve(struct sockaddr_in *src_in, }; struct sockaddr_in dst_tmp = *dst_in; in_port_t src_port; - struct sockaddr *saddr; + struct sockaddr *saddr = NULL; struct rtentry *rte; struct ifnet *ifp; int error; @@ -333,11 +333,6 @@ static int addr4_resolve(struct sockaddr_in *src_in, /* get destination network interface from route */ ifp = rte->rt_ifp; dev_hold(ifp); - - /* update source address */ - src_port = src_in->sin_port; - memcpy(src_in, saddr, rdma_addr_size(saddr)); - src_in->sin_port = src_port; /* preserve port number */ break; default: break; @@ -369,6 +364,15 @@ static int addr4_resolve(struct sockaddr_in *src_in, addr->network = RDMA_NETWORK_IPV4; } + /* + * Step 4 - update source address, if any + */ + if (saddr != NULL) { + src_port = src_in->sin_port; + memcpy(src_in, saddr, rdma_addr_size(saddr)); + src_in->sin_port = src_port; /* preserve port number */ + } + if (rte != NULL) RTFREE(rte); @@ -412,7 +416,7 @@ static int addr6_resolve(struct sockaddr_in6 *src_in, }; struct sockaddr_in6 dst_tmp = *dst_in; in_port_t src_port; - struct sockaddr *saddr; + struct sockaddr *saddr = NULL; struct rtentry *rte; struct ifnet *ifp; int error; @@ -502,10 +506,6 @@ static int addr6_resolve(struct sockaddr_in6 *src_in, /* get destination network interface from route */ ifp = rte->rt_ifp; dev_hold(ifp); - - src_port = src_in->sin6_port; - memcpy(src_in, saddr, rdma_addr_size(saddr)); - src_in->sin6_port = src_port; /* preserve port number */ break; default: break; @@ -532,6 +532,15 @@ static int addr6_resolve(struct sockaddr_in6 *src_in, goto error_put_ifp; else if (is_gw != 0) addr->network = RDMA_NETWORK_IPV6; + } + + /* + * Step 4 - update source address, if any + */ + if (saddr != NULL) { + src_port = src_in->sin6_port; + memcpy(src_in, saddr, rdma_addr_size(saddr)); + src_in->sin6_port = src_port; /* preserve port number */ } if (rte != NULL) From owner-svn-src-all@freebsd.org Thu Aug 2 08:07:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04768105CDF9; Thu, 2 Aug 2018 08:07:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ADDA479490; Thu, 2 Aug 2018 08:07:11 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8F2576F42; Thu, 2 Aug 2018 08:07:11 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7287BAL033910; Thu, 2 Aug 2018 08:07:11 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7287Bl4033909; Thu, 2 Aug 2018 08:07:11 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020807.w7287Bl4033909@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:07:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337071 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337071 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:07:12 -0000 Author: hselasky Date: Thu Aug 2 08:07:10 2018 New Revision: 337071 URL: https://svnweb.freebsd.org/changeset/base/337071 Log: MFC r336365: Add lock to multicast handlers in ibcore. When two handlers used the same object in the old schema, we blocked the process in the kernel. The new schema just returns -EBUSY. This could lead to different behaviour in applications between the old schema and the new schema. In most cases, using such handlers concurrently could lead to crashing the process. For example, if thread A destroys a QP and thread B modifies it, we could have the destruction happens before the modification. In this case, we are accessing freed memory which could lead to crashing the process. This is true for most cases. However, attaching and detaching a multicast address from QP concurrently is safe. Therefore, we preserve the original behaviour by adding a lock there. Linux commit: f48b726920d96dcd1860df06143bdea7d6d7dcc3 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c stable/11/sys/ofed/drivers/infiniband/core/uverbs.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:06:17 2018 (r337070) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:07:10 2018 (r337071) @@ -1794,6 +1794,7 @@ static int create_qp(struct ib_uverbs_file *file, init_uobj(&obj->uevent.uobject, cmd->user_handle, file->ucontext, &qp_lock_class); + mutex_init(&obj->mcast_lock); down_write(&obj->uevent.uobject.mutex); if (cmd_sz >= offsetof(typeof(*cmd), rwq_ind_tbl_handle) + sizeof(cmd->rwq_ind_tbl_handle) && @@ -3030,6 +3031,7 @@ ssize_t ib_uverbs_attach_mcast(struct ib_uverbs_file * obj = container_of(qp->uobject, struct ib_uqp_object, uevent.uobject); + mutex_lock(&obj->mcast_lock); list_for_each_entry(mcast, &obj->mcast_list, list) if (cmd.mlid == mcast->lid && !memcmp(cmd.gid, mcast->gid.raw, sizeof mcast->gid.raw)) { @@ -3053,6 +3055,7 @@ ssize_t ib_uverbs_attach_mcast(struct ib_uverbs_file * kfree(mcast); out_put: + mutex_unlock(&obj->mcast_lock); put_qp_write(qp); return ret ? ret : in_len; @@ -3076,12 +3079,13 @@ ssize_t ib_uverbs_detach_mcast(struct ib_uverbs_file * if (!qp) return -EINVAL; + obj = container_of(qp->uobject, struct ib_uqp_object, uevent.uobject); + mutex_lock(&obj->mcast_lock); + ret = ib_detach_mcast(qp, (union ib_gid *) cmd.gid, cmd.mlid); if (ret) goto out_put; - obj = container_of(qp->uobject, struct ib_uqp_object, uevent.uobject); - list_for_each_entry(mcast, &obj->mcast_list, list) if (cmd.mlid == mcast->lid && !memcmp(cmd.gid, mcast->gid.raw, sizeof mcast->gid.raw)) { @@ -3091,6 +3095,7 @@ ssize_t ib_uverbs_detach_mcast(struct ib_uverbs_file * } out_put: + mutex_unlock(&obj->mcast_lock); put_qp_write(qp); return ret ? ret : in_len; Modified: stable/11/sys/ofed/drivers/infiniband/core/uverbs.h ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/uverbs.h Thu Aug 2 08:06:17 2018 (r337070) +++ stable/11/sys/ofed/drivers/infiniband/core/uverbs.h Thu Aug 2 08:07:10 2018 (r337071) @@ -164,6 +164,8 @@ struct ib_usrq_object { struct ib_uqp_object { struct ib_uevent_object uevent; + /* lock for mcast list */ + struct mutex mcast_lock; struct list_head mcast_list; struct ib_uxrcd_object *uxrcd; }; From owner-svn-src-all@freebsd.org Thu Aug 2 08:08:03 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79C06105CEA8; Thu, 2 Aug 2018 08:08:03 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2EF7679623; Thu, 2 Aug 2018 08:08:03 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 107076F43; Thu, 2 Aug 2018 08:08:03 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72882ds034002; Thu, 2 Aug 2018 08:08:02 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728827r034001; Thu, 2 Aug 2018 08:08:02 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020808.w728827r034001@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:08:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337072 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337072 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:08:03 -0000 Author: hselasky Date: Thu Aug 2 08:08:02 2018 New Revision: 337072 URL: https://svnweb.freebsd.org/changeset/base/337072 Log: MFC r336366: If the MGID/MLID pair is not on the list return an error in ibcore. A list of MGID/MLID pairs is built when doing a multicast attach. When the multicast detach is called, the list is searched, and regardless of the search outcome, the driver detach is called. If an MGID/MLID pair is not on the list, driver detach should not be called, and an error should be returned. Calling the driver without removing an MGID/MLID pair from the list can leave the core and driver out of sync. Linux commit: 20c7840a77ddcb2ed2fbd66e8197db2868495751 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:07:10 2018 (r337071) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:08:02 2018 (r337072) @@ -3071,6 +3071,7 @@ ssize_t ib_uverbs_detach_mcast(struct ib_uverbs_file * struct ib_qp *qp; struct ib_uverbs_mcast_entry *mcast; int ret = -EINVAL; + bool found = false; if (copy_from_user(&cmd, buf, sizeof cmd)) return -EFAULT; @@ -3082,17 +3083,21 @@ ssize_t ib_uverbs_detach_mcast(struct ib_uverbs_file * obj = container_of(qp->uobject, struct ib_uqp_object, uevent.uobject); mutex_lock(&obj->mcast_lock); - ret = ib_detach_mcast(qp, (union ib_gid *) cmd.gid, cmd.mlid); - if (ret) - goto out_put; - list_for_each_entry(mcast, &obj->mcast_list, list) if (cmd.mlid == mcast->lid && !memcmp(cmd.gid, mcast->gid.raw, sizeof mcast->gid.raw)) { list_del(&mcast->list); kfree(mcast); + found = true; break; } + + if (!found) { + ret = -EINVAL; + goto out_put; + } + + ret = ib_detach_mcast(qp, (union ib_gid *)cmd.gid, cmd.mlid); out_put: mutex_unlock(&obj->mcast_lock); From owner-svn-src-all@freebsd.org Thu Aug 2 08:10:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50980105CFD6; Thu, 2 Aug 2018 08:10:07 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 06578797DF; Thu, 2 Aug 2018 08:10:07 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DCFDF6F48; Thu, 2 Aug 2018 08:10:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728A6Xw034171; Thu, 2 Aug 2018 08:10:06 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728A629034169; Thu, 2 Aug 2018 08:10:06 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020810.w728A629034169@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:10:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337073 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337073 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:10:07 -0000 Author: hselasky Date: Thu Aug 2 08:10:06 2018 New Revision: 337073 URL: https://svnweb.freebsd.org/changeset/base/337073 Log: MFC r336367: Add native FreeBSD support for multicast in ibcore. This change adds support for registering multicast addresses, both IPv4 and IPv6. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:08:02 2018 (r337072) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:10:06 2018 (r337073) @@ -346,9 +346,12 @@ static int addr4_resolve(struct sockaddr_in *src_in, ifp->if_addrlen, MAX_ADDR_LEN); error = 0; } else if (IN_MULTICAST(ntohl(dst_tmp.sin_addr.s_addr))) { + bool is_gw = (rte->rt_flags & RTF_GATEWAY) != 0; error = addr_resolve_multi(edst, ifp, (struct sockaddr *)&dst_tmp); if (error != 0) goto error_put_ifp; + else if (is_gw) + addr->network = RDMA_NETWORK_IPV4; } else if (ifp->if_flags & IFF_LOOPBACK) { memset(edst, 0, MAX_ADDR_LEN); error = 0; @@ -360,7 +363,7 @@ static int addr4_resolve(struct sockaddr_in *src_in, edst, NULL, NULL); if (error != 0) goto error_put_ifp; - else if (is_gw != 0) + else if (is_gw) addr->network = RDMA_NETWORK_IPV4; } @@ -515,10 +518,13 @@ static int addr6_resolve(struct sockaddr_in6 *src_in, * Step 3 - resolve destination MAC address */ if (IN6_IS_ADDR_MULTICAST(&dst_tmp.sin6_addr)) { + bool is_gw = (rte->rt_flags & RTF_GATEWAY) != 0; error = addr_resolve_multi(edst, ifp, (struct sockaddr *)&dst_tmp); if (error != 0) goto error_put_ifp; + else if (is_gw) + addr->network = RDMA_NETWORK_IPV6; } else if (rte->rt_ifp->if_flags & IFF_LOOPBACK) { memset(edst, 0, MAX_ADDR_LEN); error = 0; @@ -530,7 +536,7 @@ static int addr6_resolve(struct sockaddr_in6 *src_in, edst, NULL, NULL); if (error != 0) goto error_put_ifp; - else if (is_gw != 0) + else if (is_gw) addr->network = RDMA_NETWORK_IPV6; } Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:08:02 2018 (r337072) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:10:06 2018 (r337073) @@ -427,6 +427,30 @@ static inline void cma_set_ip_ver(struct cma_hdr *hdr, hdr->ip_version = (ip_ver << 4) | (hdr->ip_version & 0xF); } +static int cma_igmp_send(struct net_device *ndev, const union ib_gid *mgid, bool join) +{ + int retval; + + if (ndev) { + union { + struct sockaddr sock; + struct sockaddr_storage storage; + } addr; + + rdma_gid2ip(&addr.sock, mgid); + + CURVNET_SET_QUIET(ndev->if_vnet); + if (join) + retval = -if_addmulti(ndev, &addr.sock, NULL); + else + retval = -if_delmulti(ndev, &addr.sock); + CURVNET_RESTORE(); + } else { + retval = -ENODEV; + } + return retval; +} + static void _cma_attach_to_dev(struct rdma_id_private *id_priv, struct cma_device *cma_dev) { @@ -1621,6 +1645,9 @@ static void cma_leave_mc_groups(struct rdma_id_private ndev = dev_get_by_index(dev_addr->net, dev_addr->bound_dev_if); if (ndev) { + cma_igmp_send(ndev, + &mc->multicast.ib->rec.mgid, + false); dev_put(ndev); } } @@ -4024,7 +4051,10 @@ static int cma_iboe_join_multicast(struct rdma_id_priv if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP) { mc->multicast.ib->rec.hop_limit = IPV6_DEFAULT_HOPLIMIT; if (!send_only) { - mc->igmp_joined = true; + err = cma_igmp_send(ndev, &mc->multicast.ib->rec.mgid, + true); + if (!err) + mc->igmp_joined = true; } } } else { @@ -4129,6 +4159,9 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struc ndev = dev_get_by_index(dev_addr->net, dev_addr->bound_dev_if); if (ndev) { + cma_igmp_send(ndev, + &mc->multicast.ib->rec.mgid, + false); dev_put(ndev); } mc->igmp_joined = false; From owner-svn-src-all@freebsd.org Thu Aug 2 08:10:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88C13105D057; Thu, 2 Aug 2018 08:10:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E37E7996D; Thu, 2 Aug 2018 08:10:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1F6FC6F6C; Thu, 2 Aug 2018 08:10:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728Atw3034262; Thu, 2 Aug 2018 08:10:55 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728AsXX034259; Thu, 2 Aug 2018 08:10:54 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020810.w728AsXX034259@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:10:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337074 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337074 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:10:55 -0000 Author: hselasky Date: Thu Aug 2 08:10:54 2018 New Revision: 337074 URL: https://svnweb.freebsd.org/changeset/base/337074 Log: MFC r336368: Fix for RDMA loopback over VLAN in ibcore. Implement a more generic solution for detecting loopback. The problem was that the default netdevice was resolved for loopback also when VLAN was used. Use real network device instead of loopback device for bound device interface. How to test: ucmatose -b 127.0.0.1 -p 20090 ucmatose -s 5.6.5.1 -p 20090 Note that RDMA treats the IPv4 and IPv6 loopback addresses like any address. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:10:06 2018 (r337073) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:10:54 2018 (r337074) @@ -124,7 +124,7 @@ int rdma_copy_addr(struct rdma_dev_addr *dev_addr, str const unsigned char *dst_dev_addr) { /* check for loopback device */ - if (dev->if_type == IFT_LOOP) { + if (dev->if_flags & IFF_LOOPBACK) { dev_addr->dev_type = ARPHRD_ETHER; memset(dev_addr->src_dev_addr, 0, MAX_ADDR_LEN); memset(dev_addr->broadcast, 0, MAX_ADDR_LEN); @@ -153,19 +153,12 @@ EXPORT_SYMBOL(rdma_copy_addr); int rdma_translate_ip(const struct sockaddr *addr, struct rdma_dev_addr *dev_addr) { - struct net_device *dev = NULL; - int ret = -EADDRNOTAVAIL; + struct net_device *dev; + int ret; if (dev_addr->bound_dev_if) { dev = dev_get_by_index(dev_addr->net, dev_addr->bound_dev_if); - if (!dev) - return -ENODEV; - ret = rdma_copy_addr(dev_addr, dev, NULL); - dev_put(dev); - return ret; - } - - switch (addr->sa_family) { + } else switch (addr->sa_family) { #ifdef INET case AF_INET: dev = ip_dev_find(dev_addr->net, @@ -179,12 +172,19 @@ int rdma_translate_ip(const struct sockaddr *addr, break; #endif default: + dev = NULL; break; } if (dev != NULL) { - ret = rdma_copy_addr(dev_addr, dev, NULL); + /* disallow connections through 127.0.0.1 itself */ + if (dev->if_flags & IFF_LOOPBACK) + ret = -EINVAL; + else + ret = rdma_copy_addr(dev_addr, dev, NULL); dev_put(dev); + } else { + ret = -ENODEV; } return ret; } @@ -305,20 +305,39 @@ static int addr4_resolve(struct sockaddr_in *src_in, /* Step 2 - find outgoing network interface */ switch (type) { case ADDR_VALID: - /* check for loopback device */ - if (rte->rt_ifp->if_flags & IFF_LOOPBACK) { - ifp = rte->rt_ifp; - dev_hold(ifp); - } else if (addr->bound_dev_if != 0) { + /* get source interface */ + if (addr->bound_dev_if != 0) { ifp = dev_get_by_index(addr->net, addr->bound_dev_if); } else { ifp = ip_dev_find(addr->net, src_in->sin_addr.s_addr); } + /* check source interface */ if (ifp == NULL) { error = ENETUNREACH; goto error_rt_free; + } else if (ifp->if_flags & IFF_LOOPBACK) { + /* + * Source address cannot be a loopback device. + */ + error = EHOSTUNREACH; + goto error_put_ifp; + } else if (rte->rt_ifp->if_flags & IFF_LOOPBACK) { + if (memcmp(&src_in->sin_addr, &dst_in->sin_addr, + sizeof(src_in->sin_addr))) { + /* + * Destination is loopback, but source + * and destination address is not the + * same. + */ + error = EHOSTUNREACH; + goto error_put_ifp; + } } else if (ifp != rte->rt_ifp) { + /* + * Source and destination interfaces are + * different. + */ error = ENETUNREACH; goto error_put_ifp; } @@ -481,20 +500,39 @@ static int addr6_resolve(struct sockaddr_in6 *src_in, /* Step 2 - find outgoing network interface */ switch (type) { case ADDR_VALID: - /* check for loopback device */ - if (rte->rt_ifp->if_flags & IFF_LOOPBACK) { - ifp = rte->rt_ifp; - dev_hold(ifp); - } else if (addr->bound_dev_if != 0) { + /* get source interface */ + if (addr->bound_dev_if != 0) { ifp = dev_get_by_index(addr->net, addr->bound_dev_if); } else { ifp = ip6_dev_find(addr->net, src_in->sin6_addr); } + /* check source interface */ if (ifp == NULL) { error = ENETUNREACH; goto error_rt_free; + } else if (ifp->if_flags & IFF_LOOPBACK) { + /* + * Source address cannot be a loopback device. + */ + error = EHOSTUNREACH; + goto error_put_ifp; + } else if (rte->rt_ifp->if_flags & IFF_LOOPBACK) { + if (memcmp(&src_in->sin6_addr, &dst_in->sin6_addr, + sizeof(src_in->sin6_addr))) { + /* + * Destination is loopback, but source + * and destination address is not the + * same. + */ + error = EHOSTUNREACH; + goto error_put_ifp; + } } else if (ifp != rte->rt_ifp) { + /* + * Source and destination interfaces are + * different. + */ error = ENETUNREACH; goto error_put_ifp; } @@ -586,11 +624,14 @@ static int addr_resolve_neigh(struct ifnet *dev, if (dev->if_flags & IFF_LOOPBACK) { int ret; + /* find real device, not loopback one */ + addr->bound_dev_if = 0; + ret = rdma_translate_ip(dst_in, addr); - if (!ret) + if (ret == 0) { memcpy(addr->dst_dev_addr, addr->src_dev_addr, MAX_ADDR_LEN); - + } return ret; } @@ -603,8 +644,7 @@ static int addr_resolve_neigh(struct ifnet *dev, static int addr_resolve(struct sockaddr *src_in, const struct sockaddr *dst_in, - struct rdma_dev_addr *addr, - bool resolve_neigh) + struct rdma_dev_addr *addr) { struct net_device *ndev = NULL; u8 edst[MAX_ADDR_LEN]; @@ -613,27 +653,30 @@ static int addr_resolve(struct sockaddr *src_in, if (dst_in->sa_family != src_in->sa_family) return -EINVAL; - if (src_in->sa_family == AF_INET) { + switch (src_in->sa_family) { + case AF_INET: ret = addr4_resolve((struct sockaddr_in *)src_in, (const struct sockaddr_in *)dst_in, addr, edst, &ndev); - if (ret) - return ret; - - if (resolve_neigh) - ret = addr_resolve_neigh(ndev, dst_in, edst, addr); - } else { + break; + case AF_INET6: ret = addr6_resolve((struct sockaddr_in6 *)src_in, (const struct sockaddr_in6 *)dst_in, addr, edst, &ndev); - if (ret) - return ret; - - if (resolve_neigh) - ret = addr_resolve_neigh(ndev, dst_in, edst, addr); + break; + default: + ret = -EADDRNOTAVAIL; + break; } - addr->bound_dev_if = ndev->if_index; + /* check for error */ + if (ret != 0) + return ret; + + /* store MAC addresses and check for loopback */ + ret = addr_resolve_neigh(ndev, dst_in, edst, addr); + + /* set belonging VNET, if any */ addr->net = dev_net(ndev); dev_put(ndev); @@ -653,8 +696,7 @@ static void process_req(struct work_struct *work) if (req->status == -ENODATA) { src_in = (struct sockaddr *) &req->src_addr; dst_in = (struct sockaddr *) &req->dst_addr; - req->status = addr_resolve(src_in, dst_in, req->addr, - true); + req->status = addr_resolve(src_in, dst_in, req->addr); if (req->status && time_after_eq(jiffies, req->timeout)) req->status = -ETIMEDOUT; else if (req->status == -ENODATA) @@ -714,7 +756,7 @@ int rdma_resolve_ip(struct rdma_addr_client *client, req->client = client; atomic_inc(&client->refcount); - req->status = addr_resolve(src_in, dst_in, addr, true); + req->status = addr_resolve(src_in, dst_in, addr); switch (req->status) { case 0: req->timeout = jiffies; @@ -752,7 +794,7 @@ int rdma_resolve_ip_route(struct sockaddr *src_addr, src_in->sa_family = dst_addr->sa_family; } - return addr_resolve(src_in, dst_addr, addr, false); + return addr_resolve(src_in, dst_addr, addr); } EXPORT_SYMBOL(rdma_resolve_ip_route); Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:10:06 2018 (r337073) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:10:54 2018 (r337074) @@ -568,12 +568,12 @@ static int cma_translate_addr(struct sockaddr *addr, s static inline int cma_validate_port(struct ib_device *device, u8 port, enum ib_gid_type gid_type, - union ib_gid *gid, int dev_type, - struct vnet *net, - int bound_if_index) + union ib_gid *gid, + const struct rdma_dev_addr *dev_addr) { + const int dev_type = dev_addr->dev_type; + struct net_device *ndev; int ret = -ENODEV; - struct net_device *ndev = NULL; if ((dev_type == ARPHRD_INFINIBAND) && !rdma_protocol_ib(device, port)) return ret; @@ -582,19 +582,9 @@ static inline int cma_validate_port(struct ib_device * return ret; if (dev_type == ARPHRD_ETHER && rdma_protocol_roce(device, port)) { - ndev = dev_get_by_index(net, bound_if_index); - if (ndev && ndev->if_flags & IFF_LOOPBACK) { - pr_info("detected loopback device\n"); - dev_put(ndev); - - if (!device->get_netdev) - return -EOPNOTSUPP; - - ndev = device->get_netdev(device, port); - if (!ndev) - return -ENODEV; - } + ndev = dev_get_by_index(dev_addr->net, dev_addr->bound_dev_if); } else { + ndev = NULL; gid_type = IB_GID_TYPE_IB; } @@ -636,10 +626,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_ ret = cma_validate_port(cma_dev->device, port, rdma_protocol_ib(cma_dev->device, port) ? IB_GID_TYPE_IB : - listen_id_priv->gid_type, gidp, - dev_addr->dev_type, - dev_addr->net, - dev_addr->bound_dev_if); + listen_id_priv->gid_type, gidp, dev_addr); if (!ret) { id_priv->id.port_num = port; goto out; @@ -660,9 +647,7 @@ static int cma_acquire_dev(struct rdma_id_private *id_ rdma_protocol_ib(cma_dev->device, port) ? IB_GID_TYPE_IB : cma_dev->default_gid_type[port - 1], - gidp, dev_addr->dev_type, - dev_addr->net, - dev_addr->bound_dev_if); + gidp, dev_addr); if (!ret) { id_priv->id.port_num = port; goto out; @@ -2521,21 +2506,6 @@ static int cma_resolve_iboe_route(struct rdma_id_priva if (!ndev) { ret = -ENODEV; goto err2; - } - - if (ndev->if_flags & IFF_LOOPBACK) { - dev_put(ndev); - if (!id_priv->id.device->get_netdev) { - ret = -EOPNOTSUPP; - goto err2; - } - - ndev = id_priv->id.device->get_netdev(id_priv->id.device, - id_priv->id.port_num); - if (!ndev) { - ret = -ENODEV; - goto err2; - } } route->path_rec->net = ndev->if_vnet; Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c Thu Aug 2 08:10:06 2018 (r337073) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c Thu Aug 2 08:10:54 2018 (r337074) @@ -696,10 +696,9 @@ int ib_init_ah_from_path(struct ib_device *device, u8 resolved_dev = dev_get_by_index(dev_addr.net, dev_addr.bound_dev_if); - if (resolved_dev->if_flags & IFF_LOOPBACK) { - dev_put(resolved_dev); - resolved_dev = idev; - dev_hold(resolved_dev); + if (!resolved_dev) { + dev_put(idev); + return -ENODEV; } ndev = ib_get_ndev_from_path(rec); rcu_read_lock(); From owner-svn-src-all@freebsd.org Thu Aug 2 08:12:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9536105D21B; Thu, 2 Aug 2018 08:12:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F00579C8E; Thu, 2 Aug 2018 08:12:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7032C6FBA; Thu, 2 Aug 2018 08:12:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728C1qe034396; Thu, 2 Aug 2018 08:12:01 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728C1LB034395; Thu, 2 Aug 2018 08:12:01 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020812.w728C1LB034395@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:12:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337075 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337075 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:12:02 -0000 Author: hselasky Date: Thu Aug 2 08:12:01 2018 New Revision: 337075 URL: https://svnweb.freebsd.org/changeset/base/337075 Log: MFC r336369: For multicast functions in ibcore, verify that LIDs are multicast LIDs. The Infiniband spec defines "A multicast address is defined by a MGID and a MLID" (section 10.5). Add check to verify that the MLID value is in the correct address range. RoCE Annex (A16.9.10/11) declares that during attach (detach) QP to a multicast group, if the QP is associated with a RoCE port, the multicast group MLID is unused and is ignored. During attach or detach multicast, when the QP is associated with a port, it is enough to check the port's link layer and validate the LID only if it is Infiniband. Otherwise, avoid validating the multicast LID. Linux commit: 8561eae60ff9417a50fa1fb2b83ae950dc5c1e21 5236333592244557a19694a51337df6ac018f0a7 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:10:54 2018 (r337074) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:12:01 2018 (r337075) @@ -1467,13 +1467,52 @@ EXPORT_SYMBOL(ib_dealloc_fmr); /* Multicast groups */ +static bool is_valid_mcast_lid(struct ib_qp *qp, u16 lid) +{ + struct ib_qp_init_attr init_attr = {}; + struct ib_qp_attr attr = {}; + int num_eth_ports = 0; + int port; + + /* If QP state >= init, it is assigned to a port and we can check this + * port only. + */ + if (!ib_query_qp(qp, &attr, IB_QP_STATE | IB_QP_PORT, &init_attr)) { + if (attr.qp_state >= IB_QPS_INIT) { + if (rdma_port_get_link_layer(qp->device, attr.port_num) != + IB_LINK_LAYER_INFINIBAND) + return true; + goto lid_check; + } + } + + /* Can't get a quick answer, iterate over all ports */ + for (port = 0; port < qp->device->phys_port_cnt; port++) + if (rdma_port_get_link_layer(qp->device, port) != + IB_LINK_LAYER_INFINIBAND) + num_eth_ports++; + + /* If we have at lease one Ethernet port, RoCE annex declares that + * multicast LID should be ignored. We can't tell at this step if the + * QP belongs to an IB or Ethernet port. + */ + if (num_eth_ports) + return true; + + /* If all the ports are IB, we can check according to IB spec. */ +lid_check: + return !(lid < be16_to_cpu(IB_MULTICAST_LID_BASE) || + lid == be16_to_cpu(IB_LID_PERMISSIVE)); +} + int ib_attach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid) { int ret; if (!qp->device->attach_mcast) return -ENOSYS; - if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD) + if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD || + !is_valid_mcast_lid(qp, lid)) return -EINVAL; ret = qp->device->attach_mcast(qp, gid, lid); @@ -1489,7 +1528,8 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gi if (!qp->device->detach_mcast) return -ENOSYS; - if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD) + if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD || + !is_valid_mcast_lid(qp, lid)) return -EINVAL; ret = qp->device->detach_mcast(qp, gid, lid); From owner-svn-src-all@freebsd.org Thu Aug 2 08:12:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72510105D2A0; Thu, 2 Aug 2018 08:12:53 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2837E79E42; Thu, 2 Aug 2018 08:12:53 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0965670EC; Thu, 2 Aug 2018 08:12:53 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728Cq5d039218; Thu, 2 Aug 2018 08:12:52 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728Cq44039215; Thu, 2 Aug 2018 08:12:52 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020812.w728Cq44039215@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:12:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337076 - in stable/11/sys/ofed: drivers/infiniband/core include/rdma X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/ofed: drivers/infiniband/core include/rdma X-SVN-Commit-Revision: 337076 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:12:53 -0000 Author: hselasky Date: Thu Aug 2 08:12:52 2018 New Revision: 337076 URL: https://svnweb.freebsd.org/changeset/base/337076 Log: MFC r336370: Set RoCEv2 MGID according to spec in ibcore. RoCEv2 Annex states that for RoCEv2 over IPv4, the corresponding IPv4 address is encoded into the GID according to the following rule: GID= :ffff: Remove the 0xff0e prefix for RoCEv2 packets with IPv4 and leave it zeroed and change rdma_is_multicast_addr() to consider the new logic. Linux commit: be1d325a335840a86c133a56c6a911c368bac0fd 1c3aea2bc8f0b2e5b57375ead40457ff75a3a2ec Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c stable/11/sys/ofed/include/rdma/ib_addr.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:12:01 2018 (r337075) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:12:52 2018 (r337076) @@ -3947,7 +3947,8 @@ static void iboe_mcast_work_handler(struct work_struct kfree(mw); } -static void cma_iboe_set_mgid(struct sockaddr *addr, union ib_gid *mgid) +static void cma_iboe_set_mgid(struct sockaddr *addr, union ib_gid *mgid, + enum ib_gid_type gid_type) { struct sockaddr_in *sin = (struct sockaddr_in *)addr; struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)addr; @@ -3957,8 +3958,8 @@ static void cma_iboe_set_mgid(struct sockaddr *addr, u } else if (addr->sa_family == AF_INET6) { memcpy(mgid, &sin6->sin6_addr, sizeof *mgid); } else { - mgid->raw[0] = 0xff; - mgid->raw[1] = 0x0e; + mgid->raw[0] = (gid_type == IB_GID_TYPE_IB) ? 0xff : 0; + mgid->raw[1] = (gid_type == IB_GID_TYPE_IB) ? 0x0e : 0; mgid->raw[2] = 0; mgid->raw[3] = 0; mgid->raw[4] = 0; @@ -3999,7 +4000,9 @@ static int cma_iboe_join_multicast(struct rdma_id_priv goto out1; } - cma_iboe_set_mgid(addr, &mc->multicast.ib->rec.mgid); + gid_type = id_priv->cma_dev->default_gid_type[id_priv->id.port_num - + rdma_start_port(id_priv->cma_dev->device)]; + cma_iboe_set_mgid(addr, &mc->multicast.ib->rec.mgid, gid_type); mc->multicast.ib->rec.pkey = cpu_to_be16(0xffff); if (id_priv->id.ps == RDMA_PS_UDP) @@ -4015,8 +4018,6 @@ static int cma_iboe_join_multicast(struct rdma_id_priv mc->multicast.ib->rec.hop_limit = 1; mc->multicast.ib->rec.mtu = iboe_get_mtu(ndev->if_mtu); - gid_type = id_priv->cma_dev->default_gid_type[id_priv->id.port_num - - rdma_start_port(id_priv->cma_dev->device)]; if (addr->sa_family == AF_INET) { if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP) { mc->multicast.ib->rec.hop_limit = IPV6_DEFAULT_HOPLIMIT; Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:12:01 2018 (r337075) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:12:52 2018 (r337076) @@ -1511,8 +1511,9 @@ int ib_attach_mcast(struct ib_qp *qp, union ib_gid *gi if (!qp->device->attach_mcast) return -ENOSYS; - if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD || - !is_valid_mcast_lid(qp, lid)) + + if (!rdma_is_multicast_addr((struct in6_addr *)gid->raw) || + qp->qp_type != IB_QPT_UD || !is_valid_mcast_lid(qp, lid)) return -EINVAL; ret = qp->device->attach_mcast(qp, gid, lid); @@ -1528,8 +1529,9 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gi if (!qp->device->detach_mcast) return -ENOSYS; - if (gid->raw[0] != 0xff || qp->qp_type != IB_QPT_UD || - !is_valid_mcast_lid(qp, lid)) + + if (!rdma_is_multicast_addr((struct in6_addr *)gid->raw) || + qp->qp_type != IB_QPT_UD || !is_valid_mcast_lid(qp, lid)) return -EINVAL; ret = qp->device->detach_mcast(qp, gid, lid); Modified: stable/11/sys/ofed/include/rdma/ib_addr.h ============================================================================== --- stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:12:01 2018 (r337075) +++ stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:12:52 2018 (r337076) @@ -315,7 +315,13 @@ static inline void rdma_get_ll_mac(struct in6_addr *ad static inline int rdma_is_multicast_addr(struct in6_addr *addr) { - return addr->s6_addr[0] == 0xff; + __be32 ipv4_addr; + + if (addr->s6_addr[0] == 0xff) + return 1; + + ipv4_addr = addr->s6_addr32[3]; + return (ipv6_addr_v4mapped(addr) && ipv4_is_multicast(ipv4_addr)); } static inline void rdma_get_mcast_mac(struct in6_addr *addr, u8 *mac) From owner-svn-src-all@freebsd.org Thu Aug 2 08:14:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26DAA105D34A; Thu, 2 Aug 2018 08:14:01 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD3B279FB1; Thu, 2 Aug 2018 08:14:00 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AFE1070ED; Thu, 2 Aug 2018 08:14:00 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728E0kQ039325; Thu, 2 Aug 2018 08:14:00 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728E0kZ039324; Thu, 2 Aug 2018 08:14:00 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020814.w728E0kZ039324@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:14:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337077 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337077 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:14:01 -0000 Author: hselasky Date: Thu Aug 2 08:14:00 2018 New Revision: 337077 URL: https://svnweb.freebsd.org/changeset/base/337077 Log: MFC r336371: Set default GID type as RoCE when resolving RoCE route in ibcore. cma_iboe_set_mgid() is updated to reflect the RoCEv2 GID check. Linux commit: 5c181bda77f409d89ad513528eccac5f3a416474 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:12:52 2018 (r337076) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:14:00 2018 (r337077) @@ -3958,8 +3958,10 @@ static void cma_iboe_set_mgid(struct sockaddr *addr, u } else if (addr->sa_family == AF_INET6) { memcpy(mgid, &sin6->sin6_addr, sizeof *mgid); } else { - mgid->raw[0] = (gid_type == IB_GID_TYPE_IB) ? 0xff : 0; - mgid->raw[1] = (gid_type == IB_GID_TYPE_IB) ? 0x0e : 0; + mgid->raw[0] = + (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP) ? 0 : 0xff; + mgid->raw[1] = + (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP) ? 0 : 0x0e; mgid->raw[2] = 0; mgid->raw[3] = 0; mgid->raw[4] = 0; From owner-svn-src-all@freebsd.org Thu Aug 2 08:15:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DEDA105D405; Thu, 2 Aug 2018 08:15:07 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2EF47A11B; Thu, 2 Aug 2018 08:15:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A50F770EF; Thu, 2 Aug 2018 08:15:06 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728F6Bf039443; Thu, 2 Aug 2018 08:15:06 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728F50o039436; Thu, 2 Aug 2018 08:15:05 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020815.w728F50o039436@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:15:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337078 - in stable/11/sys: dev/mlx4/mlx4_ib dev/mlx5/mlx5_ib ofed/drivers/infiniband/core ofed/include/rdma X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys: dev/mlx4/mlx4_ib dev/mlx5/mlx5_ib ofed/drivers/infiniband/core ofed/include/rdma X-SVN-Commit-Revision: 337078 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:15:07 -0000 Author: hselasky Date: Thu Aug 2 08:15:05 2018 New Revision: 337078 URL: https://svnweb.freebsd.org/changeset/base/337078 Log: MFC r336372: Add support for prio-tagged traffic for RDMA in ibcore. When receiving a PCP change all GID entries are reloaded. This ensures the relevant GID entries use prio tagging, by setting VLAN present and VLAN ID to zero. The priority for prio tagged traffic is set using the regular rdma_set_service_type() function. Fake the real network device to have a VLAN ID of zero when prio tagging is enabled. This is logic is hidden inside the rdma_vlan_dev_vlan_id() function which must always be used to retrieve the VLAN ID throughout all of ibcore and the infiniband network drivers. The VLAN presence information then propagates through all of ibcore and so incoming connections will have the VLAN bit set. The incoming VLAN ID is then checked against the return value of rdma_vlan_dev_vlan_id(). Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c stable/11/sys/ofed/include/rdma/ib_addr.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c ============================================================================== --- stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/dev/mlx4/mlx4_ib/mlx4_ib_ah.c Thu Aug 2 08:15:05 2018 (r337078) @@ -96,8 +96,7 @@ static struct ib_ah *create_iboe_ah(struct ib_pd *pd, return ERR_PTR(ret); eth_zero_addr(ah->av.eth.s_mac); if (gid_attr.ndev) { - if (is_vlan_dev(gid_attr.ndev)) - vlan_tag = vlan_dev_vlan_id(gid_attr.ndev); + vlan_tag = rdma_vlan_dev_vlan_id(gid_attr.ndev); memcpy(ah->av.eth.s_mac, IF_LLADDR(gid_attr.ndev), ETH_ALEN); dev_put(gid_attr.ndev); } Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:15:05 2018 (r337078) @@ -285,14 +285,16 @@ static void ib_gid_to_mlx5_roce_addr(const union ib_gi source_l3_address); void *mlx5_addr_mac = MLX5_ADDR_OF(roce_addr_layout, mlx5_addr, source_mac_47_32); + u16 vlan_id; if (!gid) return; ether_addr_copy(mlx5_addr_mac, IF_LLADDR(attr->ndev)); - if (is_vlan_dev(attr->ndev)) { + vlan_id = rdma_vlan_dev_vlan_id(attr->ndev); + if (vlan_id != 0xffff) { MLX5_SET_RA(mlx5_addr, vlan_valid, 1); - MLX5_SET_RA(mlx5_addr, vlan_id, vlan_dev_vlan_id(attr->ndev)); + MLX5_SET_RA(mlx5_addr, vlan_id, vlan_id); } switch (attr->gid_type) { Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c Thu Aug 2 08:15:05 2018 (r337078) @@ -402,7 +402,20 @@ static struct notifier_block nb_inetaddr = { .notifier_call = inetaddr_event }; +static eventhandler_tag eh_ifnet_event; + static void +roce_ifnet_event(void *arg, struct ifnet *ifp, int event) +{ + if (event != IFNET_EVENT_PCP || is_vlan_dev(ifp)) + return; + + /* make sure GID table is reloaded */ + roce_gid_delete_all_event(ifp); + roce_gid_queue_scan_event(ifp); +} + +static void roce_rescan_device_handler(struct work_struct *_work) { struct roce_rescan_work *work = @@ -445,11 +458,18 @@ int __init roce_gid_mgmt_init(void) */ register_netdevice_notifier(&nb_inetaddr); + eh_ifnet_event = EVENTHANDLER_REGISTER(ifnet_event, + roce_ifnet_event, NULL, EVENTHANDLER_PRI_ANY); + return 0; } void __exit roce_gid_mgmt_cleanup(void) { + + if (eh_ifnet_event != NULL) + EVENTHANDLER_DEREGISTER(ifnet_event, eh_ifnet_event); + unregister_inetaddr_notifier(&nb_inetaddr); unregister_netdevice_notifier(&nb_inetaddr); Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:15:05 2018 (r337078) @@ -402,12 +402,8 @@ static bool find_gid_index(const union ib_gid *gid, if (ctx->gid_type != gid_attr->gid_type) return false; - - if ((!!(ctx->vlan_id != 0xffff) == !is_vlan_dev(gid_attr->ndev)) || - (is_vlan_dev(gid_attr->ndev) && - vlan_dev_vlan_id(gid_attr->ndev) != ctx->vlan_id)) + if (rdma_vlan_dev_vlan_id(gid_attr->ndev) != ctx->vlan_id) return false; - return true; } @@ -484,7 +480,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 po if (rdma_protocol_roce(device, port_num)) { struct ib_gid_attr dgid_attr; - const u16 vlan_id = wc->wc_flags & IB_WC_WITH_VLAN ? + const u16 vlan_id = (wc->wc_flags & IB_WC_WITH_VLAN) ? wc->vlan_id : 0xffff; if (!(wc->wc_flags & IB_WC_GRH)) Modified: stable/11/sys/ofed/include/rdma/ib_addr.h ============================================================================== --- stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:14:00 2018 (r337077) +++ stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:15:05 2018 (r337078) @@ -163,6 +163,8 @@ static inline u16 rdma_vlan_dev_vlan_id(const struct n { uint16_t tag; + if (dev->if_pcp != IFNET_PCP_NONE) + return 0x0000; /* prio-tagged traffic */ if (VLAN_TAG(__DECONST(struct ifnet *, dev), &tag) != 0) return 0xffff; return tag; @@ -342,8 +344,10 @@ static inline u16 rdma_get_vlan_id(union ib_gid *dgid) return vid < 0x1000 ? vid : 0xffff; } -static inline struct net_device *rdma_vlan_dev_real_dev(const struct net_device *dev) +static inline struct net_device *rdma_vlan_dev_real_dev(struct net_device *dev) { + if (dev->if_pcp != IFNET_PCP_NONE) + return dev; /* prio-tagged traffic */ return VLAN_TRUNKDEV(__DECONST(struct ifnet *, dev)); } From owner-svn-src-all@freebsd.org Thu Aug 2 08:17:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A932105D4C3; Thu, 2 Aug 2018 08:17:10 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE6D17A288; Thu, 2 Aug 2018 08:17:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A899170F0; Thu, 2 Aug 2018 08:17:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728H9D8039579; Thu, 2 Aug 2018 08:17:09 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728H97S039578; Thu, 2 Aug 2018 08:17:09 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020817.w728H97S039578@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:17:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337079 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337079 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:17:10 -0000 Author: hselasky Date: Thu Aug 2 08:17:09 2018 New Revision: 337079 URL: https://svnweb.freebsd.org/changeset/base/337079 Log: MFC r336373: Ensure that CM_ID exists prior to access it in ibcore. Prior to access UCMA commands, the context should be initialized and connected to CM_ID with ucma_create_id(). In case user skips this step, he can provide non-valid ctx without CM_ID and cause to multiple NULL dereferences. Also there are situations where the create_id can be raced with other user access, ensure that the context is only shared to other threads once it is fully initialized to avoid the races. Linux commit: e8980d67d6017c8eee8f9c35f782c4bd68e004c9 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:15:05 2018 (r337078) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:17:09 2018 (r337079) @@ -124,7 +124,7 @@ static inline struct ucma_context *_ucma_find_context( ctx = idr_find(&ctx_idr, id); if (!ctx) ctx = ERR_PTR(-ENOENT); - else if (ctx->file != file) + else if (ctx->file != file || !ctx->cm_id) ctx = ERR_PTR(-EINVAL); return ctx; } @@ -446,6 +446,7 @@ static ssize_t ucma_create_id(struct ucma_file *file, struct rdma_ucm_create_id cmd; struct rdma_ucm_create_id_resp resp; struct ucma_context *ctx; + struct rdma_cm_id *cm_id; enum ib_qp_type qp_type; int ret; @@ -466,10 +467,10 @@ static ssize_t ucma_create_id(struct ucma_file *file, return -ENOMEM; ctx->uid = cmd.uid; - ctx->cm_id = rdma_create_id(TD_TO_VNET(curthread), - ucma_event_handler, ctx, cmd.ps, qp_type); - if (IS_ERR(ctx->cm_id)) { - ret = PTR_ERR(ctx->cm_id); + cm_id = rdma_create_id(TD_TO_VNET(curthread), + ucma_event_handler, ctx, cmd.ps, qp_type); + if (IS_ERR(cm_id)) { + ret = PTR_ERR(cm_id); goto err1; } @@ -479,10 +480,12 @@ static ssize_t ucma_create_id(struct ucma_file *file, ret = -EFAULT; goto err2; } + + ctx->cm_id = cm_id; return 0; err2: - rdma_destroy_id(ctx->cm_id); + rdma_destroy_id(cm_id); err1: mutex_lock(&mut); idr_remove(&ctx_idr, ctx->id); From owner-svn-src-all@freebsd.org Thu Aug 2 08:18:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF982105D5C0; Thu, 2 Aug 2018 08:18:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 951AF7A426; Thu, 2 Aug 2018 08:18:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7643070F2; Thu, 2 Aug 2018 08:18:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728IC1U039680; Thu, 2 Aug 2018 08:18:12 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728ICrT039679; Thu, 2 Aug 2018 08:18:12 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020818.w728ICrT039679@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:18:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337080 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337080 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:18:13 -0000 Author: hselasky Date: Thu Aug 2 08:18:11 2018 New Revision: 337080 URL: https://svnweb.freebsd.org/changeset/base/337080 Log: MFC r336374: Avoid that ib_drain_qp() triggers an out-of-bounds stack access in ibcore. Linux commit: a1ae7d0345edd593d6725d3218434d903a0af95d Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:17:09 2018 (r337079) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:18:11 2018 (r337080) @@ -1940,7 +1940,13 @@ static void __ib_drain_sq(struct ib_qp *qp) { struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR }; struct ib_drain_cqe sdrain; - struct ib_send_wr swr = {}, *bad_swr; + struct ib_send_wr *bad_swr; + struct ib_rdma_wr swr = { + .wr = { + .opcode = IB_WR_RDMA_WRITE, + .wr_cqe = &sdrain.cqe, + }, + }; int ret; if (qp->send_cq->poll_ctx == IB_POLL_DIRECT) { @@ -1949,7 +1955,6 @@ static void __ib_drain_sq(struct ib_qp *qp) return; } - swr.wr_cqe = &sdrain.cqe; sdrain.cqe.done = ib_drain_qp_done; init_completion(&sdrain.done); @@ -1959,7 +1964,7 @@ static void __ib_drain_sq(struct ib_qp *qp) return; } - ret = ib_post_send(qp, &swr, &bad_swr); + ret = ib_post_send(qp, &swr.wr, &bad_swr); if (ret) { WARN_ONCE(ret, "failed to drain send queue: %d\n", ret); return; From owner-svn-src-all@freebsd.org Thu Aug 2 08:19:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B59A3105D65B; Thu, 2 Aug 2018 08:19:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6AF1F7A592; Thu, 2 Aug 2018 08:19:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4D30170F3; Thu, 2 Aug 2018 08:19:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728J91n039779; Thu, 2 Aug 2018 08:19:09 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728J9Xs039778; Thu, 2 Aug 2018 08:19:09 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020819.w728J9Xs039778@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:19:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337081 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337081 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:19:09 -0000 Author: hselasky Date: Thu Aug 2 08:19:08 2018 New Revision: 337081 URL: https://svnweb.freebsd.org/changeset/base/337081 Log: MFC r336375: Fix access to non-initialized CM_ID object in ibcore. The attempt to join multicast group without ensuring that CMA device exists will lead to the following crash reported by syzkaller. Linux commit: 7688f2c3bbf55e52388e37ac5d63ca471a7712e1 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:18:11 2018 (r337080) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:19:08 2018 (r337081) @@ -4064,6 +4064,9 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct struct cma_multicast *mc; int ret; + if (!id->device) + return -EINVAL; + id_priv = container_of(id, struct rdma_id_private, id); if (!cma_comp(id_priv, RDMA_CM_ADDR_BOUND) && !cma_comp(id_priv, RDMA_CM_ADDR_RESOLVED)) From owner-svn-src-all@freebsd.org Thu Aug 2 08:20:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81C54105D788; Thu, 2 Aug 2018 08:20:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 380C17A741; Thu, 2 Aug 2018 08:20:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1A51F70FA; Thu, 2 Aug 2018 08:20:12 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728KB6r039911; Thu, 2 Aug 2018 08:20:11 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728KB0B039910; Thu, 2 Aug 2018 08:20:11 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020820.w728KB0B039910@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:20:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337082 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337082 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:20:12 -0000 Author: hselasky Date: Thu Aug 2 08:20:11 2018 New Revision: 337082 URL: https://svnweb.freebsd.org/changeset/base/337082 Log: MFC r336376: Fix NULL pointer dereference during device removal in ibcore. As part of ib_uverbs_remove_one which might be triggered upon reset flow, we trigger IB_EVENT_DEVICE_FATAL event to userspace application. If device was removed after uverbs fd was opened but before ib_uverbs_get_context was called, the event file will be accessed before it was allocated, result in NULL pointer dereference: Linux commit: 870201f95fcbd19538aef630393fe9d583eff82e Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c Thu Aug 2 08:19:08 2018 (r337081) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c Thu Aug 2 08:20:11 2018 (r337082) @@ -1274,7 +1274,6 @@ static void ib_uverbs_free_hw_resources(struct ib_uver kref_get(&file->ref); mutex_unlock(&uverbs_dev->lists_mutex); - ib_uverbs_event_handler(&file->event_handler, &event); mutex_lock(&file->cleanup_mutex); ucontext = file->ucontext; @@ -1291,6 +1290,7 @@ static void ib_uverbs_free_hw_resources(struct ib_uver * for example due to freeing the resources * (e.g mmput). */ + ib_uverbs_event_handler(&file->event_handler, &event); ib_dev->disassociate_ucontext(ucontext); ib_uverbs_cleanup_ucontext(file, ucontext); } From owner-svn-src-all@freebsd.org Thu Aug 2 08:21:05 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D298F105D888; Thu, 2 Aug 2018 08:21:05 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 85C6E7AA59; Thu, 2 Aug 2018 08:21:05 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 67BF7721E; Thu, 2 Aug 2018 08:21:05 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728L5hQ040710; Thu, 2 Aug 2018 08:21:05 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728L59e040709; Thu, 2 Aug 2018 08:21:05 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020821.w728L59e040709@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:21:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337083 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337083 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:21:06 -0000 Author: hselasky Date: Thu Aug 2 08:21:04 2018 New Revision: 337083 URL: https://svnweb.freebsd.org/changeset/base/337083 Log: MFC r336377: Fix kernel panic while using XRC_TGT QP type in ibcore. Attempt to modify XRC_TGT QP type from the user space (ibv_xsrq_pingpong invocation) will trigger the following kernel panic. It is caused by the fact that such QPs missed uobject initialization. Linux commit: f45765872e7aae7b81feb3044aaf9886b21885ef Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:20:11 2018 (r337082) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:21:04 2018 (r337083) @@ -1943,6 +1943,9 @@ static int create_qp(struct ib_uverbs_file *file, atomic_inc(&attr.srq->usecnt); if (ind_tbl) atomic_inc(&ind_tbl->usecnt); + } else { + /* It is done in _ib_create_qp for other QP types */ + qp->uobject = &obj->uevent.uobject; } qp->uobject = &obj->uevent.uobject; From owner-svn-src-all@freebsd.org Thu Aug 2 08:21:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F945105D958; Thu, 2 Aug 2018 08:21:56 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 34EB57AC32; Thu, 2 Aug 2018 08:21:56 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 15EE87259; Thu, 2 Aug 2018 08:21:56 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728LtZh043789; Thu, 2 Aug 2018 08:21:55 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728LttN043788; Thu, 2 Aug 2018 08:21:55 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020821.w728LttN043788@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:21:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337084 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337084 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:21:56 -0000 Author: hselasky Date: Thu Aug 2 08:21:55 2018 New Revision: 337084 URL: https://svnweb.freebsd.org/changeset/base/337084 Log: MFC r336379: Check for a cm_id->device in all user calls that need it in ibcore. This was done by auditing all callers of ucma_get_ctx and switching the ones that unconditionally touch ->device to ucma_get_ctx_dev. This covers a little less than half of the call sites. The 11 remaining call sites to ucma_get_ctx() were manually audited. Linux commit: 4b658d1bbc16605330694bb3ef2570c465ef383d 8b77586bd8fe600d97f922c79f7222c46f37c118 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:21:04 2018 (r337083) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:21:55 2018 (r337084) @@ -151,6 +151,23 @@ static void ucma_put_ctx(struct ucma_context *ctx) complete(&ctx->comp); } +/* + * Same as ucm_get_ctx but requires that ->cm_id->device is valid, eg that the + * CM_ID is bound. + */ +static struct ucma_context *ucma_get_ctx_dev(struct ucma_file *file, int id) +{ + struct ucma_context *ctx = ucma_get_ctx(file, id); + + if (IS_ERR(ctx)) + return ctx; + if (!ctx->cm_id->device) { + ucma_put_ctx(ctx); + return ERR_PTR(-EINVAL); + } + return ctx; +} + static void ucma_close_event_id(struct work_struct *work) { struct ucma_event *uevent_close = container_of(work, struct ucma_event, close_work); @@ -712,7 +729,7 @@ static ssize_t ucma_resolve_route(struct ucma_file *fi if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; - ctx = ucma_get_ctx(file, cmd.id); + ctx = ucma_get_ctx_dev(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -1020,7 +1037,7 @@ static ssize_t ucma_connect(struct ucma_file *file, co if (!cmd.conn_param.valid) return -EINVAL; - ctx = ucma_get_ctx(file, cmd.id); + ctx = ucma_get_ctx_dev(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -1062,7 +1079,7 @@ static ssize_t ucma_accept(struct ucma_file *file, con if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; - ctx = ucma_get_ctx(file, cmd.id); + ctx = ucma_get_ctx_dev(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -1090,7 +1107,7 @@ static ssize_t ucma_reject(struct ucma_file *file, con if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; - ctx = ucma_get_ctx(file, cmd.id); + ctx = ucma_get_ctx_dev(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -1109,7 +1126,7 @@ static ssize_t ucma_disconnect(struct ucma_file *file, if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; - ctx = ucma_get_ctx(file, cmd.id); + ctx = ucma_get_ctx_dev(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -1134,7 +1151,7 @@ static ssize_t ucma_init_qp_attr(struct ucma_file *fil if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; - ctx = ucma_get_ctx(file, cmd.id); + ctx = ucma_get_ctx_dev(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -1329,7 +1346,7 @@ static ssize_t ucma_process_join(struct ucma_file *fil else return -EINVAL; - ctx = ucma_get_ctx(file, cmd->id); + ctx = ucma_get_ctx_dev(file, cmd->id); if (IS_ERR(ctx)) return PTR_ERR(ctx); From owner-svn-src-all@freebsd.org Thu Aug 2 08:22:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB363105DB63; Thu, 2 Aug 2018 08:22:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6FDF27AE42; Thu, 2 Aug 2018 08:22:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 528927288; Thu, 2 Aug 2018 08:22:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728Ms0h044606; Thu, 2 Aug 2018 08:22:54 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728MrxP044604; Thu, 2 Aug 2018 08:22:53 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020822.w728MrxP044604@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:22:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337085 - in stable/11/sys/ofed: drivers/infiniband/core include/rdma X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/ofed: drivers/infiniband/core include/rdma X-SVN-Commit-Revision: 337085 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:22:55 -0000 Author: hselasky Date: Thu Aug 2 08:22:53 2018 New Revision: 337085 URL: https://svnweb.freebsd.org/changeset/base/337085 Log: MFC r336380: Check AF family prior resolving address and introduce safer rdma_addr_size() variants in ibcore. Garbage supplied by user will cause to UCMA module provide zero memory size for memcpy(), because it wasn't checked, it will produce unpredictable results in rdma_resolve_addr(). There are several places in the ucma ABI where userspace can pass in a sockaddr but set the address family to AF_IB. When that happens, rdma_addr_size() will return a size bigger than sizeof struct sockaddr_in6, and the ucma kernel code might end up copying past the end of a buffer not sized for a struct sockaddr_ib. Fix this by introducing new variants int rdma_addr_size_in6(struct sockaddr_in6 *addr); int rdma_addr_size_kss(struct __kernel_sockaddr_storage *addr); that are type-safe for the types used in the ucma ABI and return 0 if the size computed is bigger than the size of the type passed in. We can use these new variants to check what size userspace has passed in before copying any addresses. Linux commit: 2975d5de6428ff6d9317e9948f0968f7d42e5d74 09abfe7b5b2f442a85f4c4d59ecf582ad76088d7 84652aefb347297aa08e91e283adf7b18f77c2d5 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c stable/11/sys/ofed/include/rdma/ib_addr.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:21:55 2018 (r337084) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:22:53 2018 (r337085) @@ -89,6 +89,22 @@ int rdma_addr_size(struct sockaddr *addr) } EXPORT_SYMBOL(rdma_addr_size); +int rdma_addr_size_in6(struct sockaddr_in6 *addr) +{ + int ret = rdma_addr_size((struct sockaddr *) addr); + + return ret <= sizeof(*addr) ? ret : 0; +} +EXPORT_SYMBOL(rdma_addr_size_in6); + +int rdma_addr_size_kss(struct sockaddr_storage *addr) +{ + int ret = rdma_addr_size((struct sockaddr *) addr); + + return ret <= sizeof(*addr) ? ret : 0; +} +EXPORT_SYMBOL(rdma_addr_size_kss); + static struct rdma_addr_client self; void rdma_addr_register_client(struct rdma_addr_client *client) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:21:55 2018 (r337084) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:22:53 2018 (r337085) @@ -636,6 +636,9 @@ static ssize_t ucma_bind_ip(struct ucma_file *file, co if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; + if (!rdma_addr_size_in6(&cmd.addr)) + return -EINVAL; + ctx = ucma_get_ctx(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); @@ -649,22 +652,21 @@ static ssize_t ucma_bind(struct ucma_file *file, const int in_len, int out_len) { struct rdma_ucm_bind cmd; - struct sockaddr *addr; struct ucma_context *ctx; int ret; if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; - addr = (struct sockaddr *) &cmd.addr; - if (cmd.reserved || !cmd.addr_size || (cmd.addr_size != rdma_addr_size(addr))) + if (cmd.reserved || !cmd.addr_size || + cmd.addr_size != rdma_addr_size_kss(&cmd.addr)) return -EINVAL; ctx = ucma_get_ctx(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); - ret = rdma_bind_addr(ctx->cm_id, addr); + ret = rdma_bind_addr(ctx->cm_id, (struct sockaddr *) &cmd.addr); ucma_put_ctx(ctx); return ret; } @@ -680,13 +682,16 @@ static ssize_t ucma_resolve_ip(struct ucma_file *file, if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; + if ((cmd.src_addr.sin6_family && !rdma_addr_size_in6(&cmd.src_addr)) || + !rdma_addr_size_in6(&cmd.dst_addr)) + return -EINVAL; + ctx = ucma_get_ctx(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); ret = rdma_resolve_addr(ctx->cm_id, (struct sockaddr *) &cmd.src_addr, - (struct sockaddr *) &cmd.dst_addr, - cmd.timeout_ms); + (struct sockaddr *) &cmd.dst_addr, cmd.timeout_ms); ucma_put_ctx(ctx); return ret; } @@ -696,24 +701,23 @@ static ssize_t ucma_resolve_addr(struct ucma_file *fil int in_len, int out_len) { struct rdma_ucm_resolve_addr cmd; - struct sockaddr *src, *dst; struct ucma_context *ctx; int ret; if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; - src = (struct sockaddr *) &cmd.src_addr; - dst = (struct sockaddr *) &cmd.dst_addr; - if (cmd.reserved || (cmd.src_size && (cmd.src_size != rdma_addr_size(src))) || - !cmd.dst_size || (cmd.dst_size != rdma_addr_size(dst))) + if (cmd.reserved || + (cmd.src_size && (cmd.src_size != rdma_addr_size_kss(&cmd.src_addr))) || + !cmd.dst_size || (cmd.dst_size != rdma_addr_size_kss(&cmd.dst_addr))) return -EINVAL; ctx = ucma_get_ctx(file, cmd.id); if (IS_ERR(ctx)) return PTR_ERR(ctx); - ret = rdma_resolve_addr(ctx->cm_id, src, dst, cmd.timeout_ms); + ret = rdma_resolve_addr(ctx->cm_id, (struct sockaddr *) &cmd.src_addr, + (struct sockaddr *) &cmd.dst_addr, cmd.timeout_ms); ucma_put_ctx(ctx); return ret; } @@ -1403,7 +1407,7 @@ static ssize_t ucma_join_ip_multicast(struct ucma_file join_cmd.response = cmd.response; join_cmd.uid = cmd.uid; join_cmd.id = cmd.id; - join_cmd.addr_size = rdma_addr_size((struct sockaddr *) &cmd.addr); + join_cmd.addr_size = rdma_addr_size_in6(&cmd.addr); join_cmd.join_flags = RDMA_MC_JOIN_FLAG_FULLMEMBER; memcpy(&join_cmd.addr, &cmd.addr, join_cmd.addr_size); @@ -1418,6 +1422,9 @@ static ssize_t ucma_join_multicast(struct ucma_file *f if (copy_from_user(&cmd, inbuf, sizeof(cmd))) return -EFAULT; + + if (!rdma_addr_size_kss(&cmd.addr)) + return -EINVAL; return ucma_process_join(file, &cmd, out_len); } Modified: stable/11/sys/ofed/include/rdma/ib_addr.h ============================================================================== --- stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:21:55 2018 (r337084) +++ stable/11/sys/ofed/include/rdma/ib_addr.h Thu Aug 2 08:22:53 2018 (r337085) @@ -131,6 +131,8 @@ int rdma_copy_addr(struct rdma_dev_addr *dev_addr, str const unsigned char *dst_dev_addr); int rdma_addr_size(struct sockaddr *addr); +int rdma_addr_size_in6(struct sockaddr_in6 *addr); +int rdma_addr_size_kss(struct sockaddr_storage *addr); int rdma_addr_find_l2_eth_by_grh(const union ib_gid *sgid, const union ib_gid *dgid, From owner-svn-src-all@freebsd.org Thu Aug 2 08:23:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3846105DC7D; Thu, 2 Aug 2018 08:23:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58E3C7B025; Thu, 2 Aug 2018 08:23:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3A727728A; Thu, 2 Aug 2018 08:23:55 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728Nthp044704; Thu, 2 Aug 2018 08:23:55 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728NtvB044703; Thu, 2 Aug 2018 08:23:55 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020823.w728NtvB044703@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:23:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337086 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337086 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:23:55 -0000 Author: hselasky Date: Thu Aug 2 08:23:54 2018 New Revision: 337086 URL: https://svnweb.freebsd.org/changeset/base/337086 Log: MFC r336381: Fix kernel crash during fail to initialize device in ibcore. This patch fixes the kernel crash that occurs during ib_dealloc_device() called due to provider driver fails with an error after ib_alloc_device() and before it can register using ib_register_device(). This crashed seen in tha lab as below which can occur with any IB device which fails to perform its device initialization before invoking ib_register_device(). This patch avoids touching cache and port immutable structures if device is not yet initialized. It also releases related memory when cache and port immutable data structure initialization fails during register_device() state. Linux commit: 4be3a4fa51f432ef045546d16f25c68a1ab525b9 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_device.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_device.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_device.c Thu Aug 2 08:22:53 2018 (r337085) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_device.c Thu Aug 2 08:23:54 2018 (r337086) @@ -174,8 +174,16 @@ static void ib_device_release(struct device *device) { struct ib_device *dev = container_of(device, struct ib_device, dev); - ib_cache_release_one(dev); - kfree(dev->port_immutable); + WARN_ON(dev->reg_state == IB_DEV_REGISTERED); + if (dev->reg_state == IB_DEV_UNREGISTERED) { + /* + * In IB_DEV_UNINITIALIZED state, cache or port table + * is not even created. Free cache and port table only when + * device reaches UNREGISTERED state. + */ + ib_cache_release_one(dev); + kfree(dev->port_immutable); + } kfree(dev); } @@ -347,23 +355,21 @@ int ib_register_device(struct ib_device *device, ret = ib_cache_setup_one(device); if (ret) { pr_warn("Couldn't set up InfiniBand P_Key/GID cache\n"); - goto out; + goto port_cleanup; } memset(&device->attrs, 0, sizeof(device->attrs)); ret = device->query_device(device, &device->attrs, &uhw); if (ret) { pr_warn("Couldn't query the device attributes\n"); - ib_cache_cleanup_one(device); - goto out; + goto cache_cleanup; } ret = ib_device_register_sysfs(device, port_callback); if (ret) { pr_warn("Couldn't register device %s with driver model\n", device->name); - ib_cache_cleanup_one(device); - goto out; + goto cache_cleanup; } device->reg_state = IB_DEV_REGISTERED; @@ -375,6 +381,14 @@ int ib_register_device(struct ib_device *device, down_write(&lists_rwsem); list_add_tail(&device->core_list, &device_list); up_write(&lists_rwsem); + mutex_unlock(&device_mutex); + return 0; + +cache_cleanup: + ib_cache_cleanup_one(device); + ib_cache_release_one(device); +port_cleanup: + kfree(device->port_immutable); out: mutex_unlock(&device_mutex); return ret; From owner-svn-src-all@freebsd.org Thu Aug 2 08:24:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E891D105DD4F; Thu, 2 Aug 2018 08:24:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DE417B1A3; Thu, 2 Aug 2018 08:24:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7FA33728C; Thu, 2 Aug 2018 08:24:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728OjiJ044805; Thu, 2 Aug 2018 08:24:45 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728OjGH044804; Thu, 2 Aug 2018 08:24:45 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020824.w728OjGH044804@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:24:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337087 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337087 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:24:46 -0000 Author: hselasky Date: Thu Aug 2 08:24:45 2018 New Revision: 337087 URL: https://svnweb.freebsd.org/changeset/base/337087 Log: MFC r336382: Depend on IPv6 stack to resolve link local address for RoCEv2 in ibcore. RoCEv1 does not use the IPv6 stack to resolve the link local DGID since it uses GID address. It forms the DMAC directly from the DGID. Linux commit: 56d0a7d9a0f045ee27a001762deac28c7d28e2e4 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:23:54 2018 (r337086) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:24:45 2018 (r337087) @@ -1160,7 +1160,10 @@ EXPORT_SYMBOL(ib_modify_qp_is_ok); int ib_resolve_eth_dmac(struct ib_device *device, struct ib_ah_attr *ah_attr) { - int ret = 0; + struct ib_gid_attr sgid_attr; + union ib_gid sgid; + int hop_limit; + int ret; if (ah_attr->port_num < rdma_start_port(device) || ah_attr->port_num > rdma_end_port(device)) @@ -1169,35 +1172,22 @@ int ib_resolve_eth_dmac(struct ib_device *device, if (!rdma_cap_eth_ah(device, ah_attr->port_num)) return 0; - if (rdma_link_local_addr((struct in6_addr *)ah_attr->grh.dgid.raw)) { - rdma_get_ll_mac((struct in6_addr *)ah_attr->grh.dgid.raw, - ah_attr->dmac); - } else { - union ib_gid sgid; - struct ib_gid_attr sgid_attr; - int hop_limit; + ret = ib_query_gid(device, + ah_attr->port_num, + ah_attr->grh.sgid_index, + &sgid, &sgid_attr); + if (ret != 0) + return (ret); + if (!sgid_attr.ndev) + return -ENXIO; - ret = ib_query_gid(device, - ah_attr->port_num, - ah_attr->grh.sgid_index, - &sgid, &sgid_attr); + ret = rdma_addr_find_l2_eth_by_grh(&sgid, + &ah_attr->grh.dgid, + ah_attr->dmac, + sgid_attr.ndev, &hop_limit); + dev_put(sgid_attr.ndev); - if (ret || !sgid_attr.ndev) { - if (!ret) - ret = -ENXIO; - goto out; - } - - ret = rdma_addr_find_l2_eth_by_grh(&sgid, - &ah_attr->grh.dgid, - ah_attr->dmac, - sgid_attr.ndev, &hop_limit); - - dev_put(sgid_attr.ndev); - - ah_attr->grh.hop_limit = hop_limit; - } -out: + ah_attr->grh.hop_limit = hop_limit; return ret; } EXPORT_SYMBOL(ib_resolve_eth_dmac); From owner-svn-src-all@freebsd.org Thu Aug 2 08:25:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56D91105DE42; Thu, 2 Aug 2018 08:25:49 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BAB07B32F; Thu, 2 Aug 2018 08:25:49 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E2136728E; Thu, 2 Aug 2018 08:25:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728PmdE044917; Thu, 2 Aug 2018 08:25:48 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728PmIx044913; Thu, 2 Aug 2018 08:25:48 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020825.w728PmIx044913@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:25:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337088 - in stable/11/sys/ofed: drivers/infiniband/core include/rdma X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/ofed: drivers/infiniband/core include/rdma X-SVN-Commit-Revision: 337088 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:25:49 -0000 Author: hselasky Date: Thu Aug 2 08:25:48 2018 New Revision: 337088 URL: https://svnweb.freebsd.org/changeset/base/337088 Log: MFC r336383: Check port number supplied by user verbs cmds in ibcore. The ib_uverbs_create_ah() ind ib_uverbs_modify_qp() calls receive the port number from user input as part of its attributes and assumes it is valid. Down on the stack, that parameter is used to access kernel data structures. If the value is invalid, the kernel accesses memory it should not. To prevent this, verify the port number before using it. Linux commit: 5ecce4c9b17bed4dc9cb58bfb10447307569b77b a62ab66b13a0f9bcb17b7b761f6670941ed5cd62 5a7a88f1b488e4ee49eb3d5b82612d4d9ffdf2c3 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c stable/11/sys/ofed/include/rdma/ib_verbs.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:24:45 2018 (r337087) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:25:48 2018 (r337088) @@ -983,6 +983,8 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib } else ret = iw_cm_init_qp_attr(id_priv->cm_id.iw, qp_attr, qp_attr_mask); + qp_attr->port_num = id_priv->id.port_num; + *qp_attr_mask |= IB_QP_PORT; } else ret = -ENOSYS; Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:24:45 2018 (r337087) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:25:48 2018 (r337088) @@ -2366,6 +2366,25 @@ ssize_t ib_uverbs_modify_qp(struct ib_uverbs_file *fil goto out; } + if ((cmd.attr_mask & IB_QP_PORT) && + !rdma_is_port_valid(qp->device, cmd.port_num)) { + ret = -EINVAL; + goto release_qp; + } + + if ((cmd.attr_mask & IB_QP_AV) && + !rdma_is_port_valid(qp->device, cmd.dest.port_num)) { + ret = -EINVAL; + goto release_qp; + } + + if ((cmd.attr_mask & IB_QP_ALT_PATH) && + (!rdma_is_port_valid(qp->device, cmd.alt_port_num) || + !rdma_is_port_valid(qp->device, cmd.alt_dest.port_num))) { + ret = -EINVAL; + goto release_qp; + } + attr->qp_state = cmd.qp_state; attr->cur_qp_state = cmd.cur_qp_state; attr->path_mtu = cmd.path_mtu; @@ -2895,6 +2914,9 @@ ssize_t ib_uverbs_create_ah(struct ib_uverbs_file *fil if (copy_from_user(&cmd, buf, sizeof cmd)) return -EFAULT; + + if (!rdma_is_port_valid(ib_dev, cmd.attr.port_num)) + return -EINVAL; INIT_UDATA(&udata, buf + sizeof(cmd), (unsigned long)cmd.response + sizeof(resp), Modified: stable/11/sys/ofed/include/rdma/ib_verbs.h ============================================================================== --- stable/11/sys/ofed/include/rdma/ib_verbs.h Thu Aug 2 08:24:45 2018 (r337087) +++ stable/11/sys/ofed/include/rdma/ib_verbs.h Thu Aug 2 08:25:48 2018 (r337088) @@ -2268,6 +2268,13 @@ static inline u8 rdma_end_port(const struct ib_device return rdma_cap_ib_switch(device) ? 0 : device->phys_port_cnt; } +static inline int rdma_is_port_valid(const struct ib_device *device, + unsigned int port) +{ + return (port >= rdma_start_port(device) && + port <= rdma_end_port(device)); +} + static inline bool rdma_protocol_ib(const struct ib_device *device, u8 port_num) { return device->port_immutable[port_num].core_cap_flags & RDMA_CORE_CAP_PROT_IB; From owner-svn-src-all@freebsd.org Thu Aug 2 08:26:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C287105DF06; Thu, 2 Aug 2018 08:26:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3853B7B491; Thu, 2 Aug 2018 08:26:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 160CB7290; Thu, 2 Aug 2018 08:26:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728QuQq045019; Thu, 2 Aug 2018 08:26:56 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728QuFd045018; Thu, 2 Aug 2018 08:26:56 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020826.w728QuFd045018@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:26:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337089 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337089 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:26:57 -0000 Author: hselasky Date: Thu Aug 2 08:26:56 2018 New Revision: 337089 URL: https://svnweb.freebsd.org/changeset/base/337089 Log: MFC r336384: Fix for loopback detection in address resolve logic in ibcore. When a loopback address is detected use the network interface which has the loopback flag set to trigger loopback logic in address resolve. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:25:48 2018 (r337088) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:26:56 2018 (r337089) @@ -349,6 +349,10 @@ static int addr4_resolve(struct sockaddr_in *src_in, error = EHOSTUNREACH; goto error_put_ifp; } + /* get destination network interface from route */ + dev_put(ifp); + ifp = rte->rt_ifp; + dev_hold(ifp); } else if (ifp != rte->rt_ifp) { /* * Source and destination interfaces are @@ -544,6 +548,10 @@ static int addr6_resolve(struct sockaddr_in6 *src_in, error = EHOSTUNREACH; goto error_put_ifp; } + /* get destination network interface from route */ + dev_put(ifp); + ifp = rte->rt_ifp; + dev_hold(ifp); } else if (ifp != rte->rt_ifp) { /* * Source and destination interfaces are From owner-svn-src-all@freebsd.org Thu Aug 2 08:27:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F661105DFA5; Thu, 2 Aug 2018 08:27:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54D417B5D3; Thu, 2 Aug 2018 08:27:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 35EED7292; Thu, 2 Aug 2018 08:27:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728Ronu045115; Thu, 2 Aug 2018 08:27:50 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728RoGN045114; Thu, 2 Aug 2018 08:27:50 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020827.w728RoGN045114@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:27:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337090 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337090 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:27:50 -0000 Author: hselasky Date: Thu Aug 2 08:27:49 2018 New Revision: 337090 URL: https://svnweb.freebsd.org/changeset/base/337090 Log: MFC r336385: Set IPv4 TOS and IPv6 traffic class field for RoCEv2 traffic in ibcore. The current implementation assumes a static mapping between the TOS bits and the priority code point, PCP bits. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:26:56 2018 (r337089) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:27:49 2018 (r337090) @@ -2451,15 +2451,9 @@ static int cma_resolve_iw_route(struct rdma_id_private static int iboe_tos_to_sl(struct net_device *ndev, int tos) { - /* get service level, SL, from type of service, TOS */ - int sl = tos; + /* get service level, SL, from IPv4 type of service, TOS */ + int sl = (tos >> 5) & 0x7; - /* range check input argument and map 1:1 */ - if (sl > 255) - sl = 255; - else if (sl < 0) - sl = 0; - /* final mappings are done by the vendor specific drivers */ return sl; } @@ -2543,6 +2537,7 @@ static int cma_resolve_iboe_route(struct rdma_id_priva route->path_rec->pkey = cpu_to_be16(0xffff); route->path_rec->mtu_selector = IB_SA_EQ; route->path_rec->sl = iboe_tos_to_sl(ndev, id_priv->tos); + route->path_rec->traffic_class = id_priv->tos; route->path_rec->mtu = iboe_get_mtu(ndev->if_mtu); route->path_rec->rate_selector = IB_SA_EQ; route->path_rec->rate = iboe_get_rate(ndev); From owner-svn-src-all@freebsd.org Thu Aug 2 08:28:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E9C1105E041; Thu, 2 Aug 2018 08:28:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D72607B72F; Thu, 2 Aug 2018 08:28:49 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B80007293; Thu, 2 Aug 2018 08:28:49 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728SnKh045219; Thu, 2 Aug 2018 08:28:49 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728SnI0045218; Thu, 2 Aug 2018 08:28:49 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020828.w728SnI0045218@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:28:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337091 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337091 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:28:50 -0000 Author: hselasky Date: Thu Aug 2 08:28:49 2018 New Revision: 337091 URL: https://svnweb.freebsd.org/changeset/base/337091 Log: MFC r336386: Honor port_num while resolving GID for IB link layer in ibcore. ah_attr contains the port number to which cm_id is bound. However, while searching for GID table for matching GID entry, the port number is ignored. This could cause the wrong GID to be used when the ah_attr is converted to an AH. Linux commit: 563c4ba3bd2b8b0b21c65669ec2226b1cfa1138b Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c Thu Aug 2 08:27:49 2018 (r337090) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c Thu Aug 2 08:28:49 2018 (r337091) @@ -727,21 +727,19 @@ int ib_init_ah_from_mcmember(struct ib_device *device, { int ret; u16 gid_index; - u8 p; - if (rdma_protocol_roce(device, port_num)) { - ret = ib_find_cached_gid_by_port(device, &rec->port_gid, - gid_type, port_num, - ndev, - &gid_index); - } else if (rdma_protocol_ib(device, port_num)) { - ret = ib_find_cached_gid(device, &rec->port_gid, - IB_GID_TYPE_IB, NULL, &p, - &gid_index); - } else { - ret = -EINVAL; - } + /* GID table is not based on the netdevice for IB link layer, + * so ignore ndev during search. + */ + if (rdma_protocol_ib(device, port_num)) + ndev = NULL; + else if (!rdma_protocol_roce(device, port_num)) + return -EINVAL; + ret = ib_find_cached_gid_by_port(device, &rec->port_gid, + gid_type, port_num, + ndev, + &gid_index); if (ret) return ret; From owner-svn-src-all@freebsd.org Thu Aug 2 08:29:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C820E105E123; Thu, 2 Aug 2018 08:29:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7D65B7B8CA; Thu, 2 Aug 2018 08:29:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 600F87296; Thu, 2 Aug 2018 08:29:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728TfOa045314; Thu, 2 Aug 2018 08:29:41 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728TfDU045313; Thu, 2 Aug 2018 08:29:41 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020829.w728TfDU045313@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:29:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337092 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337092 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:29:42 -0000 Author: hselasky Date: Thu Aug 2 08:29:40 2018 New Revision: 337092 URL: https://svnweb.freebsd.org/changeset/base/337092 Log: MFC r336387: Honor return status of ib_init_ah_from_mcmember() in ibcore. The return status of ib_init_ah_from_mcmember() is ignored by cma_ib_mc_handler(). Honor it and return error event if ah attribute initialization failed. Linux commit: 6d337179f28cc50ddd7e224f677b4cda70b275fc Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:28:49 2018 (r337091) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:29:40 2018 (r337092) @@ -3764,10 +3764,14 @@ static int cma_ib_mc_handler(int status, struct ib_sa_ rdma_start_port(id_priv->cma_dev->device)]; event.event = RDMA_CM_EVENT_MULTICAST_JOIN; - ib_init_ah_from_mcmember(id_priv->id.device, - id_priv->id.port_num, &multicast->rec, - ndev, gid_type, - &event.param.ud.ah_attr); + ret = ib_init_ah_from_mcmember(id_priv->id.device, + id_priv->id.port_num, + &multicast->rec, + ndev, gid_type, + &event.param.ud.ah_attr); + if (ret) + event.event = RDMA_CM_EVENT_MULTICAST_ERROR; + event.param.ud.qp_num = 0xFFFFFF; event.param.ud.qkey = be32_to_cpu(multicast->rec.qkey); if (ndev) From owner-svn-src-all@freebsd.org Thu Aug 2 08:30:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FC02105E1E0; Thu, 2 Aug 2018 08:30:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04D547BA64; Thu, 2 Aug 2018 08:30:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DA39272AC; Thu, 2 Aug 2018 08:30:44 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728UiAY045442; Thu, 2 Aug 2018 08:30:44 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728Uist045441; Thu, 2 Aug 2018 08:30:44 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020830.w728Uist045441@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:30:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337093 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337093 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:30:45 -0000 Author: hselasky Date: Thu Aug 2 08:30:44 2018 New Revision: 337093 URL: https://svnweb.freebsd.org/changeset/base/337093 Log: MFC r336388: Add support for RoCEv2 multicast in ibcore. When creating address handle from multicast GID, set MAC according to the appropriate formula instead of searching for it in the GID table: - For IPv4 multicast GID use ip_eth_mc_map(). - For IPv6 multicast GID use ipv6_eth_mc_map(). Linux commit: 9636a56fa864464896bf7d1272c701f2b9a57737 Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:29:40 2018 (r337092) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:30:44 2018 (r337093) @@ -1172,6 +1172,19 @@ int ib_resolve_eth_dmac(struct ib_device *device, if (!rdma_cap_eth_ah(device, ah_attr->port_num)) return 0; + if (rdma_is_multicast_addr((struct in6_addr *)ah_attr->grh.dgid.raw)) { + if (ipv6_addr_v4mapped((struct in6_addr *)ah_attr->grh.dgid.raw)) { + __be32 addr = 0; + + memcpy(&addr, ah_attr->grh.dgid.raw + 12, 4); + ip_eth_mc_map(addr, (char *)ah_attr->dmac); + } else { + ipv6_eth_mc_map((struct in6_addr *)ah_attr->grh.dgid.raw, + (char *)ah_attr->dmac); + } + return 0; + } + ret = ib_query_gid(device, ah_attr->port_num, ah_attr->grh.sgid_index, From owner-svn-src-all@freebsd.org Thu Aug 2 08:31:35 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C898105E292; Thu, 2 Aug 2018 08:31:35 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 123BF7BC5F; Thu, 2 Aug 2018 08:31:35 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E855272E9; Thu, 2 Aug 2018 08:31:34 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728VYep047880; Thu, 2 Aug 2018 08:31:34 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728VYWS047862; Thu, 2 Aug 2018 08:31:34 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020831.w728VYWS047862@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:31:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337094 - stable/11/sys/ofed/drivers/infiniband/core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/drivers/infiniband/core X-SVN-Commit-Revision: 337094 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:31:35 -0000 Author: hselasky Date: Thu Aug 2 08:31:34 2018 New Revision: 337094 URL: https://svnweb.freebsd.org/changeset/base/337094 Log: MFC r336389: Add support for IPv6 multicast in ibcore. This change allows us to join IPv6 multicast networks. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:30:44 2018 (r337093) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:31:34 2018 (r337094) @@ -4021,7 +4021,7 @@ static int cma_iboe_join_multicast(struct rdma_id_priv mc->multicast.ib->rec.hop_limit = 1; mc->multicast.ib->rec.mtu = iboe_get_mtu(ndev->if_mtu); - if (addr->sa_family == AF_INET) { + if (addr->sa_family == AF_INET || addr->sa_family == AF_INET6) { if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP) { mc->multicast.ib->rec.hop_limit = IPV6_DEFAULT_HOPLIMIT; if (!send_only) { From owner-svn-src-all@freebsd.org Thu Aug 2 08:32:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5910105E441; Thu, 2 Aug 2018 08:32:24 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B37E7BF32; Thu, 2 Aug 2018 08:32:24 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 636D9741C; Thu, 2 Aug 2018 08:32:24 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728WOgm049526; Thu, 2 Aug 2018 08:32:24 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728WOFm049525; Thu, 2 Aug 2018 08:32:24 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020832.w728WOFm049525@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:32:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337095 - stable/11 X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11 X-SVN-Commit-Revision: 337095 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:32:25 -0000 Author: hselasky Date: Thu Aug 2 08:32:23 2018 New Revision: 337095 URL: https://svnweb.freebsd.org/changeset/base/337095 Log: MFC r336390: Remove blank line. Sponsored by: Mellanox Technologies Modified: Directory Properties: stable/11/ (props changed) From owner-svn-src-all@freebsd.org Thu Aug 2 08:34:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D90B105E52D; Thu, 2 Aug 2018 08:34:00 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52B4A7C0C3; Thu, 2 Aug 2018 08:34:00 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1913F7427; Thu, 2 Aug 2018 08:34:00 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728Y0gE050401; Thu, 2 Aug 2018 08:34:00 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728XqK6050355; Thu, 2 Aug 2018 08:33:52 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020833.w728XqK6050355@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:33:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337096 - in stable/11/sys/ofed/drivers/infiniband: core ulp/ipoib X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/ofed/drivers/infiniband: core ulp/ipoib X-SVN-Commit-Revision: 337096 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:34:01 -0000 Author: hselasky Date: Thu Aug 2 08:33:51 2018 New Revision: 337096 URL: https://svnweb.freebsd.org/changeset/base/337096 Log: MFC r336391: Use __FBSDID() for RCS tags in ibcore. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c stable/11/sys/ofed/drivers/infiniband/core/ib_agent.c stable/11/sys/ofed/drivers/infiniband/core/ib_cache.c stable/11/sys/ofed/drivers/infiniband/core/ib_cm.c stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c stable/11/sys/ofed/drivers/infiniband/core/ib_cq.c stable/11/sys/ofed/drivers/infiniband/core/ib_device.c stable/11/sys/ofed/drivers/infiniband/core/ib_fmr_pool.c stable/11/sys/ofed/drivers/infiniband/core/ib_iwcm.c stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_msg.c stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_util.c stable/11/sys/ofed/drivers/infiniband/core/ib_mad.c stable/11/sys/ofed/drivers/infiniband/core/ib_mad_rmpp.c stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c stable/11/sys/ofed/drivers/infiniband/core/ib_packer.c stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c stable/11/sys/ofed/drivers/infiniband/core/ib_smi.c stable/11/sys/ofed/drivers/infiniband/core/ib_sysfs.c stable/11/sys/ofed/drivers/infiniband/core/ib_ucm.c stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c stable/11/sys/ofed/drivers/infiniband/core/ib_ud_header.c stable/11/sys/ofed/drivers/infiniband/core/ib_umem.c stable/11/sys/ofed/drivers/infiniband/core/ib_umem_odp.c stable/11/sys/ofed/drivers/infiniband/core/ib_umem_rbtree.c stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_marshall.c stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib.h stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_cm.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_fs.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ib.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_multicast.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_verbs.c stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_vlan.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_addr.c Thu Aug 2 08:33:51 2018 (r337096) @@ -33,9 +33,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_agent.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_agent.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_agent.c Thu Aug 2 08:33:51 2018 (r337096) @@ -35,9 +35,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cache.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cache.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cache.c Thu Aug 2 08:33:51 2018 (r337096) @@ -33,9 +33,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cm.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cm.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cm.c Thu Aug 2 08:33:51 2018 (r337096) @@ -33,9 +33,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cma.c Thu Aug 2 08:33:51 2018 (r337096) @@ -33,9 +33,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #define LINUXKPI_PARAM_PREFIX ibcore_ Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_cq.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_cq.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_cq.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,9 +30,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_device.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_device.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_device.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,9 +31,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_fmr_pool.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_fmr_pool.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_fmr_pool.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,9 +31,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_iwcm.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_iwcm.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_iwcm.c Thu Aug 2 08:33:51 2018 (r337096) @@ -35,9 +35,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_msg.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_msg.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_msg.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,9 +31,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include "iwpm_util.h" Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_util.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_util.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_iwpm_util.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,9 +31,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include "iwpm_util.h" Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_mad.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_mad.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_mad.c Thu Aug 2 08:33:51 2018 (r337096) @@ -34,9 +34,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #define LINUXKPI_PARAM_PREFIX ibcore_ #define KBUILD_MODNAME "ibcore" Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_mad_rmpp.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_mad_rmpp.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_mad_rmpp.c Thu Aug 2 08:33:51 2018 (r337096) @@ -32,9 +32,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_multicast.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,9 +30,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #define LINUXKPI_PARAM_PREFIX ibcore_ Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_packer.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_packer.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_packer.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,9 +31,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_roce_gid_mgmt.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,9 +30,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include "core_priv.h" Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_sa_query.c Thu Aug 2 08:33:51 2018 (r337096) @@ -32,9 +32,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_smi.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_smi.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_smi.c Thu Aug 2 08:33:51 2018 (r337096) @@ -36,9 +36,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include "smi.h" Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_sysfs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_sysfs.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_sysfs.c Thu Aug 2 08:33:51 2018 (r337096) @@ -32,9 +32,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include "core_priv.h" Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ucm.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_ucm.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_ucm.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,9 +31,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_ucma.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,9 +30,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_ud_header.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_ud_header.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_ud_header.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,9 +31,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_umem.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_umem.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_umem.c Thu Aug 2 08:33:51 2018 (r337096) @@ -32,9 +32,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #define LINUXKPI_PARAM_PREFIX ibcore_ Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_umem_odp.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_umem_odp.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_umem_odp.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,9 +30,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_umem_rbtree.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_umem_rbtree.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_umem_rbtree.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,9 +30,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_user_mad.c Thu Aug 2 08:33:51 2018 (r337096) @@ -33,9 +33,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #define pr_fmt(fmt) "user_mad: " fmt Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_cmd.c Thu Aug 2 08:33:51 2018 (r337096) @@ -33,9 +33,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #define LINUXKPI_PARAM_PREFIX ibcore_ Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_main.c Thu Aug 2 08:33:51 2018 (r337096) @@ -34,9 +34,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_marshall.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_marshall.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_uverbs_marshall.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,9 +30,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include Modified: stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/core/ib_verbs.c Thu Aug 2 08:33:51 2018 (r337096) @@ -36,9 +36,10 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. - * - * $FreeBSD$ */ + +#include +__FBSDID("$FreeBSD$"); #include #include Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib.h ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib.h Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib.h Thu Aug 2 08:33:51 2018 (r337096) @@ -30,6 +30,8 @@ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE * SOFTWARE. + * + * $FreeBSD$ */ #ifndef _IPOIB_H Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_cm.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_cm.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_cm.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,6 +30,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include "ipoib.h" #ifdef CONFIG_INFINIBAND_IPOIB_CM Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,6 +30,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include #include Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_fs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_fs.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_fs.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,6 +30,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include #include Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ib.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ib.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_ib.c Thu Aug 2 08:33:51 2018 (r337096) @@ -33,6 +33,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include "ipoib.h" #include Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_main.c Thu Aug 2 08:33:51 2018 (r337096) @@ -32,6 +32,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include "ipoib.h" static int ipoib_resolvemulti(struct ifnet *, struct sockaddr **, @@ -318,8 +321,8 @@ ipoib_ioctl(struct ifnet *ifp, u_long command, caddr_t break; case SIOCGIFADDR: - bcopy(IF_LLADDR(ifp), &ifr->ifr_addr.sa_data[0], - INFINIBAND_ALEN); + bcopy(IF_LLADDR(ifp), &ifr->ifr_addr.sa_data[0], + INFINIBAND_ALEN); break; case SIOCSIFMTU: Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_multicast.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_multicast.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_multicast.c Thu Aug 2 08:33:51 2018 (r337096) @@ -32,6 +32,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include "ipoib.h" #include Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_verbs.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_verbs.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_verbs.c Thu Aug 2 08:33:51 2018 (r337096) @@ -31,6 +31,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include "ipoib.h" int ipoib_mcast_attach(struct ipoib_dev_priv *priv, u16 mlid, union ib_gid *mgid, int set_qkey) Modified: stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_vlan.c ============================================================================== --- stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_vlan.c Thu Aug 2 08:32:23 2018 (r337095) +++ stable/11/sys/ofed/drivers/infiniband/ulp/ipoib/ipoib_vlan.c Thu Aug 2 08:33:51 2018 (r337096) @@ -30,6 +30,9 @@ * SOFTWARE. */ +#include +__FBSDID("$FreeBSD$"); + #include #include From owner-svn-src-all@freebsd.org Thu Aug 2 08:35:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBC96105E68B; Thu, 2 Aug 2018 08:35:32 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 920DA7C33C; Thu, 2 Aug 2018 08:35:32 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 73EB4742E; Thu, 2 Aug 2018 08:35:32 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728ZWcR050645; Thu, 2 Aug 2018 08:35:32 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728ZWN3050644; Thu, 2 Aug 2018 08:35:32 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020835.w728ZWN3050644@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:35:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337097 - stable/11/sys/ofed/include/rdma X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/ofed/include/rdma X-SVN-Commit-Revision: 337097 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:35:33 -0000 Author: hselasky Date: Thu Aug 2 08:35:32 2018 New Revision: 337097 URL: https://svnweb.freebsd.org/changeset/base/337097 Log: MFC r336964: Only NULL check the VNET pointer when VIMAGE is enabled in ibcore. Else a NULL VNET pointer should be ignored. This fixes address resolving when VIMAGE is disabled. Sponsored by: Mellanox Technologies Modified: stable/11/sys/ofed/include/rdma/ib_sa.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/ofed/include/rdma/ib_sa.h ============================================================================== --- stable/11/sys/ofed/include/rdma/ib_sa.h Thu Aug 2 08:33:51 2018 (r337096) +++ stable/11/sys/ofed/include/rdma/ib_sa.h Thu Aug 2 08:35:32 2018 (r337097) @@ -180,7 +180,11 @@ struct ib_sa_path_rec { static inline struct net_device *ib_get_ndev_from_path(struct ib_sa_path_rec *rec) { - return rec->net ? dev_get_by_index(rec->net, rec->ifindex) : NULL; +#ifdef VIMAGE + if (rec->net == NULL) + return NULL; +#endif + return dev_get_by_index(rec->net, rec->ifindex); } #define IB_SA_MCMEMBER_REC_MGID IB_SA_COMP_MASK( 0) From owner-svn-src-all@freebsd.org Thu Aug 2 08:36:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A623B105E804; Thu, 2 Aug 2018 08:36:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 473107C59C; Thu, 2 Aug 2018 08:36:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 279327439; Thu, 2 Aug 2018 08:36:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728asp0050767; Thu, 2 Aug 2018 08:36:54 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728apw5050752; Thu, 2 Aug 2018 08:36:51 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020836.w728apw5050752@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:36:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337098 - in stable/11/sys/dev/mlx5: . mlx5_core mlx5_en mlx5_ib X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/dev/mlx5: . mlx5_core mlx5_en mlx5_ib X-SVN-Commit-Revision: 337098 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:36:55 -0000 Author: hselasky Date: Thu Aug 2 08:36:51 2018 New Revision: 337098 URL: https://svnweb.freebsd.org/changeset/base/337098 Log: MFC r336392: Implement support for Differentiated Service Code Point, DSCP, in mlx5en(4). The DSCP feature is controlled using a set of sysctl(8) fields under the qos sysctl directory entry for mlx5en(4). For Routable RoCE QPs, the DSCP should be set in the QP's address path. The DSCP's value is derived from the traffic class. Linux commit: ed88451e1f2d400fd6a743d0a481631cf9f97550 Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/device.h stable/11/sys/dev/mlx5/driver.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c stable/11/sys/dev/mlx5/mlx5_en/en.h stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib.h stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c stable/11/sys/dev/mlx5/mlx5_ifc.h stable/11/sys/dev/mlx5/port.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/device.h ============================================================================== --- stable/11/sys/dev/mlx5/device.h Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/device.h Thu Aug 2 08:36:51 2018 (r337098) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2018, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -47,11 +47,15 @@ #define __mlx5_nullp(typ) ((struct mlx5_ifc_##typ##_bits *)0) #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld) #define __mlx5_bit_off(typ, fld) __offsetof(struct mlx5_ifc_##typ##_bits, fld) +#define __mlx5_16_off(typ, fld) (__mlx5_bit_off(typ, fld) / 16) #define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32) #define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64) +#define __mlx5_16_bit_off(typ, fld) (16 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0xf)) #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f)) #define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1)) #define __mlx5_dw_mask(typ, fld) (__mlx5_mask(typ, fld) << __mlx5_dw_bit_off(typ, fld)) +#define __mlx5_mask16(typ, fld) ((u16)((1ull << __mlx5_bit_sz(typ, fld)) - 1)) +#define __mlx5_16_mask(typ, fld) (__mlx5_mask16(typ, fld) << __mlx5_16_bit_off(typ, fld)) #define __mlx5_st_sz_bits(typ) sizeof(struct mlx5_ifc_##typ##_bits) #define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8) @@ -109,6 +113,19 @@ __mlx5_mask(typ, fld)) #define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld))) +#define MLX5_GET16(typ, p, fld) ((be16_to_cpu(*((__be16 *)(p) +\ +__mlx5_16_off(typ, fld))) >> __mlx5_16_bit_off(typ, fld)) & \ +__mlx5_mask16(typ, fld)) + +#define MLX5_SET16(typ, p, fld, v) do { \ + u16 _v = v; \ + BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 16); \ + *((__be16 *)(p) + __mlx5_16_off(typ, fld)) = \ + cpu_to_be16((be16_to_cpu(*((__be16 *)(p) + __mlx5_16_off(typ, fld))) & \ + (~__mlx5_16_mask(typ, fld))) | (((_v) & __mlx5_mask16(typ, fld)) \ + << __mlx5_16_bit_off(typ, fld))); \ +} while (0) + #define MLX5_GET64_BE(typ, p, fld) (*((__be64 *)(p) +\ __mlx5_64_off(typ, fld))) @@ -895,6 +912,14 @@ enum mlx5_cap_type { MLX5_CAP_NUM }; +enum mlx5_qcam_reg_groups { + MLX5_QCAM_REGS_FIRST_128 = 0x0, +}; + +enum mlx5_qcam_feature_groups { + MLX5_QCAM_FEATURE_ENHANCED_FEATURES = 0x0, +}; + /* GET Dev Caps macros */ #define MLX5_CAP_GEN(mdev, cap) \ MLX5_GET(cmd_hca_cap, mdev->hca_caps_cur[MLX5_CAP_GENERAL], cap) @@ -999,6 +1024,12 @@ enum mlx5_cap_type { #define MLX5_CAP_QOS_MAX(mdev, cap) \ MLX5_GET(qos_cap,\ mdev->hca_caps_max[MLX5_CAP_QOS], cap) + +#define MLX5_CAP_QCAM_REG(mdev, fld) \ + MLX5_GET(qcam_reg, (mdev)->caps.qcam, qos_access_reg_cap_mask.reg_cap.fld) + +#define MLX5_CAP_QCAM_FEATURE(mdev, fld) \ + MLX5_GET(qcam_reg, (mdev)->caps.qcam, qos_feature_cap_mask.feature_cap.fld) enum { MLX5_CMD_STAT_OK = 0x0, Modified: stable/11/sys/dev/mlx5/driver.h ============================================================================== --- stable/11/sys/dev/mlx5/driver.h Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/driver.h Thu Aug 2 08:36:51 2018 (r337098) @@ -119,10 +119,13 @@ enum { }; enum { + MLX5_REG_QPTS = 0x4002, MLX5_REG_QETCR = 0x4005, MLX5_REG_QPDP = 0x4007, MLX5_REG_QTCT = 0x400A, + MLX5_REG_QPDPM = 0x4013, MLX5_REG_QHLL = 0x4016, + MLX5_REG_QCAM = 0x4019, MLX5_REG_DCBX_PARAM = 0x4020, MLX5_REG_DCBX_APP = 0x4021, MLX5_REG_PCAP = 0x5001, @@ -626,6 +629,9 @@ struct mlx5_core_dev { struct mlx5_port_caps port_caps[MLX5_MAX_PORTS]; u32 hca_caps_cur[MLX5_CAP_NUM][MLX5_UN_SZ_DW(hca_cap_union)]; u32 hca_caps_max[MLX5_CAP_NUM][MLX5_UN_SZ_DW(hca_cap_union)]; + struct { + u32 qcam[MLX5_ST_SZ_DW(qcam_reg)]; + } caps; phys_addr_t iseg_base; struct mlx5_init_seg __iomem *iseg; enum mlx5_device_state state; Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:36:51 2018 (r337098) @@ -80,6 +80,8 @@ struct mlx5_core_dev; int mlx5_query_hca_caps(struct mlx5_core_dev *dev); int mlx5_query_board_id(struct mlx5_core_dev *dev); +int mlx5_query_qcam_reg(struct mlx5_core_dev *mdev, u32 *qcam, + u8 feature_group, u8 access_reg_group); int mlx5_cmd_init_hca(struct mlx5_core_dev *dev); int mlx5_cmd_teardown_hca(struct mlx5_core_dev *dev); int mlx5_cmd_force_teardown_hca(struct mlx5_core_dev *dev); Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c Thu Aug 2 08:36:51 2018 (r337098) @@ -110,6 +110,13 @@ static int mlx5_core_query_special_contexts(struct mlx return err; } +static int mlx5_get_qcam_reg(struct mlx5_core_dev *dev) +{ + return mlx5_query_qcam_reg(dev, dev->caps.qcam, + MLX5_QCAM_FEATURE_ENHANCED_FEATURES, + MLX5_QCAM_REGS_FIRST_128); +} + int mlx5_query_hca_caps(struct mlx5_core_dev *dev) { int err; @@ -185,6 +192,12 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev) if (MLX5_CAP_GEN(dev, qos)) { err = mlx5_core_get_caps(dev, MLX5_CAP_QOS); + if (err) + return err; + } + + if (MLX5_CAP_GEN(dev, qcam_reg)) { + err = mlx5_get_qcam_reg(dev); if (err) return err; } Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c Thu Aug 2 08:36:51 2018 (r337098) @@ -66,6 +66,19 @@ out: } EXPORT_SYMBOL_GPL(mlx5_core_access_reg); +int mlx5_query_qcam_reg(struct mlx5_core_dev *mdev, u32 *qcam, + u8 feature_group, u8 access_reg_group) +{ + u32 in[MLX5_ST_SZ_DW(qcam_reg)] = {}; + int sz = MLX5_ST_SZ_BYTES(qcam_reg); + + MLX5_SET(qcam_reg, in, feature_group, feature_group); + MLX5_SET(qcam_reg, in, access_reg_group, access_reg_group); + + return mlx5_core_access_reg(mdev, in, sz, qcam, sz, MLX5_REG_QCAM, 0, 0); +} +EXPORT_SYMBOL_GPL(mlx5_query_qcam_reg); + struct mlx5_reg_pcap { u8 rsvd0; u8 port_num; @@ -966,4 +979,102 @@ int mlx5_query_diagnostic_counters(struct mlx5_core_de MLX5_SET(query_diagnostic_counters_in, in, sample_index, sample_index); return mlx5_cmd_exec(mdev, in, sizeof(in), out, out_size); +} + +int mlx5_set_trust_state(struct mlx5_core_dev *mdev, u8 trust_state) +{ + u32 out[MLX5_ST_SZ_DW(qpts_reg)] = {}; + u32 in[MLX5_ST_SZ_DW(qpts_reg)] = {}; + int err; + + MLX5_SET(qpts_reg, in, local_port, 1); + MLX5_SET(qpts_reg, in, trust_state, trust_state); + + err = mlx5_core_access_reg(mdev, in, sizeof(in), out, + sizeof(out), MLX5_REG_QPTS, 0, 1); + return err; +} + +int mlx5_query_trust_state(struct mlx5_core_dev *mdev, u8 *trust_state) +{ + u32 out[MLX5_ST_SZ_DW(qpts_reg)] = {}; + u32 in[MLX5_ST_SZ_DW(qpts_reg)] = {}; + int err; + + MLX5_SET(qpts_reg, in, local_port, 1); + + err = mlx5_core_access_reg(mdev, in, sizeof(in), out, + sizeof(out), MLX5_REG_QPTS, 0, 0); + if (!err) + *trust_state = MLX5_GET(qpts_reg, out, trust_state); + + return err; +} + +int mlx5_set_dscp2prio(struct mlx5_core_dev *mdev, const u8 *dscp2prio) +{ + int sz = MLX5_ST_SZ_BYTES(qpdpm_reg); + void *qpdpm_dscp; + void *out; + void *in; + int err; + int i; + + in = kzalloc(sz, GFP_KERNEL); + out = kzalloc(sz, GFP_KERNEL); + if (!in || !out) { + err = -ENOMEM; + goto out; + } + + MLX5_SET(qpdpm_reg, in, local_port, 1); + err = mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_QPDPM, 0, 0); + if (err) + goto out; + + memcpy(in, out, sz); + MLX5_SET(qpdpm_reg, in, local_port, 1); + + /* Update the corresponding dscp entry */ + for (i = 0; i < MLX5_MAX_SUPPORTED_DSCP; i++) { + qpdpm_dscp = MLX5_ADDR_OF(qpdpm_reg, in, dscp[i]); + MLX5_SET16(qpdpm_dscp_reg, qpdpm_dscp, prio, dscp2prio[i]); + MLX5_SET16(qpdpm_dscp_reg, qpdpm_dscp, e, 1); + } + err = mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_QPDPM, 0, 1); +out: + kfree(in); + kfree(out); + return err; +} + +int mlx5_query_dscp2prio(struct mlx5_core_dev *mdev, u8 *dscp2prio) +{ + int sz = MLX5_ST_SZ_BYTES(qpdpm_reg); + void *qpdpm_dscp; + void *out; + void *in; + int err; + int i; + + in = kzalloc(sz, GFP_KERNEL); + out = kzalloc(sz, GFP_KERNEL); + if (!in || !out) { + err = -ENOMEM; + goto out; + } + + MLX5_SET(qpdpm_reg, in, local_port, 1); + err = mlx5_core_access_reg(mdev, in, sz, out, sz, MLX5_REG_QPDPM, 0, 0); + if (err) + goto out; + + for (i = 0; i < MLX5_MAX_SUPPORTED_DSCP; i++) { + qpdpm_dscp = MLX5_ADDR_OF(qpdpm_reg, out, dscp[i]); + dscp2prio[i] = MLX5_GET16(qpdpm_dscp_reg, qpdpm_dscp, prio); + } +out: + kfree(in); + kfree(out); + return err; } Modified: stable/11/sys/dev/mlx5/mlx5_en/en.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:36:51 2018 (r337098) @@ -476,6 +476,8 @@ struct mlx5e_params_ethtool { MLX5E_PARAMS(MLX5E_STATS_VAR) u64 max_bw_value[IEEE_8021QAZ_MAX_TCS]; u8 prio_tc[IEEE_8021QAZ_MAX_TCS]; + u8 dscp2prio[MLX5_MAX_SUPPORTED_DSCP]; + u8 trust_state; }; /* EEPROM Standards for plug in modules */ Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Aug 2 08:36:51 2018 (r337098) @@ -121,6 +121,30 @@ done: } static int +mlx5e_get_dscp(struct mlx5e_priv *priv) +{ + struct mlx5_core_dev *mdev = priv->mdev; + int err; + + if (MLX5_CAP_GEN(mdev, qcam_reg) == 0 || + MLX5_CAP_QCAM_REG(mdev, qpts) == 0 || + MLX5_CAP_QCAM_REG(mdev, qpdpm) == 0) + return (EOPNOTSUPP); + + PRIV_LOCK(priv); + err = -mlx5_query_dscp2prio(mdev, priv->params_ethtool.dscp2prio); + if (err) + goto done; + + err = -mlx5_query_trust_state(mdev, &priv->params_ethtool.trust_state); + if (err) + goto done; +done: + PRIV_UNLOCK(priv); + return (err); +} + +static int mlx5e_tc_maxrate_handler(SYSCTL_HANDLER_ARGS) { struct mlx5e_priv *priv = arg1; @@ -229,6 +253,84 @@ done: return (err); } +static int +mlx5e_trust_state_handler(SYSCTL_HANDLER_ARGS) +{ + struct mlx5e_priv *priv = arg1; + struct mlx5_core_dev *mdev = priv->mdev; + int err; + u8 result; + + PRIV_LOCK(priv); + result = priv->params_ethtool.trust_state; + err = sysctl_handle_8(oidp, &result, 0, req); + if (err || !req->newptr || + result == priv->params_ethtool.trust_state) + goto done; + + switch (result) { + case MLX5_QPTS_TRUST_PCP: + case MLX5_QPTS_TRUST_DSCP: + break; + case MLX5_QPTS_TRUST_BOTH: + if (!MLX5_CAP_QCAM_FEATURE(mdev, qpts_trust_both)) { + err = EOPNOTSUPP; + goto done; + } + break; + default: + err = ERANGE; + goto done; + } + + err = -mlx5_set_trust_state(mdev, result); + if (err) + goto done; + + priv->params_ethtool.trust_state = result; +done: + PRIV_UNLOCK(priv); + return (err); +} + +static int +mlx5e_dscp_prio_handler(SYSCTL_HANDLER_ARGS) +{ + struct mlx5e_priv *priv = arg1; + int prio_index = arg2; + struct mlx5_core_dev *mdev = priv->mdev; + uint8_t dscp2prio[MLX5_MAX_SUPPORTED_DSCP]; + uint8_t x; + int err; + + PRIV_LOCK(priv); + err = SYSCTL_OUT(req, priv->params_ethtool.dscp2prio + prio_index, + sizeof(priv->params_ethtool.dscp2prio) / 8); + if (err || !req->newptr) + goto done; + + memcpy(dscp2prio, priv->params_ethtool.dscp2prio, sizeof(dscp2prio)); + err = SYSCTL_IN(req, dscp2prio + prio_index, sizeof(dscp2prio) / 8); + if (err) + goto done; + for (x = 0; x != MLX5_MAX_SUPPORTED_DSCP; x++) { + if (dscp2prio[x] > 7) { + err = ERANGE; + goto done; + } + } + err = -mlx5_set_dscp2prio(mdev, dscp2prio); + if (err) + goto done; + + /* update local array */ + memcpy(priv->params_ethtool.dscp2prio, dscp2prio, + sizeof(priv->params_ethtool.dscp2prio)); +done: + PRIV_UNLOCK(priv); + return (err); +} + #define MLX5_PARAM_OFFSET(n) \ __offsetof(struct mlx5e_priv, params_ethtool.n) @@ -1009,5 +1111,21 @@ mlx5e_create_ethtool(struct mlx5e_priv *priv) OID_AUTO, name, CTLTYPE_U8 | CTLFLAG_RW | CTLFLAG_MPSAFE, priv, i, mlx5e_prio_to_tc_handler, "CU", "Set priority to traffic class"); + } + + /* DSCP support */ + if (mlx5e_get_dscp(priv) == 0) { + for (i = 0; i != MLX5_MAX_SUPPORTED_DSCP; i += 8) { + char name[32]; + snprintf(name, sizeof(name), "dscp_%d_%d_prio", i, i + 7); + SYSCTL_ADD_PROC(&priv->sysctl_ctx, SYSCTL_CHILDREN(qos_node), + OID_AUTO, name, CTLTYPE_U8 | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, + priv, i, mlx5e_dscp_prio_handler, "CU", + "Set DSCP to priority mapping, 0..7"); + } + SYSCTL_ADD_PROC(&priv->sysctl_ctx, SYSCTL_CHILDREN(qos_node), + OID_AUTO, "trust_state", CTLTYPE_U8 | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, + priv, 0, mlx5e_trust_state_handler, "CU", + "Set trust state, 1:PCP 2:DSCP 3:BOTH"); } } Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib.h Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib.h Thu Aug 2 08:36:51 2018 (r337098) @@ -944,6 +944,8 @@ int mlx5_ib_set_vf_guid(struct ib_device *device, int __be16 mlx5_get_roce_udp_sport(struct mlx5_ib_dev *dev, u8 port_num, int index); +int mlx5_get_roce_gid_type(struct mlx5_ib_dev *dev, u8 port_num, + int index, enum ib_gid_type *gid_type); /* GSI QP helper functions */ struct ib_qp *mlx5_ib_gsi_create_qp(struct ib_pd *pd, Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:36:51 2018 (r337098) @@ -380,6 +380,27 @@ __be16 mlx5_get_roce_udp_sport(struct mlx5_ib_dev *dev return cpu_to_be16(MLX5_CAP_ROCE(dev->mdev, r_roce_min_src_udp_port)); } +int mlx5_get_roce_gid_type(struct mlx5_ib_dev *dev, u8 port_num, + int index, enum ib_gid_type *gid_type) +{ + struct ib_gid_attr attr; + union ib_gid gid; + int ret; + + ret = ib_get_cached_gid(&dev->ib_dev, port_num, index, &gid, &attr); + if (ret) + return ret; + + if (!attr.ndev) + return -ENODEV; + + dev_put(attr.ndev); + + *gid_type = attr.gid_type; + + return 0; +} + static int mlx5_use_mad_ifc(struct mlx5_ib_dev *dev) { if (MLX5_CAP_GEN(dev->mdev, port_type) == MLX5_CAP_PORT_TYPE_IB) Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_qp.c Thu Aug 2 08:36:51 2018 (r337098) @@ -2195,6 +2195,7 @@ static int mlx5_set_path(struct mlx5_ib_dev *dev, stru { enum rdma_link_layer ll = rdma_port_get_link_layer(&dev->ib_dev, port); int err; + enum ib_gid_type gid_type; if (attr_mask & IB_QP_PKEY_INDEX) path->pkey_index = cpu_to_be16(alt ? attr->alt_pkey_index : @@ -2213,10 +2214,16 @@ static int mlx5_set_path(struct mlx5_ib_dev *dev, stru if (ll == IB_LINK_LAYER_ETHERNET) { if (!(ah->ah_flags & IB_AH_GRH)) return -EINVAL; + err = mlx5_get_roce_gid_type(dev, port, ah->grh.sgid_index, + &gid_type); + if (err) + return err; memcpy(path->rmac, ah->dmac, sizeof(ah->dmac)); path->udp_sport = mlx5_get_roce_udp_sport(dev, port, ah->grh.sgid_index); path->dci_cfi_prio_sl = (ah->sl & 0x7) << 4; + if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP) + path->ecn_dscp = (ah->grh.traffic_class >> 2) & 0x3f; } else { path->fl_free_ar = (path_flags & MLX5_PATH_FLAG_FL) ? 0x80 : 0; path->fl_free_ar |= Modified: stable/11/sys/dev/mlx5/mlx5_ifc.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ifc.h Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/mlx5_ifc.h Thu Aug 2 08:36:51 2018 (r337098) @@ -962,7 +962,8 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 cc_modify_allowed[0x1]; u8 start_pad[0x1]; u8 cache_line_128byte[0x1]; - u8 reserved_15[0xb]; + u8 reserved_at_165[0xa]; + u8 qcam_reg[0x1]; u8 gid_table_size[0x10]; u8 out_of_seq_cnt[0x1]; @@ -8383,6 +8384,43 @@ struct mlx5_ifc_peir_reg_bits { u8 error_type[0x8]; }; +struct mlx5_ifc_qcam_access_reg_cap_mask { + u8 qcam_access_reg_cap_mask_127_to_20[0x6C]; + u8 qpdpm[0x1]; + u8 qcam_access_reg_cap_mask_18_to_4[0x0F]; + u8 qdpm[0x1]; + u8 qpts[0x1]; + u8 qcap[0x1]; + u8 qcam_access_reg_cap_mask_0[0x1]; +}; + +struct mlx5_ifc_qcam_qos_feature_cap_mask { + u8 qcam_qos_feature_cap_mask_127_to_1[0x7F]; + u8 qpts_trust_both[0x1]; +}; + +struct mlx5_ifc_qcam_reg_bits { + u8 reserved_at_0[0x8]; + u8 feature_group[0x8]; + u8 reserved_at_10[0x8]; + u8 access_reg_group[0x8]; + u8 reserved_at_20[0x20]; + + union { + struct mlx5_ifc_qcam_access_reg_cap_mask reg_cap; + u8 reserved_at_0[0x80]; + } qos_access_reg_cap_mask; + + u8 reserved_at_c0[0x80]; + + union { + struct mlx5_ifc_qcam_qos_feature_cap_mask feature_cap; + u8 reserved_at_0[0x80]; + } qos_feature_cap_mask; + + u8 reserved_at_1c0[0x80]; +}; + struct mlx5_ifc_pcap_reg_bits { u8 reserved_0[0x8]; u8 local_port[0x8]; @@ -9695,5 +9733,24 @@ union mlx5_ifc_uplink_pci_interface_document_bits { u8 reserved_0[0x20120]; }; +struct mlx5_ifc_qpdpm_dscp_reg_bits { + u8 e[0x1]; + u8 reserved_at_01[0x0b]; + u8 prio[0x04]; +}; + +struct mlx5_ifc_qpdpm_reg_bits { + u8 reserved_at_0[0x8]; + u8 local_port[0x8]; + u8 reserved_at_10[0x10]; + struct mlx5_ifc_qpdpm_dscp_reg_bits dscp[64]; +}; + +struct mlx5_ifc_qpts_reg_bits { + u8 reserved_at_0[0x8]; + u8 local_port[0x8]; + u8 reserved_at_10[0x2d]; + u8 trust_state[0x3]; +}; #endif /* MLX5_IFC_H */ Modified: stable/11/sys/dev/mlx5/port.h ============================================================================== --- stable/11/sys/dev/mlx5/port.h Thu Aug 2 08:35:32 2018 (r337097) +++ stable/11/sys/dev/mlx5/port.h Thu Aug 2 08:36:51 2018 (r337098) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2016, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2016-2018, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -102,6 +102,12 @@ enum mlx5e_connector_type { MLX5E_CONNECTOR_TYPE_NUMBER, }; +enum mlx5_qpts_trust_state { + MLX5_QPTS_TRUST_PCP = 1, + MLX5_QPTS_TRUST_DSCP = 2, + MLX5_QPTS_TRUST_BOTH = 3, +}; + #define MLX5E_PROT_MASK(link_mode) (1 << (link_mode)) #define PORT_MODULE_EVENT_MODULE_STATUS_MASK 0xF @@ -155,4 +161,11 @@ int mlx5_query_port_prio_tc(struct mlx5_core_dev *mdev u8 prio, u8 *tc); int mlx5_set_port_prio_tc(struct mlx5_core_dev *mdev, int prio_index, const u8 prio_tc); +int mlx5_set_trust_state(struct mlx5_core_dev *mdev, u8 trust_state); +int mlx5_query_trust_state(struct mlx5_core_dev *mdev, u8 *trust_state); + +#define MLX5_MAX_SUPPORTED_DSCP 64 +int mlx5_set_dscp2prio(struct mlx5_core_dev *mdev, const u8 *dscp2prio); +int mlx5_query_dscp2prio(struct mlx5_core_dev *mdev, u8 *dscp2prio); + #endif /* __MLX5_PORT_H__ */ From owner-svn-src-all@freebsd.org Thu Aug 2 08:37:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68A67105E8EC; Thu, 2 Aug 2018 08:37:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1E7FF7C732; Thu, 2 Aug 2018 08:37:45 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F2A1D743A; Thu, 2 Aug 2018 08:37:44 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728bid4050860; Thu, 2 Aug 2018 08:37:44 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728bieA050859; Thu, 2 Aug 2018 08:37:44 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020837.w728bieA050859@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:37:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337099 - stable/11/sys/dev/mlx5/mlx5_ib X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_ib X-SVN-Commit-Revision: 337099 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:37:45 -0000 Author: hselasky Date: Thu Aug 2 08:37:44 2018 New Revision: 337099 URL: https://svnweb.freebsd.org/changeset/base/337099 Log: MFC r336393: Use static device naming instead of dynamic one in mlx5ib. When resetting mlx5core instances it can happen that the order of attach and detach for mlx5ib instances is changed. Take the unit number for mlx5_%d from the parent PCI device, similarly to what is done in mlx5en(4), so that there is a direct relationship between mce and mlx5_. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:36:51 2018 (r337098) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:37:44 2018 (r337099) @@ -2957,7 +2957,6 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev) struct mlx5_ib_dev *dev; enum rdma_link_layer ll; int port_type_cap; - const char *name; int err; int i; @@ -2990,9 +2989,7 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev) MLX5_INIT_DOORBELL_LOCK(&dev->uar_lock); - name = "mlx5_%d"; - - strlcpy(dev->ib_dev.name, name, IB_DEVICE_NAME_MAX); + snprintf(dev->ib_dev.name, IB_DEVICE_NAME_MAX, "mlx5_%d", device_get_unit(mdev->pdev->dev.bsddev)); dev->ib_dev.owner = THIS_MODULE; dev->ib_dev.node_type = RDMA_NODE_IB_CA; dev->ib_dev.local_dma_lkey = 0 /* not supported for now */; From owner-svn-src-all@freebsd.org Thu Aug 2 08:38:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 652F2105E9E0; Thu, 2 Aug 2018 08:38:38 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 199B67C892; Thu, 2 Aug 2018 08:38:38 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EECC9743C; Thu, 2 Aug 2018 08:38:37 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728cbaV050970; Thu, 2 Aug 2018 08:38:37 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728cbb4050969; Thu, 2 Aug 2018 08:38:37 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020838.w728cbb4050969@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:38:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337100 - stable/11/sys/dev/mlx5/mlx5_ib X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_ib X-SVN-Commit-Revision: 337100 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:38:38 -0000 Author: hselasky Date: Thu Aug 2 08:38:37 2018 New Revision: 337100 URL: https://svnweb.freebsd.org/changeset/base/337100 Log: MFC r336394: Don't pass unsupported events to ibcore from mlx5ib. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:37:44 2018 (r337099) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:38:37 2018 (r337100) @@ -2381,14 +2381,15 @@ static void mlx5_ib_event(struct mlx5_core_dev *dev, v break; default: - break; + /* unsupported event */ + return; } ibev.device = &ibdev->ib_dev; ibev.element.port_num = port; if (port < 1 || port > ibdev->num_ports) { - mlx5_ib_warn(ibdev, "warning: event on port %d\n", port); + mlx5_ib_warn(ibdev, "warning: event(%d) on port %d\n", event, port); return; } From owner-svn-src-all@freebsd.org Thu Aug 2 08:39:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B806105EB4F; Thu, 2 Aug 2018 08:39:49 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C531C7CB42; Thu, 2 Aug 2018 08:39:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A6DB67441; Thu, 2 Aug 2018 08:39:48 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728dmS2051197; Thu, 2 Aug 2018 08:39:48 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728dmTZ051196; Thu, 2 Aug 2018 08:39:48 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020839.w728dmTZ051196@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:39:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337101 - stable/11/sys/dev/mlx5/mlx5_ib X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_ib X-SVN-Commit-Revision: 337101 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:39:49 -0000 Author: hselasky Date: Thu Aug 2 08:39:48 2018 New Revision: 337101 URL: https://svnweb.freebsd.org/changeset/base/337101 Log: MFC r336395: Update version information for the mlx5ib module. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:38:37 2018 (r337100) +++ stable/11/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c Thu Aug 2 08:39:48 2018 (r337101) @@ -51,8 +51,10 @@ #include "mlx5_ib.h" #define DRIVER_NAME "mlx5_ib" -#define DRIVER_VERSION "3.4.1-BETA" -#define DRIVER_RELDATE "October 2017" +#ifndef DRIVER_VERSION +#define DRIVER_VERSION "3.4.2" +#endif +#define DRIVER_RELDATE "July 2018" MODULE_DESCRIPTION("Mellanox Connect-IB HCA IB driver"); MODULE_LICENSE("Dual BSD/GPL"); @@ -65,8 +67,8 @@ static int deprecated_prof_sel = 2; module_param_named(prof_sel, deprecated_prof_sel, int, 0444); MODULE_PARM_DESC(prof_sel, "profile selector. Deprecated here. Moved to module mlx5_core"); -static char mlx5_version[] = - DRIVER_NAME ": Mellanox Connect-IB Infiniband driver v" +static const char mlx5_version[] = + DRIVER_NAME ": Mellanox Connect-IB Infiniband driver " DRIVER_VERSION " (" DRIVER_RELDATE ")\n"; enum { From owner-svn-src-all@freebsd.org Thu Aug 2 08:40:47 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECAC9105EC22; Thu, 2 Aug 2018 08:40:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B35D7CCE4; Thu, 2 Aug 2018 08:40:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6CC967455; Thu, 2 Aug 2018 08:40:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728ekFw051322; Thu, 2 Aug 2018 08:40:46 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728ekSA051321; Thu, 2 Aug 2018 08:40:46 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020840.w728ekSA051321@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:40:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337102 - stable/11/sys/dev/mlx5/mlx5_core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_core X-SVN-Commit-Revision: 337102 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:40:47 -0000 Author: hselasky Date: Thu Aug 2 08:40:46 2018 New Revision: 337102 URL: https://svnweb.freebsd.org/changeset/base/337102 Log: MFC r336396: Remove redundant newline character in mlx5core. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c Thu Aug 2 08:39:48 2018 (r337101) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c Thu Aug 2 08:40:46 2018 (r337102) @@ -159,8 +159,7 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev) return err; } - if ( - MLX5_CAP_GEN(dev, eswitch_flow_table)) { + if (MLX5_CAP_GEN(dev, eswitch_flow_table)) { err = mlx5_core_get_caps(dev, MLX5_CAP_ESWITCH_FLOW_TABLE); if (err) return err; From owner-svn-src-all@freebsd.org Thu Aug 2 08:42:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEEA6105EE6B; Thu, 2 Aug 2018 08:42:42 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70F007D0BE; Thu, 2 Aug 2018 08:42:42 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 51DC875D1; Thu, 2 Aug 2018 08:42:42 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728ggR3056083; Thu, 2 Aug 2018 08:42:42 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728geCx056075; Thu, 2 Aug 2018 08:42:40 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020842.w728geCx056075@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:42:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337103 - in stable/11/sys: conf dev/mlx5 dev/mlx5/mlx5_core modules/mlx5 X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys: conf dev/mlx5 dev/mlx5/mlx5_core modules/mlx5 X-SVN-Commit-Revision: 337103 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:42:43 -0000 Author: hselasky Date: Thu Aug 2 08:42:40 2018 New Revision: 337103 URL: https://svnweb.freebsd.org/changeset/base/337103 Log: MFC r336397: Refactor access to CR-space into using VSC APIs in mlx5core. Remove no longer used files and APIs. Sponsored by: Mellanox Technologies Deleted: stable/11/sys/dev/mlx5/mlx5_core/mlx5_crspace.c Modified: stable/11/sys/conf/files stable/11/sys/dev/mlx5/driver.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c stable/11/sys/modules/mlx5/Makefile Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/conf/files ============================================================================== --- stable/11/sys/conf/files Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/conf/files Thu Aug 2 08:42:40 2018 (r337103) @@ -4542,8 +4542,6 @@ dev/mlx5/mlx5_core/mlx5_cmd.c optional mlx5 pci \ compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_cq.c optional mlx5 pci \ compile-with "${OFED_C}" -dev/mlx5/mlx5_core/mlx5_crspace.c optional mlx5 pci \ - compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_diagnostics.c optional mlx5 pci \ compile-with "${OFED_C}" dev/mlx5/mlx5_core/mlx5_eq.c optional mlx5 pci \ Modified: stable/11/sys/dev/mlx5/driver.h ============================================================================== --- stable/11/sys/dev/mlx5/driver.h Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/driver.h Thu Aug 2 08:42:40 2018 (r337103) @@ -656,7 +656,6 @@ struct mlx5_core_dev { struct mlx5_flow_root_namespace *sniffer_tx_root_ns; u32 num_q_counter_allocated[MLX5_INTERFACE_NUMBER]; struct mlx5_dump_data *dump_data; - u32 vsec_addr; }; enum { @@ -1029,8 +1028,11 @@ int mlx5_vsc_find_cap(struct mlx5_core_dev *mdev); int mlx5_vsc_lock(struct mlx5_core_dev *mdev); void mlx5_vsc_unlock(struct mlx5_core_dev *mdev); int mlx5_vsc_set_space(struct mlx5_core_dev *mdev, u16 space); -int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, u32 *data); +int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, const u32 *data); int mlx5_vsc_read(struct mlx5_core_dev *mdev, u32 addr, u32 *data); +int mlx5_vsc_lock_addr_space(struct mlx5_core_dev *mdev, u32 addr); +int mlx5_vsc_unlock_addr_space(struct mlx5_core_dev *mdev, u32 addr); + static inline u32 mlx5_mkey_to_idx(u32 mkey) { return mkey >> 8; Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:42:40 2018 (r337103) @@ -70,12 +70,6 @@ enum mlx5_semaphore_space_address { MLX5_SEMAPHORE_SW_RESET = 0x20, }; -enum { - UNLOCK = 0, - LOCK = 1, - CAP_ID = 0x9, -}; - struct mlx5_core_dev; int mlx5_query_hca_caps(struct mlx5_core_dev *dev); @@ -109,8 +103,4 @@ struct mlx5_crspace_regmap { extern struct pci_driver mlx5_core_driver; -void mlx5_vsec_init(struct mlx5_core_dev *dev); -int mlx5_pciconf_cap9_sem(struct mlx5_core_dev *dev, int state); -int mlx5_pciconf_set_sem_addr_space(struct mlx5_core_dev *dev, - u32 sem_space_address, int state); #endif /* __MLX5_CORE_H__ */ Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:42:40 2018 (r337103) @@ -59,19 +59,19 @@ enum { MLX5_SENSOR_FW_SYND_RFR = 5, }; -static int lock_sem_sw_reset(struct mlx5_core_dev *dev, int state) +static int lock_sem_sw_reset(struct mlx5_core_dev *dev) { - int ret, err; + int ret; /* Lock GW access */ - ret = mlx5_pciconf_cap9_sem(dev, LOCK); + ret = -mlx5_vsc_lock(dev); if (ret) { - mlx5_core_warn(dev, "Timed out locking gateway %d, %d\n", state, ret); + mlx5_core_warn(dev, "Timed out locking gateway %d\n", ret); return ret; } - ret = mlx5_pciconf_set_sem_addr_space(dev, MLX5_SEMAPHORE_SW_RESET, state); - if (ret && state == LOCK) { + ret = -mlx5_vsc_lock_addr_space(dev, MLX5_SEMAPHORE_SW_RESET); + if (ret) { if (ret == -EBUSY) mlx5_core_dbg(dev, "SW reset FW semaphore already locked, another function will handle the reset\n"); else @@ -79,13 +79,30 @@ static int lock_sem_sw_reset(struct mlx5_core_dev *dev } /* Unlock GW access */ - err = mlx5_pciconf_cap9_sem(dev, UNLOCK); - if (err) - mlx5_core_warn(dev, "Timed out unlocking gateway: state %d, err %d\n", state, err); + mlx5_vsc_unlock(dev); return ret; } +static int unlock_sem_sw_reset(struct mlx5_core_dev *dev) +{ + int ret; + + /* Lock GW access */ + ret = -mlx5_vsc_lock(dev); + if (ret) { + mlx5_core_warn(dev, "Timed out locking gateway %d\n", ret); + return ret; + } + + ret = -mlx5_vsc_unlock_addr_space(dev, MLX5_SEMAPHORE_SW_RESET); + + /* Unlock GW access */ + mlx5_vsc_unlock(dev); + + return ret; +} + static u8 get_nic_mode(struct mlx5_core_dev *dev) { return (ioread32be(&dev->iseg->cmdq_addr_l_sz) >> 8) & 7; @@ -223,7 +240,7 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, if (fatal_error == MLX5_SENSOR_FW_SYND_RFR) { /* Get cr-dump and reset FW semaphore */ if (mlx5_core_is_pf(dev)) - lock = lock_sem_sw_reset(dev, LOCK); + lock = lock_sem_sw_reset(dev); /* Execute cr-dump and SW reset */ if (lock != -EBUSY) { @@ -249,7 +266,7 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, /* Release FW semaphore if you are the lock owner */ if (!lock) - lock_sem_sw_reset(dev, UNLOCK); + unlock_sem_sw_reset(dev); mlx5_core_err(dev, "system error event triggered\n"); Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:42:40 2018 (r337103) @@ -873,7 +873,9 @@ static int mlx5_init_once(struct mlx5_core_dev *dev, s struct pci_dev *pdev = dev->pdev; int err; - mlx5_vsec_init(dev); + err = mlx5_vsc_find_cap(dev); + if (err) + dev_err(&pdev->dev, "Unable to find vendor specific capabilities\n"); err = mlx5_query_hca_caps(dev); if (err) { Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_vsc.c Thu Aug 2 08:42:40 2018 (r337103) @@ -29,6 +29,8 @@ #include #include +#define MLX5_SEMAPHORE_SPACE_DOMAIN 0xA + struct mlx5_ifc_vsc_space_bits { u8 status[0x3]; u8 reserved0[0xd]; @@ -139,7 +141,7 @@ int mlx5_vsc_set_space(struct mlx5_core_dev *mdev, u16 return 0; } -int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, u32 *data) +int mlx5_vsc_write(struct mlx5_core_dev *mdev, u32 addr, const u32 *data) { device_t dev = mdev->pdev->dev.bsddev; int vsc_addr = mdev->vsc_addr; @@ -185,6 +187,60 @@ int mlx5_vsc_read(struct mlx5_core_dev *mdev, u32 addr } *data = pci_read_config(dev, vsc_addr + MLX5_VSC_DATA_OFFSET, 4); + + return 0; +} + +int mlx5_vsc_lock_addr_space(struct mlx5_core_dev *mdev, u32 addr) +{ + device_t dev = mdev->pdev->dev.bsddev; + int vsc_addr = mdev->vsc_addr; + u32 data; + int ret; + u32 id; + + ret = mlx5_vsc_set_space(mdev, MLX5_SEMAPHORE_SPACE_DOMAIN); + if (ret) + return ret; + + /* Get a unique ID based on the counter */ + id = pci_read_config(dev, vsc_addr + MLX5_VSC_COUNTER_OFFSET, 4); + + /* Try to modify lock */ + ret = mlx5_vsc_write(mdev, addr, &id); + if (ret) + return ret; + + /* Verify */ + ret = mlx5_vsc_read(mdev, addr, &data); + if (ret) + return ret; + if (data != id) + return EBUSY; + + return 0; +} + +int mlx5_vsc_unlock_addr_space(struct mlx5_core_dev *mdev, u32 addr) +{ + u32 data = 0; + int ret; + + ret = mlx5_vsc_set_space(mdev, MLX5_SEMAPHORE_SPACE_DOMAIN); + if (ret) + return ret; + + /* Try to modify lock */ + ret = mlx5_vsc_write(mdev, addr, &data); + if (ret) + return ret; + + /* Verify */ + ret = mlx5_vsc_read(mdev, addr, &data); + if (ret) + return ret; + if (data != 0) + return EBUSY; return 0; } Modified: stable/11/sys/modules/mlx5/Makefile ============================================================================== --- stable/11/sys/modules/mlx5/Makefile Thu Aug 2 08:40:46 2018 (r337102) +++ stable/11/sys/modules/mlx5/Makefile Thu Aug 2 08:42:40 2018 (r337103) @@ -6,7 +6,6 @@ SRCS= \ mlx5_alloc.c \ mlx5_cmd.c \ mlx5_cq.c \ -mlx5_crspace.c \ mlx5_diagnostics.c \ mlx5_eq.c \ mlx5_fs_cmd.c \ From owner-svn-src-all@freebsd.org Thu Aug 2 08:43:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E76E2105EF16; Thu, 2 Aug 2018 08:43:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 976797D23C; Thu, 2 Aug 2018 08:43:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7830375D2; Thu, 2 Aug 2018 08:43:54 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728hsHS056189; Thu, 2 Aug 2018 08:43:54 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728hsCa056188; Thu, 2 Aug 2018 08:43:54 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020843.w728hsCa056188@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:43:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337104 - stable/11/sys/dev/mlx5/mlx5_core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_core X-SVN-Commit-Revision: 337104 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:43:55 -0000 Author: hselasky Date: Thu Aug 2 08:43:54 2018 New Revision: 337104 URL: https://svnweb.freebsd.org/changeset/base/337104 Log: MFC r336398: Make sure the state variable is set atomically instead of using a mutex in mlx5core. Device detach and setting error state may deadlock over the interface mutex like this: a) Detach code in mlx5en waits until error state is set while the interface mutex is locked. b) The set error handler needs to lock the interface mutex before it can set the error state. The solution is to use atomics to set the error state. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:42:40 2018 (r337103) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:43:54 2018 (r337104) @@ -219,21 +219,19 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, u32 fatal_error; int lock = -EBUSY; - mutex_lock(&dev->intf_state_mutex); - if (dev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR) { - goto unlock; - return; - } - fatal_error = check_fatal_sensors(dev); if (fatal_error || force) { + if (xchg(&dev->state, MLX5_DEVICE_STATE_INTERNAL_ERROR) == + MLX5_DEVICE_STATE_INTERNAL_ERROR) + return; if (!force) mlx5_core_err(dev, "internal state error detected\n"); - dev->state = MLX5_DEVICE_STATE_INTERNAL_ERROR; mlx5_trigger_cmd_completions(dev); } + mutex_lock(&dev->intf_state_mutex); + if (force) goto err_state_done; @@ -272,7 +270,6 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, err_state_done: mlx5_core_event(dev, MLX5_DEV_EVENT_SYS_ERROR, 0); -unlock: mutex_unlock(&dev->intf_state_mutex); } From owner-svn-src-all@freebsd.org Thu Aug 2 08:44:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7AED2105EFCE; Thu, 2 Aug 2018 08:44:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 315B37D3D4; Thu, 2 Aug 2018 08:44:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1266C75D3; Thu, 2 Aug 2018 08:44:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728ie1U056303; Thu, 2 Aug 2018 08:44:40 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728ieQ0056302; Thu, 2 Aug 2018 08:44:40 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020844.w728ieQ0056302@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:44:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337105 - stable/11/sys/dev/mlx5/mlx5_core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_core X-SVN-Commit-Revision: 337105 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:44:41 -0000 Author: hselasky Date: Thu Aug 2 08:44:40 2018 New Revision: 337105 URL: https://svnweb.freebsd.org/changeset/base/337105 Log: MFC r336399: Remove redundant call to mlx5_vsc_find_cap() in mlx5core. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:43:54 2018 (r337104) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:44:40 2018 (r337105) @@ -846,10 +846,7 @@ static int mlx5_pci_init(struct mlx5_core_dev *dev, st goto err_clr_master; } - if (mlx5_vsc_find_cap(dev)) - dev_err(&pdev->dev, "Unable to find vendor specific capabilities\n"); - - return 0; + return 0; err_clr_master: pci_clear_master(dev->pdev); From owner-svn-src-all@freebsd.org Thu Aug 2 08:45:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50D19105F0E7; Thu, 2 Aug 2018 08:45:33 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 057107D5C2; Thu, 2 Aug 2018 08:45:33 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DAC3C75D4; Thu, 2 Aug 2018 08:45:32 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728jWXu056407; Thu, 2 Aug 2018 08:45:32 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728jW48056406; Thu, 2 Aug 2018 08:45:32 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020845.w728jW48056406@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:45:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337106 - stable/11/sys/dev/mlx5/mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_en X-SVN-Commit-Revision: 337106 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:45:33 -0000 Author: hselasky Date: Thu Aug 2 08:45:32 2018 New Revision: 337106 URL: https://svnweb.freebsd.org/changeset/base/337106 Log: MFC r336401: Correctly write atomic variable in mlx5en(4). Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:44:40 2018 (r337105) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:45:32 2018 (r337106) @@ -1232,7 +1232,7 @@ mlx5e_open_sq(struct mlx5e_channel *c, if (err) goto err_disable_sq; - atomic_store_rel_int(&sq->queue_state, MLX5E_SQ_READY); + WRITE_ONCE(sq->queue_state, MLX5E_SQ_READY); return (0); From owner-svn-src-all@freebsd.org Thu Aug 2 08:46:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0C93105F205; Thu, 2 Aug 2018 08:46:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 665007D748; Thu, 2 Aug 2018 08:46:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 483F275D6; Thu, 2 Aug 2018 08:46:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728kSkl056503; Thu, 2 Aug 2018 08:46:28 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728kSj9056502; Thu, 2 Aug 2018 08:46:28 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020846.w728kSj9056502@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:46:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337107 - stable/11/sys/dev/mlx5/mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_en X-SVN-Commit-Revision: 337107 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:46:28 -0000 Author: hselasky Date: Thu Aug 2 08:46:27 2018 New Revision: 337107 URL: https://svnweb.freebsd.org/changeset/base/337107 Log: MFC r336402: Do not hint about 'trust both' mode when the mlx5en(4) hardware does not support it. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Aug 2 08:45:32 2018 (r337106) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Aug 2 08:46:27 2018 (r337107) @@ -1123,9 +1123,14 @@ mlx5e_create_ethtool(struct mlx5e_priv *priv) priv, i, mlx5e_dscp_prio_handler, "CU", "Set DSCP to priority mapping, 0..7"); } +#define A "Set trust state, 1:PCP 2:DSCP" +#define B " 3:BOTH" SYSCTL_ADD_PROC(&priv->sysctl_ctx, SYSCTL_CHILDREN(qos_node), OID_AUTO, "trust_state", CTLTYPE_U8 | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, priv, 0, mlx5e_trust_state_handler, "CU", - "Set trust state, 1:PCP 2:DSCP 3:BOTH"); + MLX5_CAP_QCAM_FEATURE(mdev, qpts_trust_both) ? + A B : A); +#undef B +#undef A } } From owner-svn-src-all@freebsd.org Thu Aug 2 08:47:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BB1D105F2F3; Thu, 2 Aug 2018 08:47:25 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 312D87D8F7; Thu, 2 Aug 2018 08:47:25 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1265F75DA; Thu, 2 Aug 2018 08:47:25 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728lO2l056606; Thu, 2 Aug 2018 08:47:24 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728lOg6056603; Thu, 2 Aug 2018 08:47:24 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020847.w728lOg6056603@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:47:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337108 - stable/11/sys/dev/mlx5/mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_en X-SVN-Commit-Revision: 337108 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:47:25 -0000 Author: hselasky Date: Thu Aug 2 08:47:24 2018 New Revision: 337108 URL: https://svnweb.freebsd.org/changeset/base/337108 Log: MFC r336403: Add context numbers for HW elements in mlx5en(4). To access the data, set sysctl dev.mce.N.conf.debug_stats to 1. This enables the sysctl node dev.mce.N.hw_ctx_debug. Its content is the mapping of each channel' number to used receive queue and associated completion queue, set of the transmit queues numbers and corresponding completion queues. Trimmed example output: channel 30 rq 188 cq 1085 channel 30 tc 0 sq 187 cq 1084 channel 31 rq 191 cq 1087 channel 31 tc 0 sq 190 cq 1086 Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_en/en.h stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_en/en.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:46:27 2018 (r337107) +++ stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:47:24 2018 (r337108) @@ -759,6 +759,7 @@ struct mlx5e_priv { struct sysctl_oid *sysctl_hw; int sysctl_debug; struct mlx5e_stats stats; + struct sysctl_ctx_list sysctl_ctx_channel_debug; int counter_set_id; struct workqueue_struct *wq; Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Aug 2 08:46:27 2018 (r337107) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c Thu Aug 2 08:47:24 2018 (r337108) @@ -916,27 +916,75 @@ static const char *mlx5e_port_stats_debug_desc[] = { }; static int +mlx5e_ethtool_debug_channel_info(SYSCTL_HANDLER_ARGS) +{ + struct mlx5e_priv *priv; + struct sbuf sb; + struct mlx5e_channel *c; + struct mlx5e_sq *sq; + struct mlx5e_rq *rq; + int error, i, tc; + + priv = arg1; + error = sysctl_wire_old_buffer(req, 0); + if (error != 0) + return (error); + if (sbuf_new_for_sysctl(&sb, NULL, 128, req) == NULL) + return (ENOMEM); + sbuf_clear_flags(&sb, SBUF_INCLUDENUL); + + PRIV_LOCK(priv); + if (test_bit(MLX5E_STATE_OPENED, &priv->state) == 0) + goto out; + for (i = 0; i < priv->params.num_channels; i++) { + c = priv->channel[i]; + rq = &c->rq; + sbuf_printf(&sb, "channel %d rq %d cq %d\n", + c->ix, rq->rqn, rq->cq.mcq.cqn); + for (tc = 0; tc < c->num_tc; tc++) { + sq = &c->sq[tc]; + sbuf_printf(&sb, "channel %d tc %d sq %d cq %d\n", + c->ix, tc, sq->sqn, sq->cq.mcq.cqn); + } + } +out: + PRIV_UNLOCK(priv); + error = sbuf_finish(&sb); + sbuf_delete(&sb); + return (error); +} + +static int mlx5e_ethtool_debug_stats(SYSCTL_HANDLER_ARGS) { struct mlx5e_priv *priv = arg1; - int error; - int sys_debug; + int error, sys_debug; sys_debug = priv->sysctl_debug; error = sysctl_handle_int(oidp, &priv->sysctl_debug, 0, req); - if (error || !req->newptr) + if (error != 0 || !req->newptr) return (error); - priv->sysctl_debug = !!priv->sysctl_debug; + priv->sysctl_debug = priv->sysctl_debug != 0; if (sys_debug == priv->sysctl_debug) - return (error); - if (priv->sysctl_debug) + return (0); + + PRIV_LOCK(priv); + if (priv->sysctl_debug) { mlx5e_create_stats(&priv->stats.port_stats_debug.ctx, SYSCTL_CHILDREN(priv->sysctl_ifnet), "debug_stats", mlx5e_port_stats_debug_desc, MLX5E_PORT_STATS_DEBUG_NUM, priv->stats.port_stats_debug.arg); - else + SYSCTL_ADD_PROC(&priv->sysctl_ctx_channel_debug, + SYSCTL_CHILDREN(priv->sysctl_ifnet), OID_AUTO, + "hw_ctx_debug", + CTLFLAG_RD | CTLFLAG_MPSAFE | CTLTYPE_STRING, priv, 0, + mlx5e_ethtool_debug_channel_info, "S", ""); + } else { sysctl_ctx_free(&priv->stats.port_stats_debug.ctx); - return (error); + sysctl_ctx_free(&priv->sysctl_ctx_channel_debug); + } + PRIV_UNLOCK(priv); + return (0); } static void Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:46:27 2018 (r337107) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:47:24 2018 (r337108) @@ -3407,6 +3407,8 @@ mlx5e_create_ifp(struct mlx5_core_dev *mdev) if (ifp->if_capenable & IFCAP_TXCSUM_IPV6) ifp->if_hwassist |= (CSUM_UDP_IPV6 | CSUM_TCP_IPV6); + sysctl_ctx_init(&priv->sysctl_ctx_channel_debug); + /* ifnet sysctl tree */ sysctl_ctx_init(&priv->sysctl_ctx); priv->sysctl_ifnet = SYSCTL_ADD_NODE(&priv->sysctl_ctx, SYSCTL_STATIC_CHILDREN(_dev), @@ -3565,6 +3567,7 @@ err_free_wq: err_free_sysctl: sysctl_ctx_free(&priv->sysctl_ctx); + sysctl_ctx_free(&priv->sysctl_ctx_channel_debug); if_free(ifp); @@ -3612,8 +3615,10 @@ mlx5e_destroy_ifp(struct mlx5_core_dev *mdev, void *vp if_free(ifp); /* destroy all remaining sysctl nodes */ - if (priv->sysctl_debug) + if (priv->sysctl_debug) { + sysctl_ctx_free(&priv->sysctl_ctx_channel_debug); sysctl_ctx_free(&priv->stats.port_stats_debug.ctx); + } sysctl_ctx_free(&priv->stats.vport.ctx); sysctl_ctx_free(&priv->stats.pport.ctx); sysctl_ctx_free(&priv->sysctl_ctx); From owner-svn-src-all@freebsd.org Thu Aug 2 08:48:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 039DA105F3BA; Thu, 2 Aug 2018 08:48:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACC2D7DA7C; Thu, 2 Aug 2018 08:48:27 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8DD3975DB; Thu, 2 Aug 2018 08:48:27 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728mRtV056707; Thu, 2 Aug 2018 08:48:27 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728mRYm056706; Thu, 2 Aug 2018 08:48:27 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020848.w728mRYm056706@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:48:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337109 - stable/11/sys/dev/mlx5/mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_en X-SVN-Commit-Revision: 337109 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:48:28 -0000 Author: hselasky Date: Thu Aug 2 08:48:27 2018 New Revision: 337109 URL: https://svnweb.freebsd.org/changeset/base/337109 Log: MFC r336404: Enable both receive and transmit pauseframes by default in mlx5en(4). Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:47:24 2018 (r337108) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:48:27 2018 (r337109) @@ -3252,8 +3252,8 @@ mlx5e_setup_pauseframes(struct mlx5e_priv *priv) char path[96]; int error; - /* Only receiving pauseframes is enabled by default */ - priv->params.tx_pauseframe_control = 0; + /* enable pauseframes by default */ + priv->params.tx_pauseframe_control = 1; priv->params.rx_pauseframe_control = 1; /* disable ports flow control, PFC, by default */ From owner-svn-src-all@freebsd.org Thu Aug 2 08:49:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CF3C105F530; Thu, 2 Aug 2018 08:49:37 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C66A27DC3A; Thu, 2 Aug 2018 08:49:36 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A789C75DC; Thu, 2 Aug 2018 08:49:36 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728naXM056812; Thu, 2 Aug 2018 08:49:36 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728naAe056810; Thu, 2 Aug 2018 08:49:36 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020849.w728naAe056810@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:49:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337110 - stable/11/sys/dev/mlx5/mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_en X-SVN-Commit-Revision: 337110 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:49:37 -0000 Author: hselasky Date: Thu Aug 2 08:49:35 2018 New Revision: 337110 URL: https://svnweb.freebsd.org/changeset/base/337110 Log: MFC r336407: Handle jumbo frames without requiring big clusters in mlx5en(4). The scatter list is formed by the chunks of MCLBYTES each, and larger than default packets are returned to the stack as the mbuf chain. Submitted by: kib@ Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_en/en.h stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_en/en.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:48:27 2018 (r337109) +++ stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:49:35 2018 (r337110) @@ -80,8 +80,19 @@ #define MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE 0xa #define MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE 0xe -/* freeBSD HW LRO is limited by 16KB - the size of max mbuf */ +#define MLX5E_MAX_RX_SEGS 7 + +#ifndef MLX5E_MAX_RX_BYTES +#define MLX5E_MAX_RX_BYTES MCLBYTES +#endif + +#if (MLX5E_MAX_RX_SEGS == 1) +/* FreeBSD HW LRO is limited by 16KB - the size of max mbuf */ #define MLX5E_PARAMS_DEFAULT_LRO_WQE_SZ MJUM16BYTES +#else +#define MLX5E_PARAMS_DEFAULT_LRO_WQE_SZ \ + MIN(65535, MLX5E_MAX_RX_SEGS * MLX5E_MAX_RX_BYTES) +#endif #define MLX5E_PARAMS_DEFAULT_RX_CQ_MODERATION_USEC 0x10 #define MLX5E_PARAMS_DEFAULT_RX_CQ_MODERATION_USEC_FROM_CQE 0x3 #define MLX5E_PARAMS_DEFAULT_RX_CQ_MODERATION_PKTS 0x20 @@ -532,6 +543,7 @@ struct mlx5e_rq { struct mtx mtx; bus_dma_tag_t dma_tag; u32 wqe_sz; + u32 nsegs; struct mlx5e_rq_mbuf *mbuf; struct ifnet *ifp; struct mlx5e_rq_stats stats; @@ -782,8 +794,11 @@ struct mlx5e_tx_wqe { struct mlx5e_rx_wqe { struct mlx5_wqe_srq_next_seg next; - struct mlx5_wqe_data_seg data; + struct mlx5_wqe_data_seg data[]; }; + +/* the size of the structure above must be power of two */ +CTASSERT(powerof2(sizeof(struct mlx5e_rx_wqe))); struct mlx5e_eeprom { int lock_bit; Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:48:27 2018 (r337109) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:49:35 2018 (r337110) @@ -33,9 +33,12 @@ #ifndef ETH_DRIVER_VERSION #define ETH_DRIVER_VERSION "3.4.1" #endif + char mlx5e_version[] = "Mellanox Ethernet driver" " (" ETH_DRIVER_VERSION ")"; +static int mlx5e_get_wqe_sz(struct mlx5e_priv *priv, u32 *wqe_sz, u32 *nsegs); + struct mlx5e_channel_param { struct mlx5e_rq_param rq; struct mlx5e_sq_param sq; @@ -705,7 +708,12 @@ mlx5e_create_rq(struct mlx5e_channel *c, int wq_sz; int err; int i; + u32 nsegs, wqe_sz; + err = mlx5e_get_wqe_sz(priv, &wqe_sz, &nsegs); + if (err != 0) + goto done; + /* Create DMA descriptor TAG */ if ((err = -bus_dma_tag_create( bus_get_dma_tag(mdev->pdev->dev.bsddev), @@ -714,9 +722,9 @@ mlx5e_create_rq(struct mlx5e_channel *c, BUS_SPACE_MAXADDR, /* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ NULL, NULL, /* filter, filterarg */ - MJUM16BYTES, /* maxsize */ - 1, /* nsegments */ - MJUM16BYTES, /* maxsegsize */ + nsegs * MLX5E_MAX_RX_BYTES, /* maxsize */ + nsegs, /* nsegments */ + nsegs * MLX5E_MAX_RX_BYTES, /* maxsegsize */ 0, /* flags */ NULL, NULL, /* lockfunc, lockfuncarg */ &rq->dma_tag))) @@ -729,23 +737,9 @@ mlx5e_create_rq(struct mlx5e_channel *c, rq->wq.db = &rq->wq.db[MLX5_RCV_DBR]; - if (priv->params.hw_lro_en) { - rq->wqe_sz = priv->params.lro_wqe_sz; - } else { - rq->wqe_sz = MLX5E_SW2MB_MTU(priv->ifp->if_mtu); - } - if (rq->wqe_sz > MJUM16BYTES) { - err = -ENOMEM; + err = mlx5e_get_wqe_sz(priv, &rq->wqe_sz, &rq->nsegs); + if (err != 0) goto err_rq_wq_destroy; - } else if (rq->wqe_sz > MJUM9BYTES) { - rq->wqe_sz = MJUM16BYTES; - } else if (rq->wqe_sz > MJUMPAGESIZE) { - rq->wqe_sz = MJUM9BYTES; - } else if (rq->wqe_sz > MCLBYTES) { - rq->wqe_sz = MJUMPAGESIZE; - } else { - rq->wqe_sz = MCLBYTES; - } wq_sz = mlx5_wq_ll_get_size(&rq->wq); @@ -756,7 +750,11 @@ mlx5e_create_rq(struct mlx5e_channel *c, rq->mbuf = malloc(wq_sz * sizeof(rq->mbuf[0]), M_MLX5EN, M_WAITOK | M_ZERO); for (i = 0; i != wq_sz; i++) { struct mlx5e_rx_wqe *wqe = mlx5_wq_ll_get_wqe(&rq->wq, i); +#if (MLX5E_MAX_RX_SEGS == 1) uint32_t byte_count = rq->wqe_sz - MLX5E_NET_IP_ALIGN; +#else + int j; +#endif err = -bus_dmamap_create(rq->dma_tag, 0, &rq->mbuf[i].dma_map); if (err != 0) { @@ -764,8 +762,15 @@ mlx5e_create_rq(struct mlx5e_channel *c, bus_dmamap_destroy(rq->dma_tag, rq->mbuf[i].dma_map); goto err_rq_mbuf_free; } - wqe->data.lkey = c->mkey_be; - wqe->data.byte_count = cpu_to_be32(byte_count | MLX5_HW_START_PADDING); + + /* set value for constant fields */ +#if (MLX5E_MAX_RX_SEGS == 1) + wqe->data[0].lkey = c->mkey_be; + wqe->data[0].byte_count = cpu_to_be32(byte_count | MLX5_HW_START_PADDING); +#else + for (j = 0; j < rq->nsegs; j++) + wqe->data[j].lkey = c->mkey_be; +#endif } rq->ifp = c->ifp; @@ -1704,16 +1709,51 @@ mlx5e_close_channel_wait(struct mlx5e_channel *volatil free(c, M_MLX5EN); } +static int +mlx5e_get_wqe_sz(struct mlx5e_priv *priv, u32 *wqe_sz, u32 *nsegs) +{ + u32 r, n; + + r = priv->params.hw_lro_en ? priv->params.lro_wqe_sz : + MLX5E_SW2MB_MTU(priv->ifp->if_mtu); + if (r > MJUM16BYTES) + return (-ENOMEM); + + if (r > MJUM9BYTES) + r = MJUM16BYTES; + else if (r > MJUMPAGESIZE) + r = MJUM9BYTES; + else if (r > MCLBYTES) + r = MJUMPAGESIZE; + else + r = MCLBYTES; + + /* + * n + 1 must be a power of two, because stride size must be. + * Stride size is 16 * (n + 1), as the first segment is + * control. + */ + for (n = howmany(r, MLX5E_MAX_RX_BYTES); !powerof2(n + 1); n++) + ; + + *wqe_sz = r; + *nsegs = n; + return (0); +} + static void mlx5e_build_rq_param(struct mlx5e_priv *priv, struct mlx5e_rq_param *param) { void *rqc = param->rqc; void *wq = MLX5_ADDR_OF(rqc, rqc, wq); + u32 wqe_sz, nsegs; + mlx5e_get_wqe_sz(priv, &wqe_sz, &nsegs); MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_LINKED_LIST); MLX5_SET(wq, wq, end_padding_mode, MLX5_WQ_END_PAD_MODE_ALIGN); - MLX5_SET(wq, wq, log_wq_stride, ilog2(sizeof(struct mlx5e_rx_wqe))); + MLX5_SET(wq, wq, log_wq_stride, ilog2(sizeof(struct mlx5e_rx_wqe) + + nsegs * sizeof(struct mlx5_wqe_data_seg))); MLX5_SET(wq, wq, log_wq_sz, priv->params.log_rq_size); MLX5_SET(wq, wq, pd, priv->pdn); Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Thu Aug 2 08:48:27 2018 (r337109) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Thu Aug 2 08:49:35 2018 (r337110) @@ -32,21 +32,47 @@ static inline int mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, struct mlx5e_rx_wqe *wqe, u16 ix) { - bus_dma_segment_t segs[1]; + bus_dma_segment_t segs[rq->nsegs]; struct mbuf *mb; int nsegs; int err; - +#if (MLX5E_MAX_RX_SEGS != 1) + struct mbuf *mb_head; + int i; +#endif if (rq->mbuf[ix].mbuf != NULL) return (0); +#if (MLX5E_MAX_RX_SEGS == 1) mb = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, rq->wqe_sz); if (unlikely(!mb)) return (-ENOMEM); - /* set initial mbuf length */ mb->m_pkthdr.len = mb->m_len = rq->wqe_sz; +#else + mb_head = mb = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, + MLX5E_MAX_RX_BYTES); + if (unlikely(mb == NULL)) + return (-ENOMEM); + mb->m_len = MLX5E_MAX_RX_BYTES; + mb->m_pkthdr.len = MLX5E_MAX_RX_BYTES; + + for (i = 1; i < rq->nsegs; i++) { + if (mb_head->m_pkthdr.len >= rq->wqe_sz) + break; + mb = mb->m_next = m_getjcl(M_NOWAIT, MT_DATA, 0, + MLX5E_MAX_RX_BYTES); + if (unlikely(mb == NULL)) { + m_freem(mb_head); + return (-ENOMEM); + } + mb->m_len = MLX5E_MAX_RX_BYTES; + mb_head->m_pkthdr.len += MLX5E_MAX_RX_BYTES; + } + /* rewind to first mbuf in chain */ + mb = mb_head; +#endif /* get IP header aligned */ m_adj(mb, MLX5E_NET_IP_ALIGN); @@ -54,12 +80,26 @@ mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, mb, segs, &nsegs, BUS_DMA_NOWAIT); if (err != 0) goto err_free_mbuf; - if (unlikely(nsegs != 1)) { + if (unlikely(nsegs == 0)) { bus_dmamap_unload(rq->dma_tag, rq->mbuf[ix].dma_map); err = -ENOMEM; goto err_free_mbuf; } - wqe->data.addr = cpu_to_be64(segs[0].ds_addr); +#if (MLX5E_MAX_RX_SEGS == 1) + wqe->data[0].addr = cpu_to_be64(segs[0].ds_addr); +#else + wqe->data[0].addr = cpu_to_be64(segs[0].ds_addr); + wqe->data[0].byte_count = cpu_to_be32(segs[0].ds_len | + MLX5_HW_START_PADDING); + for (i = 1; i != nsegs; i++) { + wqe->data[i].addr = cpu_to_be64(segs[i].ds_addr); + wqe->data[i].byte_count = cpu_to_be32(segs[i].ds_len); + } + for (; i < rq->nsegs; i++) { + wqe->data[i].addr = 0; + wqe->data[i].byte_count = 0; + } +#endif rq->mbuf[ix].mbuf = mb; rq->mbuf[ix].data = mb->m_data; @@ -185,6 +225,9 @@ mlx5e_build_rx_mbuf(struct mlx5_cqe64 *cqe, u32 cqe_bcnt) { struct ifnet *ifp = rq->ifp; +#if (MLX5E_MAX_RX_SEGS != 1) + struct mbuf *mb_head; +#endif int lro_num_seg; /* HW LRO session aggregated packets counter */ lro_num_seg = be32_to_cpu(cqe->srqn) >> 24; @@ -194,7 +237,26 @@ mlx5e_build_rx_mbuf(struct mlx5_cqe64 *cqe, rq->stats.lro_bytes += cqe_bcnt; } +#if (MLX5E_MAX_RX_SEGS == 1) mb->m_pkthdr.len = mb->m_len = cqe_bcnt; +#else + mb->m_pkthdr.len = cqe_bcnt; + for (mb_head = mb; mb != NULL; mb = mb->m_next) { + if (mb->m_len > cqe_bcnt) + mb->m_len = cqe_bcnt; + cqe_bcnt -= mb->m_len; + if (likely(cqe_bcnt == 0)) { + if (likely(mb->m_next != NULL)) { + /* trim off empty mbufs */ + m_freem(mb->m_next); + mb->m_next = NULL; + } + break; + } + } + /* rewind to first mbuf in chain */ + mb = mb_head; +#endif /* check if a Toeplitz hash was computed */ if (cqe->rss_hash_type != 0) { mb->m_pkthdr.flowid = be32_to_cpu(cqe->rss_hash_result); @@ -357,6 +419,10 @@ mlx5e_poll_rx_cq(struct mlx5e_rq *rq, int budget) } if ((MHLEN - MLX5E_NET_IP_ALIGN) >= byte_cnt && (mb = m_gethdr(M_NOWAIT, MT_DATA)) != NULL) { +#if (MLX5E_MAX_RX_SEGS != 1) + /* set maximum mbuf length */ + mb->m_len = MHLEN - MLX5E_NET_IP_ALIGN; +#endif /* get IP header aligned */ mb->m_data += MLX5E_NET_IP_ALIGN; From owner-svn-src-all@freebsd.org Thu Aug 2 08:50:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75249105F5FE; Thu, 2 Aug 2018 08:50:29 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B7F17DDA8; Thu, 2 Aug 2018 08:50:29 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E992C75E1; Thu, 2 Aug 2018 08:50:28 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728oSkc056942; Thu, 2 Aug 2018 08:50:28 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728oSeS056941; Thu, 2 Aug 2018 08:50:28 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020850.w728oSeS056941@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:50:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337111 - stable/11 X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11 X-SVN-Commit-Revision: 337111 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:50:29 -0000 Author: hselasky Date: Thu Aug 2 08:50:28 2018 New Revision: 337111 URL: https://svnweb.freebsd.org/changeset/base/337111 Log: MFC r336408: Add missing newline. Sponsored by: Mellanox Technologies Modified: Directory Properties: stable/11/ (props changed) From owner-svn-src-all@freebsd.org Thu Aug 2 08:51:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E56A6105F95C; Thu, 2 Aug 2018 08:51:56 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 942517E10A; Thu, 2 Aug 2018 08:51:56 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7537E7756; Thu, 2 Aug 2018 08:51:56 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728puCa060856; Thu, 2 Aug 2018 08:51:56 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728pusZ060855; Thu, 2 Aug 2018 08:51:56 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020851.w728pusZ060855@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:51:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337112 - stable/11/sys/dev/mlx5/mlx5_core X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_core X-SVN-Commit-Revision: 337112 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:51:57 -0000 Author: hselasky Date: Thu Aug 2 08:51:55 2018 New Revision: 337112 URL: https://svnweb.freebsd.org/changeset/base/337112 Log: MFC r336410: Add module parameter to limit number of MSIX EQ vectors in mlx5en(4). For setups having a large amount of PCI devices, it makes sense to limit the number of MSIX vectors per PCI device, in order to avoid running out of IRQ vectors. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:50:28 2018 (r337111) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_main.c Thu Aug 2 08:51:55 2018 (r337112) @@ -61,6 +61,10 @@ static int prof_sel = MLX5_DEFAULT_PROF; module_param_named(prof_sel, prof_sel, int, 0444); MODULE_PARM_DESC(prof_sel, "profile selector. Valid range 0 - 2"); +static int mlx5_core_msix_eqvec; +module_param_named(msix_eqvec, mlx5_core_msix_eqvec, int, 0644); +MODULE_PARM_DESC(msix_eqvec, "Maximum number of MSIX event queue vectors"); + #define NUMA_NO_NODE -1 static LIST_HEAD(intf_list); @@ -239,11 +243,15 @@ static int mlx5_enable_msix(struct mlx5_core_dev *dev) struct mlx5_priv *priv = &dev->priv; struct mlx5_eq_table *table = &priv->eq_table; int num_eqs = 1 << MLX5_CAP_GEN(dev, log_max_eq); - int nvec; + int limit = mlx5_core_msix_eqvec; + int nvec = MLX5_EQ_VEC_COMP_BASE; int i; - nvec = MLX5_CAP_GEN(dev, num_ports) * num_online_cpus() + - MLX5_EQ_VEC_COMP_BASE; + if (limit > 0) + nvec += limit; + else + nvec += MLX5_CAP_GEN(dev, num_ports) * num_online_cpus(); + nvec = min_t(int, nvec, num_eqs); if (nvec <= MLX5_EQ_VEC_COMP_BASE) return -ENOMEM; From owner-svn-src-all@freebsd.org Thu Aug 2 08:52:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE6DA105FB5F; Thu, 2 Aug 2018 08:52:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 990F97E3D6; Thu, 2 Aug 2018 08:52:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7A6D57783; Thu, 2 Aug 2018 08:52:50 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728qoxM061882; Thu, 2 Aug 2018 08:52:50 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728qo9b061881; Thu, 2 Aug 2018 08:52:50 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020852.w728qo9b061881@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:52:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337113 - stable/11/sys/dev/mlx5/mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/dev/mlx5/mlx5_en X-SVN-Commit-Revision: 337113 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:52:51 -0000 Author: hselasky Date: Thu Aug 2 08:52:49 2018 New Revision: 337113 URL: https://svnweb.freebsd.org/changeset/base/337113 Log: MFC r336411: Use a mbuf header instead of a mbuf cluster for debugging interrupts in mlx5en(4). Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Thu Aug 2 08:51:55 2018 (r337112) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Thu Aug 2 08:52:49 2018 (r337113) @@ -468,7 +468,10 @@ mlx5e_rx_cq_comp(struct mlx5_core_cq *mcq) int i = 0; #ifdef HAVE_PER_CQ_EVENT_PACKET - struct mbuf *mb = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, rq->wqe_sz); +#if (MHLEN < 15) +#error "MHLEN is too small" +#endif + struct mbuf *mb = m_gethdr(M_NOWAIT, MT_DATA); if (mb != NULL) { /* this code is used for debugging purpose only */ From owner-svn-src-all@freebsd.org Thu Aug 2 08:55:22 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0DF8105FC68; Thu, 2 Aug 2018 08:55:21 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4D157E563; Thu, 2 Aug 2018 08:55:21 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 817E27784; Thu, 2 Aug 2018 08:55:21 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728tLFc062110; Thu, 2 Aug 2018 08:55:21 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728tKZp062103; Thu, 2 Aug 2018 08:55:20 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020855.w728tKZp062103@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:55:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337114 - in stable/11/sys/dev/mlx5: . mlx5_core mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/dev/mlx5: . mlx5_core mlx5_en X-SVN-Commit-Revision: 337114 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:55:22 -0000 Author: hselasky Date: Thu Aug 2 08:55:19 2018 New Revision: 337114 URL: https://svnweb.freebsd.org/changeset/base/337114 Log: MFC r336450: Do not inline transmit headers and use HW VLAN tagging if supported by mlx5en(4). Query the minimal inline mode supported by the card. When creating a send queue, cache the queried mode and optimize the transmit if no inlining is required. In this case, we can avoid touching the headers cache line and avoid dirtying several more lines by copying headers into the send WQEs. Also, if no inline headers are used, hardware assists in the VLAN tag framing. Submitted by: kib@, slavash@ Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/device.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_vport.c stable/11/sys/dev/mlx5/mlx5_en/en.h stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c stable/11/sys/dev/mlx5/qp.h stable/11/sys/dev/mlx5/vport.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/device.h ============================================================================== --- stable/11/sys/dev/mlx5/device.h Thu Aug 2 08:52:49 2018 (r337113) +++ stable/11/sys/dev/mlx5/device.h Thu Aug 2 08:55:19 2018 (r337114) @@ -1091,6 +1091,13 @@ enum { MLX5_CMD_HCA_CAP_MIN_WQE_INLINE_MODE_NOT_REQUIRED = 0x2 }; +enum mlx5_inline_modes { + MLX5_INLINE_MODE_NONE, + MLX5_INLINE_MODE_L2, + MLX5_INLINE_MODE_IP, + MLX5_INLINE_MODE_TCP_UDP, +}; + enum { MLX5_QUERY_VPORT_STATE_OUT_STATE_FOLLOW = 0x2, }; Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_vport.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_vport.c Thu Aug 2 08:52:49 2018 (r337113) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_vport.c Thu Aug 2 08:55:19 2018 (r337114) @@ -208,6 +208,58 @@ int mlx5_vport_query_out_of_rx_buffer(struct mlx5_core return err; } +int mlx5_query_nic_vport_min_inline(struct mlx5_core_dev *mdev, + u16 vport, u8 *min_inline) +{ + u32 out[MLX5_ST_SZ_DW(query_nic_vport_context_out)] = {0}; + int err; + + err = mlx5_query_nic_vport_context(mdev, vport, out, sizeof(out)); + if (!err) + *min_inline = MLX5_GET(query_nic_vport_context_out, out, + nic_vport_context.min_wqe_inline_mode); + return err; +} +EXPORT_SYMBOL_GPL(mlx5_query_nic_vport_min_inline); + +void mlx5_query_min_inline(struct mlx5_core_dev *mdev, + u8 *min_inline_mode) +{ + switch (MLX5_CAP_ETH(mdev, wqe_inline_mode)) { + case MLX5_CAP_INLINE_MODE_L2: + *min_inline_mode = MLX5_INLINE_MODE_L2; + break; + case MLX5_CAP_INLINE_MODE_VPORT_CONTEXT: + mlx5_query_nic_vport_min_inline(mdev, 0, min_inline_mode); + break; + case MLX5_CAP_INLINE_MODE_NOT_REQUIRED: + *min_inline_mode = MLX5_INLINE_MODE_NONE; + break; + } +} +EXPORT_SYMBOL_GPL(mlx5_query_min_inline); + +int mlx5_modify_nic_vport_min_inline(struct mlx5_core_dev *mdev, + u16 vport, u8 min_inline) +{ + u32 in[MLX5_ST_SZ_DW(modify_nic_vport_context_in)] = {0}; + int inlen = MLX5_ST_SZ_BYTES(modify_nic_vport_context_in); + void *nic_vport_ctx; + + MLX5_SET(modify_nic_vport_context_in, in, + field_select.min_wqe_inline_mode, 1); + MLX5_SET(modify_nic_vport_context_in, in, vport_number, vport); + MLX5_SET(modify_nic_vport_context_in, in, other_vport, 1); + + nic_vport_ctx = MLX5_ADDR_OF(modify_nic_vport_context_in, + in, nic_vport_context); + MLX5_SET(nic_vport_context, nic_vport_ctx, + min_wqe_inline_mode, min_inline); + + return mlx5_modify_nic_vport_context(mdev, in, inlen); +} +EXPORT_SYMBOL_GPL(mlx5_modify_nic_vport_min_inline); + int mlx5_query_nic_vport_mac_address(struct mlx5_core_dev *mdev, u16 vport, u8 *addr) { Modified: stable/11/sys/dev/mlx5/mlx5_en/en.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:52:49 2018 (r337113) +++ stable/11/sys/dev/mlx5/mlx5_en/en.h Thu Aug 2 08:55:19 2018 (r337114) @@ -450,6 +450,8 @@ struct mlx5e_params { u32 rx_pauseframe_control __aligned(4); u32 tx_priority_flow_control __aligned(4); u32 rx_priority_flow_control __aligned(4); + u16 tx_max_inline; + u8 tx_min_inline_mode; }; #define MLX5E_PARAMS(m) \ @@ -612,6 +614,9 @@ struct mlx5e_sq { u32 sqn; u32 bf_buf_size; u32 mkey_be; + u16 max_inline; + u8 min_inline_mode; + u8 vlan_inline_cap; /* control path */ struct mlx5_wq_ctrl wq_ctrl; @@ -909,5 +914,6 @@ void mlx5e_drain_sq(struct mlx5e_sq *); void mlx5e_modify_tx_dma(struct mlx5e_priv *priv, uint8_t value); void mlx5e_modify_rx_dma(struct mlx5e_priv *priv, uint8_t value); void mlx5e_resume_sq(struct mlx5e_sq *sq); +u8 mlx5e_params_calculate_tx_min_inline(struct mlx5_core_dev *mdev); #endif /* _MLX5_EN_H_ */ Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:52:49 2018 (r337113) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:55:19 2018 (r337114) @@ -1069,6 +1069,9 @@ mlx5e_create_sq(struct mlx5e_channel *c, sq->ifp = priv->ifp; sq->priv = priv; sq->tc = tc; + sq->max_inline = priv->params.tx_max_inline; + sq->min_inline_mode = priv->params.tx_min_inline_mode; + sq->vlan_inline_cap = MLX5_CAP_ETH(mdev, wqe_vlan_insert); /* check if we should allocate a second packet buffer */ if (priv->params_ethtool.tx_bufring_disable == 0) { @@ -2906,6 +2909,16 @@ mlx5e_check_required_hca_cap(struct mlx5_core_dev *mde return (0); } +static u16 +mlx5e_get_max_inline_cap(struct mlx5_core_dev *mdev) +{ + int bf_buf_size = (1 << MLX5_CAP_GEN(mdev, log_bf_reg_size)) / 2; + + return bf_buf_size - + sizeof(struct mlx5e_tx_wqe) + + 2 /*sizeof(mlx5e_tx_wqe.inline_hdr_start)*/; +} + static void mlx5e_build_ifp_priv(struct mlx5_core_dev *mdev, struct mlx5e_priv *priv, @@ -2941,6 +2954,8 @@ mlx5e_build_ifp_priv(struct mlx5_core_dev *mdev, priv->params.num_tc = 1; priv->params.default_vlan_prio = 0; priv->counter_set_id = -1; + priv->params.tx_max_inline = mlx5e_get_max_inline_cap(mdev); + mlx5_query_min_inline(mdev, &priv->params.tx_min_inline_mode); /* * hw lro is currently defaulted to off. when it won't anymore we @@ -3199,6 +3214,20 @@ mlx5e_modify_rx_dma(struct mlx5e_priv *priv, uint8_t v else mlx5e_enable_rx_dma(priv->channel[i]); } +} + +u8 +mlx5e_params_calculate_tx_min_inline(struct mlx5_core_dev *mdev) +{ + u8 min_inline_mode; + + min_inline_mode = MLX5_INLINE_MODE_L2; + mlx5_query_min_inline(mdev, &min_inline_mode); + if (min_inline_mode == MLX5_INLINE_MODE_NONE && + !MLX5_CAP_ETH(mdev, wqe_vlan_insert)) + min_inline_mode = MLX5_INLINE_MODE_L2; + + return (min_inline_mode); } static void Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c Thu Aug 2 08:52:49 2018 (r337113) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_tx.c Thu Aug 2 08:55:19 2018 (r337114) @@ -137,7 +137,43 @@ mlx5e_select_queue(struct ifnet *ifp, struct mbuf *mb) static inline u16 mlx5e_get_inline_hdr_size(struct mlx5e_sq *sq, struct mbuf *mb) { - return (MIN(MLX5E_MAX_TX_INLINE, mb->m_len)); + + switch(sq->min_inline_mode) { + case MLX5_INLINE_MODE_NONE: + /* + * When inline mode is NONE, we do not need to copy + * headers into WQEs, except when vlan tag framing is + * requested. Hardware might offload vlan tagging on + * transmit. This is a separate capability, which is + * known to be disabled on ConnectX-5 due to a hardware + * bug RM 931383. If vlan_inline_cap is not present and + * the packet has vlan tag, fall back to inlining. + */ + if ((mb->m_flags & M_VLANTAG) != 0 && + sq->vlan_inline_cap == 0) + break; + return (0); + case MLX5_INLINE_MODE_L2: + /* + * Due to hardware limitations, when trust mode is + * DSCP, the hardware may request MLX5_INLINE_MODE_L2 + * while it really needs all L2 headers and the 4 first + * bytes of the IP header (which include the + * TOS/traffic-class). + * + * To avoid doing a firmware command for querying the + * trust state and parsing the mbuf for doing + * unnecessary checks (VLAN/eth_type) in the fast path, + * we are going for the worth case (22 Bytes) if + * the mb->m_pkthdr.len allows it. + */ + if (mb->m_pkthdr.len > ETHER_HDR_LEN + + ETHER_VLAN_ENCAP_LEN + 4) + return (MIN(sq->max_inline, ETHER_HDR_LEN + + ETHER_VLAN_ENCAP_LEN + 4)); + break; + } + return (MIN(sq->max_inline, mb->m_pkthdr.len)); } static int @@ -275,37 +311,47 @@ mlx5e_sq_xmit(struct mlx5e_sq *sq, struct mbuf **mbp) sq->mbuf[pi].num_bytes = max_t (unsigned int, mb->m_pkthdr.len, ETHER_MIN_LEN - ETHER_CRC_LEN); } - if (mb->m_flags & M_VLANTAG) { - struct ether_vlan_header *eh = - (struct ether_vlan_header *)wqe->eth.inline_hdr_start; - - /* Range checks */ - if (ihs > (MLX5E_MAX_TX_INLINE - ETHER_VLAN_ENCAP_LEN)) - ihs = (MLX5E_MAX_TX_INLINE - ETHER_VLAN_ENCAP_LEN); - else if (ihs < ETHER_HDR_LEN) { - err = EINVAL; - goto tx_drop; + if (ihs == 0) { + if ((mb->m_flags & M_VLANTAG) != 0) { + wqe->eth.vlan_cmd = htons(0x8000); /* bit 0 CVLAN */ + wqe->eth.vlan_hdr = htons(mb->m_pkthdr.ether_vtag); + } else { + wqe->eth.inline_hdr_sz = 0; } - m_copydata(mb, 0, ETHER_HDR_LEN, (caddr_t)eh); - m_adj(mb, ETHER_HDR_LEN); - /* Insert 4 bytes VLAN tag into data stream */ - eh->evl_proto = eh->evl_encap_proto; - eh->evl_encap_proto = htons(ETHERTYPE_VLAN); - eh->evl_tag = htons(mb->m_pkthdr.ether_vtag); - /* Copy rest of header data, if any */ - m_copydata(mb, 0, ihs - ETHER_HDR_LEN, (caddr_t)(eh + 1)); - m_adj(mb, ihs - ETHER_HDR_LEN); - /* Extend header by 4 bytes */ - ihs += ETHER_VLAN_ENCAP_LEN; } else { - m_copydata(mb, 0, ihs, wqe->eth.inline_hdr_start); - m_adj(mb, ihs); + if ((mb->m_flags & M_VLANTAG) != 0) { + struct ether_vlan_header *eh = (struct ether_vlan_header + *)wqe->eth.inline_hdr_start; + + /* Range checks */ + if (ihs > (MLX5E_MAX_TX_INLINE - ETHER_VLAN_ENCAP_LEN)) + ihs = (MLX5E_MAX_TX_INLINE - + ETHER_VLAN_ENCAP_LEN); + else if (ihs < ETHER_HDR_LEN) { + err = EINVAL; + goto tx_drop; + } + m_copydata(mb, 0, ETHER_HDR_LEN, (caddr_t)eh); + m_adj(mb, ETHER_HDR_LEN); + /* Insert 4 bytes VLAN tag into data stream */ + eh->evl_proto = eh->evl_encap_proto; + eh->evl_encap_proto = htons(ETHERTYPE_VLAN); + eh->evl_tag = htons(mb->m_pkthdr.ether_vtag); + /* Copy rest of header data, if any */ + m_copydata(mb, 0, ihs - ETHER_HDR_LEN, (caddr_t)(eh + + 1)); + m_adj(mb, ihs - ETHER_HDR_LEN); + /* Extend header by 4 bytes */ + ihs += ETHER_VLAN_ENCAP_LEN; + } else { + m_copydata(mb, 0, ihs, wqe->eth.inline_hdr_start); + m_adj(mb, ihs); + } + wqe->eth.inline_hdr_sz = cpu_to_be16(ihs); } - wqe->eth.inline_hdr_sz = cpu_to_be16(ihs); - ds_cnt = sizeof(*wqe) / MLX5_SEND_WQE_DS; - if (likely(ihs > sizeof(wqe->eth.inline_hdr_start))) { + if (ihs > sizeof(wqe->eth.inline_hdr_start)) { ds_cnt += DIV_ROUND_UP(ihs - sizeof(wqe->eth.inline_hdr_start), MLX5_SEND_WQE_DS); } Modified: stable/11/sys/dev/mlx5/qp.h ============================================================================== --- stable/11/sys/dev/mlx5/qp.h Thu Aug 2 08:52:49 2018 (r337113) +++ stable/11/sys/dev/mlx5/qp.h Thu Aug 2 08:55:19 2018 (r337114) @@ -240,8 +240,16 @@ struct mlx5_wqe_eth_seg { u8 swp_flags; __be16 mss; __be32 rsvd2; - __be16 inline_hdr_sz; - u8 inline_hdr_start[2]; + union { + struct { + __be16 inline_hdr_sz; + u8 inline_hdr_start[2]; + }; + struct { + __be16 vlan_cmd; + __be16 vlan_hdr; + }; + }; }; struct mlx5_wqe_xrc_seg { Modified: stable/11/sys/dev/mlx5/vport.h ============================================================================== --- stable/11/sys/dev/mlx5/vport.h Thu Aug 2 08:52:49 2018 (r337113) +++ stable/11/sys/dev/mlx5/vport.h Thu Aug 2 08:55:19 2018 (r337114) @@ -29,6 +29,13 @@ #define __MLX5_VPORT_H__ #include + +enum { + MLX5_CAP_INLINE_MODE_L2, + MLX5_CAP_INLINE_MODE_VPORT_CONTEXT, + MLX5_CAP_INLINE_MODE_NOT_REQUIRED, +}; + int mlx5_vport_alloc_q_counter(struct mlx5_core_dev *mdev, int client_id, u16 *counter_set_id); int mlx5_vport_dealloc_q_counter(struct mlx5_core_dev *mdev, int client_id, @@ -79,6 +86,11 @@ int mlx5_modify_nic_vport_mac_address(struct mlx5_core u16 vport, u8 mac[ETH_ALEN]); int mlx5_set_nic_vport_current_mac(struct mlx5_core_dev *mdev, int vport, bool other_vport, u8 *addr); +int mlx5_query_nic_vport_min_inline(struct mlx5_core_dev *mdev, + u16 vport, u8 *min_inline); +void mlx5_query_min_inline(struct mlx5_core_dev *mdev, u8 *min_inline); +int mlx5_modify_nic_vport_min_inline(struct mlx5_core_dev *mdev, + u16 vport, u8 min_inline); int mlx5_modify_nic_vport_port_guid(struct mlx5_core_dev *mdev, u32 vport, u64 port_guid); int mlx5_modify_nic_vport_node_guid(struct mlx5_core_dev *mdev, From owner-svn-src-all@freebsd.org Thu Aug 2 08:56:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36610105FCFF; Thu, 2 Aug 2018 08:56:32 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF5877E6B6; Thu, 2 Aug 2018 08:56:31 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C002A7785; Thu, 2 Aug 2018 08:56:31 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728uVXJ062236; Thu, 2 Aug 2018 08:56:31 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728uRlT062213; Thu, 2 Aug 2018 08:56:27 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020856.w728uRlT062213@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:56:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337115 - in stable/11/sys/dev/mlx5: . mlx5_core mlx5_en X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: in stable/11/sys/dev/mlx5: . mlx5_core mlx5_en X-SVN-Commit-Revision: 337115 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:56:32 -0000 Author: hselasky Date: Thu Aug 2 08:56:27 2018 New Revision: 337115 URL: https://svnweb.freebsd.org/changeset/base/337115 Log: MFC r336451: Update version information for the mlx5 and mlx5en(4) modules. While at it bump some copyright dates. Sponsored by: Mellanox Technologies Modified: stable/11/sys/dev/mlx5/cq.h stable/11/sys/dev/mlx5/doorbell.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h stable/11/sys/dev/mlx5/mlx5_core/mlx5_cq.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_mad.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_mcg.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_mr.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_pd.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_qp.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_srq.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_uar.c stable/11/sys/dev/mlx5/mlx5_core/mlx5_wq.c stable/11/sys/dev/mlx5/mlx5_core/wq.h stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c stable/11/sys/dev/mlx5/mlx5_rdma_if.h stable/11/sys/dev/mlx5/qp.h stable/11/sys/dev/mlx5/srq.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mlx5/cq.h ============================================================================== --- stable/11/sys/dev/mlx5/cq.h Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/cq.h Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/doorbell.h ============================================================================== --- stable/11/sys/dev/mlx5/doorbell.h Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/doorbell.h Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_core.h Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -34,9 +34,9 @@ #define DRIVER_NAME "mlx5_core" #ifndef DRIVER_VERSION -#define DRIVER_VERSION "3.4.1" +#define DRIVER_VERSION "3.4.2" #endif -#define DRIVER_RELDATE "February 2018" +#define DRIVER_RELDATE "July 2018" extern int mlx5_core_debug_mask; Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_cq.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_cq.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_cq.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_fw.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_health.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_mad.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_mad.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_mad.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_mcg.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_mcg.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_mcg.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_mr.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_mr.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_mr.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_pd.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_pd.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_pd.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_port.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_qp.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_qp.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_qp.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_srq.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_srq.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_srq.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_uar.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_uar.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_uar.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/mlx5_wq.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/mlx5_wq.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/mlx5_wq.c Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_core/wq.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_core/wq.h Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_core/wq.h Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Thu Aug 2 08:56:27 2018 (r337115) @@ -31,7 +31,7 @@ #include #ifndef ETH_DRIVER_VERSION -#define ETH_DRIVER_VERSION "3.4.1" +#define ETH_DRIVER_VERSION "3.4.2" #endif char mlx5e_version[] = "Mellanox Ethernet driver" Modified: stable/11/sys/dev/mlx5/mlx5_rdma_if.h ============================================================================== --- stable/11/sys/dev/mlx5/mlx5_rdma_if.h Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/mlx5_rdma_if.h Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/qp.h ============================================================================== --- stable/11/sys/dev/mlx5/qp.h Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/qp.h Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions Modified: stable/11/sys/dev/mlx5/srq.h ============================================================================== --- stable/11/sys/dev/mlx5/srq.h Thu Aug 2 08:55:19 2018 (r337114) +++ stable/11/sys/dev/mlx5/srq.h Thu Aug 2 08:56:27 2018 (r337115) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved. + * Copyright (c) 2013-2017, Mellanox Technologies, Ltd. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions From owner-svn-src-all@freebsd.org Thu Aug 2 08:57:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2F45105FD82; Thu, 2 Aug 2018 08:57:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7EA97E801; Thu, 2 Aug 2018 08:57:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 88E797786; Thu, 2 Aug 2018 08:57:41 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728vfNw062363; Thu, 2 Aug 2018 08:57:41 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728vfq9062361; Thu, 2 Aug 2018 08:57:41 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020857.w728vfq9062361@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 08:57:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337116 - stable/11/contrib/ofed/libibumad X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/contrib/ofed/libibumad X-SVN-Commit-Revision: 337116 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:57:42 -0000 Author: hselasky Date: Thu Aug 2 08:57:40 2018 New Revision: 337116 URL: https://svnweb.freebsd.org/changeset/base/337116 Log: MFC r336452: Add ability to parse sysfs paths under FreeBSD in libibumad. Add the ability to to parse sysfs paths to sysctl nodes by replacing '/' with '.' Submitted by: slavash@ Sponsored by: Mellanox Technologies Modified: stable/11/contrib/ofed/libibumad/sysfs.c stable/11/contrib/ofed/libibumad/sysfs.h stable/11/contrib/ofed/libibumad/umad.c Directory Properties: stable/11/ (props changed) Modified: stable/11/contrib/ofed/libibumad/sysfs.c ============================================================================== --- stable/11/contrib/ofed/libibumad/sysfs.c Thu Aug 2 08:56:27 2018 (r337115) +++ stable/11/contrib/ofed/libibumad/sysfs.c Thu Aug 2 08:57:40 2018 (r337116) @@ -62,12 +62,8 @@ int sys_read_string(const char *dir_name, const char * snprintf(path, sizeof(path), "%s/%s", dir_name, file_name); - for (s = &path[0]; *s != '\0'; s++) - if (*s == '/') - *s = '.'; - len = max_len; - if (sysctlbyname(&path[1], str, &len, NULL, 0) == -1) + if (sysctlbyname(PATH_TO_SYS(path), str, &len, NULL, 0) == -1) return ret_code(); str[(len < max_len) ? len : max_len - 1] = 0; @@ -170,11 +166,8 @@ sys_scandir(const char *dirname, struct dirent ***name int i; *namelist = NULL; - /* Skip the leading / */ - strncpy(name, &dirname[1], sizeof(name)); - for (s = &name[0]; *s != '\0'; s++) - if (*s == '/') - *s = '.'; + if (strlcpy(name, PATH_TO_SYS(dirname), sizeof(name)) >= sizeof(name)) + return (-EINVAL); /* * Resolve the path. */ @@ -259,7 +252,7 @@ out: if (cnt && compar) qsort(names, cnt, sizeof(struct dirent *), (int (*)(const void *, const void *))compar); - + *namelist = names; return (cnt); Modified: stable/11/contrib/ofed/libibumad/sysfs.h ============================================================================== --- stable/11/contrib/ofed/libibumad/sysfs.h Thu Aug 2 08:56:27 2018 (r337115) +++ stable/11/contrib/ofed/libibumad/sysfs.h Thu Aug 2 08:57:40 2018 (r337116) @@ -34,6 +34,8 @@ #define _UMAD_SYSFS_H #include +#include +#include #include #include @@ -48,5 +50,38 @@ extern int sys_read_uint(const char *dir_name, const c extern int sys_scandir(const char *dirname, struct dirent ***namelist, int (*select)(const struct dirent *), int (*compar)(const struct dirent **, const struct dirent **)); + +#ifdef __FreeBSD__ +static inline const char * +path_to_sysctl(const char *path, int out_len, char *out) +{ + const char *retval = out; + + /* Validate that out is at least as long as the original path */ + if (out_len < (strlen(path) + 1)) + return NULL; + + while (*path == '/') + path++; + + while (*path) { + if (*path == '/') { + if (*(path + 1) == '/') + *out = '.'; + else + *out++ = '.'; + } else + *out++ = *path; + path++; + } + *out = 0; + return (retval); +} + +#define PATH_TO_SYS(str) \ + path_to_sysctl(str, strlen(str) + 1, alloca(strlen(str) + 1)) +#else +#define PATH_TO_SYS(str) str +#endif #endif /* _UMAD_SYSFS_H */ Modified: stable/11/contrib/ofed/libibumad/umad.c ============================================================================== --- stable/11/contrib/ofed/libibumad/umad.c Thu Aug 2 08:56:27 2018 (r337115) +++ stable/11/contrib/ofed/libibumad/umad.c Thu Aug 2 08:57:40 2018 (r337116) @@ -509,14 +509,14 @@ int umad_init(void) TRACE("umad_init"); if (sys_read_uint(IB_UMAD_ABI_DIR, IB_UMAD_ABI_FILE, &abi_version) < 0) { IBWARN - ("can't read ABI version from %s/%s (%m): is ib_umad module loaded?", - IB_UMAD_ABI_DIR, IB_UMAD_ABI_FILE); + ("can't read ABI version from %s (%m): is ibcore module loaded?", + PATH_TO_SYS(IB_UMAD_ABI_DIR "/" IB_UMAD_ABI_FILE)); return -1; } if (abi_version < IB_UMAD_ABI_VERSION) { IBWARN - ("wrong ABI version: %s/%s is %d but library minimal ABI is %d", - IB_UMAD_ABI_DIR, IB_UMAD_ABI_FILE, abi_version, + ("wrong ABI version: %s is %d but library minimal ABI is %d", + PATH_TO_SYS(IB_UMAD_ABI_DIR "/" IB_UMAD_ABI_FILE), abi_version, IB_UMAD_ABI_VERSION); return -1; } From owner-svn-src-all@freebsd.org Thu Aug 2 08:58:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC255105FE15; Thu, 2 Aug 2018 08:58:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 90B507E95B; Thu, 2 Aug 2018 08:58:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 717F57789; Thu, 2 Aug 2018 08:58:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w728wXC5062468; Thu, 2 Aug 2018 08:58:33 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w728wXSE062467; Thu, 2 Aug 2018 08:58:33 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201808020858.w728wXSE062467@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Thu, 2 Aug 2018 08:58:33 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337117 - stable/11/sys/dev/acpica X-SVN-Group: stable-11 X-SVN-Commit-Author: avg X-SVN-Commit-Paths: stable/11/sys/dev/acpica X-SVN-Commit-Revision: 337117 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 08:58:34 -0000 Author: avg Date: Thu Aug 2 08:58:33 2018 New Revision: 337117 URL: https://svnweb.freebsd.org/changeset/base/337117 Log: MFC r334479: call AcpiLeaveSleepStatePrep after re-enabling interrupts Modified: stable/11/sys/dev/acpica/acpi.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/acpica/acpi.c ============================================================================== --- stable/11/sys/dev/acpica/acpi.c Thu Aug 2 08:57:40 2018 (r337116) +++ stable/11/sys/dev/acpica/acpi.c Thu Aug 2 08:58:33 2018 (r337117) @@ -2928,8 +2928,6 @@ acpi_EnterSleepState(struct acpi_softc *sc, int state) if (sleep_result == 1 && state != ACPI_STATE_S4) AcpiWriteBitRegister(ACPI_BITREG_SCI_ENABLE, ACPI_ENABLE_EVENT); - AcpiLeaveSleepStatePrep(state); - if (sleep_result == 1 && state == ACPI_STATE_S3) { /* * Prevent mis-interpretation of the wakeup by power button @@ -2958,6 +2956,8 @@ acpi_EnterSleepState(struct acpi_softc *sc, int state) /* call acpi_wakeup_machdep() again with interrupt enabled */ acpi_wakeup_machdep(sc, state, sleep_result, 1); + AcpiLeaveSleepStatePrep(state); + if (sleep_result == -1) goto backout; @@ -2966,8 +2966,8 @@ acpi_EnterSleepState(struct acpi_softc *sc, int state) AcpiEnable(); } else { status = AcpiEnterSleepState(state); - AcpiLeaveSleepStatePrep(state); intr_restore(intr); + AcpiLeaveSleepStatePrep(state); if (ACPI_FAILURE(status)) { device_printf(sc->acpi_dev, "AcpiEnterSleepState failed - %s\n", AcpiFormatException(status)); From owner-svn-src-all@freebsd.org Thu Aug 2 09:00:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E00F10600A5; Thu, 2 Aug 2018 09:00:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4E78E7EACC; Thu, 2 Aug 2018 09:00:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 16770778E; Thu, 2 Aug 2018 09:00:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7290902062679; Thu, 2 Aug 2018 09:00:09 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w729094l062677; Thu, 2 Aug 2018 09:00:09 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201808020900.w729094l062677@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Thu, 2 Aug 2018 09:00:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337118 - stable/11/sys/x86/x86 X-SVN-Group: stable-11 X-SVN-Commit-Author: avg X-SVN-Commit-Paths: stable/11/sys/x86/x86 X-SVN-Commit-Revision: 337118 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 09:00:10 -0000 Author: avg Date: Thu Aug 2 09:00:09 2018 New Revision: 337118 URL: https://svnweb.freebsd.org/changeset/base/337118 Log: MFC r334786: x86: reorganize code that deals with unexpected NMI-s Modified: stable/11/sys/x86/x86/cpu_machdep.c stable/11/sys/x86/x86/mp_x86.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/x86/x86/cpu_machdep.c ============================================================================== --- stable/11/sys/x86/x86/cpu_machdep.c Thu Aug 2 08:58:33 2018 (r337117) +++ stable/11/sys/x86/x86/cpu_machdep.c Thu Aug 2 09:00:09 2018 (r337118) @@ -768,33 +768,34 @@ SYSCTL_INT(_machdep, OID_AUTO, kdb_on_nmi, CTLFLAG_RWT "Go to KDB on NMI with unknown source"); #endif -#ifdef DEV_ISA void nmi_call_kdb(u_int cpu, u_int type, struct trapframe *frame) { + bool claimed = false; +#ifdef DEV_ISA /* machine/parity/power fail/"kitchen sink" faults */ - if (isa_nmi(frame->tf_err) == 0) { + if (isa_nmi(frame->tf_err)) { + claimed = true; + if (panic_on_nmi) + panic("NMI indicates hardware failure"); + } +#endif /* DEV_ISA */ #ifdef KDB + if (!claimed && kdb_on_nmi) { /* * NMI can be hooked up to a pushbutton for debugging. */ - if (kdb_on_nmi) { - printf("NMI/cpu%d ... going to debugger\n", cpu); - kdb_trap(type, 0, frame); - } -#endif /* KDB */ - } else if (panic_on_nmi) { - panic("NMI indicates hardware failure"); + printf("NMI/cpu%d ... going to debugger\n", cpu); + kdb_trap(type, 0, frame); } +#endif /* KDB */ } -#endif void nmi_handle_intr(u_int type, struct trapframe *frame) { -#ifdef DEV_ISA #ifdef SMP if (nmi_is_broadcast) { nmi_call_kdb_smp(type, frame); @@ -802,7 +803,6 @@ nmi_handle_intr(u_int type, struct trapframe *frame) } #endif nmi_call_kdb(PCPU_GET(cpuid), type, frame); -#endif } int hw_ibrs_active; Modified: stable/11/sys/x86/x86/mp_x86.c ============================================================================== --- stable/11/sys/x86/x86/mp_x86.c Thu Aug 2 08:58:33 2018 (r337117) +++ stable/11/sys/x86/x86/mp_x86.c Thu Aug 2 09:00:09 2018 (r337118) @@ -31,7 +31,6 @@ __FBSDID("$FreeBSD$"); #include "opt_apic.h" #endif #include "opt_cpu.h" -#include "opt_isa.h" #include "opt_kstack_pages.h" #include "opt_pmap.h" #include "opt_sched.h" @@ -1233,7 +1232,6 @@ ipi_nmi_handler(void) return (0); } -#ifdef DEV_ISA int nmi_kdb_lock; void @@ -1257,7 +1255,6 @@ nmi_call_kdb_smp(u_int type, struct trapframe *frame) if (call_post) cpustop_handler_post(cpu); } -#endif /* * Handle an IPI_STOP by saving our current context and spinning until we From owner-svn-src-all@freebsd.org Thu Aug 2 09:02:21 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 545481060296; Thu, 2 Aug 2018 09:02:21 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 08F757F379; Thu, 2 Aug 2018 09:02:21 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DF8A4790F; Thu, 2 Aug 2018 09:02:20 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7292Ksr067243; Thu, 2 Aug 2018 09:02:20 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7292Jpc067238; Thu, 2 Aug 2018 09:02:19 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020902.w7292Jpc067238@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 09:02:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337119 - stable/11/contrib/ofed/libibverbs/examples X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/contrib/ofed/libibverbs/examples X-SVN-Commit-Revision: 337119 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 09:02:21 -0000 Author: hselasky Date: Thu Aug 2 09:02:19 2018 New Revision: 337119 URL: https://svnweb.freebsd.org/changeset/base/337119 Log: MFC r336453: Use unspecified address family when connecting as a client in libibverbs example utilities. This allows connecting to both IPv4 and IPv6 and reverts some FreeBSD only patches. Sponsored by: Mellanox Technologies Modified: stable/11/contrib/ofed/libibverbs/examples/rc_pingpong.c stable/11/contrib/ofed/libibverbs/examples/srq_pingpong.c stable/11/contrib/ofed/libibverbs/examples/uc_pingpong.c stable/11/contrib/ofed/libibverbs/examples/ud_pingpong.c stable/11/contrib/ofed/libibverbs/examples/xsrq_pingpong.c Directory Properties: stable/11/ (props changed) Modified: stable/11/contrib/ofed/libibverbs/examples/rc_pingpong.c ============================================================================== --- stable/11/contrib/ofed/libibverbs/examples/rc_pingpong.c Thu Aug 2 09:00:09 2018 (r337118) +++ stable/11/contrib/ofed/libibverbs/examples/rc_pingpong.c Thu Aug 2 09:02:19 2018 (r337119) @@ -154,7 +154,7 @@ static struct pingpong_dest *pp_client_exch_dest(const { struct addrinfo *res, *t; struct addrinfo hints = { - .ai_family = AF_INET, + .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; char *service; Modified: stable/11/contrib/ofed/libibverbs/examples/srq_pingpong.c ============================================================================== --- stable/11/contrib/ofed/libibverbs/examples/srq_pingpong.c Thu Aug 2 09:00:09 2018 (r337118) +++ stable/11/contrib/ofed/libibverbs/examples/srq_pingpong.c Thu Aug 2 09:02:19 2018 (r337119) @@ -147,7 +147,7 @@ static struct pingpong_dest *pp_client_exch_dest(const { struct addrinfo *res, *t; struct addrinfo hints = { - .ai_family = AF_INET, + .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; char *service; Modified: stable/11/contrib/ofed/libibverbs/examples/uc_pingpong.c ============================================================================== --- stable/11/contrib/ofed/libibverbs/examples/uc_pingpong.c Thu Aug 2 09:00:09 2018 (r337118) +++ stable/11/contrib/ofed/libibverbs/examples/uc_pingpong.c Thu Aug 2 09:02:19 2018 (r337119) @@ -127,7 +127,7 @@ static struct pingpong_dest *pp_client_exch_dest(const { struct addrinfo *res, *t; struct addrinfo hints = { - .ai_family = AF_INET, + .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; char *service; Modified: stable/11/contrib/ofed/libibverbs/examples/ud_pingpong.c ============================================================================== --- stable/11/contrib/ofed/libibverbs/examples/ud_pingpong.c Thu Aug 2 09:00:09 2018 (r337118) +++ stable/11/contrib/ofed/libibverbs/examples/ud_pingpong.c Thu Aug 2 09:02:19 2018 (r337119) @@ -127,7 +127,7 @@ static struct pingpong_dest *pp_client_exch_dest(const { struct addrinfo *res, *t; struct addrinfo hints = { - .ai_family = AF_INET, + .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; char *service; Modified: stable/11/contrib/ofed/libibverbs/examples/xsrq_pingpong.c ============================================================================== --- stable/11/contrib/ofed/libibverbs/examples/xsrq_pingpong.c Thu Aug 2 09:00:09 2018 (r337118) +++ stable/11/contrib/ofed/libibverbs/examples/xsrq_pingpong.c Thu Aug 2 09:02:19 2018 (r337119) @@ -544,7 +544,7 @@ static int pp_client_connect(const char *servername, i int ret; int sockfd = -1; struct addrinfo hints = { - .ai_family = AF_INET, + .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; From owner-svn-src-all@freebsd.org Thu Aug 2 09:03:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96BD1106030C; Thu, 2 Aug 2018 09:03:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BDA97F4CD; Thu, 2 Aug 2018 09:03:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2BD40791F; Thu, 2 Aug 2018 09:03:09 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72939S8067339; Thu, 2 Aug 2018 09:03:09 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w729395w067338; Thu, 2 Aug 2018 09:03:09 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808020903.w729395w067338@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Thu, 2 Aug 2018 09:03:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337120 - stable/11/sys/compat/linuxkpi/common/include/asm X-SVN-Group: stable-11 X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: stable/11/sys/compat/linuxkpi/common/include/asm X-SVN-Commit-Revision: 337120 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 09:03:09 -0000 Author: hselasky Date: Thu Aug 2 09:03:08 2018 New Revision: 337120 URL: https://svnweb.freebsd.org/changeset/base/337120 Log: MFC r337056: Don't refer to non-existing atomic functions, even though not compiled, in the LinuxKPI. Found by: rpokala @ Sponsored by: Mellanox Technologies Modified: stable/11/sys/compat/linuxkpi/common/include/asm/atomic.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/compat/linuxkpi/common/include/asm/atomic.h ============================================================================== --- stable/11/sys/compat/linuxkpi/common/include/asm/atomic.h Thu Aug 2 09:02:19 2018 (r337119) +++ stable/11/sys/compat/linuxkpi/common/include/asm/atomic.h Thu Aug 2 09:03:08 2018 (r337120) @@ -2,7 +2,7 @@ * Copyright (c) 2010 Isilon Systems, Inc. * Copyright (c) 2010 iX Systems, Inc. * Copyright (c) 2010 Panasas, Inc. - * Copyright (c) 2013-2017 Mellanox Technologies, Ltd. + * Copyright (c) 2013-2018 Mellanox Technologies, Ltd. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -159,6 +159,21 @@ atomic_cmpxchg(atomic_t *v, int old, int new) return (ret); } +#if defined(__i386__) || defined(__amd64__) +#define LINUXKPI_ATOMIC_8(...) __VA_ARGS__ +#define LINUXKPI_ATOMIC_16(...) __VA_ARGS__ +#else +#define LINUXKPI_ATOMIC_8(...) +#define LINUXKPI_ATOMIC_16(...) +#endif + +#if defined(__amd64__) || (defined(__ARM_ARCH) && (__ARM_ARCH >= 6)) || \ + defined(__aarch64__) || defined(__powerpc64__) || defined(__riscv) +#define LINUXKPI_ATOMIC_64(...) __VA_ARGS__ +#else +#define LINUXKPI_ATOMIC_64(...) +#endif + #define cmpxchg(ptr, old, new) ({ \ union { \ __typeof(*(ptr)) val; \ @@ -168,30 +183,39 @@ atomic_cmpxchg(atomic_t *v, int old, int new) u64 u64[0]; \ } __ret = { .val = (old) }, __new = { .val = (new) }; \ \ - CTASSERT(sizeof(__ret.val) == 1 || sizeof(__ret.val) == 2 || \ - sizeof(__ret.val) == 4 || sizeof(__ret.val) == 8); \ + CTASSERT( \ + LINUXKPI_ATOMIC_8(sizeof(__ret.val) == 1 ||) \ + LINUXKPI_ATOMIC_16(sizeof(__ret.val) == 2 ||) \ + LINUXKPI_ATOMIC_64(sizeof(__ret.val) == 8 ||) \ + sizeof(__ret.val) == 4); \ \ switch (sizeof(__ret.val)) { \ + LINUXKPI_ATOMIC_8( \ case 1: \ while (!atomic_fcmpset_8((volatile u8 *)(ptr), \ __ret.u8, __new.u8[0]) && __ret.val == (old)) \ ; \ break; \ + ) \ + LINUXKPI_ATOMIC_16( \ case 2: \ while (!atomic_fcmpset_16((volatile u16 *)(ptr), \ __ret.u16, __new.u16[0]) && __ret.val == (old)) \ ; \ break; \ + ) \ case 4: \ while (!atomic_fcmpset_32((volatile u32 *)(ptr), \ __ret.u32, __new.u32[0]) && __ret.val == (old)) \ ; \ break; \ + LINUXKPI_ATOMIC_64( \ case 8: \ while (!atomic_fcmpset_64((volatile u64 *)(ptr), \ __ret.u64, __new.u64[0]) && __ret.val == (old)) \ ; \ break; \ + ) \ } \ __ret.val; \ }) @@ -207,30 +231,39 @@ atomic_cmpxchg(atomic_t *v, int old, int new) u64 u64[0]; \ } __ret, __new = { .val = (new) }; \ \ - CTASSERT(sizeof(__ret.val) == 1 || sizeof(__ret.val) == 2 || \ - sizeof(__ret.val) == 4 || sizeof(__ret.val) == 8); \ + CTASSERT( \ + LINUXKPI_ATOMIC_8(sizeof(__ret.val) == 1 ||) \ + LINUXKPI_ATOMIC_16(sizeof(__ret.val) == 2 ||) \ + LINUXKPI_ATOMIC_64(sizeof(__ret.val) == 8 ||) \ + sizeof(__ret.val) == 4); \ \ switch (sizeof(__ret.val)) { \ + LINUXKPI_ATOMIC_8( \ case 1: \ __ret.val = READ_ONCE(*ptr); \ while (!atomic_fcmpset_8((volatile u8 *)(ptr), \ __ret.u8, __new.u8[0])) \ ; \ break; \ + ) \ + LINUXKPI_ATOMIC_16( \ case 2: \ __ret.val = READ_ONCE(*ptr); \ while (!atomic_fcmpset_16((volatile u16 *)(ptr), \ __ret.u16, __new.u16[0])) \ ; \ break; \ + ) \ case 4: \ __ret.u32[0] = atomic_swap_32((volatile u32 *)(ptr), \ __new.u32[0]); \ break; \ + LINUXKPI_ATOMIC_64( \ case 8: \ __ret.u64[0] = atomic_swap_64((volatile u64 *)(ptr), \ __new.u64[0]); \ break; \ + ) \ } \ __ret.val; \ }) From owner-svn-src-all@freebsd.org Thu Aug 2 09:29:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 609DE1060B48; Thu, 2 Aug 2018 09:29:42 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F420380024; Thu, 2 Aug 2018 09:29:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C88A57CC6; Thu, 2 Aug 2018 09:29:41 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w729Tfh7078066; Thu, 2 Aug 2018 09:29:41 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w729TeNm078059; Thu, 2 Aug 2018 09:29:40 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201808020929.w729TeNm078059@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Thu, 2 Aug 2018 09:29:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337121 - in stable/11/sys: cam cam/ata cam/nvme cam/scsi dev/acpica/Osd dev/pccbb kern X-SVN-Group: stable-11 X-SVN-Commit-Author: avg X-SVN-Commit-Paths: in stable/11/sys: cam cam/ata cam/nvme cam/scsi dev/acpica/Osd dev/pccbb kern X-SVN-Commit-Revision: 337121 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 09:29:42 -0000 Author: avg Date: Thu Aug 2 09:29:39 2018 New Revision: 337121 URL: https://svnweb.freebsd.org/changeset/base/337121 Log: MFC r335934: remove unneeded inclusion of sys/interrupt.h from several files Modified: stable/11/sys/cam/ata/ata_xpt.c stable/11/sys/cam/cam_xpt.c stable/11/sys/cam/nvme/nvme_xpt.c stable/11/sys/cam/scsi/scsi_xpt.c stable/11/sys/dev/acpica/Osd/OsdSchedule.c stable/11/sys/dev/pccbb/pccbb.c stable/11/sys/kern/subr_bus.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/cam/ata/ata_xpt.c ============================================================================== --- stable/11/sys/cam/ata/ata_xpt.c Thu Aug 2 09:03:08 2018 (r337120) +++ stable/11/sys/cam/ata/ata_xpt.c Thu Aug 2 09:29:39 2018 (r337121) @@ -37,7 +37,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include Modified: stable/11/sys/cam/cam_xpt.c ============================================================================== --- stable/11/sys/cam/cam_xpt.c Thu Aug 2 09:03:08 2018 (r337120) +++ stable/11/sys/cam/cam_xpt.c Thu Aug 2 09:29:39 2018 (r337121) @@ -39,7 +39,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include #include Modified: stable/11/sys/cam/nvme/nvme_xpt.c ============================================================================== --- stable/11/sys/cam/nvme/nvme_xpt.c Thu Aug 2 09:03:08 2018 (r337120) +++ stable/11/sys/cam/nvme/nvme_xpt.c Thu Aug 2 09:29:39 2018 (r337121) @@ -39,7 +39,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include Modified: stable/11/sys/cam/scsi/scsi_xpt.c ============================================================================== --- stable/11/sys/cam/scsi/scsi_xpt.c Thu Aug 2 09:03:08 2018 (r337120) +++ stable/11/sys/cam/scsi/scsi_xpt.c Thu Aug 2 09:29:39 2018 (r337121) @@ -40,7 +40,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include Modified: stable/11/sys/dev/acpica/Osd/OsdSchedule.c ============================================================================== --- stable/11/sys/dev/acpica/Osd/OsdSchedule.c Thu Aug 2 09:03:08 2018 (r337120) +++ stable/11/sys/dev/acpica/Osd/OsdSchedule.c Thu Aug 2 09:29:39 2018 (r337121) @@ -37,7 +37,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include #include Modified: stable/11/sys/dev/pccbb/pccbb.c ============================================================================== --- stable/11/sys/dev/pccbb/pccbb.c Thu Aug 2 09:03:08 2018 (r337120) +++ stable/11/sys/dev/pccbb/pccbb.c Thu Aug 2 09:29:39 2018 (r337121) @@ -84,7 +84,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include #include Modified: stable/11/sys/kern/subr_bus.c ============================================================================== --- stable/11/sys/kern/subr_bus.c Thu Aug 2 09:03:08 2018 (r337120) +++ stable/11/sys/kern/subr_bus.c Thu Aug 2 09:29:39 2018 (r337121) @@ -54,7 +54,6 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include #include #include From owner-svn-src-all@freebsd.org Thu Aug 2 09:31:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D4F71060DA6; Thu, 2 Aug 2018 09:31:51 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7298802FC; Thu, 2 Aug 2018 09:31:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A97477E18; Thu, 2 Aug 2018 09:31:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w729VofX079028; Thu, 2 Aug 2018 09:31:50 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w729VoGH079027; Thu, 2 Aug 2018 09:31:50 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201808020931.w729VoGH079027@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Thu, 2 Aug 2018 09:31:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337122 - stable/11/sys/security/audit X-SVN-Group: stable-11 X-SVN-Commit-Author: avg X-SVN-Commit-Paths: stable/11/sys/security/audit X-SVN-Commit-Revision: 337122 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 09:31:51 -0000 Author: avg Date: Thu Aug 2 09:31:50 2018 New Revision: 337122 URL: https://svnweb.freebsd.org/changeset/base/337122 Log: MFC r336641: fix incorrect operator in the AUDITPIPE_SET_QLIMIT bounds check Modified: stable/11/sys/security/audit/audit_pipe.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/security/audit/audit_pipe.c ============================================================================== --- stable/11/sys/security/audit/audit_pipe.c Thu Aug 2 09:29:39 2018 (r337121) +++ stable/11/sys/security/audit/audit_pipe.c Thu Aug 2 09:31:50 2018 (r337122) @@ -756,7 +756,7 @@ audit_pipe_ioctl(struct cdev *dev, u_long cmd, caddr_t case AUDITPIPE_SET_QLIMIT: /* Lockless integer write. */ - if (*(u_int *)data >= AUDIT_PIPE_QLIMIT_MIN || + if (*(u_int *)data >= AUDIT_PIPE_QLIMIT_MIN && *(u_int *)data <= AUDIT_PIPE_QLIMIT_MAX) { ap->ap_qlimit = *(u_int *)data; error = 0; From owner-svn-src-all@freebsd.org Thu Aug 2 09:41:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3ACF10611AD; Thu, 2 Aug 2018 09:41:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69146809B9; Thu, 2 Aug 2018 09:41:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4A5D27ECB; Thu, 2 Aug 2018 09:41:56 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w729fuwQ086419; Thu, 2 Aug 2018 09:41:56 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w729fuqP086418; Thu, 2 Aug 2018 09:41:56 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201808020941.w729fuqP086418@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Thu, 2 Aug 2018 09:41:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337123 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: avg X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 337123 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 09:41:56 -0000 Author: avg Date: Thu Aug 2 09:41:55 2018 New Revision: 337123 URL: https://svnweb.freebsd.org/changeset/base/337123 Log: fix a typo resulting in a wrong variable in kern_syscall_deregister The difference is between sysent, a global, and sysents, a function parameter. Modified: head/sys/kern/kern_syscalls.c Modified: head/sys/kern/kern_syscalls.c ============================================================================== --- head/sys/kern/kern_syscalls.c Thu Aug 2 09:31:50 2018 (r337122) +++ head/sys/kern/kern_syscalls.c Thu Aug 2 09:41:55 2018 (r337123) @@ -148,7 +148,7 @@ kern_syscall_deregister(struct sysent *sysents, int of if ((se->sy_thrcnt & SY_THR_STATIC) != 0) return (EINVAL); syscall_thread_drain(se); - sysent[offset] = *old_sysent; + sysents[offset] = *old_sysent; return (0); } From owner-svn-src-all@freebsd.org Thu Aug 2 11:55:17 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 516B2106421D; Thu, 2 Aug 2018 11:55:17 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EBCE28467A; Thu, 2 Aug 2018 11:55:16 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C7DFC1155C; Thu, 2 Aug 2018 11:55:16 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72BtGxJ055036; Thu, 2 Aug 2018 11:55:16 GMT (envelope-from trasz@FreeBSD.org) Received: (from trasz@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72BtGjU055035; Thu, 2 Aug 2018 11:55:16 GMT (envelope-from trasz@FreeBSD.org) Message-Id: <201808021155.w72BtGjU055035@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: trasz set sender to trasz@FreeBSD.org using -f From: Edward Tomasz Napierala Date: Thu, 2 Aug 2018 11:55:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337124 - head/libexec/rtld-elf/tests X-SVN-Group: head X-SVN-Commit-Author: trasz X-SVN-Commit-Paths: head/libexec/rtld-elf/tests X-SVN-Commit-Revision: 337124 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 11:55:17 -0000 Author: trasz Date: Thu Aug 2 11:55:16 2018 New Revision: 337124 URL: https://svnweb.freebsd.org/changeset/base/337124 Log: Fix regression tests broken by r337067. MFC after: 2 weeks Sponsored by: DARPA, AFRL Modified: head/libexec/rtld-elf/tests/ld_library_pathfds.c Modified: head/libexec/rtld-elf/tests/ld_library_pathfds.c ============================================================================== --- head/libexec/rtld-elf/tests/ld_library_pathfds.c Thu Aug 2 09:41:55 2018 (r337123) +++ head/libexec/rtld-elf/tests/ld_library_pathfds.c Thu Aug 2 11:55:16 2018 (r337124) @@ -186,7 +186,7 @@ expect_missing_library(int binary, char *pathfds) { char * const env[] = { pathfds, NULL }; try_to_run(binary, 1, env, "", - "Shared object \"libpythagoras.so.0\" not found," + "ld-elf.so.1: Shared object \"libpythagoras.so.0\" not found," " required by \"target\"\n"); } From owner-svn-src-all@freebsd.org Thu Aug 2 12:08:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6537110651AA; Thu, 2 Aug 2018 12:08:53 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14FB08512D; Thu, 2 Aug 2018 12:08:53 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DC9DF11709; Thu, 2 Aug 2018 12:08:52 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72C8qSI060203; Thu, 2 Aug 2018 12:08:52 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72C8qZR060202; Thu, 2 Aug 2018 12:08:52 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808021208.w72C8qZR060202@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Thu, 2 Aug 2018 12:08:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337125 - head/sys/riscv/riscv X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/sys/riscv/riscv X-SVN-Commit-Revision: 337125 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:08:53 -0000 Author: br Date: Thu Aug 2 12:08:52 2018 New Revision: 337125 URL: https://svnweb.freebsd.org/changeset/base/337125 Log: o Correctly set user tls base: consider TP_OFFSET. o Ensure tp (thread pointer) saved before copying the pcb. Sponsored by: DARPA, AFRL Modified: head/sys/riscv/riscv/vm_machdep.c Modified: head/sys/riscv/riscv/vm_machdep.c ============================================================================== --- head/sys/riscv/riscv/vm_machdep.c Thu Aug 2 11:55:16 2018 (r337124) +++ head/sys/riscv/riscv/vm_machdep.c Thu Aug 2 12:08:52 2018 (r337125) @@ -55,6 +55,10 @@ __FBSDID("$FreeBSD$"); #include #include +#if __riscv_xlen == 64 +#define TP_OFFSET 16 /* sizeof(struct tcb) */ +#endif + /* * Finish a fork operation, with process p2 nearly set up. * Copy and update the pcb, set up the stack so that the child @@ -65,10 +69,23 @@ cpu_fork(struct thread *td1, struct proc *p2, struct t { struct pcb *pcb2; struct trapframe *tf; + register_t val; if ((flags & RFPROC) == 0) return; + if (td1 == curthread) { + /* + * Save the tp. These normally happen in cpu_switch, + * but if userland changes this then forks this may + * not have happened. + */ + __asm __volatile("mv %0, tp" : "=&r"(val)); + td1->td_pcb->pcb_tp = val; + + /* RISCVTODO: save the FPU state here */ + } + pcb2 = (struct pcb *)(td2->td_kstack + td2->td_kstack_pages * PAGE_SIZE) - 1; @@ -198,7 +215,9 @@ cpu_set_user_tls(struct thread *td, void *tls_base) return (EINVAL); pcb = td->td_pcb; - pcb->pcb_tp = (register_t)tls_base; + pcb->pcb_tp = (register_t)tls_base + TP_OFFSET; + if (td == curthread) + __asm __volatile("mv tp, %0" :: "r"(pcb->pcb_tp)); return (0); } From owner-svn-src-all@freebsd.org Thu Aug 2 12:12:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3C7D1065589; Thu, 2 Aug 2018 12:12:13 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 91F3A858BD; Thu, 2 Aug 2018 12:12:13 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 72F3A11889; Thu, 2 Aug 2018 12:12:13 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72CCDN7065143; Thu, 2 Aug 2018 12:12:13 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72CCDG1065142; Thu, 2 Aug 2018 12:12:13 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808021212.w72CCDG1065142@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Thu, 2 Aug 2018 12:12:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337126 - head/sys/riscv/riscv X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/sys/riscv/riscv X-SVN-Commit-Revision: 337126 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:12:14 -0000 Author: br Date: Thu Aug 2 12:12:13 2018 New Revision: 337126 URL: https://svnweb.freebsd.org/changeset/base/337126 Log: o Don't overwrite tp in fork_trampoline(). o Save and restore tp in cpu_switch(). o Restore tp in cpu_throw(). o Save tp in savectx(). This makes libthr tests happy. In particular fork_test:fork. Sponsored by: DARPA, AFRL Modified: head/sys/riscv/riscv/swtch.S Modified: head/sys/riscv/riscv/swtch.S ============================================================================== --- head/sys/riscv/riscv/swtch.S Thu Aug 2 12:08:52 2018 (r337125) +++ head/sys/riscv/riscv/swtch.S Thu Aug 2 12:12:13 2018 (r337126) @@ -179,6 +179,7 @@ ENTRY(cpu_throw) /* Load registers */ ld ra, (PCB_RA)(x13) ld sp, (PCB_SP)(x13) + ld tp, (PCB_TP)(x13) /* s[0-11] */ ld s0, (PCB_S + 0 * 8)(x13) @@ -231,6 +232,7 @@ ENTRY(cpu_switch) /* Store ra, sp and the callee-saved registers */ sd ra, (PCB_RA)(x13) sd sp, (PCB_SP)(x13) + sd tp, (PCB_TP)(x13) /* s[0-11] */ sd s0, (PCB_S + 0 * 8)(x13) @@ -303,6 +305,7 @@ ENTRY(cpu_switch) #endif /* Restore the registers */ + ld tp, (PCB_TP)(x13) ld ra, (PCB_RA)(x13) ld sp, (PCB_SP)(x13) @@ -392,7 +395,6 @@ ENTRY(fork_trampoline) ld a7, (TF_A + 7 * 8)(sp) /* Load user ra and sp */ - ld tp, (TF_TP)(sp) ld ra, (TF_RA)(sp) /* @@ -416,6 +418,7 @@ ENTRY(savectx) /* Store ra, sp and the callee-saved registers */ sd ra, (PCB_RA)(a0) sd sp, (PCB_SP)(a0) + sd tp, (PCB_TP)(a0) /* s[0-11] */ sd s0, (PCB_S + 0 * 8)(a0) From owner-svn-src-all@freebsd.org Thu Aug 2 12:13:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 82F2B10656F5; Thu, 2 Aug 2018 12:13:53 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F9D185BB3; Thu, 2 Aug 2018 12:13:53 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0E0011189C; Thu, 2 Aug 2018 12:13:53 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72CDqO7065425; Thu, 2 Aug 2018 12:13:52 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72CDqQV065424; Thu, 2 Aug 2018 12:13:52 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808021213.w72CDqQV065424@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Thu, 2 Aug 2018 12:13:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337127 - head/sys/riscv/riscv X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/sys/riscv/riscv X-SVN-Commit-Revision: 337127 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:13:53 -0000 Author: br Date: Thu Aug 2 12:13:52 2018 New Revision: 337127 URL: https://svnweb.freebsd.org/changeset/base/337127 Log: Don't overwrite tp in set_mcontext(). This makes libthr/swapcontext_test:swapcontext1 happy. Sponsored by: DARPA, AFRL Modified: head/sys/riscv/riscv/machdep.c Modified: head/sys/riscv/riscv/machdep.c ============================================================================== --- head/sys/riscv/riscv/machdep.c Thu Aug 2 12:12:13 2018 (r337126) +++ head/sys/riscv/riscv/machdep.c Thu Aug 2 12:13:52 2018 (r337127) @@ -342,7 +342,6 @@ set_mcontext(struct thread *td, mcontext_t *mcp) tf->tf_ra = mcp->mc_gpregs.gp_ra; tf->tf_sp = mcp->mc_gpregs.gp_sp; tf->tf_gp = mcp->mc_gpregs.gp_gp; - tf->tf_tp = mcp->mc_gpregs.gp_tp; tf->tf_sepc = mcp->mc_gpregs.gp_sepc; tf->tf_sstatus = mcp->mc_gpregs.gp_sstatus; From owner-svn-src-all@freebsd.org Thu Aug 2 12:21:40 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D1F81065F4B; Thu, 2 Aug 2018 12:21:40 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E75368646C; Thu, 2 Aug 2018 12:21:39 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C8851118FA; Thu, 2 Aug 2018 12:21:39 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72CLdrE068949; Thu, 2 Aug 2018 12:21:39 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72CLdsP068948; Thu, 2 Aug 2018 12:21:39 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808021221.w72CLdsP068948@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Thu, 2 Aug 2018 12:21:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337128 - head/libexec/rtld-elf/riscv X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/libexec/rtld-elf/riscv X-SVN-Commit-Revision: 337128 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:21:40 -0000 Author: br Date: Thu Aug 2 12:21:39 2018 New Revision: 337128 URL: https://svnweb.freebsd.org/changeset/base/337128 Log: Save and restore floating-point arguments. Sponsored by: DARPA, AFRL Modified: head/libexec/rtld-elf/riscv/rtld_start.S Modified: head/libexec/rtld-elf/riscv/rtld_start.S ============================================================================== --- head/libexec/rtld-elf/riscv/rtld_start.S Thu Aug 2 12:13:52 2018 (r337127) +++ head/libexec/rtld-elf/riscv/rtld_start.S Thu Aug 2 12:21:39 2018 (r337128) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2015 Ruslan Bukin + * Copyright (c) 2015-2018 Ruslan Bukin * All rights reserved. * * This software was developed by SRI International and the University of @@ -64,7 +64,7 @@ END(.rtld_start) */ ENTRY(_rtld_bind_start) /* Save the arguments and ra */ - addi sp, sp, -(8 * 25) + addi sp, sp, -(8 * 17) sd a0, (8 * 0)(sp) sd a1, (8 * 1)(sp) sd a2, (8 * 2)(sp) @@ -74,17 +74,17 @@ ENTRY(_rtld_bind_start) sd a6, (8 * 6)(sp) sd a7, (8 * 7)(sp) sd ra, (8 * 8)(sp) -#if 0 - /* RISCVTODO VFP */ + +#ifdef __riscv_float_abi_double /* Save any floating-point arguments */ - fsq fa0, (8 * 9)(sp) - fsq fa1, (8 * 11)(sp) - fsq fa2, (8 * 13)(sp) - fsq fa3, (8 * 15)(sp) - fsq fa4, (8 * 17)(sp) - fsq fa5, (8 * 19)(sp) - fsq fa6, (8 * 21)(sp) - fsq fa7, (8 * 23)(sp) + fsd fa0, (8 * 9)(sp) + fsd fa1, (8 * 10)(sp) + fsd fa2, (8 * 11)(sp) + fsd fa3, (8 * 12)(sp) + fsd fa4, (8 * 13)(sp) + fsd fa5, (8 * 14)(sp) + fsd fa6, (8 * 15)(sp) + fsd fa7, (8 * 16)(sp) #endif /* Reloc offset is 3x of the .got.plt offset */ @@ -110,19 +110,19 @@ ENTRY(_rtld_bind_start) ld a6, (8 * 6)(sp) ld a7, (8 * 7)(sp) ld ra, (8 * 8)(sp) -#if 0 - /* RISCVTODO VFP */ + +#ifdef __riscv_float_abi_double /* Restore floating-point arguments */ - flq fa0, (8 * 9)(sp) - flq fa1, (8 * 11)(sp) - flq fa2, (8 * 13)(sp) - flq fa3, (8 * 15)(sp) - flq fa4, (8 * 17)(sp) - flq fa5, (8 * 19)(sp) - flq fa6, (8 * 21)(sp) - flq fa7, (8 * 23)(sp) + fld fa0, (8 * 9)(sp) + fld fa1, (8 * 10)(sp) + fld fa2, (8 * 11)(sp) + fld fa3, (8 * 12)(sp) + fld fa4, (8 * 13)(sp) + fld fa5, (8 * 14)(sp) + fld fa6, (8 * 15)(sp) + fld fa7, (8 * 16)(sp) #endif - addi sp, sp, (8 * 25) + addi sp, sp, (8 * 17) /* Call into the correct function */ jr t0 From owner-svn-src-all@freebsd.org Thu Aug 2 12:24:35 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 807CD10661BA; Thu, 2 Aug 2018 12:24:35 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2789B866EC; Thu, 2 Aug 2018 12:24:35 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 02C5011A32; Thu, 2 Aug 2018 12:24:35 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72COYJ8070603; Thu, 2 Aug 2018 12:24:34 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72COYr3070602; Thu, 2 Aug 2018 12:24:34 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808021224.w72COYr3070602@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Thu, 2 Aug 2018 12:24:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337129 - head/lib/libthread_db/arch/riscv X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/lib/libthread_db/arch/riscv X-SVN-Commit-Revision: 337129 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:24:35 -0000 Author: br Date: Thu Aug 2 12:24:34 2018 New Revision: 337129 URL: https://svnweb.freebsd.org/changeset/base/337129 Log: Implement pt_fpreg_to_ucontext(), pt_ucontext_to_fpreg(). Sponsored by: DARPA, AFRL Modified: head/lib/libthread_db/arch/riscv/libpthread_md.c Modified: head/lib/libthread_db/arch/riscv/libpthread_md.c ============================================================================== --- head/lib/libthread_db/arch/riscv/libpthread_md.c Thu Aug 2 12:21:39 2018 (r337128) +++ head/lib/libthread_db/arch/riscv/libpthread_md.c Thu Aug 2 12:24:34 2018 (r337129) @@ -80,15 +80,21 @@ pt_ucontext_to_reg(const ucontext_t *uc, struct reg *r void pt_fpreg_to_ucontext(const struct fpreg *r __unused, ucontext_t *uc __unused) { + mcontext_t *mc = &uc->uc_mcontext; - /* RISCVTODO */ + memcpy(&mc->mc_fpregs, r, sizeof(*r)); + mc->mc_flags |= _MC_FP_VALID; } void pt_ucontext_to_fpreg(const ucontext_t *uc __unused, struct fpreg *r __unused) { + const mcontext_t *mc = &uc->uc_mcontext; - /* RISCVTODO */ + if (mc->mc_flags & _MC_FP_VALID) + memcpy(r, &mc->mc_fpregs, sizeof(*r)); + else + memset(r, 0, sizeof(*r)); } void From owner-svn-src-all@freebsd.org Thu Aug 2 12:39:24 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A80C1066696; Thu, 2 Aug 2018 12:39:24 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF8B786D23; Thu, 2 Aug 2018 12:39:23 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B592811BC8; Thu, 2 Aug 2018 12:39:23 +0000 (UTC) (envelope-from emaste@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72CdNRg076123; Thu, 2 Aug 2018 12:39:23 GMT (envelope-from emaste@FreeBSD.org) Received: (from emaste@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72CdNfe076122; Thu, 2 Aug 2018 12:39:23 GMT (envelope-from emaste@FreeBSD.org) Message-Id: <201808021239.w72CdNfe076122@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: emaste set sender to emaste@FreeBSD.org using -f From: Ed Maste Date: Thu, 2 Aug 2018 12:39:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337130 - head/share/man/man5 X-SVN-Group: head X-SVN-Commit-Author: emaste X-SVN-Commit-Paths: head/share/man/man5 X-SVN-Commit-Revision: 337130 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:39:24 -0000 Author: emaste Date: Thu Aug 2 12:39:23 2018 New Revision: 337130 URL: https://svnweb.freebsd.org/changeset/base/337130 Log: Regen src.conf.5 after r337057, LLD_BOOTSTRAP on armv7 Modified: head/share/man/man5/src.conf.5 Modified: head/share/man/man5/src.conf.5 ============================================================================== --- head/share/man/man5/src.conf.5 Thu Aug 2 12:24:34 2018 (r337129) +++ head/share/man/man5/src.conf.5 Thu Aug 2 12:39:23 2018 (r337130) @@ -1,6 +1,6 @@ .\" DO NOT EDIT-- this file is @generated by tools/build/options/makeman. .\" $FreeBSD$ -.Dd July 30, 2018 +.Dd August 2, 2018 .Dt SRC.CONF 5 .Os .Sh NAME @@ -1001,12 +1001,12 @@ To be able to build the system, either Binutils or LLD enabled unless an alternate linker is provided via XLD. .Pp This is a default setting on -arm/arm, arm/armv6, arm/armv7, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. +arm/arm, arm/armv6, mips/mipsel, mips/mips, mips/mips64el, mips/mips64, mips/mipsn32, mips/mipselhf, mips/mipshf, mips/mips64elhf, mips/mips64hf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. .It Va WITH_LLD_BOOTSTRAP Set to build the LLD linker during the bootstrap phase of the build. .Pp This is a default setting on -amd64/amd64, arm64/aarch64 and i386/i386. +amd64/amd64, arm/armv7, arm64/aarch64 and i386/i386. .It Va WITHOUT_LLD_IS_LD Set to use GNU binutils ld as the system linker, instead of LLVM's LLD. .Pp From owner-svn-src-all@freebsd.org Thu Aug 2 12:39:58 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA70310666D8; Thu, 2 Aug 2018 12:39:58 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EBE686E68; Thu, 2 Aug 2018 12:39:58 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-it0-x233.google.com with SMTP id e14-v6so2983211itf.1; Thu, 02 Aug 2018 05:39:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=zTYK0DUC2GkSkd3ueu/Idl6odDS3gOZUVycEHrbBzSA=; b=pj49cIQh3cypJak5GkgScIBa7cHfT/eAbSXGJmLHNRk8oabYw1V/D98h+GyKc44+98 4b0Z8az69KzxI/rXWmcxt9Z/k5NpdFrnUTLCTclMCq6VrrAWOg3b3o6UTT1/mznUrVkC VUC1SiGDZ4pDPBfByr9/S6OMa8Hh9BhJQednO39weoS6VyWE8ApqoLMX2IiQX8k5i1Oo 4qfrPjymx4m0luDwKxR1y4sEz2B8jSgeOqRDc7hQ488Wikg+KqEsP9E0FvkmaSiurIKB RDPWYIzFE2ilauzSssMn5EjmfIMw/tpPlZQSBzpEY3NzOaXYXCbCqH5ymzETotmZ7uzK w0vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=zTYK0DUC2GkSkd3ueu/Idl6odDS3gOZUVycEHrbBzSA=; b=saZtRCyRedWmvwFPhggJ+Dy419ciakxhnUl+Mv8+lckgFLcWmmLvlvb7CEDDucYCQq scWrqGtS2V+B0P1WTx0UML1RsXY6EYGYD9SwcH256NEubRgeeg+tRPhC266oWfupjZDY R8dnkWzWj7O39hUHFXTe0zcphVNBrMyNX5jqMgUlDzm3KpXCo6JE/R+PNjNV1+uVtDHy LuEuDy0gZEJVP32hq3MX5Y3m1RhVUQw/mpIyDN8Qc7JdNydsIl/ubIxd0zYddDLBrdd3 UibPGZ+fYaBnVPYbeixVj/CPUeeEUctGdQGvntqCu6RBJewfpm8XpuRnQf3s6xHp1pH/ C8+w== X-Gm-Message-State: AOUpUlHj3qjLQ+yE3cylv/UUpJmzfqmxGm7sfaBlxYgFBLgTw7M928F+ 7O6Xolm6MG0/WKp+9lItCpqVNCsiAJsC1AbNgQWfQQ== X-Google-Smtp-Source: AAOMgpfjf8ivRAbYH9J1eWmUFyjsSbIjdZbk3/r1BX8QWXCWm99tF67obVcxMTf093Nct4Ut66A91nojtGWFxvHOTNA= X-Received: by 2002:a24:c888:: with SMTP id w130-v6mr2452701itf.78.1533213597611; Thu, 02 Aug 2018 05:39:57 -0700 (PDT) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 2002:a6b:4a08:0:0:0:0:0 with HTTP; Thu, 2 Aug 2018 05:39:37 -0700 (PDT) In-Reply-To: <201806140115.w5E1FJH6099583@repo.freebsd.org> References: <201806140115.w5E1FJH6099583@repo.freebsd.org> From: Ed Maste Date: Thu, 2 Aug 2018 08:39:37 -0400 X-Google-Sender-Auth: -6bznNtVcohL7S_KeA8p9ucYK74 Message-ID: Subject: Re: svn commit: r335101 - in head: sbin sbin/camcontrol share/mk To: Warner Losh Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:39:59 -0000 On 13 June 2018 at 21:15, Warner Losh wrote: > Author: imp > Date: Thu Jun 14 01:15:19 2018 > New Revision: 335101 > URL: https://svnweb.freebsd.org/changeset/base/335101 > > Log: > NVME support is only for x86 and powerpc64. > > Implement MK_NVME ... This is missing WITH_NVME and WITHOUT_NVME descriptions in tools/build/options. From owner-svn-src-all@freebsd.org Thu Aug 2 12:43:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B2F21066904; Thu, 2 Aug 2018 12:43:41 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 511B68728C; Thu, 2 Aug 2018 12:43:41 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3333311D6B; Thu, 2 Aug 2018 12:43:41 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Che5P081064; Thu, 2 Aug 2018 12:43:40 GMT (envelope-from tsoome@FreeBSD.org) Received: (from tsoome@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Cheef081063; Thu, 2 Aug 2018 12:43:40 GMT (envelope-from tsoome@FreeBSD.org) Message-Id: <201808021243.w72Cheef081063@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tsoome set sender to tsoome@FreeBSD.org using -f From: Toomas Soome Date: Thu, 2 Aug 2018 12:43:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337131 - head/stand/efi/loader X-SVN-Group: head X-SVN-Commit-Author: tsoome X-SVN-Commit-Paths: head/stand/efi/loader X-SVN-Commit-Revision: 337131 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 12:43:41 -0000 Author: tsoome Date: Thu Aug 2 12:43:40 2018 New Revision: 337131 URL: https://svnweb.freebsd.org/changeset/base/337131 Log: loader.efi: efipart_inithandles() failure should not prevent devsw initialization The efipart_inithandles() can return error code when there are no block devices, however, we still should walk and initialize devsw. Modified: head/stand/efi/loader/main.c Modified: head/stand/efi/loader/main.c ============================================================================== --- head/stand/efi/loader/main.c Thu Aug 2 12:39:23 2018 (r337130) +++ head/stand/efi/loader/main.c Thu Aug 2 12:43:40 2018 (r337131) @@ -861,12 +861,15 @@ main(int argc, CHAR16 *argv[]) * Scan the BLOCK IO MEDIA handles then * march through the device switch probing for things. */ - if ((i = efipart_inithandles()) == 0) { - for (i = 0; devsw[i] != NULL; i++) - if (devsw[i]->dv_init != NULL) - (devsw[i]->dv_init)(); - } else - printf("efipart_inithandles failed %d, expect failures", i); + i = efipart_inithandles(); + if (i != 0) { + printf("efipart_inithandles failed with ERRNO %d, expect " + "failures", i); + } + + for (i = 0; devsw[i] != NULL; i++) + if (devsw[i]->dv_init != NULL) + (devsw[i]->dv_init)(); printf("%s\n", bootprog_info); printf(" Command line arguments:"); From owner-svn-src-all@freebsd.org Thu Aug 2 15:17:00 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3EB281069FC8; Thu, 2 Aug 2018 15:17:00 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E92658CB24; Thu, 2 Aug 2018 15:16:59 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C9AD813629; Thu, 2 Aug 2018 15:16:59 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72FGx4S057542; Thu, 2 Aug 2018 15:16:59 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72FGxZX057541; Thu, 2 Aug 2018 15:16:59 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808021516.w72FGxZX057541@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Thu, 2 Aug 2018 15:16:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337132 - stable/11/sys/kern X-SVN-Group: stable-11 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: stable/11/sys/kern X-SVN-Commit-Revision: 337132 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 15:17:00 -0000 Author: markj Date: Thu Aug 2 15:16:59 2018 New Revision: 337132 URL: https://svnweb.freebsd.org/changeset/base/337132 Log: MFC r336922: Remove a redundant check. Modified: stable/11/sys/kern/uipc_sockbuf.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/kern/uipc_sockbuf.c ============================================================================== --- stable/11/sys/kern/uipc_sockbuf.c Thu Aug 2 12:43:40 2018 (r337131) +++ stable/11/sys/kern/uipc_sockbuf.c Thu Aug 2 15:16:59 2018 (r337132) @@ -897,8 +897,6 @@ sbappendcontrol_locked(struct sockbuf *sb, struct mbuf SOCKBUF_LOCK_ASSERT(sb); - if (control == NULL) - panic("sbappendcontrol_locked"); space = m_length(control, &n) + m_length(m0, NULL); if (space > sbspace(sb)) From owner-svn-src-all@freebsd.org Thu Aug 2 15:17:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 172C41069FD4; Thu, 2 Aug 2018 15:17:02 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B33478CB25; Thu, 2 Aug 2018 15:17:01 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 1476310A87D; Thu, 2 Aug 2018 11:16:59 -0400 (EDT) Subject: Re: svn commit: r337127 - head/sys/riscv/riscv To: Ruslan Bukin , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201808021213.w72CDqQV065424@repo.freebsd.org> From: John Baldwin Message-ID: <7842b7e4-d14e-805f-719e-b393fc80b74c@FreeBSD.org> Date: Thu, 2 Aug 2018 08:16:59 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <201808021213.w72CDqQV065424@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 02 Aug 2018 11:17:00 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 15:17:02 -0000 On 8/2/18 5:13 AM, Ruslan Bukin wrote: > Author: br > Date: Thu Aug 2 12:13:52 2018 > New Revision: 337127 > URL: https://svnweb.freebsd.org/changeset/base/337127 > > Log: > Don't overwrite tp in set_mcontext(). > > This makes libthr/swapcontext_test:swapcontext1 happy. > > Sponsored by: DARPA, AFRL This seems a bit odd. Is get_mcontext() not setting gp_tp correctly, or is makecontext() not preserving it perhaps? In general I feel like we should be saving/restoring 'tp' in the trapframe on kernel entry/exit like other GPRs instead of in the pcb and then including it in 'struct reg'. This makes it possible for debuggers to handle TLS variables more sanely (and in general we need to fix the various struct reg in FreeBSD architectures to include TLS registers when possible, such as adding fs_base and gs_base on x86). It would be nice if RISC-V started off on the correct foot here. -- John Baldwin From owner-svn-src-all@freebsd.org Thu Aug 2 15:50:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD514106AB1A; Thu, 2 Aug 2018 15:50:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3E76A8DD5A; Thu, 2 Aug 2018 15:50:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w72FoU88010029 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 2 Aug 2018 18:50:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w72FoU88010029 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w72FoUer010027; Thu, 2 Aug 2018 18:50:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 2 Aug 2018 18:50:30 +0300 From: Konstantin Belousov To: John Baldwin Cc: Ruslan Bukin , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r337127 - head/sys/riscv/riscv Message-ID: <20180802155030.GA40119@kib.kiev.ua> References: <201808021213.w72CDqQV065424@repo.freebsd.org> <7842b7e4-d14e-805f-719e-b393fc80b74c@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7842b7e4-d14e-805f-719e-b393fc80b74c@FreeBSD.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 15:50:41 -0000 On Thu, Aug 02, 2018 at 08:16:59AM -0700, John Baldwin wrote: > On 8/2/18 5:13 AM, Ruslan Bukin wrote: > > Author: br > > Date: Thu Aug 2 12:13:52 2018 > > New Revision: 337127 > > URL: https://svnweb.freebsd.org/changeset/base/337127 > > > > Log: > > Don't overwrite tp in set_mcontext(). > > > > This makes libthr/swapcontext_test:swapcontext1 happy. > > > > Sponsored by: DARPA, AFRL > > This seems a bit odd. Is get_mcontext() not setting gp_tp correctly, or is > makecontext() not preserving it perhaps? > > In general I feel like we should be saving/restoring 'tp' in the trapframe > on kernel entry/exit like other GPRs instead of in the pcb and then > including it in 'struct reg'. This makes it possible for debuggers to > handle TLS variables more sanely (and in general we need to fix the various > struct reg in FreeBSD architectures to include TLS registers when possible, > such as adding fs_base and gs_base on x86). It would be nice if RISC-V > started off on the correct foot here. I wanted to do this on x86, but I gave up due to the ABI issues. When FSGSBASE instructions are supported on amd64, we save base on context switch and restore on return to usermode. From owner-svn-src-all@freebsd.org Thu Aug 2 16:03:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76D10106B30E; Thu, 2 Aug 2018 16:03:48 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A8ED8E7BE; Thu, 2 Aug 2018 16:03:48 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0AE5A13E64; Thu, 2 Aug 2018 16:03:48 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72G3ls5082903; Thu, 2 Aug 2018 16:03:47 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72G3lBs082902; Thu, 2 Aug 2018 16:03:47 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808021603.w72G3lBs082902@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Thu, 2 Aug 2018 16:03:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337133 - head/usr.bin/procstat/tests X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/usr.bin/procstat/tests X-SVN-Commit-Revision: 337133 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 16:03:48 -0000 Author: markj Date: Thu Aug 2 16:03:47 2018 New Revision: 337133 URL: https://svnweb.freebsd.org/changeset/base/337133 Log: Add a rudimentary test for procstat kstack. MFC after: 1 week Modified: head/usr.bin/procstat/tests/procstat_test.sh Modified: head/usr.bin/procstat/tests/procstat_test.sh ============================================================================== --- head/usr.bin/procstat/tests/procstat_test.sh Thu Aug 2 15:16:59 2018 (r337132) +++ head/usr.bin/procstat/tests/procstat_test.sh Thu Aug 2 16:03:47 2018 (r337133) @@ -144,10 +144,25 @@ file_descriptor_body() atf_check -o match:"$line_re" awk 'NR > 1' procstat.out } +atf_test_case kernel_stacks +kernel_stacks_head() +{ + atf_set "descr" "Captures kernel stacks for all visible threads" +} +kernel_stacks_body() +{ + atf_check -o save:procstat.out procstat -a kstack + atf_check -o not-empty awk '{if ($3 == "procstat") print}' procstat.out + + atf_check -o save:procstat.out procstat -kka + atf_check -o not-empty awk '{if ($3 == "procstat") print}' procstat.out +} + atf_init_test_cases() { atf_add_test_case binary_info atf_add_test_case command_line_arguments atf_add_test_case environment atf_add_test_case file_descriptor + atf_add_test_case kernel_stacks } From owner-svn-src-all@freebsd.org Thu Aug 2 17:32:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43238106D689; Thu, 2 Aug 2018 17:32:56 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E41117210A; Thu, 2 Aug 2018 17:32:55 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ACFF114D2E; Thu, 2 Aug 2018 17:32:55 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HWtHO029235; Thu, 2 Aug 2018 17:32:55 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HWtgt029234; Thu, 2 Aug 2018 17:32:55 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021732.w72HWtgt029234@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:32:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337138 - vendor/llvm/llvm-trunk-r338536 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/llvm/llvm-trunk-r338536 X-SVN-Commit-Revision: 337138 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:32:56 -0000 Author: dim Date: Thu Aug 2 17:32:55 2018 New Revision: 337138 URL: https://svnweb.freebsd.org/changeset/base/337138 Log: Tag llvm trunk r338536. Added: vendor/llvm/llvm-trunk-r338536/ - copied from r337137, vendor/llvm/dist/ From owner-svn-src-all@freebsd.org Thu Aug 2 17:32:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56D81106D680; Thu, 2 Aug 2018 17:32:52 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09229720FF; Thu, 2 Aug 2018 17:32:52 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C4F7214D2D; Thu, 2 Aug 2018 17:32:51 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HWpGD029189; Thu, 2 Aug 2018 17:32:51 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HWiRj029156; Thu, 2 Aug 2018 17:32:44 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021732.w72HWiRj029156@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:32:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337137 - in vendor/llvm/dist: cmake/modules docs docs/CommandGuide include/llvm include/llvm/ADT include/llvm/Analysis include/llvm/BinaryFormat include/llvm/CodeGen include/llvm/CodeG... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/llvm/dist: cmake/modules docs docs/CommandGuide include/llvm include/llvm/ADT include/llvm/Analysis include/llvm/BinaryFormat include/llvm/CodeGen include/llvm/CodeGen/GlobalISel include/llv... X-SVN-Commit-Revision: 337137 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:32:53 -0000 Author: dim Date: Thu Aug 2 17:32:43 2018 New Revision: 337137 URL: https://svnweb.freebsd.org/changeset/base/337137 Log: Vendor import of llvm trunk r338536: https://llvm.org/svn/llvm-project/llvm/trunk@338536 Added: vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDebugAddr.h (contents, props changed) vendor/llvm/dist/lib/DebugInfo/DWARF/DWARFDebugAddr.cpp (contents, props changed) vendor/llvm/dist/lib/Transforms/Vectorize/VPlanDominatorTree.h (contents, props changed) vendor/llvm/dist/lib/Transforms/Vectorize/VPlanLoopInfo.h (contents, props changed) vendor/llvm/dist/test/Analysis/BasicAA/phi-values-usage.ll vendor/llvm/dist/test/CodeGen/AArch64/GlobalISel/irtranslator-block-order.ll vendor/llvm/dist/test/CodeGen/AArch64/GlobalISel/legalize-blockaddress.mir vendor/llvm/dist/test/CodeGen/AArch64/GlobalISel/select-blockaddress.mir vendor/llvm/dist/test/CodeGen/AArch64/machine-outliner-default.mir vendor/llvm/dist/test/CodeGen/AArch64/machine-outliner-regsave.mir vendor/llvm/dist/test/CodeGen/AMDGPU/fmaxnum.r600.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fminnum.r600.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.image.sample.ltolz.ll vendor/llvm/dist/test/CodeGen/Hexagon/bit-cmp0.mir vendor/llvm/dist/test/CodeGen/Mips/GlobalISel/instruction-select/gloal_address.mir vendor/llvm/dist/test/CodeGen/Mips/GlobalISel/irtranslator/global_address.ll vendor/llvm/dist/test/CodeGen/Mips/GlobalISel/legalizer/global_address.mir vendor/llvm/dist/test/CodeGen/Mips/GlobalISel/llvm-ir/global_address.ll vendor/llvm/dist/test/CodeGen/Mips/GlobalISel/regbankselect/global_address.mir vendor/llvm/dist/test/CodeGen/X86/pmaddubsw.ll vendor/llvm/dist/test/CodeGen/X86/win_coreclr_chkstk_liveins.mir vendor/llvm/dist/test/DebugInfo/PDB/pdb-invalid-type.test vendor/llvm/dist/test/DebugInfo/PDB/using-namespace.test vendor/llvm/dist/test/DebugInfo/RISCV/ vendor/llvm/dist/test/DebugInfo/RISCV/lit.local.cfg vendor/llvm/dist/test/DebugInfo/RISCV/relax-debug-line.ll vendor/llvm/dist/test/DebugInfo/X86/debug_addr.ll vendor/llvm/dist/test/Demangle/ms-cxx11.test vendor/llvm/dist/test/Demangle/ms-nested-scopes.test vendor/llvm/dist/test/Demangle/ms-return-qualifiers.test vendor/llvm/dist/test/Demangle/ms-template-callback.test vendor/llvm/dist/test/MC/AArch64/SVE/brka-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brka.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkas-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkas.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkb-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkb.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkbs-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkbs.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkn-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkn.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkns-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkns.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpa-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpa.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpas-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpas.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpb-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpb.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpbs-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/brkpbs.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/ctermeq-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/ctermeq.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/ctermne-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/ctermne.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/movprfx-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/movprfx.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/pfalse-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/pfalse.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/pfirst-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/pfirst.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/pnext-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/pnext.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/ptest-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/ptest.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/rev-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/sel-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/uqdecp-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilele-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilele.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilelo-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilelo.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilels-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilels.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilelt-diagnostics.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/SVE/whilelt.s (contents, props changed) vendor/llvm/dist/test/MC/AArch64/inst-directive-other.s (contents, props changed) vendor/llvm/dist/test/MC/ARM/inst-directive-other.s (contents, props changed) vendor/llvm/dist/test/MC/ARM/inst-thumb-suffixes-auto.s (contents, props changed) vendor/llvm/dist/test/Transforms/GlobalOpt/globalsra-multigep.ll vendor/llvm/dist/test/Transforms/InstCombine/pow-cbrt.ll vendor/llvm/dist/test/Transforms/InstCombine/select-binop-icmp.ll vendor/llvm/dist/test/Transforms/InstSimplify/select-and-cmp.ll vendor/llvm/dist/test/Transforms/InstSimplify/select-or-cmp.ll vendor/llvm/dist/test/Transforms/SLPVectorizer/AArch64/PR38339.ll vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_64bit_address.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_absent.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_address_size_mismatch.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_address_size_not_multiple.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_dwarf4.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_dwarf64.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_empty.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_invalid_addr_size.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_segment_selector.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_small_length_field.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_too_small_for_length_field.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_too_small_for_section.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_unsupported_version.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-dwarfdump/X86/debug_addr_version_mismatch.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/Atom/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/Broadwell/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/BtVer2/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/Generic/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/Haswell/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/SLM/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/SandyBridge/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/SkylakeClient/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/SkylakeServer/resources-x86_32.s (contents, props changed) vendor/llvm/dist/test/tools/llvm-mca/X86/Znver1/resources-x86_32.s (contents, props changed) vendor/llvm/dist/unittests/Transforms/Vectorize/VPlanDominatorTreeTest.cpp (contents, props changed) vendor/llvm/dist/unittests/Transforms/Vectorize/VPlanLoopInfoTest.cpp (contents, props changed) Deleted: vendor/llvm/dist/tools/llvm-mca/README.txt Modified: vendor/llvm/dist/cmake/modules/AddLLVM.cmake vendor/llvm/dist/docs/CommandGuide/llvm-mca.rst vendor/llvm/dist/docs/GettingStarted.rst vendor/llvm/dist/docs/LangRef.rst vendor/llvm/dist/docs/SourceLevelDebugging.rst vendor/llvm/dist/include/llvm/ADT/DenseSet.h vendor/llvm/dist/include/llvm/Analysis/BasicAliasAnalysis.h vendor/llvm/dist/include/llvm/Analysis/LoopAccessAnalysis.h vendor/llvm/dist/include/llvm/Analysis/MemoryDependenceAnalysis.h vendor/llvm/dist/include/llvm/Analysis/MustExecute.h vendor/llvm/dist/include/llvm/Analysis/TargetTransformInfoImpl.h vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h vendor/llvm/dist/include/llvm/BinaryFormat/Dwarf.def vendor/llvm/dist/include/llvm/BinaryFormat/ELF.h vendor/llvm/dist/include/llvm/CodeGen/GCStrategy.h vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/LegalizerInfo.h vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h vendor/llvm/dist/include/llvm/CodeGen/MachORelocation.h vendor/llvm/dist/include/llvm/CodeGen/MachineModuleInfo.h vendor/llvm/dist/include/llvm/CodeGen/MachineOutliner.h vendor/llvm/dist/include/llvm/CodeGen/ScheduleDAG.h vendor/llvm/dist/include/llvm/CodeGen/StackMaps.h vendor/llvm/dist/include/llvm/CodeGen/TargetLowering.h vendor/llvm/dist/include/llvm/CodeGen/TargetPassConfig.h vendor/llvm/dist/include/llvm/CodeGen/TargetRegisterInfo.h vendor/llvm/dist/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def vendor/llvm/dist/include/llvm/DebugInfo/CodeView/SymbolRecord.h vendor/llvm/dist/include/llvm/DebugInfo/DIContext.h vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFContext.h vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDataExtractor.h vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDie.h vendor/llvm/dist/include/llvm/ExecutionEngine/Orc/RPCSerialization.h vendor/llvm/dist/include/llvm/IR/Attributes.td vendor/llvm/dist/include/llvm/IR/Instruction.h vendor/llvm/dist/include/llvm/IR/Instructions.h vendor/llvm/dist/include/llvm/IR/Intrinsics.td vendor/llvm/dist/include/llvm/IR/IntrinsicsAMDGPU.td vendor/llvm/dist/include/llvm/IR/IntrinsicsARM.td vendor/llvm/dist/include/llvm/IR/IntrinsicsPowerPC.td vendor/llvm/dist/include/llvm/IR/LegacyPassManagers.h vendor/llvm/dist/include/llvm/IR/Statepoint.h vendor/llvm/dist/include/llvm/IR/User.h vendor/llvm/dist/include/llvm/LinkAllIR.h vendor/llvm/dist/include/llvm/MC/MCDwarf.h vendor/llvm/dist/include/llvm/MC/MCFragment.h vendor/llvm/dist/include/llvm/MC/MCInstrAnalysis.h vendor/llvm/dist/include/llvm/MC/MCParser/AsmCond.h vendor/llvm/dist/include/llvm/MC/MCStreamer.h vendor/llvm/dist/include/llvm/Object/MachO.h vendor/llvm/dist/include/llvm/PassAnalysisSupport.h vendor/llvm/dist/include/llvm/PassRegistry.h vendor/llvm/dist/include/llvm/ProfileData/Coverage/CoverageMapping.h vendor/llvm/dist/include/llvm/Support/ARMBuildAttributes.h vendor/llvm/dist/include/llvm/Support/DataExtractor.h vendor/llvm/dist/include/llvm/Support/GenericDomTree.h vendor/llvm/dist/include/llvm/Support/MemoryBuffer.h vendor/llvm/dist/include/llvm/Support/SmallVectorMemoryBuffer.h vendor/llvm/dist/include/llvm/Support/TargetOpcodes.def vendor/llvm/dist/include/llvm/Support/xxhash.h vendor/llvm/dist/include/llvm/Target/GenericOpcodes.td vendor/llvm/dist/include/llvm/Target/TargetCallingConv.td vendor/llvm/dist/include/llvm/Target/TargetInstrPredicate.td vendor/llvm/dist/include/llvm/Transforms/Scalar/SpeculativeExecution.h vendor/llvm/dist/include/llvm/Transforms/Utils/CodeExtractor.h vendor/llvm/dist/include/llvm/Transforms/Utils/FunctionComparator.h vendor/llvm/dist/include/llvm/Transforms/Utils/SymbolRewriter.h vendor/llvm/dist/lib/Analysis/AliasSetTracker.cpp vendor/llvm/dist/lib/Analysis/BasicAliasAnalysis.cpp vendor/llvm/dist/lib/Analysis/CFGPrinter.cpp vendor/llvm/dist/lib/Analysis/CallGraph.cpp vendor/llvm/dist/lib/Analysis/CallGraphSCCPass.cpp vendor/llvm/dist/lib/Analysis/DemandedBits.cpp vendor/llvm/dist/lib/Analysis/GlobalsModRef.cpp vendor/llvm/dist/lib/Analysis/InstructionSimplify.cpp vendor/llvm/dist/lib/Analysis/LazyValueInfo.cpp vendor/llvm/dist/lib/Analysis/LoopAccessAnalysis.cpp vendor/llvm/dist/lib/Analysis/MemDepPrinter.cpp vendor/llvm/dist/lib/Analysis/MemoryDependenceAnalysis.cpp vendor/llvm/dist/lib/Analysis/MustExecute.cpp vendor/llvm/dist/lib/Analysis/ScalarEvolution.cpp vendor/llvm/dist/lib/Analysis/TargetTransformInfo.cpp vendor/llvm/dist/lib/Analysis/ValueTracking.cpp vendor/llvm/dist/lib/AsmParser/LLParser.cpp vendor/llvm/dist/lib/Bitcode/Writer/BitcodeWriter.cpp vendor/llvm/dist/lib/CodeGen/AntiDepBreaker.h vendor/llvm/dist/lib/CodeGen/AsmPrinter/AddressPool.cpp vendor/llvm/dist/lib/CodeGen/AsmPrinter/AddressPool.h vendor/llvm/dist/lib/CodeGen/AsmPrinter/DwarfDebug.cpp vendor/llvm/dist/lib/CodeGen/AsmPrinter/DwarfDebug.h vendor/llvm/dist/lib/CodeGen/AsmPrinter/DwarfExpression.h vendor/llvm/dist/lib/CodeGen/AsmPrinter/DwarfFile.cpp vendor/llvm/dist/lib/CodeGen/AsmPrinter/DwarfUnit.cpp vendor/llvm/dist/lib/CodeGen/AtomicExpandPass.cpp vendor/llvm/dist/lib/CodeGen/BuiltinGCs.cpp vendor/llvm/dist/lib/CodeGen/CriticalAntiDepBreaker.cpp vendor/llvm/dist/lib/CodeGen/GCMetadata.cpp vendor/llvm/dist/lib/CodeGen/GlobalISel/IRTranslator.cpp vendor/llvm/dist/lib/CodeGen/GlobalISel/MachineIRBuilder.cpp vendor/llvm/dist/lib/CodeGen/GlobalMerge.cpp vendor/llvm/dist/lib/CodeGen/IntrinsicLowering.cpp vendor/llvm/dist/lib/CodeGen/LiveDebugValues.cpp vendor/llvm/dist/lib/CodeGen/MachineModuleInfo.cpp vendor/llvm/dist/lib/CodeGen/MachineOutliner.cpp vendor/llvm/dist/lib/CodeGen/MachineRegisterInfo.cpp vendor/llvm/dist/lib/CodeGen/MachineSSAUpdater.cpp vendor/llvm/dist/lib/CodeGen/MachineSink.cpp vendor/llvm/dist/lib/CodeGen/MachineTraceMetrics.cpp vendor/llvm/dist/lib/CodeGen/MachineVerifier.cpp vendor/llvm/dist/lib/CodeGen/RegisterScavenging.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/DAGCombiner.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/FastISel.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/FunctionLoweringInfo.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/LegalizeFloatTypes.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/LegalizeTypes.h vendor/llvm/dist/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h vendor/llvm/dist/lib/CodeGen/SelectionDAG/StatepointLowering.cpp vendor/llvm/dist/lib/CodeGen/SelectionDAG/TargetLowering.cpp vendor/llvm/dist/lib/CodeGen/ShadowStackGCLowering.cpp vendor/llvm/dist/lib/CodeGen/SplitKit.h vendor/llvm/dist/lib/CodeGen/TargetLoweringBase.cpp vendor/llvm/dist/lib/CodeGen/TargetLoweringObjectFileImpl.cpp vendor/llvm/dist/lib/CodeGen/TargetPassConfig.cpp vendor/llvm/dist/lib/CodeGen/WinEHPrepare.cpp vendor/llvm/dist/lib/DebugInfo/CodeView/RecordName.cpp vendor/llvm/dist/lib/DebugInfo/CodeView/SymbolDumper.cpp vendor/llvm/dist/lib/DebugInfo/CodeView/SymbolRecordMapping.cpp vendor/llvm/dist/lib/DebugInfo/CodeView/TypeIndexDiscovery.cpp vendor/llvm/dist/lib/DebugInfo/CodeView/TypeStreamMerger.cpp vendor/llvm/dist/lib/DebugInfo/DWARF/CMakeLists.txt vendor/llvm/dist/lib/DebugInfo/DWARF/DWARFAbbreviationDeclaration.cpp vendor/llvm/dist/lib/DebugInfo/DWARF/DWARFContext.cpp vendor/llvm/dist/lib/DebugInfo/Symbolize/SymbolizableObjectFile.cpp vendor/llvm/dist/lib/Demangle/ItaniumDemangle.cpp vendor/llvm/dist/lib/Demangle/MicrosoftDemangle.cpp vendor/llvm/dist/lib/Demangle/StringView.h vendor/llvm/dist/lib/ExecutionEngine/ExecutionEngineBindings.cpp vendor/llvm/dist/lib/ExecutionEngine/IntelJITEvents/ittnotify_config.h vendor/llvm/dist/lib/ExecutionEngine/IntelJITEvents/jitprofiling.h vendor/llvm/dist/lib/ExecutionEngine/Interpreter/Execution.cpp vendor/llvm/dist/lib/ExecutionEngine/Interpreter/Interpreter.h vendor/llvm/dist/lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp vendor/llvm/dist/lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp vendor/llvm/dist/lib/FuzzMutate/FuzzerCLI.cpp vendor/llvm/dist/lib/IR/Attributes.cpp vendor/llvm/dist/lib/IR/AutoUpgrade.cpp vendor/llvm/dist/lib/IR/Function.cpp vendor/llvm/dist/lib/IR/InlineAsm.cpp vendor/llvm/dist/lib/IR/Instructions.cpp vendor/llvm/dist/lib/IR/LLVMContextImpl.h vendor/llvm/dist/lib/IR/SymbolTableListTraitsImpl.h vendor/llvm/dist/lib/IR/ValueSymbolTable.cpp vendor/llvm/dist/lib/LTO/ThinLTOCodeGenerator.cpp vendor/llvm/dist/lib/MC/MCAsmStreamer.cpp vendor/llvm/dist/lib/MC/MCAssembler.cpp vendor/llvm/dist/lib/MC/MCDisassembler/Disassembler.cpp vendor/llvm/dist/lib/MC/MCDisassembler/Disassembler.h vendor/llvm/dist/lib/MC/MCDwarf.cpp vendor/llvm/dist/lib/MC/MCInstrAnalysis.cpp vendor/llvm/dist/lib/MC/MCObjectFileInfo.cpp vendor/llvm/dist/lib/MC/MCParser/ELFAsmParser.cpp vendor/llvm/dist/lib/MC/MCStreamer.cpp vendor/llvm/dist/lib/MC/MachObjectWriter.cpp vendor/llvm/dist/lib/Object/COFFObjectFile.cpp vendor/llvm/dist/lib/ObjectYAML/CodeViewYAMLSymbols.cpp vendor/llvm/dist/lib/Support/APFloat.cpp vendor/llvm/dist/lib/Support/ConvertUTF.cpp vendor/llvm/dist/lib/Support/CrashRecoveryContext.cpp vendor/llvm/dist/lib/Support/DAGDeltaAlgorithm.cpp vendor/llvm/dist/lib/Support/Errno.cpp vendor/llvm/dist/lib/Support/FoldingSet.cpp vendor/llvm/dist/lib/Support/FormattedStream.cpp vendor/llvm/dist/lib/Support/ManagedStatic.cpp vendor/llvm/dist/lib/Support/MemoryBuffer.cpp vendor/llvm/dist/lib/Support/Path.cpp vendor/llvm/dist/lib/Support/PrettyStackTrace.cpp vendor/llvm/dist/lib/Support/SourceMgr.cpp vendor/llvm/dist/lib/Support/StringPool.cpp vendor/llvm/dist/lib/Support/StringRef.cpp vendor/llvm/dist/lib/Support/TargetRegistry.cpp vendor/llvm/dist/lib/Support/Windows/Path.inc vendor/llvm/dist/lib/Support/YAMLParser.cpp vendor/llvm/dist/lib/Support/regex_impl.h vendor/llvm/dist/lib/Support/xxhash.cpp vendor/llvm/dist/lib/TableGen/StringMatcher.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64FastISel.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64ISelLowering.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64ISelLowering.h vendor/llvm/dist/lib/Target/AArch64/AArch64InstrFormats.td vendor/llvm/dist/lib/Target/AArch64/AArch64InstrInfo.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64InstrInfo.h vendor/llvm/dist/lib/Target/AArch64/AArch64InstructionSelector.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64LegalizerInfo.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64MachineFunctionInfo.h vendor/llvm/dist/lib/Target/AArch64/AArch64RegisterInfo.td vendor/llvm/dist/lib/Target/AArch64/AArch64SVEInstrInfo.td vendor/llvm/dist/lib/Target/AArch64/AArch64TargetMachine.cpp vendor/llvm/dist/lib/Target/AArch64/AArch64TargetTransformInfo.cpp vendor/llvm/dist/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp vendor/llvm/dist/lib/Target/AArch64/MCTargetDesc/AArch64TargetStreamer.cpp vendor/llvm/dist/lib/Target/AArch64/SVEInstrFormats.td vendor/llvm/dist/lib/Target/AMDGPU/AMDGPUISelLowering.cpp vendor/llvm/dist/lib/Target/AMDGPU/AMDGPUInstrInfo.td vendor/llvm/dist/lib/Target/AMDGPU/AMDGPUInstructions.td vendor/llvm/dist/lib/Target/AMDGPU/AMDGPULowerKernelArguments.cpp vendor/llvm/dist/lib/Target/AMDGPU/MIMGInstructions.td vendor/llvm/dist/lib/Target/AMDGPU/SIISelLowering.cpp vendor/llvm/dist/lib/Target/AMDGPU/SIISelLowering.h vendor/llvm/dist/lib/Target/AMDGPU/SIInsertSkips.cpp vendor/llvm/dist/lib/Target/AMDGPU/SIInstrInfo.cpp vendor/llvm/dist/lib/Target/AMDGPU/SIInstrInfo.h vendor/llvm/dist/lib/Target/AMDGPU/SIInstructions.td vendor/llvm/dist/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp vendor/llvm/dist/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h vendor/llvm/dist/lib/Target/AMDGPU/VOP3PInstructions.td vendor/llvm/dist/lib/Target/ARM/ARMAsmPrinter.cpp vendor/llvm/dist/lib/Target/ARM/ARMBaseRegisterInfo.cpp vendor/llvm/dist/lib/Target/ARM/ARMCallingConv.h vendor/llvm/dist/lib/Target/ARM/ARMConstantIslandPass.cpp vendor/llvm/dist/lib/Target/ARM/ARMConstantPoolValue.h vendor/llvm/dist/lib/Target/ARM/ARMFastISel.cpp vendor/llvm/dist/lib/Target/ARM/ARMFrameLowering.cpp vendor/llvm/dist/lib/Target/ARM/ARMISelDAGToDAG.cpp vendor/llvm/dist/lib/Target/ARM/ARMISelLowering.cpp vendor/llvm/dist/lib/Target/ARM/ARMLoadStoreOptimizer.cpp vendor/llvm/dist/lib/Target/ARM/ARMMachineFunctionInfo.h vendor/llvm/dist/lib/Target/ARM/ARMSelectionDAGInfo.cpp vendor/llvm/dist/lib/Target/ARM/ARMTargetTransformInfo.cpp vendor/llvm/dist/lib/Target/ARM/ARMTargetTransformInfo.h vendor/llvm/dist/lib/Target/ARM/AsmParser/ARMAsmParser.cpp vendor/llvm/dist/lib/Target/ARM/Disassembler/ARMDisassembler.cpp vendor/llvm/dist/lib/Target/ARM/InstPrinter/ARMInstPrinter.cpp vendor/llvm/dist/lib/Target/ARM/MCTargetDesc/ARMELFObjectWriter.cpp vendor/llvm/dist/lib/Target/ARM/MCTargetDesc/ARMMCCodeEmitter.cpp vendor/llvm/dist/lib/Target/ARM/MCTargetDesc/ARMTargetStreamer.cpp vendor/llvm/dist/lib/Target/ARM/MLxExpansionPass.cpp vendor/llvm/dist/lib/Target/ARM/Thumb1FrameLowering.cpp vendor/llvm/dist/lib/Target/AVR/AVRISelLowering.cpp vendor/llvm/dist/lib/Target/Hexagon/HexagonBitSimplify.cpp vendor/llvm/dist/lib/Target/Hexagon/HexagonBitTracker.cpp vendor/llvm/dist/lib/Target/Hexagon/HexagonBitTracker.h vendor/llvm/dist/lib/Target/MSP430/MCTargetDesc/MSP430MCAsmInfo.h vendor/llvm/dist/lib/Target/Mips/AsmParser/MipsAsmParser.cpp vendor/llvm/dist/lib/Target/Mips/MCTargetDesc/MipsFixupKinds.h vendor/llvm/dist/lib/Target/Mips/MipsAsmPrinter.cpp vendor/llvm/dist/lib/Target/Mips/MipsCallLowering.cpp vendor/llvm/dist/lib/Target/Mips/MipsConstantIslandPass.cpp vendor/llvm/dist/lib/Target/Mips/MipsFastISel.cpp vendor/llvm/dist/lib/Target/Mips/MipsISelLowering.cpp vendor/llvm/dist/lib/Target/Mips/MipsISelLowering.h vendor/llvm/dist/lib/Target/Mips/MipsInstructionSelector.cpp vendor/llvm/dist/lib/Target/Mips/MipsLegalizerInfo.cpp vendor/llvm/dist/lib/Target/Mips/MipsRegisterBankInfo.cpp vendor/llvm/dist/lib/Target/Mips/MipsSubtarget.h vendor/llvm/dist/lib/Target/NVPTX/NVPTXAsmPrinter.h vendor/llvm/dist/lib/Target/NVPTX/NVPTXImageOptimizer.cpp vendor/llvm/dist/lib/Target/NVPTX/NVPTXMachineFunctionInfo.h vendor/llvm/dist/lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp vendor/llvm/dist/lib/Target/PowerPC/InstPrinter/PPCInstPrinter.h vendor/llvm/dist/lib/Target/PowerPC/MCTargetDesc/PPCMCAsmInfo.cpp vendor/llvm/dist/lib/Target/PowerPC/MCTargetDesc/PPCMCCodeEmitter.cpp vendor/llvm/dist/lib/Target/PowerPC/MCTargetDesc/PPCPredicates.h vendor/llvm/dist/lib/Target/PowerPC/PPC.h vendor/llvm/dist/lib/Target/PowerPC/PPCBranchSelector.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCEarlyReturn.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCFastISel.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCFrameLowering.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCHazardRecognizers.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCISelLowering.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCISelLowering.h vendor/llvm/dist/lib/Target/PowerPC/PPCInstrInfo.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCLoopPreIncPrep.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCMCInstLower.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCMIPeephole.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCMachineFunctionInfo.h vendor/llvm/dist/lib/Target/PowerPC/PPCRegisterInfo.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCTargetTransformInfo.cpp vendor/llvm/dist/lib/Target/PowerPC/PPCVSXSwapRemoval.cpp vendor/llvm/dist/lib/Target/Sparc/AsmParser/SparcAsmParser.cpp vendor/llvm/dist/lib/Target/Sparc/Disassembler/SparcDisassembler.cpp vendor/llvm/dist/lib/Target/Sparc/InstPrinter/SparcInstPrinter.cpp vendor/llvm/dist/lib/Target/Sparc/Sparc.h vendor/llvm/dist/lib/Target/Sparc/SparcISelLowering.h vendor/llvm/dist/lib/Target/Sparc/SparcInstrInfo.cpp vendor/llvm/dist/lib/Target/Sparc/SparcTargetMachine.cpp vendor/llvm/dist/lib/Target/SystemZ/SystemZHazardRecognizer.cpp vendor/llvm/dist/lib/Target/SystemZ/SystemZHazardRecognizer.h vendor/llvm/dist/lib/Target/SystemZ/SystemZISelLowering.cpp vendor/llvm/dist/lib/Target/SystemZ/SystemZISelLowering.h vendor/llvm/dist/lib/Target/SystemZ/SystemZInstrInfo.td vendor/llvm/dist/lib/Target/SystemZ/SystemZMachineScheduler.cpp vendor/llvm/dist/lib/Target/SystemZ/SystemZMachineScheduler.h vendor/llvm/dist/lib/Target/SystemZ/SystemZOperands.td vendor/llvm/dist/lib/Target/SystemZ/SystemZOperators.td vendor/llvm/dist/lib/Target/SystemZ/SystemZTargetTransformInfo.cpp vendor/llvm/dist/lib/Target/Target.cpp vendor/llvm/dist/lib/Target/TargetLoweringObjectFile.cpp vendor/llvm/dist/lib/Target/X86/AsmParser/X86AsmParser.cpp vendor/llvm/dist/lib/Target/X86/InstPrinter/X86ATTInstPrinter.cpp vendor/llvm/dist/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp vendor/llvm/dist/lib/Target/X86/X86CallingConv.h vendor/llvm/dist/lib/Target/X86/X86CmovConversion.cpp vendor/llvm/dist/lib/Target/X86/X86FastISel.cpp vendor/llvm/dist/lib/Target/X86/X86FixupLEAs.cpp vendor/llvm/dist/lib/Target/X86/X86FlagsCopyLowering.cpp vendor/llvm/dist/lib/Target/X86/X86FloatingPoint.cpp vendor/llvm/dist/lib/Target/X86/X86FrameLowering.cpp vendor/llvm/dist/lib/Target/X86/X86ISelLowering.cpp vendor/llvm/dist/lib/Target/X86/X86ISelLowering.h vendor/llvm/dist/lib/Target/X86/X86InstrFoldTables.cpp vendor/llvm/dist/lib/Target/X86/X86InstrInfo.cpp vendor/llvm/dist/lib/Target/X86/X86InstrInfo.td vendor/llvm/dist/lib/Target/X86/X86InstrShiftRotate.td vendor/llvm/dist/lib/Target/X86/X86SchedBroadwell.td vendor/llvm/dist/lib/Target/X86/X86SchedHaswell.td vendor/llvm/dist/lib/Target/X86/X86SchedSandyBridge.td vendor/llvm/dist/lib/Target/X86/X86SchedSkylakeClient.td vendor/llvm/dist/lib/Target/X86/X86SchedSkylakeServer.td vendor/llvm/dist/lib/Target/X86/X86Schedule.td vendor/llvm/dist/lib/Target/X86/X86ScheduleAtom.td vendor/llvm/dist/lib/Target/X86/X86ScheduleBtVer2.td vendor/llvm/dist/lib/Target/X86/X86ScheduleSLM.td vendor/llvm/dist/lib/Target/X86/X86ScheduleZnver1.td vendor/llvm/dist/lib/Target/X86/X86Subtarget.h vendor/llvm/dist/lib/Target/X86/X86TargetTransformInfo.cpp vendor/llvm/dist/lib/Target/XCore/XCoreAsmPrinter.cpp vendor/llvm/dist/lib/Target/XCore/XCoreInstrInfo.cpp vendor/llvm/dist/lib/Target/XCore/XCoreMachineFunctionInfo.h vendor/llvm/dist/lib/Target/XCore/XCoreRegisterInfo.cpp vendor/llvm/dist/lib/Target/XCore/XCoreRegisterInfo.h vendor/llvm/dist/lib/Target/XCore/XCoreSubtarget.h vendor/llvm/dist/lib/Transforms/IPO/DeadArgumentElimination.cpp vendor/llvm/dist/lib/Transforms/IPO/FunctionAttrs.cpp vendor/llvm/dist/lib/Transforms/IPO/GlobalOpt.cpp vendor/llvm/dist/lib/Transforms/IPO/IPConstantPropagation.cpp vendor/llvm/dist/lib/Transforms/IPO/MergeFunctions.cpp vendor/llvm/dist/lib/Transforms/IPO/PruneEH.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstCombineAddSub.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstCombineCasts.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstCombineSelect.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstCombineShifts.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstCombineVectorOps.cpp vendor/llvm/dist/lib/Transforms/InstCombine/InstructionCombining.cpp vendor/llvm/dist/lib/Transforms/Instrumentation/AddressSanitizer.cpp vendor/llvm/dist/lib/Transforms/Instrumentation/GCOVProfiling.cpp vendor/llvm/dist/lib/Transforms/Instrumentation/InstrProfiling.cpp vendor/llvm/dist/lib/Transforms/Scalar/AlignmentFromAssumptions.cpp vendor/llvm/dist/lib/Transforms/Scalar/ConstantHoisting.cpp vendor/llvm/dist/lib/Transforms/Scalar/CorrelatedValuePropagation.cpp vendor/llvm/dist/lib/Transforms/Scalar/DeadStoreElimination.cpp vendor/llvm/dist/lib/Transforms/Scalar/EarlyCSE.cpp vendor/llvm/dist/lib/Transforms/Scalar/GVNSink.cpp vendor/llvm/dist/lib/Transforms/Scalar/GuardWidening.cpp vendor/llvm/dist/lib/Transforms/Scalar/InductiveRangeCheckElimination.cpp vendor/llvm/dist/lib/Transforms/Scalar/LICM.cpp vendor/llvm/dist/lib/Transforms/Scalar/LoopIdiomRecognize.cpp vendor/llvm/dist/lib/Transforms/Scalar/LoopPredication.cpp vendor/llvm/dist/lib/Transforms/Scalar/LoopUnrollPass.cpp vendor/llvm/dist/lib/Transforms/Scalar/LoopUnswitch.cpp vendor/llvm/dist/lib/Transforms/Scalar/NewGVN.cpp vendor/llvm/dist/lib/Transforms/Scalar/Reassociate.cpp vendor/llvm/dist/lib/Transforms/Scalar/RewriteStatepointsForGC.cpp vendor/llvm/dist/lib/Transforms/Scalar/SROA.cpp vendor/llvm/dist/lib/Transforms/Scalar/SimpleLoopUnswitch.cpp vendor/llvm/dist/lib/Transforms/Utils/BuildLibCalls.cpp vendor/llvm/dist/lib/Transforms/Utils/CallPromotionUtils.cpp vendor/llvm/dist/lib/Transforms/Utils/CloneFunction.cpp vendor/llvm/dist/lib/Transforms/Utils/CloneModule.cpp vendor/llvm/dist/lib/Transforms/Utils/CodeExtractor.cpp vendor/llvm/dist/lib/Transforms/Utils/InlineFunction.cpp vendor/llvm/dist/lib/Transforms/Utils/IntegerDivision.cpp vendor/llvm/dist/lib/Transforms/Utils/LCSSA.cpp vendor/llvm/dist/lib/Transforms/Utils/LoopUnrollPeel.cpp vendor/llvm/dist/lib/Transforms/Utils/MetaRenamer.cpp vendor/llvm/dist/lib/Transforms/Utils/SSAUpdater.cpp vendor/llvm/dist/lib/Transforms/Utils/SimplifyIndVar.cpp vendor/llvm/dist/lib/Transforms/Utils/SimplifyLibCalls.cpp vendor/llvm/dist/lib/Transforms/Utils/SymbolRewriter.cpp vendor/llvm/dist/lib/Transforms/Utils/UnifyFunctionExitNodes.cpp vendor/llvm/dist/lib/Transforms/Vectorize/LoopVectorize.cpp vendor/llvm/dist/lib/Transforms/Vectorize/SLPVectorizer.cpp vendor/llvm/dist/lib/Transforms/Vectorize/VPlan.cpp vendor/llvm/dist/lib/Transforms/Vectorize/VPlan.h vendor/llvm/dist/lib/Transforms/Vectorize/VPlanHCFGBuilder.cpp vendor/llvm/dist/lib/Transforms/Vectorize/VPlanHCFGBuilder.h vendor/llvm/dist/test/Analysis/BasicAA/invalidation.ll vendor/llvm/dist/test/Analysis/BasicAA/phi-aa.ll vendor/llvm/dist/test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll vendor/llvm/dist/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll vendor/llvm/dist/test/CodeGen/AArch64/O3-pipeline.ll vendor/llvm/dist/test/CodeGen/AArch64/arm64-cse.ll vendor/llvm/dist/test/CodeGen/AArch64/arm64-memset-to-bzero.ll vendor/llvm/dist/test/CodeGen/AArch64/arm64-opt-remarks-lazy-bfi.ll vendor/llvm/dist/test/CodeGen/AArch64/cond-sel.ll vendor/llvm/dist/test/CodeGen/AArch64/machine-outliner-flags.ll vendor/llvm/dist/test/CodeGen/AArch64/machine-outliner.ll vendor/llvm/dist/test/CodeGen/AArch64/machine-outliner.mir vendor/llvm/dist/test/CodeGen/AArch64/max-jump-table.ll vendor/llvm/dist/test/CodeGen/AArch64/rotate-extract.ll vendor/llvm/dist/test/CodeGen/AArch64/signbit-shift.ll vendor/llvm/dist/test/CodeGen/AMDGPU/bfi_int.ll vendor/llvm/dist/test/CodeGen/AMDGPU/call-argument-types.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fcanonicalize-elimination.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fcanonicalize.f16.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fcanonicalize.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fmax3.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fmaxnum.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fmin3.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fminnum.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fmul-2-combine-multi-use.ll vendor/llvm/dist/test/CodeGen/AMDGPU/fneg-combines.ll vendor/llvm/dist/test/CodeGen/AMDGPU/function-args.ll vendor/llvm/dist/test/CodeGen/AMDGPU/function-returns.ll vendor/llvm/dist/test/CodeGen/AMDGPU/kernel-args.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.exp.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.fdot2.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.sdot2.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.sdot4.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.sdot8.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.sendmsg.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.udot2.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.udot4.ll vendor/llvm/dist/test/CodeGen/AMDGPU/llvm.amdgcn.udot8.ll vendor/llvm/dist/test/CodeGen/AMDGPU/lower-kernargs.ll vendor/llvm/dist/test/CodeGen/AMDGPU/mad-mix-lo.ll vendor/llvm/dist/test/CodeGen/AMDGPU/mad-mix.ll vendor/llvm/dist/test/CodeGen/AMDGPU/mul.i16.ll vendor/llvm/dist/test/CodeGen/AMDGPU/r600.extract-lowbits.ll vendor/llvm/dist/test/CodeGen/AMDGPU/skip-if-dead.ll vendor/llvm/dist/test/CodeGen/AMDGPU/store-global.ll vendor/llvm/dist/test/CodeGen/AMDGPU/store-private.ll vendor/llvm/dist/test/CodeGen/AMDGPU/zero_extend.ll vendor/llvm/dist/test/CodeGen/ARM/aggregate-padding.ll vendor/llvm/dist/test/CodeGen/ARM/inline-asm-operand-implicit-cast.ll vendor/llvm/dist/test/CodeGen/ARM/inlineasm-64bit.ll vendor/llvm/dist/test/CodeGen/ARM/machine-cse-cmp.ll vendor/llvm/dist/test/CodeGen/Mips/const-mult.ll vendor/llvm/dist/test/CodeGen/PowerPC/signbit-shift.ll vendor/llvm/dist/test/CodeGen/RISCV/tail-calls.ll vendor/llvm/dist/test/CodeGen/SystemZ/shift-12.ll vendor/llvm/dist/test/CodeGen/SystemZ/vec-cmp-cmp-logic-select.ll vendor/llvm/dist/test/CodeGen/X86/atom-fixup-lea2.ll vendor/llvm/dist/test/CodeGen/X86/combine-sdiv.ll vendor/llvm/dist/test/CodeGen/X86/combine-shl.ll vendor/llvm/dist/test/CodeGen/X86/dagcombine-select.ll vendor/llvm/dist/test/CodeGen/X86/fast-isel-fold-mem.ll vendor/llvm/dist/test/CodeGen/X86/fast-isel-select.ll vendor/llvm/dist/test/CodeGen/X86/fast-isel-sext-zext.ll vendor/llvm/dist/test/CodeGen/X86/flags-copy-lowering.mir vendor/llvm/dist/test/CodeGen/X86/lea-opt.ll vendor/llvm/dist/test/CodeGen/X86/machine-outliner-tailcalls.ll vendor/llvm/dist/test/CodeGen/X86/mul-constant-i16.ll vendor/llvm/dist/test/CodeGen/X86/mul-constant-i32.ll vendor/llvm/dist/test/CodeGen/X86/mul-constant-i64.ll vendor/llvm/dist/test/CodeGen/X86/pku.ll vendor/llvm/dist/test/CodeGen/X86/rem.ll vendor/llvm/dist/test/CodeGen/X86/rotate-extract-vector.ll vendor/llvm/dist/test/CodeGen/X86/rotate-extract.ll vendor/llvm/dist/test/CodeGen/X86/signbit-shift.ll vendor/llvm/dist/test/CodeGen/X86/speculative-load-hardening.ll vendor/llvm/dist/test/CodeGen/X86/vector-idiv-sdiv-128.ll vendor/llvm/dist/test/CodeGen/X86/vector-idiv-sdiv-256.ll vendor/llvm/dist/test/CodeGen/X86/vector-idiv-sdiv-512.ll vendor/llvm/dist/test/CodeGen/X86/vector-idiv-udiv-128.ll vendor/llvm/dist/test/CodeGen/X86/vector-idiv-udiv-256.ll vendor/llvm/dist/test/CodeGen/X86/vector-idiv-udiv-512.ll vendor/llvm/dist/test/CodeGen/X86/vector-shift-lshr-128.ll vendor/llvm/dist/test/CodeGen/X86/vector-shift-lshr-256.ll vendor/llvm/dist/test/CodeGen/X86/win_coreclr_chkstk.ll vendor/llvm/dist/test/DebugInfo/X86/accel-tables-dwarf5.ll vendor/llvm/dist/test/DebugInfo/X86/accel-tables.ll vendor/llvm/dist/test/Demangle/ms-mangle.test vendor/llvm/dist/test/Instrumentation/InstrProfiling/linkage.ll vendor/llvm/dist/test/Instrumentation/InstrProfiling/platform.ll vendor/llvm/dist/test/MC/AArch64/SVE/abs.s vendor/llvm/dist/test/MC/AArch64/SVE/add-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/add.s vendor/llvm/dist/test/MC/AArch64/SVE/adr-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/and-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/and.s vendor/llvm/dist/test/MC/AArch64/SVE/andv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/asr-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/asr.s vendor/llvm/dist/test/MC/AArch64/SVE/asrd.s vendor/llvm/dist/test/MC/AArch64/SVE/asrr.s vendor/llvm/dist/test/MC/AArch64/SVE/bic-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/bic.s vendor/llvm/dist/test/MC/AArch64/SVE/clasta-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/clasta.s vendor/llvm/dist/test/MC/AArch64/SVE/clastb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/clastb.s vendor/llvm/dist/test/MC/AArch64/SVE/cls.s vendor/llvm/dist/test/MC/AArch64/SVE/clz.s vendor/llvm/dist/test/MC/AArch64/SVE/cmpeq-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmpge-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmpgt-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmphi-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmphs-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmple-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmplo-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmpls-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmplt-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cmpne-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cnot.s vendor/llvm/dist/test/MC/AArch64/SVE/cnt.s vendor/llvm/dist/test/MC/AArch64/SVE/compact-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/cpy.s vendor/llvm/dist/test/MC/AArch64/SVE/decp-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/decp.s vendor/llvm/dist/test/MC/AArch64/SVE/dup-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/dupm-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/eon-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/eon.s vendor/llvm/dist/test/MC/AArch64/SVE/eor-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/eor.s vendor/llvm/dist/test/MC/AArch64/SVE/eorv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ext-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ext.s vendor/llvm/dist/test/MC/AArch64/SVE/fabd.s vendor/llvm/dist/test/MC/AArch64/SVE/fabs.s vendor/llvm/dist/test/MC/AArch64/SVE/facge-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/facgt-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/facle-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/faclt-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fadd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fadd.s vendor/llvm/dist/test/MC/AArch64/SVE/fadda-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/faddv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcadd.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmeq-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmge-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmgt-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmla-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmla.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmle-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmlt-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmne-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcmuo-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fcpy.s vendor/llvm/dist/test/MC/AArch64/SVE/fcvt.s vendor/llvm/dist/test/MC/AArch64/SVE/fcvtzs.s vendor/llvm/dist/test/MC/AArch64/SVE/fcvtzu.s vendor/llvm/dist/test/MC/AArch64/SVE/fdiv.s vendor/llvm/dist/test/MC/AArch64/SVE/fdivr.s vendor/llvm/dist/test/MC/AArch64/SVE/fdup-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fexpa-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmad.s vendor/llvm/dist/test/MC/AArch64/SVE/fmax.s vendor/llvm/dist/test/MC/AArch64/SVE/fmaxnm.s vendor/llvm/dist/test/MC/AArch64/SVE/fmaxnmv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmaxv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmin.s vendor/llvm/dist/test/MC/AArch64/SVE/fminnm.s vendor/llvm/dist/test/MC/AArch64/SVE/fminnmv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fminv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmla-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmla.s vendor/llvm/dist/test/MC/AArch64/SVE/fmls-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmls.s vendor/llvm/dist/test/MC/AArch64/SVE/fmov-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmov.s vendor/llvm/dist/test/MC/AArch64/SVE/fmsb.s vendor/llvm/dist/test/MC/AArch64/SVE/fmul-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fmul.s vendor/llvm/dist/test/MC/AArch64/SVE/fmulx.s vendor/llvm/dist/test/MC/AArch64/SVE/fneg.s vendor/llvm/dist/test/MC/AArch64/SVE/fnmad.s vendor/llvm/dist/test/MC/AArch64/SVE/fnmla.s vendor/llvm/dist/test/MC/AArch64/SVE/fnmls.s vendor/llvm/dist/test/MC/AArch64/SVE/fnmsb.s vendor/llvm/dist/test/MC/AArch64/SVE/frecpe-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/frecps-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/frecpx.s vendor/llvm/dist/test/MC/AArch64/SVE/frinta.s vendor/llvm/dist/test/MC/AArch64/SVE/frinti.s vendor/llvm/dist/test/MC/AArch64/SVE/frintm.s vendor/llvm/dist/test/MC/AArch64/SVE/frintn.s vendor/llvm/dist/test/MC/AArch64/SVE/frintp.s vendor/llvm/dist/test/MC/AArch64/SVE/frintx.s vendor/llvm/dist/test/MC/AArch64/SVE/frintz.s vendor/llvm/dist/test/MC/AArch64/SVE/frsqrte-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/frsqrts-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fscale.s vendor/llvm/dist/test/MC/AArch64/SVE/fsqrt.s vendor/llvm/dist/test/MC/AArch64/SVE/fsub-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/fsub.s vendor/llvm/dist/test/MC/AArch64/SVE/fsubr.s vendor/llvm/dist/test/MC/AArch64/SVE/ftmad-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ftmad.s vendor/llvm/dist/test/MC/AArch64/SVE/ftsmul-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ftssel-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/incd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/incd.s vendor/llvm/dist/test/MC/AArch64/SVE/inch-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/inch.s vendor/llvm/dist/test/MC/AArch64/SVE/incp-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/incp.s vendor/llvm/dist/test/MC/AArch64/SVE/incw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/incw.s vendor/llvm/dist/test/MC/AArch64/SVE/index-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/insr-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/insr.s vendor/llvm/dist/test/MC/AArch64/SVE/lasta-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/lastb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rh-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rqb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rqd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rqh-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rqw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rsb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rsh-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rsw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1rw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1sb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1sh-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1sw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld1w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld2b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld2d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld2h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld2w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld3b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld3d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld3h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld3w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld4b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld4d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld4h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ld4w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldff1b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldff1d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldff1h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldff1sb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldff1sh-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldff1sw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldff1w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnf1b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnf1d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnf1h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnf1sb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnf1sh-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnf1sw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnf1w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnt1b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnt1d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnt1h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ldnt1w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/lsl-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/lsl.s vendor/llvm/dist/test/MC/AArch64/SVE/lslr.s vendor/llvm/dist/test/MC/AArch64/SVE/lsr-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/lsr.s vendor/llvm/dist/test/MC/AArch64/SVE/lsrr.s vendor/llvm/dist/test/MC/AArch64/SVE/mad.s vendor/llvm/dist/test/MC/AArch64/SVE/mla.s vendor/llvm/dist/test/MC/AArch64/SVE/mls.s vendor/llvm/dist/test/MC/AArch64/SVE/mov-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/mov.s vendor/llvm/dist/test/MC/AArch64/SVE/msb.s vendor/llvm/dist/test/MC/AArch64/SVE/mul-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/mul.s vendor/llvm/dist/test/MC/AArch64/SVE/neg.s vendor/llvm/dist/test/MC/AArch64/SVE/not.s vendor/llvm/dist/test/MC/AArch64/SVE/orn-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/orn.s vendor/llvm/dist/test/MC/AArch64/SVE/orr-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/orr.s vendor/llvm/dist/test/MC/AArch64/SVE/orv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/prfb-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/prfd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/prfh-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/prfw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/rbit.s vendor/llvm/dist/test/MC/AArch64/SVE/revb.s vendor/llvm/dist/test/MC/AArch64/SVE/revh.s vendor/llvm/dist/test/MC/AArch64/SVE/revw.s vendor/llvm/dist/test/MC/AArch64/SVE/sabd.s vendor/llvm/dist/test/MC/AArch64/SVE/saddv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/scvtf.s vendor/llvm/dist/test/MC/AArch64/SVE/sdiv.s vendor/llvm/dist/test/MC/AArch64/SVE/sdivr.s vendor/llvm/dist/test/MC/AArch64/SVE/sdot-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sdot.s vendor/llvm/dist/test/MC/AArch64/SVE/smax-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/smax.s vendor/llvm/dist/test/MC/AArch64/SVE/smaxv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/smin-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/smin.s vendor/llvm/dist/test/MC/AArch64/SVE/sminv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/smulh.s vendor/llvm/dist/test/MC/AArch64/SVE/splice-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/splice.s vendor/llvm/dist/test/MC/AArch64/SVE/sqadd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqadd.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdecd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdecd.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdech-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdech.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdecp-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdecp.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdecw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqdecw.s vendor/llvm/dist/test/MC/AArch64/SVE/sqincd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqincd.s vendor/llvm/dist/test/MC/AArch64/SVE/sqinch-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqinch.s vendor/llvm/dist/test/MC/AArch64/SVE/sqincp-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqincp.s vendor/llvm/dist/test/MC/AArch64/SVE/sqincw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqincw.s vendor/llvm/dist/test/MC/AArch64/SVE/sqsub-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sqsub.s vendor/llvm/dist/test/MC/AArch64/SVE/st1b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st1d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st1h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st1w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st2b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st2d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st2h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st2w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st3b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st3d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st3h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st3w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st4b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st4d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st4h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/st4w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/stnt1b-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/stnt1d-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/stnt1h-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/stnt1w-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sub-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sub.s vendor/llvm/dist/test/MC/AArch64/SVE/subr-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/subr.s vendor/llvm/dist/test/MC/AArch64/SVE/sunpkhi-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sunpklo-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/sxtb.s vendor/llvm/dist/test/MC/AArch64/SVE/sxth.s vendor/llvm/dist/test/MC/AArch64/SVE/sxtw.s vendor/llvm/dist/test/MC/AArch64/SVE/tbl-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/trn1-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/trn2-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uabd.s vendor/llvm/dist/test/MC/AArch64/SVE/uaddv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/ucvtf.s vendor/llvm/dist/test/MC/AArch64/SVE/udiv.s vendor/llvm/dist/test/MC/AArch64/SVE/udivr.s vendor/llvm/dist/test/MC/AArch64/SVE/udot-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/udot.s vendor/llvm/dist/test/MC/AArch64/SVE/umax-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/umax.s vendor/llvm/dist/test/MC/AArch64/SVE/umaxv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/umin-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/umin.s vendor/llvm/dist/test/MC/AArch64/SVE/uminv-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/umulh.s vendor/llvm/dist/test/MC/AArch64/SVE/uqadd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqadd.s vendor/llvm/dist/test/MC/AArch64/SVE/uqdecd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqdecd.s vendor/llvm/dist/test/MC/AArch64/SVE/uqdech-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqdech.s vendor/llvm/dist/test/MC/AArch64/SVE/uqdecp.s vendor/llvm/dist/test/MC/AArch64/SVE/uqdecw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqdecw.s vendor/llvm/dist/test/MC/AArch64/SVE/uqincd-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqincd.s vendor/llvm/dist/test/MC/AArch64/SVE/uqinch-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqinch.s vendor/llvm/dist/test/MC/AArch64/SVE/uqincp-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqincp.s vendor/llvm/dist/test/MC/AArch64/SVE/uqincw-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqincw.s vendor/llvm/dist/test/MC/AArch64/SVE/uqsub-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uqsub.s vendor/llvm/dist/test/MC/AArch64/SVE/uunpkhi-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uunpklo-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uxtb.s vendor/llvm/dist/test/MC/AArch64/SVE/uxth.s vendor/llvm/dist/test/MC/AArch64/SVE/uxtw.s vendor/llvm/dist/test/MC/AArch64/SVE/uzp1-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/uzp2-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/zip1-diagnostics.s vendor/llvm/dist/test/MC/AArch64/SVE/zip2-diagnostics.s vendor/llvm/dist/test/MC/AArch64/arm64-directive_loh.s vendor/llvm/dist/test/MC/ARM/directive-unsupported.s vendor/llvm/dist/test/MC/ARM/inst-thumb-suffixes.s vendor/llvm/dist/test/MC/WebAssembly/debug-info.ll vendor/llvm/dist/test/Other/new-pm-defaults.ll vendor/llvm/dist/test/Other/new-pm-lto-defaults.ll vendor/llvm/dist/test/Other/new-pm-thinlto-defaults.ll vendor/llvm/dist/test/Other/opt-O2-pipeline.ll vendor/llvm/dist/test/Other/opt-O3-pipeline.ll vendor/llvm/dist/test/Other/opt-Os-pipeline.ll vendor/llvm/dist/test/Transforms/GVN/PRE/pre-after-rle.ll vendor/llvm/dist/test/Transforms/GlobalOpt/globalsra-partial.ll vendor/llvm/dist/test/Transforms/Inline/attributes.ll vendor/llvm/dist/test/Transforms/InstCombine/and-xor-or.ll vendor/llvm/dist/test/Transforms/InstCombine/and2.ll vendor/llvm/dist/test/Transforms/InstCombine/double-float-shrink-1.ll vendor/llvm/dist/test/Transforms/InstCombine/gep-addrspace.ll vendor/llvm/dist/test/Transforms/InstCombine/pow-1.ll vendor/llvm/dist/test/Transforms/InstCombine/pow-sqrt.ll vendor/llvm/dist/test/Transforms/InstCombine/sub-not.ll vendor/llvm/dist/test/Transforms/InstCombine/xor.ll vendor/llvm/dist/test/Transforms/InstSimplify/AndOrXor.ll vendor/llvm/dist/test/Transforms/InstSimplify/call.ll vendor/llvm/dist/test/Transforms/InstSimplify/shift.ll vendor/llvm/dist/test/Transforms/LCSSA/basictest.ll vendor/llvm/dist/test/Transforms/SCCP/preserve-analysis.ll vendor/llvm/dist/test/Transforms/SimplifyCFG/merge-cond-stores.ll vendor/llvm/dist/test/tools/dsymutil/X86/accelerator.test vendor/llvm/dist/test/tools/dsymutil/X86/update-one-CU.test vendor/llvm/dist/test/tools/dsymutil/X86/update.test vendor/llvm/dist/test/tools/llvm-mca/X86/Atom/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/Broadwell/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/BtVer2/dependency-breaking-cmp.s vendor/llvm/dist/test/tools/llvm-mca/X86/BtVer2/dependency-breaking-pcmpeq.s vendor/llvm/dist/test/tools/llvm-mca/X86/BtVer2/dependency-breaking-sbb-2.s vendor/llvm/dist/test/tools/llvm-mca/X86/BtVer2/one-idioms.s vendor/llvm/dist/test/tools/llvm-mca/X86/BtVer2/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/Generic/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/Haswell/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/SLM/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/SandyBridge/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/SkylakeClient/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/SkylakeServer/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-mca/X86/Znver1/resources-x86_64.s vendor/llvm/dist/test/tools/llvm-objcopy/strip-debug.test vendor/llvm/dist/tools/dsymutil/DwarfLinker.cpp vendor/llvm/dist/tools/dsymutil/DwarfLinker.h vendor/llvm/dist/tools/dsymutil/MachOUtils.cpp vendor/llvm/dist/tools/dsymutil/MachOUtils.h vendor/llvm/dist/tools/dsymutil/dsymutil.cpp vendor/llvm/dist/tools/llvm-mca/DispatchStage.cpp vendor/llvm/dist/tools/llvm-mca/DispatchStage.h vendor/llvm/dist/tools/llvm-mca/InstrBuilder.cpp vendor/llvm/dist/tools/llvm-mca/Instruction.h vendor/llvm/dist/tools/llvm-mca/RetireControlUnit.h vendor/llvm/dist/tools/llvm-mca/RetireStage.cpp vendor/llvm/dist/tools/llvm-mca/llvm-mca.cpp vendor/llvm/dist/tools/llvm-objcopy/llvm-objcopy.cpp vendor/llvm/dist/tools/llvm-pdbutil/MinimalSymbolDumper.cpp vendor/llvm/dist/unittests/DebugInfo/CodeView/TypeIndexDiscoveryTest.cpp vendor/llvm/dist/unittests/DebugInfo/DWARF/DWARFDebugInfoTest.cpp vendor/llvm/dist/unittests/ExecutionEngine/Orc/RPCUtilsTest.cpp vendor/llvm/dist/unittests/Transforms/Vectorize/CMakeLists.txt vendor/llvm/dist/unittests/Transforms/Vectorize/VPlanTestBase.h vendor/llvm/dist/utils/LLVMVisualizers/llvm.natvis vendor/llvm/dist/utils/TableGen/CodeGenDAGPatterns.cpp Modified: vendor/llvm/dist/cmake/modules/AddLLVM.cmake ============================================================================== --- vendor/llvm/dist/cmake/modules/AddLLVM.cmake Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/cmake/modules/AddLLVM.cmake Thu Aug 2 17:32:43 2018 (r337137) @@ -867,6 +867,7 @@ if(NOT LLVM_TOOLCHAIN_TOOLS) llvm-ranlib llvm-lib llvm-objdump + llvm-rc ) endif() Modified: vendor/llvm/dist/docs/CommandGuide/llvm-mca.rst ============================================================================== --- vendor/llvm/dist/docs/CommandGuide/llvm-mca.rst Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/docs/CommandGuide/llvm-mca.rst Thu Aug 2 17:32:43 2018 (r337137) @@ -114,8 +114,8 @@ option specifies "``-``", then the output will also be .. option:: -register-file-size= Specify the size of the register file. When specified, this flag limits how - many temporary registers are available for register renaming purposes. A value - of zero for this flag means "unlimited number of temporary registers". + many physical registers are available for register renaming purposes. A value + of zero for this flag means "unlimited number of physical registers". .. option:: -iterations= @@ -207,23 +207,23 @@ EXIT STATUS :program:`llvm-mca` returns 0 on success. Otherwise, an error message is printed to standard error, and the tool returns 1. -HOW MCA WORKS -------------- +HOW LLVM-MCA WORKS +------------------ -MCA takes assembly code as input. The assembly code is parsed into a sequence -of MCInst with the help of the existing LLVM target assembly parsers. The -parsed sequence of MCInst is then analyzed by a ``Pipeline`` module to generate -a performance report. +:program:`llvm-mca` takes assembly code as input. The assembly code is parsed +into a sequence of MCInst with the help of the existing LLVM target assembly +parsers. The parsed sequence of MCInst is then analyzed by a ``Pipeline`` module +to generate a performance report. The Pipeline module simulates the execution of the machine code sequence in a loop of iterations (default is 100). During this process, the pipeline collects a number of execution related statistics. At the end of this process, the pipeline generates and prints a report from the collected statistics. -Here is an example of a performance report generated by MCA for a dot-product -of two packed float vectors of four elements. The analysis is conducted for -target x86, cpu btver2. The following result can be produced via the following -command using the example located at +Here is an example of a performance report generated by the tool for a +dot-product of two packed float vectors of four elements. The analysis is +conducted for target x86, cpu btver2. The following result can be produced via +the following command using the example located at ``test/tools/llvm-mca/X86/BtVer2/dot-product.s``: .. code-block:: bash @@ -287,11 +287,31 @@ for a total of 900 dynamically executed instructions. The report is structured in three main sections. The first section collects a few performance numbers; the goal of this section is to give a very quick overview of the performance throughput. In this example, the two important -performance indicators are the predicted total number of cycles, and the IPC. -IPC is probably the most important throughput indicator. A big delta between -the Dispatch Width and the computed IPC is an indicator of potential -performance issues. +performance indicators are **IPC** and **Block RThroughput** (Block Reciprocal +Throughput). +IPC is computed dividing the total number of simulated instructions by the total +number of cycles. A delta between Dispatch Width and IPC is an indicator of a +performance issue. In the absence of loop-carried data dependencies, the +observed IPC tends to a theoretical maximum which can be computed by dividing +the number of instructions of a single iteration by the *Block RThroughput*. + +IPC is bounded from above by the dispatch width. That is because the dispatch +width limits the maximum size of a dispatch group. IPC is also limited by the +amount of hardware parallelism. The availability of hardware resources affects +the resource pressure distribution, and it limits the number of instructions +that can be executed in parallel every cycle. A delta between Dispatch +Width and the theoretical maximum IPC is an indicator of a performance +bottleneck caused by the lack of hardware resources. In general, the lower the +Block RThroughput, the better. + +In this example, ``Instructions per iteration/Block RThroughput`` is 1.50. Since +there are no loop-carried dependencies, the observed IPC is expected to approach +1.50 when the number of iterations tends to infinity. The delta between the +Dispatch Width (2.00), and the theoretical maximum IPC (1.50) is an indicator of +a performance bottleneck caused by the lack of hardware resources, and the +*Resource pressure view* can help to identify the problematic resource usage. + The second section of the report shows the latency and reciprocal throughput of every instruction in the sequence. That section also reports extra information related to the number of micro opcodes, and opcode properties @@ -316,7 +336,7 @@ pressure should be uniformly distributed between multi Timeline View ^^^^^^^^^^^^^ -MCA's timeline view produces a detailed report of each instruction's state +The timeline view produces a detailed report of each instruction's state transitions through an instruction pipeline. This view is enabled by the command line option ``-timeline``. As instructions transition through the various stages of the pipeline, their states are depicted in the view report. @@ -331,7 +351,7 @@ These states are represented by the following characte Below is the timeline view for a subset of the dot-product example located in ``test/tools/llvm-mca/X86/BtVer2/dot-product.s`` and processed by -MCA using the following command: +:program:`llvm-mca` using the following command: .. code-block:: bash @@ -366,7 +386,7 @@ MCA using the following command: 2. 3 5.7 0.0 0.0 vhaddps %xmm3, %xmm3, %xmm4 The timeline view is interesting because it shows instruction state changes -during execution. It also gives an idea of how MCA processes instructions +during execution. It also gives an idea of how the tool processes instructions executed on the target, and how their timing information might be calculated. The timeline view is structured in two tables. The first table shows @@ -411,12 +431,12 @@ Parallelism). In the dot-product example, there are anti-dependencies introduced by instructions from different iterations. However, those dependencies can be removed at register renaming stage (at the cost of allocating register aliases, -and therefore consuming temporary registers). +and therefore consuming physical registers). Table *Average Wait times* helps diagnose performance issues that are caused by the presence of long latency instructions and potentially long data dependencies -which may limit the ILP. Note that MCA, by default, assumes at least 1cy -between the dispatch event and the issue event. +which may limit the ILP. Note that :program:`llvm-mca`, by default, assumes at +least 1cy between the dispatch event and the issue event. When the performance is limited by data dependencies and/or long latency instructions, the number of cycles spent while in the *ready* state is expected @@ -549,3 +569,177 @@ statistics are displayed by using the command option ` In this example, we can conclude that the IPC is mostly limited by data dependencies, and not by resource pressure. + +Instruction Flow +^^^^^^^^^^^^^^^^ +This section describes the instruction flow through MCA's default out-of-order +pipeline, as well as the functional units involved in the process. + +The default pipeline implements the following sequence of stages used to +process instructions. + +* Dispatch (Instruction is dispatched to the schedulers). +* Issue (Instruction is issued to the processor pipelines). +* Write Back (Instruction is executed, and results are written back). +* Retire (Instruction is retired; writes are architecturally committed). + +The default pipeline only models the out-of-order portion of a processor. +Therefore, the instruction fetch and decode stages are not modeled. Performance +bottlenecks in the frontend are not diagnosed. MCA assumes that instructions +have all been decoded and placed into a queue. Also, MCA does not model branch +prediction. + +Instruction Dispatch +"""""""""""""""""""" +During the dispatch stage, instructions are picked in program order from a +queue of already decoded instructions, and dispatched in groups to the +simulated hardware schedulers. + +The size of a dispatch group depends on the availability of the simulated +hardware resources. The processor dispatch width defaults to the value +of the ``IssueWidth`` in LLVM's scheduling model. + +An instruction can be dispatched if: + +* The size of the dispatch group is smaller than processor's dispatch width. +* There are enough entries in the reorder buffer. +* There are enough physical registers to do register renaming. +* The schedulers are not full. + +Scheduling models can optionally specify which register files are available on +the processor. MCA uses that information to initialize register file +descriptors. Users can limit the number of physical registers that are +globally available for register renaming by using the command option +``-register-file-size``. A value of zero for this option means *unbounded*. +By knowing how many registers are available for renaming, MCA can predict +dispatch stalls caused by the lack of registers. + +The number of reorder buffer entries consumed by an instruction depends on the +number of micro-opcodes specified by the target scheduling model. MCA's +reorder buffer's purpose is to track the progress of instructions that are +"in-flight," and to retire instructions in program order. The number of +entries in the reorder buffer defaults to the `MicroOpBufferSize` provided by +the target scheduling model. + +Instructions that are dispatched to the schedulers consume scheduler buffer +entries. :program:`llvm-mca` queries the scheduling model to determine the set +of buffered resources consumed by an instruction. Buffered resources are +treated like scheduler resources. + +Instruction Issue +""""""""""""""""" +Each processor scheduler implements a buffer of instructions. An instruction +has to wait in the scheduler's buffer until input register operands become +available. Only at that point, does the instruction becomes eligible for +execution and may be issued (potentially out-of-order) for execution. +Instruction latencies are computed by :program:`llvm-mca` with the help of the +scheduling model. + +:program:`llvm-mca`'s scheduler is designed to simulate multiple processor +schedulers. The scheduler is responsible for tracking data dependencies, and +dynamically selecting which processor resources are consumed by instructions. +It delegates the management of processor resource units and resource groups to a +resource manager. The resource manager is responsible for selecting resource +units that are consumed by instructions. For example, if an instruction +consumes 1cy of a resource group, the resource manager selects one of the +available units from the group; by default, the resource manager uses a +round-robin selector to guarantee that resource usage is uniformly distributed +between all units of a group. + +:program:`llvm-mca`'s scheduler implements three instruction queues: + +* WaitQueue: a queue of instructions whose operands are not ready. +* ReadyQueue: a queue of instructions ready to execute. +* IssuedQueue: a queue of instructions executing. + +Depending on the operand availability, instructions that are dispatched to the +scheduler are either placed into the WaitQueue or into the ReadyQueue. + +Every cycle, the scheduler checks if instructions can be moved from the +WaitQueue to the ReadyQueue, and if instructions from the ReadyQueue can be +issued to the underlying pipelines. The algorithm prioritizes older instructions +over younger instructions. + +Write-Back and Retire Stage +""""""""""""""""""""""""""" +Issued instructions are moved from the ReadyQueue to the IssuedQueue. There, +instructions wait until they reach the write-back stage. At that point, they +get removed from the queue and the retire control unit is notified. + +When instructions are executed, the retire control unit flags the +instruction as "ready to retire." + +Instructions are retired in program order. The register file is notified of +the retirement so that it can free the physical registers that were allocated +for the instruction during the register renaming stage. + +Load/Store Unit and Memory Consistency Model +"""""""""""""""""""""""""""""""""""""""""""" +To simulate an out-of-order execution of memory operations, :program:`llvm-mca` +utilizes a simulated load/store unit (LSUnit) to simulate the speculative +execution of loads and stores. + +Each load (or store) consumes an entry in the load (or store) queue. Users can +specify flags ``-lqueue`` and ``-squeue`` to limit the number of entries in the +load and store queues respectively. The queues are unbounded by default. + +The LSUnit implements a relaxed consistency model for memory loads and stores. +The rules are: + +1. A younger load is allowed to pass an older load only if there are no + intervening stores or barriers between the two loads. +2. A younger load is allowed to pass an older store provided that the load does + not alias with the store. +3. A younger store is not allowed to pass an older store. +4. A younger store is not allowed to pass an older load. + +By default, the LSUnit optimistically assumes that loads do not alias +(`-noalias=true`) store operations. Under this assumption, younger loads are +always allowed to pass older stores. Essentially, the LSUnit does not attempt +to run any alias analysis to predict when loads and stores do not alias with +each other. + +Note that, in the case of write-combining memory, rule 3 could be relaxed to +allow reordering of non-aliasing store operations. That being said, at the +moment, there is no way to further relax the memory model (``-noalias`` is the +only option). Essentially, there is no option to specify a different memory +type (e.g., write-back, write-combining, write-through; etc.) and consequently +to weaken, or strengthen, the memory model. + +Other limitations are: + +* The LSUnit does not know when store-to-load forwarding may occur. +* The LSUnit does not know anything about cache hierarchy and memory types. +* The LSUnit does not know how to identify serializing operations and memory + fences. + +The LSUnit does not attempt to predict if a load or store hits or misses the L1 +cache. It only knows if an instruction "MayLoad" and/or "MayStore." For +loads, the scheduling model provides an "optimistic" load-to-use latency (which +usually matches the load-to-use latency for when there is a hit in the L1D). + +:program:`llvm-mca` does not know about serializing operations or memory-barrier +like instructions. The LSUnit conservatively assumes that an instruction which +has both "MayLoad" and unmodeled side effects behaves like a "soft" +load-barrier. That means, it serializes loads without forcing a flush of the +load queue. Similarly, instructions that "MayStore" and have unmodeled side +effects are treated like store barriers. A full memory barrier is a "MayLoad" +and "MayStore" instruction with unmodeled side effects. This is inaccurate, but +it is the best that we can do at the moment with the current information +available in LLVM. + +A load/store barrier consumes one entry of the load/store queue. A load/store +barrier enforces ordering of loads/stores. A younger load cannot pass a load +barrier. Also, a younger store cannot pass a store barrier. A younger load +has to wait for the memory/load barrier to execute. A load/store barrier is +"executed" when it becomes the oldest entry in the load/store queue(s). That +also means, by construction, all of the older loads/stores have been executed. + +In conclusion, the full set of load/store consistency rules are: + +#. A store may not pass a previous store. +#. A store may not pass a previous load (regardless of ``-noalias``). +#. A store has to wait until an older store barrier is fully executed. +#. A load may pass a previous load. +#. A load may not pass a previous store unless ``-noalias`` is set. +#. A load has to wait until an older load barrier is fully executed. Modified: vendor/llvm/dist/docs/GettingStarted.rst ============================================================================== --- vendor/llvm/dist/docs/GettingStarted.rst Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/docs/GettingStarted.rst Thu Aug 2 17:32:43 2018 (r337137) @@ -838,7 +838,7 @@ To configure LLVM, follow these steps: .. code-block:: console - % cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=prefix=/install/path + % cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/install/path [other options] SRC_ROOT Compiling the LLVM Suite Source Code Modified: vendor/llvm/dist/docs/LangRef.rst ============================================================================== --- vendor/llvm/dist/docs/LangRef.rst Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/docs/LangRef.rst Thu Aug 2 17:32:43 2018 (r337137) @@ -4588,9 +4588,12 @@ DIExpression ``DIExpression`` nodes represent expressions that are inspired by the DWARF expression language. They are used in :ref:`debug intrinsics` (such as ``llvm.dbg.declare`` and ``llvm.dbg.value``) to describe how the -referenced LLVM variable relates to the source language variable. +referenced LLVM variable relates to the source language variable. Debug +intrinsics are interpreted left-to-right: start by pushing the value/address +operand of the intrinsic onto a stack, then repeatedly push and evaluate +opcodes from the DIExpression until the final variable description is produced. -The current supported vocabulary is limited: +The current supported opcode vocabulary is limited: - ``DW_OP_deref`` dereferences the top of the expression stack. - ``DW_OP_plus`` pops the last two entries from the expression stack, adds @@ -4610,12 +4613,30 @@ The current supported vocabulary is limited: - ``DW_OP_stack_value`` marks a constant value. DWARF specifies three kinds of simple location descriptions: Register, memory, -and implicit location descriptions. Register and memory location descriptions -describe the *location* of a source variable (in the sense that a debugger might -modify its value), whereas implicit locations describe merely the *value* of a -source variable. DIExpressions also follow this model: A DIExpression that -doesn't have a trailing ``DW_OP_stack_value`` will describe an *address* when -combined with a concrete location. +and implicit location descriptions. Note that a location description is +defined over certain ranges of a program, i.e the location of a variable may +change over the course of the program. Register and memory location +descriptions describe the *concrete location* of a source variable (in the +sense that a debugger might modify its value), whereas *implicit locations* +describe merely the actual *value* of a source variable which might not exist +in registers or in memory (see ``DW_OP_stack_value``). + +A ``llvm.dbg.addr`` or ``llvm.dbg.declare`` intrinsic describes an indirect +value (the address) of a source variable. The first operand of the intrinsic +must be an address of some kind. A DIExpression attached to the intrinsic +refines this address to produce a concrete location for the source variable. + +A ``llvm.dbg.value`` intrinsic describes the direct value of a source variable. +The first operand of the intrinsic may be a direct or indirect value. A +DIExpresion attached to the intrinsic refines the first operand to produce a +direct value. For example, if the first operand is an indirect value, it may be +necessary to insert ``DW_OP_deref`` into the DIExpresion in order to produce a +valid debug intrinsic. + +.. note:: + + A DIExpression is interpreted in the same way regardless of which kind of + debug intrinsic it's attached to. .. code-block:: text Modified: vendor/llvm/dist/docs/SourceLevelDebugging.rst ============================================================================== --- vendor/llvm/dist/docs/SourceLevelDebugging.rst Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/docs/SourceLevelDebugging.rst Thu Aug 2 17:32:43 2018 (r337137) @@ -244,6 +244,11 @@ argument is a `local variable `_. +An `llvm.dbg.value` intrinsic describes the *value* of a source variable +directly, not its address. Note that the value operand of this intrinsic may +be indirect (i.e, a pointer to the source variable), provided that interpreting +the complex expression derives the direct value. + Object lifetimes and scoping ============================ Modified: vendor/llvm/dist/include/llvm/ADT/DenseSet.h ============================================================================== --- vendor/llvm/dist/include/llvm/ADT/DenseSet.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/ADT/DenseSet.h Thu Aug 2 17:32:43 2018 (r337137) @@ -17,7 +17,7 @@ #include "llvm/ADT/DenseMap.h" #include "llvm/ADT/DenseMapInfo.h" #include "llvm/Support/type_traits.h" -#include +#include #include #include #include Modified: vendor/llvm/dist/include/llvm/Analysis/BasicAliasAnalysis.h ============================================================================== --- vendor/llvm/dist/include/llvm/Analysis/BasicAliasAnalysis.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/Analysis/BasicAliasAnalysis.h Thu Aug 2 17:32:43 2018 (r337137) @@ -43,6 +43,7 @@ class LoopInfo; class PHINode; class SelectInst; class TargetLibraryInfo; +class PhiValues; class Value; /// This is the AA result object for the basic, local, and stateless alias @@ -60,19 +61,22 @@ class BasicAAResult : public AAResultBase VL, const DataL SmallVectorImpl &SortedIndices); /// Returns true if the memory operations \p A and \p B are consecutive. -/// This is a simple API that does not depend on the analysis pass. +/// This is a simple API that does not depend on the analysis pass. bool isConsecutiveAccess(Value *A, Value *B, const DataLayout &DL, ScalarEvolution &SE, bool CheckType = true); @@ -734,7 +734,7 @@ class LoopAccessLegacyAnalysis : public FunctionPass { /// accesses of a loop. /// /// It runs the analysis for a loop on demand. This can be initiated by -/// querying the loop access info via AM.getResult. +/// querying the loop access info via AM.getResult. /// getResult return a LoopAccessInfo object. See this class for the /// specifics of what information is provided. class LoopAccessAnalysis Modified: vendor/llvm/dist/include/llvm/Analysis/MemoryDependenceAnalysis.h ============================================================================== --- vendor/llvm/dist/include/llvm/Analysis/MemoryDependenceAnalysis.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/Analysis/MemoryDependenceAnalysis.h Thu Aug 2 17:32:43 2018 (r337137) @@ -44,6 +44,7 @@ class Instruction; class LoadInst; class PHITransAddr; class TargetLibraryInfo; +class PhiValues; class Value; /// A memory dependence query can return one of three different answers. @@ -360,13 +361,14 @@ class MemoryDependenceResults { (private) AssumptionCache &AC; const TargetLibraryInfo &TLI; DominatorTree &DT; + PhiValues &PV; PredIteratorCache PredCache; public: MemoryDependenceResults(AliasAnalysis &AA, AssumptionCache &AC, const TargetLibraryInfo &TLI, - DominatorTree &DT) - : AA(AA), AC(AC), TLI(TLI), DT(DT) {} + DominatorTree &DT, PhiValues &PV) + : AA(AA), AC(AC), TLI(TLI), DT(DT), PV(PV) {} /// Handle invalidation in the new PM. bool invalidate(Function &F, const PreservedAnalyses &PA, Modified: vendor/llvm/dist/include/llvm/Analysis/MustExecute.h ============================================================================== --- vendor/llvm/dist/include/llvm/Analysis/MustExecute.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/Analysis/MustExecute.h Thu Aug 2 17:32:43 2018 (r337137) @@ -10,7 +10,7 @@ /// Contains a collection of routines for determining if a given instruction is /// guaranteed to execute if a given point in control flow is reached. The most /// common example is an instruction within a loop being provably executed if we -/// branch to the header of it's containing loop. +/// branch to the header of it's containing loop. /// //===----------------------------------------------------------------------===// @@ -58,7 +58,7 @@ void computeLoopSafetyInfo(LoopSafetyInfo *, Loop *); bool isGuaranteedToExecute(const Instruction &Inst, const DominatorTree *DT, const Loop *CurLoop, const LoopSafetyInfo *SafetyInfo); - + } #endif Modified: vendor/llvm/dist/include/llvm/Analysis/TargetTransformInfoImpl.h ============================================================================== --- vendor/llvm/dist/include/llvm/Analysis/TargetTransformInfoImpl.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/Analysis/TargetTransformInfoImpl.h Thu Aug 2 17:32:43 2018 (r337137) @@ -326,7 +326,7 @@ class TargetTransformInfoImplBase { (public) bool haveFastSqrt(Type *Ty) { return false; } bool isFCmpOrdCheaperThanFCmpZero(Type *Ty) { return true; } - + unsigned getFPOpCost(Type *Ty) { return TargetTransformInfo::TCC_Basic; } int getIntImmCodeSizeCost(unsigned Opcode, unsigned Idx, const APInt &Imm, Modified: vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h ============================================================================== --- vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/Analysis/ValueTracking.h Thu Aug 2 17:32:43 2018 (r337137) @@ -464,7 +464,7 @@ class Value; /// This is equivelent to saying that all instructions within the basic block /// are guaranteed to transfer execution to their successor within the basic /// block. This has the same assumptions w.r.t. undefined behavior as the - /// instruction variant of this function. + /// instruction variant of this function. bool isGuaranteedToTransferExecutionToSuccessor(const BasicBlock *BB); /// Return true if this function can prove that the instruction I Modified: vendor/llvm/dist/include/llvm/BinaryFormat/Dwarf.def ============================================================================== --- vendor/llvm/dist/include/llvm/BinaryFormat/Dwarf.def Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/BinaryFormat/Dwarf.def Thu Aug 2 17:32:43 2018 (r337137) @@ -856,6 +856,7 @@ HANDLE_DW_UT(0x06, split_type) // TODO: Add Mach-O and COFF names. // Official DWARF sections. HANDLE_DWARF_SECTION(DebugAbbrev, ".debug_abbrev", "debug-abbrev") +HANDLE_DWARF_SECTION(DebugAddr, ".debug_addr", "debug-addr") HANDLE_DWARF_SECTION(DebugAranges, ".debug_aranges", "debug-aranges") HANDLE_DWARF_SECTION(DebugInfo, ".debug_info", "debug-info") HANDLE_DWARF_SECTION(DebugTypes, ".debug_types", "debug-types") Modified: vendor/llvm/dist/include/llvm/BinaryFormat/ELF.h ============================================================================== --- vendor/llvm/dist/include/llvm/BinaryFormat/ELF.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/BinaryFormat/ELF.h Thu Aug 2 17:32:43 2018 (r337137) @@ -413,8 +413,10 @@ enum { // ARM Specific e_flags enum : unsigned { - EF_ARM_SOFT_FLOAT = 0x00000200U, - EF_ARM_VFP_FLOAT = 0x00000400U, + EF_ARM_SOFT_FLOAT = 0x00000200U, // Legacy pre EABI_VER5 + EF_ARM_ABI_FLOAT_SOFT = 0x00000200U, // EABI_VER5 + EF_ARM_VFP_FLOAT = 0x00000400U, // Legacy pre EABI_VER5 + EF_ARM_ABI_FLOAT_HARD = 0x00000400U, // EABI_VER5 EF_ARM_EABI_UNKNOWN = 0x00000000U, EF_ARM_EABI_VER1 = 0x01000000U, EF_ARM_EABI_VER2 = 0x02000000U, Modified: vendor/llvm/dist/include/llvm/CodeGen/GCStrategy.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/GCStrategy.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/GCStrategy.h Thu Aug 2 17:32:43 2018 (r337137) @@ -104,12 +104,12 @@ class GCStrategy { (public) const std::string &getName() const { return Name; } /// By default, write barriers are replaced with simple store - /// instructions. If true, you must provide a custom pass to lower + /// instructions. If true, you must provide a custom pass to lower /// calls to \@llvm.gcwrite. bool customWriteBarrier() const { return CustomWriteBarriers; } /// By default, read barriers are replaced with simple load - /// instructions. If true, you must provide a custom pass to lower + /// instructions. If true, you must provide a custom pass to lower /// calls to \@llvm.gcread. bool customReadBarrier() const { return CustomReadBarriers; } @@ -146,7 +146,7 @@ class GCStrategy { (public) } /// By default, roots are left for the code generator so it can generate a - /// stack map. If true, you must provide a custom pass to lower + /// stack map. If true, you must provide a custom pass to lower /// calls to \@llvm.gcroot. bool customRoots() const { return CustomRoots; } Modified: vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/LegalizerInfo.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/LegalizerInfo.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/LegalizerInfo.h Thu Aug 2 17:32:43 2018 (r337137) @@ -786,7 +786,7 @@ class LegalizerInfo { (public) /// setAction ({G_ADD, 0, LLT::scalar(32)}, Legal); /// setLegalizeScalarToDifferentSizeStrategy( /// G_ADD, 0, widenToLargerTypesAndNarrowToLargest); - /// will end up defining getAction({G_ADD, 0, T}) to return the following + /// will end up defining getAction({G_ADD, 0, T}) to return the following /// actions for different scalar types T: /// LLT::scalar(1)..LLT::scalar(31): {WidenScalar, 0, LLT::scalar(32)} /// LLT::scalar(32): {Legal, 0, LLT::scalar(32)} @@ -814,7 +814,7 @@ class LegalizerInfo { (public) VectorElementSizeChangeStrategies[OpcodeIdx][TypeIdx] = S; } - /// A SizeChangeStrategy for the common case where legalization for a + /// A SizeChangeStrategy for the common case where legalization for a /// particular operation consists of only supporting a specific set of type /// sizes. E.g. /// setAction ({G_DIV, 0, LLT::scalar(32)}, Legal); Modified: vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h Thu Aug 2 17:32:43 2018 (r337137) @@ -942,6 +942,16 @@ class MachineIRBuilderBase { (public) /// \return a MachineInstrBuilder for the newly created instruction. MachineInstrBuilder buildAtomicRMWUmin(unsigned OldValRes, unsigned Addr, unsigned Val, MachineMemOperand &MMO); + + /// Build and insert \p Res = G_BLOCK_ADDR \p BA + /// + /// G_BLOCK_ADDR computes the address of a basic block. + /// + /// \pre setBasicBlock or setMI must have been called. + /// \pre \p Res must be a generic virtual register of a pointer type. + /// + /// \return The newly created instruction. + MachineInstrBuilder buildBlockAddress(unsigned Res, const BlockAddress *BA); }; /// A CRTP class that contains methods for building instructions that can Modified: vendor/llvm/dist/include/llvm/CodeGen/MachORelocation.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/MachORelocation.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/MachORelocation.h Thu Aug 2 17:32:43 2018 (r337137) @@ -27,15 +27,15 @@ namespace llvm { uint32_t r_symbolnum; // symbol index if r_extern == 1 else section index bool r_pcrel; // was relocated pc-relative already uint8_t r_length; // length = 2 ^ r_length - bool r_extern; // + bool r_extern; // uint8_t r_type; // if not 0, machine-specific relocation type. bool r_scattered; // 1 = scattered, 0 = non-scattered int32_t r_value; // the value the item to be relocated is referring // to. - public: + public: uint32_t getPackedFields() const { if (r_scattered) - return (1 << 31) | (r_pcrel << 30) | ((r_length & 3) << 28) | + return (1 << 31) | (r_pcrel << 30) | ((r_length & 3) << 28) | ((r_type & 15) << 24) | (r_address & 0x00FFFFFF); else return (r_symbolnum << 8) | (r_pcrel << 7) | ((r_length & 3) << 5) | @@ -45,8 +45,8 @@ namespace llvm { uint32_t getRawAddress() const { return r_address; } MachORelocation(uint32_t addr, uint32_t index, bool pcrel, uint8_t len, - bool ext, uint8_t type, bool scattered = false, - int32_t value = 0) : + bool ext, uint8_t type, bool scattered = false, + int32_t value = 0) : r_address(addr), r_symbolnum(index), r_pcrel(pcrel), r_length(len), r_extern(ext), r_type(type), r_scattered(scattered), r_value(value) {} }; Modified: vendor/llvm/dist/include/llvm/CodeGen/MachineModuleInfo.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/MachineModuleInfo.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/MachineModuleInfo.h Thu Aug 2 17:32:43 2018 (r337137) @@ -105,7 +105,7 @@ class MachineModuleInfo : public ImmutablePass { /// basic block's address of label. MMIAddrLabelMap *AddrLabelSymbols; - // TODO: Ideally, what we'd like is to have a switch that allows emitting + // TODO: Ideally, what we'd like is to have a switch that allows emitting // synchronous (precise at call-sites only) CFA into .eh_frame. However, // even under this switch, we'd like .debug_frame to be precise when using // -g. At this moment, there's no way to specify that some CFI directives Modified: vendor/llvm/dist/include/llvm/CodeGen/MachineOutliner.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/MachineOutliner.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/MachineOutliner.h Thu Aug 2 17:32:43 2018 (r337137) @@ -19,6 +19,7 @@ #include "llvm/CodeGen/LiveRegUnits.h" #include "llvm/CodeGen/MachineFunction.h" #include "llvm/CodeGen/TargetRegisterInfo.h" +#include "llvm/CodeGen/LivePhysRegs.h" namespace llvm { namespace outliner { @@ -74,6 +75,13 @@ struct Candidate { (public) /// cost model information. LiveRegUnits LRU; + /// Contains the accumulated register liveness information for the + /// instructions in this \p Candidate. + /// + /// This is optionally used by the target to determine which registers have + /// been used across the sequence. + LiveRegUnits UsedInSequence; + /// Return the number of instructions in this Candidate. unsigned getLength() const { return Len; } @@ -137,6 +145,12 @@ struct Candidate { (public) // outlining candidate. std::for_each(MBB->rbegin(), (MachineBasicBlock::reverse_iterator)front(), [this](MachineInstr &MI) { LRU.stepBackward(MI); }); + + // Walk over the sequence itself and figure out which registers were used + // in the sequence. + UsedInSequence.init(TRI); + std::for_each(front(), std::next(back()), + [this](MachineInstr &MI) { UsedInSequence.accumulate(MI); }); } }; Modified: vendor/llvm/dist/include/llvm/CodeGen/ScheduleDAG.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/ScheduleDAG.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/ScheduleDAG.h Thu Aug 2 17:32:43 2018 (r337137) @@ -252,7 +252,7 @@ class TargetRegisterInfo; MachineInstr *Instr = nullptr; ///< Alternatively, a MachineInstr. public: - SUnit *OrigNode = nullptr; ///< If not this, the node from which this node + SUnit *OrigNode = nullptr; ///< If not this, the node from which this node /// was cloned. (SD scheduling only) const MCSchedClassDesc *SchedClass = Modified: vendor/llvm/dist/include/llvm/CodeGen/StackMaps.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/StackMaps.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/StackMaps.h Thu Aug 2 17:32:43 2018 (r337137) @@ -156,7 +156,7 @@ class StatepointOpers { // TODO:: we should change the STATEPOINT representation so that CC and // Flags should be part of meta operands, with args and deopt operands, and // gc operands all prefixed by their length and a type code. This would be - // much more consistent. + // much more consistent. public: // These values are aboolute offsets into the operands of the statepoint // instruction. Modified: vendor/llvm/dist/include/llvm/CodeGen/TargetLowering.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/TargetLowering.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/TargetLowering.h Thu Aug 2 17:32:43 2018 (r337137) @@ -718,7 +718,7 @@ class TargetLoweringBase { (public) /// always broken down into scalars in some contexts. This occurs even if the /// vector type is legal. virtual unsigned getVectorTypeBreakdownForCallingConv( - LLVMContext &Context, EVT VT, EVT &IntermediateVT, + LLVMContext &Context, CallingConv::ID CC, EVT VT, EVT &IntermediateVT, unsigned &NumIntermediates, MVT &RegisterVT) const { return getVectorTypeBreakdown(Context, VT, IntermediateVT, NumIntermediates, RegisterVT); @@ -1174,7 +1174,7 @@ class TargetLoweringBase { (public) /// are legal for some operations and not for other operations. /// For MIPS all vector types must be passed through the integer register set. virtual MVT getRegisterTypeForCallingConv(LLVMContext &Context, - EVT VT) const { + CallingConv::ID CC, EVT VT) const { return getRegisterType(Context, VT); } @@ -1182,6 +1182,7 @@ class TargetLoweringBase { (public) /// this occurs when a vector type is used, as vector are passed through the /// integer register set. virtual unsigned getNumRegistersForCallingConv(LLVMContext &Context, + CallingConv::ID CC, EVT VT) const { return getNumRegisters(Context, VT); } @@ -3489,10 +3490,10 @@ class TargetLowering : public TargetLoweringBase { (pu // SDValue BuildSDIV(SDNode *N, const APInt &Divisor, SelectionDAG &DAG, bool IsAfterLegalization, - std::vector *Created) const; + SmallVectorImpl &Created) const; SDValue BuildUDIV(SDNode *N, const APInt &Divisor, SelectionDAG &DAG, bool IsAfterLegalization, - std::vector *Created) const; + SmallVectorImpl &Created) const; /// Targets may override this function to provide custom SDIV lowering for /// power-of-2 denominators. If the target returns an empty SDValue, LLVM @@ -3500,7 +3501,7 @@ class TargetLowering : public TargetLoweringBase { (pu /// operations. virtual SDValue BuildSDIVPow2(SDNode *N, const APInt &Divisor, SelectionDAG &DAG, - std::vector *Created) const; + SmallVectorImpl &Created) const; /// Indicate whether this target prefers to combine FDIVs with the same /// divisor. If the transform should never be done, return zero. If the @@ -3690,7 +3691,7 @@ class TargetLowering : public TargetLoweringBase { (pu /// Given an LLVM IR type and return type attributes, compute the return value /// EVTs and flags, and optionally also the offsets, if the return value is /// being lowered to memory. -void GetReturnInfo(Type *ReturnType, AttributeList attr, +void GetReturnInfo(CallingConv::ID CC, Type *ReturnType, AttributeList attr, SmallVectorImpl &Outs, const TargetLowering &TLI, const DataLayout &DL); Modified: vendor/llvm/dist/include/llvm/CodeGen/TargetPassConfig.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/TargetPassConfig.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/TargetPassConfig.h Thu Aug 2 17:32:43 2018 (r337137) @@ -16,7 +16,7 @@ #include "llvm/Pass.h" #include "llvm/Support/CodeGen.h" -#include +#include #include namespace llvm { Modified: vendor/llvm/dist/include/llvm/CodeGen/TargetRegisterInfo.h ============================================================================== --- vendor/llvm/dist/include/llvm/CodeGen/TargetRegisterInfo.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/CodeGen/TargetRegisterInfo.h Thu Aug 2 17:32:43 2018 (r337137) @@ -456,7 +456,7 @@ class TargetRegisterInfo : public MCRegisterInfo { (pu /// stack frame offset. The first register is closest to the incoming stack /// pointer if stack grows down, and vice versa. /// Notice: This function does not take into account disabled CSRs. - /// In most cases you will want to use instead the function + /// In most cases you will want to use instead the function /// getCalleeSavedRegs that is implemented in MachineRegisterInfo. virtual const MCPhysReg* getCalleeSavedRegs(const MachineFunction *MF) const = 0; @@ -518,7 +518,7 @@ class TargetRegisterInfo : public MCRegisterInfo { (pu /// guaranteed to be restored before any uses. This is useful for targets that /// have call sequences where a GOT register may be updated by the caller /// prior to a call and is guaranteed to be restored (also by the caller) - /// after the call. + /// after the call. virtual bool isCallerPreservedPhysReg(unsigned PhysReg, const MachineFunction &MF) const { return false; Modified: vendor/llvm/dist/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def ============================================================================== --- vendor/llvm/dist/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def Thu Aug 2 17:32:43 2018 (r337137) @@ -143,7 +143,6 @@ CV_SYMBOL(S_MANSLOT , 0x1120) CV_SYMBOL(S_MANMANYREG , 0x1121) CV_SYMBOL(S_MANREGREL , 0x1122) CV_SYMBOL(S_MANMANYREG2 , 0x1123) -CV_SYMBOL(S_UNAMESPACE , 0x1124) CV_SYMBOL(S_DATAREF , 0x1126) CV_SYMBOL(S_ANNOTATIONREF , 0x1128) CV_SYMBOL(S_TOKENREF , 0x1129) @@ -255,6 +254,7 @@ SYMBOL_RECORD_ALIAS(S_GMANDATA , 0x111d, ManagedG SYMBOL_RECORD(S_LTHREAD32 , 0x1112, ThreadLocalDataSym) SYMBOL_RECORD_ALIAS(S_GTHREAD32 , 0x1113, GlobalTLS, ThreadLocalDataSym) +SYMBOL_RECORD(S_UNAMESPACE , 0x1124, UsingNamespaceSym) #undef CV_SYMBOL #undef SYMBOL_RECORD Modified: vendor/llvm/dist/include/llvm/DebugInfo/CodeView/SymbolRecord.h ============================================================================== --- vendor/llvm/dist/include/llvm/DebugInfo/CodeView/SymbolRecord.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/DebugInfo/CodeView/SymbolRecord.h Thu Aug 2 17:32:43 2018 (r337137) @@ -942,6 +942,19 @@ class ThreadLocalDataSym : public SymbolRecord { (publ uint32_t RecordOffset; }; +// S_UNAMESPACE +class UsingNamespaceSym : public SymbolRecord { +public: + explicit UsingNamespaceSym(SymbolRecordKind Kind) : SymbolRecord(Kind) {} + explicit UsingNamespaceSym(uint32_t RecordOffset) + : SymbolRecord(SymbolRecordKind::RegRelativeSym), + RecordOffset(RecordOffset) {} + + StringRef Name; + + uint32_t RecordOffset; +}; + // S_ANNOTATION using CVSymbol = CVRecord; Modified: vendor/llvm/dist/include/llvm/DebugInfo/DIContext.h ============================================================================== --- vendor/llvm/dist/include/llvm/DebugInfo/DIContext.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/DebugInfo/DIContext.h Thu Aug 2 17:32:43 2018 (r337137) @@ -154,6 +154,8 @@ enum DIDumpType : unsigned { struct DIDumpOptions { unsigned DumpType = DIDT_All; unsigned RecurseDepth = -1U; + uint16_t Version = 0; // DWARF version to assume when extracting. + uint8_t AddrSize = 4; // Address byte size to assume when extracting. bool ShowAddresses = true; bool ShowChildren = false; bool ShowParents = false; Modified: vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFContext.h ============================================================================== --- vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFContext.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFContext.h Thu Aug 2 17:32:43 2018 (r337137) @@ -323,6 +323,10 @@ class DWARFContext : public DIContext { (public) /// have initialized the relevant target descriptions. Error loadRegisterInfo(const object::ObjectFile &Obj); + /// Get address size from CUs. + /// TODO: refactor compile_units() to make this const. + uint8_t getCUAddrSize(); + private: /// Return the compile unit which contains instruction with provided /// address. Modified: vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDataExtractor.h ============================================================================== --- vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDataExtractor.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDataExtractor.h Thu Aug 2 17:32:43 2018 (r337137) @@ -51,6 +51,8 @@ class DWARFDataExtractor : public DataExtractor { (pub /// reflect the absolute address of this pointer. Optional getEncodedPointer(uint32_t *Offset, uint8_t Encoding, uint64_t AbsPosOffset = 0) const; + + size_t size() const { return Section == nullptr ? 0 : Section->Data.size(); } }; } // end namespace llvm Added: vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDebugAddr.h ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDebugAddr.h Thu Aug 2 17:32:43 2018 (r337137) @@ -0,0 +1,98 @@ +//===- DWARFDebugAddr.h -------------------------------------*- C++ -*-===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===------------------------------------------------------------------===// + +#ifndef LLVM_DEBUGINFO_DWARFDEBUGADDR_H +#define LLVM_DEBUGINFO_DWARFDEBUGADDR_H + +#include "llvm/BinaryFormat/Dwarf.h" +#include "llvm/DebugInfo/DIContext.h" +#include "llvm/DebugInfo/DWARF/DWARFDataExtractor.h" +#include "llvm/Support/Errc.h" +#include "llvm/Support/Error.h" +#include +#include +#include + +namespace llvm { + +class Error; +class raw_ostream; + +/// A class representing an address table as specified in DWARF v5. +/// The table consists of a header followed by an array of address values from +/// .debug_addr section. +class DWARFDebugAddrTable { +public: + struct Header { + /// The total length of the entries for this table, not including the length + /// field itself. + uint32_t Length = 0; + /// The DWARF version number. + uint16_t Version = 5; + /// The size in bytes of an address on the target architecture. For + /// segmented addressing, this is the size of the offset portion of the + /// address. + uint8_t AddrSize; + /// The size in bytes of a segment selector on the target architecture. + /// If the target system uses a flat address space, this value is 0. + uint8_t SegSize = 0; + }; + +private: + dwarf::DwarfFormat Format; + uint32_t HeaderOffset; + Header HeaderData; + uint32_t DataSize = 0; + std::vector Addrs; + +public: + void clear(); + + /// Extract an entire table, including all addresses. + Error extract(DWARFDataExtractor Data, uint32_t *OffsetPtr, + uint16_t Version, uint8_t AddrSize, + std::function WarnCallback); + + uint32_t getHeaderOffset() const { return HeaderOffset; } + uint8_t getAddrSize() const { return HeaderData.AddrSize; } + void dump(raw_ostream &OS, DIDumpOptions DumpOpts = {}) const; + + /// Return the address based on a given index. + Expected getAddrEntry(uint32_t Index) const; + + /// Return the size of the table header including the length + /// but not including the addresses. + uint8_t getHeaderSize() const { + switch (Format) { + case dwarf::DwarfFormat::DWARF32: + return 8; // 4 + 2 + 1 + 1 + case dwarf::DwarfFormat::DWARF64: + return 16; // 12 + 2 + 1 + 1 + } + llvm_unreachable("Invalid DWARF format (expected DWARF32 or DWARF64)"); + } + + /// Returns the length of this table, including the length field, or 0 if the + /// length has not been determined (e.g. because the table has not yet been + /// parsed, or there was a problem in parsing). + uint32_t getLength() const; + + /// Verify that the given length is valid for this table. + bool hasValidLength() const { return getLength() != 0; } + + /// Invalidate Length field to stop further processing. + void invalidateLength() { HeaderData.Length = 0; } + + /// Returns the length of the array of addresses. + uint32_t getDataSize() const; +}; + +} // end namespace llvm + +#endif // LLVM_DEBUGINFO_DWARFDEBUGADDR_H Modified: vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDie.h ============================================================================== --- vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDie.h Thu Aug 2 17:06:03 2018 (r337136) +++ vendor/llvm/dist/include/llvm/DebugInfo/DWARF/DWARFDie.h Thu Aug 2 17:32:43 2018 (r337137) @@ -46,7 +46,7 @@ class DWARFDie { public: DWARFDie() = default; - DWARFDie(DWARFUnit *Unit, const DWARFDebugInfoEntry * D) : U(Unit), Die(D) {} + DWARFDie(DWARFUnit *Unit, const DWARFDebugInfoEntry *D) : U(Unit), Die(D) {} bool isValid() const { return U && Die; } explicit operator bool() const { return isValid(); } @@ -82,9 +82,7 @@ class DWARFDie { *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E781106D6E0; Thu, 2 Aug 2018 17:33:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 387577230C; Thu, 2 Aug 2018 17:33:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1961F14D2F; Thu, 2 Aug 2018 17:33:13 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXDnN029324; Thu, 2 Aug 2018 17:33:13 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXCXH029319; Thu, 2 Aug 2018 17:33:12 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXCXH029319@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337139 - in vendor/clang/dist: docs include/clang-c include/clang/ARCMigrate include/clang/AST include/clang/ASTMatchers include/clang/Analysis include/clang/Analysis/Analyses include/... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/clang/dist: docs include/clang-c include/clang/ARCMigrate include/clang/AST include/clang/ASTMatchers include/clang/Analysis include/clang/Analysis/Analyses include/clang/Analysis/DomainSpec... X-SVN-Commit-Revision: 337139 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:14 -0000 Author: dim Date: Thu Aug 2 17:33:11 2018 New Revision: 337139 URL: https://svnweb.freebsd.org/changeset/base/337139 Log: Vendor import of clang trunk r338536: https://llvm.org/svn/llvm-project/cfe/trunk@338536 Added: vendor/clang/dist/lib/Driver/ToolChains/RISCV.cpp (contents, props changed) vendor/clang/dist/lib/Driver/ToolChains/RISCV.h (contents, props changed) vendor/clang/dist/test/Analysis/cfg-rich-constructors.mm vendor/clang/dist/test/Analysis/copy-elision.cpp (contents, props changed) vendor/clang/dist/test/Analysis/cstring-ranges.c (contents, props changed) vendor/clang/dist/test/Analysis/lifetime-extension.mm vendor/clang/dist/test/Analysis/temporaries.mm vendor/clang/dist/test/CodeGen/aapcs-align.cc (contents, props changed) vendor/clang/dist/test/CodeGen/aapcs64-align.cc (contents, props changed) vendor/clang/dist/test/CodeGen/catch-implicit-integer-truncations.c (contents, props changed) vendor/clang/dist/test/CodeGenCXX/Inputs/override-layout-nameless-struct-union.layout vendor/clang/dist/test/CodeGenCXX/Inputs/override-layout-packed-base.layout vendor/clang/dist/test/CodeGenCXX/castexpr-basepathsize-threshold.cpp (contents, props changed) vendor/clang/dist/test/CodeGenCXX/catch-implicit-integer-truncations.cpp (contents, props changed) vendor/clang/dist/test/CodeGenCXX/override-layout-nameless-struct-union.cpp (contents, props changed) vendor/clang/dist/test/CodeGenCXX/override-layout-packed-base.cpp (contents, props changed) vendor/clang/dist/test/CodeGenCXX/ubsan-new-checks.cpp (contents, props changed) vendor/clang/dist/test/CodeGenOpenCL/builtins-amdgcn-dl-insts-err-clamp.cl vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/bin/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/bin/riscv32-unknown-elf-ld (contents, props changed) vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/lib/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/lib/gcc/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/lib/gcc/riscv32-unknown-elf/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/lib/gcc/riscv32-unknown-elf/8.0.1/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/lib/gcc/riscv32-unknown-elf/8.0.1/crtbegin.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/lib/gcc/riscv32-unknown-elf/8.0.1/crtend.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/riscv32-unknown-elf/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/riscv32-unknown-elf/include/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/riscv32-unknown-elf/include/c++/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/riscv32-unknown-elf/include/c++/8.0.1/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/riscv32-unknown-elf/include/c++/8.0.1/.keep vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/riscv32-unknown-elf/lib/ vendor/clang/dist/test/Driver/Inputs/basic_riscv32_tree/riscv32-unknown-elf/lib/crt0.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/include/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/include/c++/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/include/c++/6.3.0/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/include/c++/6.3.0/backward/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/include/c++/6.3.0/backward/.keep vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/aarch64-oe-linux/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/aarch64-oe-linux/6.3.0/ vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/aarch64-oe-linux/6.3.0/crtbegin.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/aarch64-oe-linux/6.3.0/crtend.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/crt1.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/crti.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_aarch64_linux_tree/usr/lib64/crtn.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/include/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/include/c++/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/include/c++/6.3.0/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/include/c++/6.3.0/backward/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/include/c++/6.3.0/backward/.keep vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/arm-oe-linux-gnueabi/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/arm-oe-linux-gnueabi/6.3.0/ vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/arm-oe-linux-gnueabi/6.3.0/crtbegin.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/arm-oe-linux-gnueabi/6.3.0/crtend.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/crt1.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/crti.o (contents, props changed) vendor/clang/dist/test/Driver/Inputs/openembedded_arm_linux_tree/usr/lib/crtn.o (contents, props changed) vendor/clang/dist/test/Driver/cuda-unsupported-debug-options.cu vendor/clang/dist/test/Driver/openmp-unsupported-debug-options.c (contents, props changed) vendor/clang/dist/test/Index/complete-access-checks-crash.cpp (contents, props changed) vendor/clang/dist/test/SemaCXX/attr-lifetimebound.cpp (contents, props changed) vendor/clang/dist/test/SemaObjCXX/class-templ-error-null-init.mm Deleted: vendor/clang/dist/test/Analysis/cxx17-mandatory-elision.cpp vendor/clang/dist/test/Driver/Inputs/resource_dir/lib/darwin/ Modified: vendor/clang/dist/docs/ClangFormatStyleOptions.rst vendor/clang/dist/docs/ReleaseNotes.rst vendor/clang/dist/docs/UndefinedBehaviorSanitizer.rst vendor/clang/dist/docs/UsersManual.rst vendor/clang/dist/include/clang-c/Index.h vendor/clang/dist/include/clang/ARCMigrate/FileRemapper.h vendor/clang/dist/include/clang/AST/ASTContext.h vendor/clang/dist/include/clang/AST/ASTDiagnostic.h vendor/clang/dist/include/clang/AST/ASTImporter.h vendor/clang/dist/include/clang/AST/ASTLambda.h vendor/clang/dist/include/clang/AST/ASTMutationListener.h vendor/clang/dist/include/clang/AST/Attr.h vendor/clang/dist/include/clang/AST/AttrIterator.h vendor/clang/dist/include/clang/AST/BaseSubobject.h vendor/clang/dist/include/clang/AST/CXXInheritance.h vendor/clang/dist/include/clang/AST/CanonicalType.h vendor/clang/dist/include/clang/AST/CharUnits.h vendor/clang/dist/include/clang/AST/Comment.h vendor/clang/dist/include/clang/AST/CommentCommandTraits.h vendor/clang/dist/include/clang/AST/CommentLexer.h vendor/clang/dist/include/clang/AST/CommentSema.h vendor/clang/dist/include/clang/AST/Decl.h vendor/clang/dist/include/clang/AST/DeclBase.h vendor/clang/dist/include/clang/AST/DeclCXX.h vendor/clang/dist/include/clang/AST/DeclFriend.h vendor/clang/dist/include/clang/AST/DeclLookups.h vendor/clang/dist/include/clang/AST/DeclObjC.h vendor/clang/dist/include/clang/AST/DeclTemplate.h vendor/clang/dist/include/clang/AST/DeclarationName.h vendor/clang/dist/include/clang/AST/DependentDiagnostic.h vendor/clang/dist/include/clang/AST/EvaluatedExprVisitor.h vendor/clang/dist/include/clang/AST/Expr.h vendor/clang/dist/include/clang/AST/ExprCXX.h vendor/clang/dist/include/clang/AST/ExprObjC.h vendor/clang/dist/include/clang/AST/ExternalASTMerger.h vendor/clang/dist/include/clang/AST/ExternalASTSource.h vendor/clang/dist/include/clang/AST/GlobalDecl.h vendor/clang/dist/include/clang/AST/LambdaCapture.h vendor/clang/dist/include/clang/AST/Mangle.h vendor/clang/dist/include/clang/AST/NSAPI.h vendor/clang/dist/include/clang/AST/NestedNameSpecifier.h vendor/clang/dist/include/clang/AST/OperationKinds.def vendor/clang/dist/include/clang/AST/OperationKinds.h vendor/clang/dist/include/clang/AST/PrettyPrinter.h vendor/clang/dist/include/clang/AST/RecordLayout.h vendor/clang/dist/include/clang/AST/Stmt.h vendor/clang/dist/include/clang/AST/StmtIterator.h vendor/clang/dist/include/clang/AST/StmtObjC.h vendor/clang/dist/include/clang/AST/StmtOpenMP.h vendor/clang/dist/include/clang/AST/TemplateBase.h vendor/clang/dist/include/clang/AST/TemplateName.h vendor/clang/dist/include/clang/AST/Type.h vendor/clang/dist/include/clang/AST/TypeLoc.h vendor/clang/dist/include/clang/AST/TypeOrdering.h vendor/clang/dist/include/clang/AST/UnresolvedSet.h vendor/clang/dist/include/clang/AST/VTTBuilder.h vendor/clang/dist/include/clang/AST/VTableBuilder.h vendor/clang/dist/include/clang/ASTMatchers/ASTMatchers.h vendor/clang/dist/include/clang/Analysis/Analyses/CFGReachabilityAnalysis.h vendor/clang/dist/include/clang/Analysis/Analyses/Consumed.h vendor/clang/dist/include/clang/Analysis/Analyses/Dominators.h vendor/clang/dist/include/clang/Analysis/Analyses/FormatString.h vendor/clang/dist/include/clang/Analysis/Analyses/LiveVariables.h vendor/clang/dist/include/clang/Analysis/Analyses/ReachableCode.h vendor/clang/dist/include/clang/Analysis/Analyses/ThreadSafetyCommon.h vendor/clang/dist/include/clang/Analysis/AnalysisDeclContext.h vendor/clang/dist/include/clang/Analysis/CFG.h vendor/clang/dist/include/clang/Analysis/CFGStmtMap.h vendor/clang/dist/include/clang/Analysis/CloneDetection.h vendor/clang/dist/include/clang/Analysis/ConstructionContext.h vendor/clang/dist/include/clang/Analysis/DomainSpecific/CocoaConventions.h vendor/clang/dist/include/clang/Analysis/DomainSpecific/ObjCNoReturn.h vendor/clang/dist/include/clang/Analysis/ProgramPoint.h vendor/clang/dist/include/clang/Analysis/Support/BumpVector.h vendor/clang/dist/include/clang/Basic/ABI.h vendor/clang/dist/include/clang/Basic/AllDiagnostics.h vendor/clang/dist/include/clang/Basic/Attr.td vendor/clang/dist/include/clang/Basic/AttrDocs.td vendor/clang/dist/include/clang/Basic/Builtins.def vendor/clang/dist/include/clang/Basic/BuiltinsAMDGPU.def vendor/clang/dist/include/clang/Basic/BuiltinsPPC.def vendor/clang/dist/include/clang/Basic/Diagnostic.h vendor/clang/dist/include/clang/Basic/DiagnosticASTKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticCommentKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticCommonKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticDriverKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticFrontendKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticGroups.td vendor/clang/dist/include/clang/Basic/DiagnosticIDs.h vendor/clang/dist/include/clang/Basic/DiagnosticLexKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticOptions.def vendor/clang/dist/include/clang/Basic/DiagnosticOptions.h vendor/clang/dist/include/clang/Basic/DiagnosticParseKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticSemaKinds.td vendor/clang/dist/include/clang/Basic/DiagnosticSerializationKinds.td vendor/clang/dist/include/clang/Basic/FileSystemStatCache.h vendor/clang/dist/include/clang/Basic/IdentifierTable.h vendor/clang/dist/include/clang/Basic/LLVM.h vendor/clang/dist/include/clang/Basic/LangOptions.h vendor/clang/dist/include/clang/Basic/Linkage.h vendor/clang/dist/include/clang/Basic/Module.h vendor/clang/dist/include/clang/Basic/ObjCRuntime.h vendor/clang/dist/include/clang/Basic/OperatorKinds.h vendor/clang/dist/include/clang/Basic/Sanitizers.def vendor/clang/dist/include/clang/Basic/Sanitizers.h vendor/clang/dist/include/clang/Basic/SourceLocation.h vendor/clang/dist/include/clang/Basic/SourceManager.h vendor/clang/dist/include/clang/Basic/SourceManagerInternals.h vendor/clang/dist/include/clang/Basic/Specifiers.h vendor/clang/dist/include/clang/Basic/StmtNodes.td vendor/clang/dist/include/clang/Basic/TargetInfo.h vendor/clang/dist/include/clang/Basic/TemplateKinds.h vendor/clang/dist/include/clang/Basic/arm_neon.td vendor/clang/dist/include/clang/Basic/arm_neon_incl.td vendor/clang/dist/include/clang/Driver/CC1Options.td vendor/clang/dist/include/clang/Driver/Driver.h vendor/clang/dist/include/clang/Driver/ToolChain.h vendor/clang/dist/include/clang/Edit/Commit.h vendor/clang/dist/include/clang/Edit/EditedSource.h vendor/clang/dist/include/clang/Edit/Rewriters.h vendor/clang/dist/include/clang/Frontend/ASTUnit.h vendor/clang/dist/include/clang/Frontend/CompilerInstance.h vendor/clang/dist/include/clang/Frontend/CompilerInvocation.h vendor/clang/dist/include/clang/Frontend/DiagnosticRenderer.h vendor/clang/dist/include/clang/Frontend/FrontendActions.h vendor/clang/dist/include/clang/Frontend/LayoutOverrideSource.h vendor/clang/dist/include/clang/Frontend/LogDiagnosticPrinter.h vendor/clang/dist/include/clang/Frontend/PCHContainerOperations.h vendor/clang/dist/include/clang/Frontend/VerifyDiagnosticConsumer.h vendor/clang/dist/include/clang/Lex/CodeCompletionHandler.h vendor/clang/dist/include/clang/Lex/DirectoryLookup.h vendor/clang/dist/include/clang/Lex/ExternalPreprocessorSource.h vendor/clang/dist/include/clang/Lex/HeaderSearch.h vendor/clang/dist/include/clang/Lex/HeaderSearchOptions.h vendor/clang/dist/include/clang/Lex/LiteralSupport.h vendor/clang/dist/include/clang/Lex/MacroArgs.h vendor/clang/dist/include/clang/Lex/MacroInfo.h vendor/clang/dist/include/clang/Lex/ModuleLoader.h vendor/clang/dist/include/clang/Lex/ModuleMap.h vendor/clang/dist/include/clang/Lex/PPCallbacks.h vendor/clang/dist/include/clang/Lex/PPConditionalDirectiveRecord.h vendor/clang/dist/include/clang/Lex/PTHLexer.h vendor/clang/dist/include/clang/Lex/Pragma.h vendor/clang/dist/include/clang/Lex/PreprocessingRecord.h vendor/clang/dist/include/clang/Lex/Preprocessor.h vendor/clang/dist/include/clang/Lex/PreprocessorLexer.h vendor/clang/dist/include/clang/Lex/PreprocessorOptions.h vendor/clang/dist/include/clang/Lex/Token.h vendor/clang/dist/include/clang/Lex/TokenConcatenation.h vendor/clang/dist/include/clang/Lex/TokenLexer.h vendor/clang/dist/include/clang/Lex/VariadicMacroSupport.h vendor/clang/dist/include/clang/Parse/ParseAST.h vendor/clang/dist/include/clang/Parse/Parser.h vendor/clang/dist/include/clang/Parse/RAIIObjectsForParser.h vendor/clang/dist/include/clang/Sema/DeclSpec.h vendor/clang/dist/include/clang/Sema/DelayedDiagnostic.h vendor/clang/dist/include/clang/Sema/ExternalSemaSource.h vendor/clang/dist/include/clang/Sema/IdentifierResolver.h vendor/clang/dist/include/clang/Sema/Initialization.h vendor/clang/dist/include/clang/Sema/Lookup.h vendor/clang/dist/include/clang/Sema/MultiplexExternalSemaSource.h vendor/clang/dist/include/clang/Sema/Overload.h vendor/clang/dist/include/clang/Sema/ParsedAttr.h vendor/clang/dist/include/clang/Sema/ParsedTemplate.h vendor/clang/dist/include/clang/Sema/Scope.h vendor/clang/dist/include/clang/Sema/ScopeInfo.h vendor/clang/dist/include/clang/Sema/SemaInternal.h vendor/clang/dist/include/clang/Sema/SemaLambda.h vendor/clang/dist/include/clang/Sema/Template.h vendor/clang/dist/include/clang/Serialization/ASTBitCodes.h vendor/clang/dist/include/clang/Serialization/ASTReader.h vendor/clang/dist/include/clang/Serialization/ASTWriter.h vendor/clang/dist/include/clang/Serialization/ContinuousRangeMap.h vendor/clang/dist/include/clang/Serialization/GlobalModuleIndex.h vendor/clang/dist/include/clang/Serialization/Module.h vendor/clang/dist/include/clang/Serialization/ModuleFileExtension.h vendor/clang/dist/include/clang/Serialization/ModuleManager.h vendor/clang/dist/include/clang/StaticAnalyzer/Checkers/Checkers.td vendor/clang/dist/include/clang/StaticAnalyzer/Checkers/ObjCRetainCount.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/AnalyzerOptions.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/BugReporter/BugReporter.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/BugReporter/BugReporterVisitors.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/BugReporter/BugType.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/BugReporter/PathDiagnostic.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/Checker.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/CheckerManager.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/CheckerOptInfo.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/CheckerRegistry.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/IssueHash.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/APSIntType.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/AnalysisManager.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/BasicValueFactory.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/BlockCounter.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/CallEvent.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/CheckerContext.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/CoreEngine.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/Environment.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/ExplodedGraph.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/ExprEngine.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/MemRegion.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/ProgramState.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/SValBuilder.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/SVals.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/Store.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/StoreRef.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/SubEngine.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/SummaryManager.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/SymbolManager.h vendor/clang/dist/include/clang/StaticAnalyzer/Core/PathSensitive/WorkList.h vendor/clang/dist/lib/ARCMigrate/ARCMT.cpp vendor/clang/dist/lib/ARCMigrate/FileRemapper.cpp vendor/clang/dist/lib/ARCMigrate/Internals.h vendor/clang/dist/lib/ARCMigrate/ObjCMT.cpp vendor/clang/dist/lib/ARCMigrate/PlistReporter.cpp vendor/clang/dist/lib/ARCMigrate/TransARCAssign.cpp vendor/clang/dist/lib/ARCMigrate/TransAutoreleasePool.cpp vendor/clang/dist/lib/ARCMigrate/TransBlockObjCVariable.cpp vendor/clang/dist/lib/ARCMigrate/TransEmptyStatementsAndDealloc.cpp vendor/clang/dist/lib/ARCMigrate/TransGCAttrs.cpp vendor/clang/dist/lib/ARCMigrate/TransProperties.cpp vendor/clang/dist/lib/ARCMigrate/TransProtectedScope.cpp vendor/clang/dist/lib/ARCMigrate/TransRetainReleaseDealloc.cpp vendor/clang/dist/lib/ARCMigrate/TransUnbridgedCasts.cpp vendor/clang/dist/lib/ARCMigrate/TransZeroOutPropsInDealloc.cpp vendor/clang/dist/lib/ARCMigrate/TransformActions.cpp vendor/clang/dist/lib/ARCMigrate/Transforms.cpp vendor/clang/dist/lib/ARCMigrate/Transforms.h vendor/clang/dist/lib/AST/ASTContext.cpp vendor/clang/dist/lib/AST/ASTDiagnostic.cpp vendor/clang/dist/lib/AST/ASTDumper.cpp vendor/clang/dist/lib/AST/ASTImporter.cpp vendor/clang/dist/lib/AST/ASTStructuralEquivalence.cpp vendor/clang/dist/lib/AST/CXXInheritance.cpp vendor/clang/dist/lib/AST/Comment.cpp vendor/clang/dist/lib/AST/CommentLexer.cpp vendor/clang/dist/lib/AST/Decl.cpp vendor/clang/dist/lib/AST/DeclBase.cpp vendor/clang/dist/lib/AST/DeclCXX.cpp vendor/clang/dist/lib/AST/DeclObjC.cpp vendor/clang/dist/lib/AST/DeclPrinter.cpp vendor/clang/dist/lib/AST/DeclTemplate.cpp vendor/clang/dist/lib/AST/DeclarationName.cpp vendor/clang/dist/lib/AST/Expr.cpp vendor/clang/dist/lib/AST/ExprCXX.cpp vendor/clang/dist/lib/AST/ExprClassification.cpp vendor/clang/dist/lib/AST/ExprConstant.cpp vendor/clang/dist/lib/AST/ExternalASTSource.cpp vendor/clang/dist/lib/AST/ItaniumCXXABI.cpp vendor/clang/dist/lib/AST/ItaniumMangle.cpp vendor/clang/dist/lib/AST/Mangle.cpp vendor/clang/dist/lib/AST/MicrosoftMangle.cpp vendor/clang/dist/lib/AST/NSAPI.cpp vendor/clang/dist/lib/AST/NestedNameSpecifier.cpp vendor/clang/dist/lib/AST/ParentMap.cpp vendor/clang/dist/lib/AST/RecordLayout.cpp vendor/clang/dist/lib/AST/RecordLayoutBuilder.cpp vendor/clang/dist/lib/AST/StmtOpenMP.cpp vendor/clang/dist/lib/AST/StmtPrinter.cpp vendor/clang/dist/lib/AST/StmtProfile.cpp vendor/clang/dist/lib/AST/TemplateBase.cpp vendor/clang/dist/lib/AST/TemplateName.cpp vendor/clang/dist/lib/AST/Type.cpp vendor/clang/dist/lib/AST/TypeLoc.cpp vendor/clang/dist/lib/AST/TypePrinter.cpp vendor/clang/dist/lib/AST/VTTBuilder.cpp vendor/clang/dist/lib/AST/VTableBuilder.cpp vendor/clang/dist/lib/Analysis/AnalysisDeclContext.cpp vendor/clang/dist/lib/Analysis/BodyFarm.cpp vendor/clang/dist/lib/Analysis/CFG.cpp vendor/clang/dist/lib/Analysis/CFGReachabilityAnalysis.cpp vendor/clang/dist/lib/Analysis/CFGStmtMap.cpp vendor/clang/dist/lib/Analysis/CallGraph.cpp vendor/clang/dist/lib/Analysis/CocoaConventions.cpp vendor/clang/dist/lib/Analysis/ConstructionContext.cpp vendor/clang/dist/lib/Analysis/Consumed.cpp vendor/clang/dist/lib/Analysis/FormatString.cpp vendor/clang/dist/lib/Analysis/FormatStringParsing.h vendor/clang/dist/lib/Analysis/LiveVariables.cpp vendor/clang/dist/lib/Analysis/ObjCNoReturn.cpp vendor/clang/dist/lib/Analysis/PostOrderCFGView.cpp vendor/clang/dist/lib/Analysis/PrintfFormatString.cpp vendor/clang/dist/lib/Analysis/ProgramPoint.cpp vendor/clang/dist/lib/Analysis/ReachableCode.cpp vendor/clang/dist/lib/Analysis/ScanfFormatString.cpp vendor/clang/dist/lib/Analysis/UninitializedValues.cpp vendor/clang/dist/lib/Basic/Diagnostic.cpp vendor/clang/dist/lib/Basic/DiagnosticIDs.cpp vendor/clang/dist/lib/Basic/FileManager.cpp vendor/clang/dist/lib/Basic/FileSystemStatCache.cpp vendor/clang/dist/lib/Basic/IdentifierTable.cpp vendor/clang/dist/lib/Basic/Module.cpp vendor/clang/dist/lib/Basic/ObjCRuntime.cpp vendor/clang/dist/lib/Basic/SourceLocation.cpp vendor/clang/dist/lib/Basic/SourceManager.cpp vendor/clang/dist/lib/Basic/Targets/ARM.cpp vendor/clang/dist/lib/Basic/Targets/X86.cpp vendor/clang/dist/lib/Basic/Targets/X86.h vendor/clang/dist/lib/Basic/Warnings.cpp vendor/clang/dist/lib/CodeGen/CGBlocks.cpp vendor/clang/dist/lib/CodeGen/CGBlocks.h vendor/clang/dist/lib/CodeGen/CGBuilder.h vendor/clang/dist/lib/CodeGen/CGBuiltin.cpp vendor/clang/dist/lib/CodeGen/CGCXX.cpp vendor/clang/dist/lib/CodeGen/CGCXXABI.cpp vendor/clang/dist/lib/CodeGen/CGCall.cpp vendor/clang/dist/lib/CodeGen/CGCall.h vendor/clang/dist/lib/CodeGen/CGClass.cpp vendor/clang/dist/lib/CodeGen/CGCleanup.cpp vendor/clang/dist/lib/CodeGen/CGCleanup.h vendor/clang/dist/lib/CodeGen/CGDebugInfo.cpp vendor/clang/dist/lib/CodeGen/CGDeclCXX.cpp vendor/clang/dist/lib/CodeGen/CGException.cpp vendor/clang/dist/lib/CodeGen/CGExpr.cpp vendor/clang/dist/lib/CodeGen/CGExprAgg.cpp vendor/clang/dist/lib/CodeGen/CGExprCXX.cpp vendor/clang/dist/lib/CodeGen/CGExprConstant.cpp vendor/clang/dist/lib/CodeGen/CGExprScalar.cpp vendor/clang/dist/lib/CodeGen/CGLoopInfo.cpp vendor/clang/dist/lib/CodeGen/CGObjC.cpp vendor/clang/dist/lib/CodeGen/CGObjCGNU.cpp vendor/clang/dist/lib/CodeGen/CGObjCMac.cpp vendor/clang/dist/lib/CodeGen/CGObjCRuntime.cpp vendor/clang/dist/lib/CodeGen/CGObjCRuntime.h vendor/clang/dist/lib/CodeGen/CGOpenMPRuntime.cpp vendor/clang/dist/lib/CodeGen/CGRecordLayoutBuilder.cpp vendor/clang/dist/lib/CodeGen/CGVTT.cpp vendor/clang/dist/lib/CodeGen/CGVTables.cpp vendor/clang/dist/lib/CodeGen/CGVTables.h vendor/clang/dist/lib/CodeGen/CGValue.h vendor/clang/dist/lib/CodeGen/CodeGenAction.cpp vendor/clang/dist/lib/CodeGen/CodeGenFunction.h vendor/clang/dist/lib/CodeGen/CodeGenModule.cpp vendor/clang/dist/lib/CodeGen/CodeGenModule.h vendor/clang/dist/lib/CodeGen/CodeGenTypes.cpp vendor/clang/dist/lib/CodeGen/CodeGenTypes.h vendor/clang/dist/lib/CodeGen/ConstantInitBuilder.cpp vendor/clang/dist/lib/CodeGen/ItaniumCXXABI.cpp vendor/clang/dist/lib/CodeGen/MicrosoftCXXABI.cpp vendor/clang/dist/lib/CodeGen/SwiftCallingConv.cpp vendor/clang/dist/lib/CodeGen/TargetInfo.cpp vendor/clang/dist/lib/Driver/CMakeLists.txt vendor/clang/dist/lib/Driver/Driver.cpp vendor/clang/dist/lib/Driver/SanitizerArgs.cpp vendor/clang/dist/lib/Driver/ToolChain.cpp vendor/clang/dist/lib/Driver/ToolChains/Arch/RISCV.cpp vendor/clang/dist/lib/Driver/ToolChains/Clang.cpp vendor/clang/dist/lib/Driver/ToolChains/Cuda.cpp vendor/clang/dist/lib/Driver/ToolChains/Cuda.h vendor/clang/dist/lib/Driver/ToolChains/Darwin.cpp vendor/clang/dist/lib/Driver/ToolChains/Darwin.h vendor/clang/dist/lib/Driver/ToolChains/Gnu.cpp vendor/clang/dist/lib/Driver/ToolChains/Linux.cpp vendor/clang/dist/lib/Edit/Commit.cpp vendor/clang/dist/lib/Edit/RewriteObjCFoundationAPI.cpp vendor/clang/dist/lib/Format/BreakableToken.cpp vendor/clang/dist/lib/Format/BreakableToken.h vendor/clang/dist/lib/Format/ContinuationIndenter.cpp vendor/clang/dist/lib/Format/ContinuationIndenter.h vendor/clang/dist/lib/Format/Format.cpp vendor/clang/dist/lib/Frontend/ASTConsumers.cpp vendor/clang/dist/lib/Frontend/ASTMerge.cpp vendor/clang/dist/lib/Frontend/ChainedIncludesSource.cpp vendor/clang/dist/lib/Frontend/CompilerInstance.cpp vendor/clang/dist/lib/Frontend/DependencyFile.cpp vendor/clang/dist/lib/Frontend/DependencyGraph.cpp vendor/clang/dist/lib/Frontend/DiagnosticRenderer.cpp vendor/clang/dist/lib/Frontend/FrontendAction.cpp vendor/clang/dist/lib/Frontend/FrontendActions.cpp vendor/clang/dist/lib/Frontend/LayoutOverrideSource.cpp vendor/clang/dist/lib/Frontend/MultiplexConsumer.cpp vendor/clang/dist/lib/Frontend/PrintPreprocessedOutput.cpp vendor/clang/dist/lib/Frontend/Rewrite/FixItRewriter.cpp vendor/clang/dist/lib/Frontend/Rewrite/FrontendActions.cpp vendor/clang/dist/lib/Frontend/Rewrite/RewriteModernObjC.cpp vendor/clang/dist/lib/Frontend/Rewrite/RewriteObjC.cpp vendor/clang/dist/lib/Frontend/SerializedDiagnosticPrinter.cpp vendor/clang/dist/lib/Frontend/TestModuleFileExtension.cpp vendor/clang/dist/lib/Frontend/TextDiagnostic.cpp vendor/clang/dist/lib/FrontendTool/ExecuteCompilerInvocation.cpp vendor/clang/dist/lib/Headers/avx512fintrin.h vendor/clang/dist/lib/Headers/avx512vlcdintrin.h vendor/clang/dist/lib/Headers/clzerointrin.h vendor/clang/dist/lib/Headers/lwpintrin.h vendor/clang/dist/lib/Headers/opencl-c.h vendor/clang/dist/lib/Index/IndexBody.cpp vendor/clang/dist/lib/Index/IndexTypeSourceInfo.cpp vendor/clang/dist/lib/Index/IndexingContext.cpp vendor/clang/dist/lib/Index/USRGeneration.cpp vendor/clang/dist/lib/Lex/HeaderSearch.cpp vendor/clang/dist/lib/Lex/LiteralSupport.cpp vendor/clang/dist/lib/Lex/MacroArgs.cpp vendor/clang/dist/lib/Lex/ModuleMap.cpp vendor/clang/dist/lib/Lex/PPDirectives.cpp vendor/clang/dist/lib/Lex/PPExpressions.cpp vendor/clang/dist/lib/Lex/PPLexerChange.cpp vendor/clang/dist/lib/Lex/PPMacroExpansion.cpp vendor/clang/dist/lib/Lex/Pragma.cpp vendor/clang/dist/lib/Lex/PreprocessingRecord.cpp vendor/clang/dist/lib/Lex/Preprocessor.cpp vendor/clang/dist/lib/Lex/TokenLexer.cpp vendor/clang/dist/lib/Parse/ParseAST.cpp vendor/clang/dist/lib/Parse/ParseCXXInlineMethods.cpp vendor/clang/dist/lib/Parse/ParseDecl.cpp vendor/clang/dist/lib/Parse/ParseDeclCXX.cpp vendor/clang/dist/lib/Parse/ParseExpr.cpp vendor/clang/dist/lib/Parse/ParseExprCXX.cpp vendor/clang/dist/lib/Parse/ParseInit.cpp vendor/clang/dist/lib/Parse/ParseObjc.cpp vendor/clang/dist/lib/Parse/ParsePragma.cpp vendor/clang/dist/lib/Parse/ParseStmtAsm.cpp vendor/clang/dist/lib/Parse/ParseTemplate.cpp vendor/clang/dist/lib/Parse/ParseTentative.cpp vendor/clang/dist/lib/Parse/Parser.cpp vendor/clang/dist/lib/Rewrite/DeltaTree.cpp vendor/clang/dist/lib/Rewrite/HTMLRewrite.cpp vendor/clang/dist/lib/Rewrite/Rewriter.cpp vendor/clang/dist/lib/Sema/AnalysisBasedWarnings.cpp vendor/clang/dist/lib/Sema/CMakeLists.txt vendor/clang/dist/lib/Sema/CodeCompleteConsumer.cpp vendor/clang/dist/lib/Sema/DeclSpec.cpp vendor/clang/dist/lib/Sema/DelayedDiagnostic.cpp vendor/clang/dist/lib/Sema/IdentifierResolver.cpp vendor/clang/dist/lib/Sema/MultiplexExternalSemaSource.cpp vendor/clang/dist/lib/Sema/Sema.cpp vendor/clang/dist/lib/Sema/SemaAccess.cpp vendor/clang/dist/lib/Sema/SemaAttr.cpp vendor/clang/dist/lib/Sema/SemaCXXScopeSpec.cpp vendor/clang/dist/lib/Sema/SemaCast.cpp vendor/clang/dist/lib/Sema/SemaChecking.cpp vendor/clang/dist/lib/Sema/SemaCodeComplete.cpp vendor/clang/dist/lib/Sema/SemaDecl.cpp vendor/clang/dist/lib/Sema/SemaDeclAttr.cpp vendor/clang/dist/lib/Sema/SemaDeclObjC.cpp vendor/clang/dist/lib/Sema/SemaExceptionSpec.cpp vendor/clang/dist/lib/Sema/SemaExpr.cpp vendor/clang/dist/lib/Sema/SemaExprMember.cpp vendor/clang/dist/lib/Sema/SemaExprObjC.cpp vendor/clang/dist/lib/Sema/SemaInit.cpp vendor/clang/dist/lib/Sema/SemaLambda.cpp vendor/clang/dist/lib/Sema/SemaLookup.cpp vendor/clang/dist/lib/Sema/SemaObjCProperty.cpp vendor/clang/dist/lib/Sema/SemaPseudoObject.cpp vendor/clang/dist/lib/Sema/SemaStmt.cpp vendor/clang/dist/lib/Sema/SemaStmtAsm.cpp vendor/clang/dist/lib/Sema/SemaTemplate.cpp vendor/clang/dist/lib/Sema/SemaTemplateDeduction.cpp vendor/clang/dist/lib/Sema/SemaTemplateInstantiate.cpp vendor/clang/dist/lib/Sema/SemaTemplateInstantiateDecl.cpp vendor/clang/dist/lib/Sema/SemaTemplateVariadic.cpp vendor/clang/dist/lib/Sema/SemaType.cpp vendor/clang/dist/lib/Sema/TreeTransform.h vendor/clang/dist/lib/Sema/TypeLocBuilder.h vendor/clang/dist/lib/Serialization/ASTReader.cpp vendor/clang/dist/lib/Serialization/ASTReaderDecl.cpp vendor/clang/dist/lib/Serialization/ASTReaderInternals.h vendor/clang/dist/lib/Serialization/ASTWriterDecl.cpp vendor/clang/dist/lib/Serialization/ASTWriterStmt.cpp vendor/clang/dist/lib/Serialization/Module.cpp vendor/clang/dist/lib/Serialization/ModuleManager.cpp vendor/clang/dist/lib/StaticAnalyzer/Checkers/CStringChecker.cpp vendor/clang/dist/lib/StaticAnalyzer/Checkers/InnerPointerChecker.cpp vendor/clang/dist/lib/StaticAnalyzer/Checkers/IteratorChecker.cpp vendor/clang/dist/lib/StaticAnalyzer/Checkers/MallocChecker.cpp vendor/clang/dist/lib/StaticAnalyzer/Checkers/RetainCountChecker.cpp vendor/clang/dist/lib/StaticAnalyzer/Checkers/RunLoopAutoreleaseLeakChecker.cpp vendor/clang/dist/lib/StaticAnalyzer/Checkers/StackAddrEscapeChecker.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/BugReporterVisitors.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/CallEvent.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/ExprEngine.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/ExprEngineC.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/ExprEngineCXX.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/IssueHash.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/ProgramState.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/SimpleSValBuilder.cpp vendor/clang/dist/lib/StaticAnalyzer/Core/Store.cpp vendor/clang/dist/test/Analysis/Checkers/RunLoopAutoreleaseLeakChecker.m vendor/clang/dist/test/Analysis/casts.c vendor/clang/dist/test/Analysis/cfg-rich-constructors.cpp vendor/clang/dist/test/Analysis/inner-pointer.cpp vendor/clang/dist/test/Analysis/temporaries.cpp vendor/clang/dist/test/CodeGen/arm-arguments.c vendor/clang/dist/test/CodeGen/builtin-memfns.c vendor/clang/dist/test/CodeGen/exceptions-seh-finally.c vendor/clang/dist/test/CodeGen/exceptions-seh.c vendor/clang/dist/test/CodeGen/mips-unsigned-ext-var.c vendor/clang/dist/test/CodeGen/mips-varargs.c vendor/clang/dist/test/CodeGen/mips-vector-arg.c vendor/clang/dist/test/CodeGenCUDA/device-var-init.cu vendor/clang/dist/test/CodeGenCXX/ctor-dtor-alias.cpp vendor/clang/dist/test/CodeGenCXX/float16-declarations.cpp vendor/clang/dist/test/CodeGenOpenCL/blocks.cl vendor/clang/dist/test/CodeGenOpenCL/builtins-amdgcn-dl-insts-err.cl vendor/clang/dist/test/CodeGenOpenCL/builtins-amdgcn-dl-insts.cl vendor/clang/dist/test/Driver/aarch64-cpus.c vendor/clang/dist/test/Driver/arm-cortex-cpus.c vendor/clang/dist/test/Driver/baremetal.cpp vendor/clang/dist/test/Driver/cuda-dwarf-2.cu vendor/clang/dist/test/Driver/darwin-asan-nofortify.c vendor/clang/dist/test/Driver/darwin-sanitizer-ld.c vendor/clang/dist/test/Driver/fsanitize.c vendor/clang/dist/test/Driver/fuzzer.c vendor/clang/dist/test/Driver/linux-header-search.cpp vendor/clang/dist/test/Driver/linux-ld.c vendor/clang/dist/test/Driver/openmp-offload-gpu.c vendor/clang/dist/test/Driver/riscv32-toolchain.c vendor/clang/dist/test/Driver/sanitizer-ld.c vendor/clang/dist/test/Frontend/clang-abi-compat.cpp vendor/clang/dist/test/Modules/resolution-change.m vendor/clang/dist/test/OpenMP/declare_target_codegen.cpp vendor/clang/dist/test/OpenMP/nvptx_declare_target_var_ctor_dtor_codegen.cpp vendor/clang/dist/test/OpenMP/target_codegen.cpp vendor/clang/dist/test/OpenMP/target_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_parallel_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_for_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_for_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_parallel_for_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_for_simd_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_for_simd_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_parallel_for_simd_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_if_codegen.cpp vendor/clang/dist/test/OpenMP/target_parallel_num_threads_codegen.cpp vendor/clang/dist/test/OpenMP/target_simd_codegen.cpp vendor/clang/dist/test/OpenMP/target_simd_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_simd_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_teams_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_parallel_for_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_parallel_for_simd_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_parallel_for_simd_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_simd_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_simd_codegen_registration.cpp vendor/clang/dist/test/OpenMP/target_teams_distribute_simd_depend_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_num_teams_codegen.cpp vendor/clang/dist/test/OpenMP/target_teams_thread_limit_codegen.cpp vendor/clang/dist/test/PCH/coroutines.cpp vendor/clang/dist/test/PCH/cxx-templates.cpp vendor/clang/dist/test/PCH/cxx-templates.h vendor/clang/dist/test/Parser/cxx1z-fold-expressions.cpp vendor/clang/dist/test/SemaCUDA/device-var-init.cu vendor/clang/dist/test/SemaCXX/constexpr-string.cpp vendor/clang/dist/test/SemaCXX/cxx1z-class-template-argument-deduction.cpp vendor/clang/dist/test/SemaCXX/warn-dangling-local.cpp vendor/clang/dist/test/SemaObjCXX/noescape.mm vendor/clang/dist/test/SemaOpenCL/invalid-kernel-parameters.cl vendor/clang/dist/unittests/Format/FormatTestComments.cpp vendor/clang/dist/unittests/Format/FormatTestJS.cpp vendor/clang/dist/utils/TableGen/ClangAttrEmitter.cpp vendor/clang/dist/utils/analyzer/CmpRuns.py vendor/clang/dist/utils/analyzer/SATestBuild.py Modified: vendor/clang/dist/docs/ClangFormatStyleOptions.rst ============================================================================== --- vendor/clang/dist/docs/ClangFormatStyleOptions.rst Thu Aug 2 17:32:55 2018 (r337138) +++ vendor/clang/dist/docs/ClangFormatStyleOptions.rst Thu Aug 2 17:33:11 2018 (r337139) @@ -1134,17 +1134,21 @@ the configuration (without a prefix: ``Auto``). .. code-block:: c++ true: - SomeClass::Constructor() - : aaaaaaaa(aaaaaaaa), aaaaaaaa(aaaaaaaa), aaaaaaaa(aaaaaaaaaaaaaaaaaaaaaaaaa) { - return 0; - } + FitsOnOneLine::Constructor() + : aaaaaaaaaaaaa(aaaaaaaaaaaaaa), aaaaaaaaaaaaa(aaaaaaaaaaaaaa) {} + DoesntFit::Constructor() + : aaaaaaaaaaaaa(aaaaaaaaaaaaaa), + aaaaaaaaaaaaa(aaaaaaaaaaaaaa), + aaaaaaaaaaaaa(aaaaaaaaaaaaaa) {} + false: - SomeClass::Constructor() - : aaaaaaaa(aaaaaaaa), aaaaaaaa(aaaaaaaa), - aaaaaaaa(aaaaaaaaaaaaaaaaaaaaaaaaa) { - return 0; - } + FitsOnOneLine::Constructor() + : aaaaaaaaaaaaa(aaaaaaaaaaaaaa), aaaaaaaaaaaaa(aaaaaaaaaaaaaa) {} + + DoesntFit::Constructor() + : aaaaaaaaaaaaa(aaaaaaaaaaaaaa), aaaaaaaaaaaaa(aaaaaaaaaaaaaa), + aaaaaaaaaaaaa(aaaaaaaaaaaaaa) {} **ConstructorInitializerIndentWidth** (``unsigned``) The number of characters to use for indentation of constructor Modified: vendor/clang/dist/docs/ReleaseNotes.rst ============================================================================== --- vendor/clang/dist/docs/ReleaseNotes.rst Thu Aug 2 17:32:55 2018 (r337138) +++ vendor/clang/dist/docs/ReleaseNotes.rst Thu Aug 2 17:33:11 2018 (r337139) @@ -46,7 +46,9 @@ sections with improvements to Clang's support for thos Major New Features ------------------ -- ... +- A new Implicit Conversion Sanitizer (``-fsanitize=implicit-conversion``) group + was added. Please refer to the :ref:`release-notes-ubsan` section of the + release notes for the details. Improvements to Clang's diagnostics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -280,10 +282,36 @@ Static Analyzer ... +.. _release-notes-ubsan: + Undefined Behavior Sanitizer (UBSan) ------------------------------------ -* ... +* A new Implicit Conversion Sanitizer (``-fsanitize=implicit-conversion``) group + was added. + + Currently, only one type of issues is caught - implicit integer truncation + (``-fsanitize=implicit-integer-truncation``), also known as integer demotion. + While there is a ``-Wconversion`` diagnostic group that catches this kind of + issues, it is both noisy, and does not catch **all** the cases. + + .. code-block:: c++ + + unsigned char store = 0; + + bool consume(unsigned int val); + + void test(unsigned long val) { + if (consume(val)) // the value may have been silently truncated. + store = store + 768; // before addition, 'store' was promoted to int. + (void)consume((unsigned int)val); // OK, the truncation is explicit. + } + + Just like other ``-fsanitize=integer`` checks, these issues are **not** + undefined behaviour. But they are not *always* intentional, and are somewhat + hard to track down. This group is **not** enabled by ``-fsanitize=undefined``, + but the ``-fsanitize=implicit-integer-truncation`` check + is enabled by ``-fsanitize=integer``. Core Analysis Improvements ========================== Modified: vendor/clang/dist/docs/UndefinedBehaviorSanitizer.rst ============================================================================== --- vendor/clang/dist/docs/UndefinedBehaviorSanitizer.rst Thu Aug 2 17:32:55 2018 (r337138) +++ vendor/clang/dist/docs/UndefinedBehaviorSanitizer.rst Thu Aug 2 17:33:11 2018 (r337139) @@ -89,6 +89,12 @@ Available checks are: - ``-fsanitize=function``: Indirect call of a function through a function pointer of the wrong type (Darwin/Linux, C++ and x86/x86_64 only). + - ``-fsanitize=implicit-integer-truncation``: Implicit conversion from + integer of larger bit width to smaller bit width, if that results in data + loss. That is, if the demoted value, after casting back to the original + width, is not equal to the original value before the downcast. + Issues caught by this sanitizer are not undefined behavior, + but are often unintentional. - ``-fsanitize=integer-divide-by-zero``: Integer division by zero. - ``-fsanitize=nonnull-attribute``: Passing null pointer as a function parameter which is declared to never be null. @@ -121,15 +127,21 @@ Available checks are: unsigned overflow in C++. You can use ``-fsanitize=shift-base`` or ``-fsanitize=shift-exponent`` to check only left-hand side or right-hand side of shift operation, respectively. - - ``-fsanitize=signed-integer-overflow``: Signed integer overflow, - including all the checks added by ``-ftrapv``, and checking for - overflow in signed division (``INT_MIN / -1``). + - ``-fsanitize=signed-integer-overflow``: Signed integer overflow, where the + result of a signed integer computation cannot be represented in its type. + This includes all the checks covered by ``-ftrapv``, as well as checks for + signed division overflow (``INT_MIN/-1``), but not checks for + lossy implicit conversions performed before the computation + (see ``-fsanitize=implicit-conversion``). Both of these two issues are + handled by ``-fsanitize=implicit-conversion`` group of checks. - ``-fsanitize=unreachable``: If control flow reaches an unreachable program point. - - ``-fsanitize=unsigned-integer-overflow``: Unsigned integer - overflows. Note that unlike signed integer overflow, unsigned integer - is not undefined behavior. However, while it has well-defined semantics, - it is often unintentional, so UBSan offers to catch it. + - ``-fsanitize=unsigned-integer-overflow``: Unsigned integer overflow, where + the result of an unsigned integer computation cannot be represented in its + type. Unlike signed integer overflow, this is not undefined behavior, but + it is often unintentional. This sanitizer does not check for lossy implicit + conversions performed before such a computation + (see ``-fsanitize=implicit-conversion``). - ``-fsanitize=vla-bound``: A variable-length array whose bound does not evaluate to a positive value. - ``-fsanitize=vptr``: Use of an object whose vptr indicates that it is of @@ -140,11 +152,17 @@ Available checks are: You can also use the following check groups: - ``-fsanitize=undefined``: All of the checks listed above other than - ``unsigned-integer-overflow`` and the ``nullability-*`` checks. + ``unsigned-integer-overflow``, ``implicit-conversion`` and the + ``nullability-*`` group of checks. - ``-fsanitize=undefined-trap``: Deprecated alias of ``-fsanitize=undefined``. - ``-fsanitize=integer``: Checks for undefined or suspicious integer behavior (e.g. unsigned integer overflow). + Enables ``signed-integer-overflow``, ``unsigned-integer-overflow``, + ``shift``, ``integer-divide-by-zero``, and ``implicit-integer-truncation``. + - ``-fsanitize=implicit-conversion``: Checks for suspicious behaviours of + implicit conversions. + Currently, only ``-fsanitize=implicit-integer-truncation`` is implemented. - ``-fsanitize=nullability``: Enables ``nullability-arg``, ``nullability-assign``, and ``nullability-return``. While violating nullability does not have undefined behavior, it is often unintentional, Modified: vendor/clang/dist/docs/UsersManual.rst ============================================================================== --- vendor/clang/dist/docs/UsersManual.rst Thu Aug 2 17:32:55 2018 (r337138) +++ vendor/clang/dist/docs/UsersManual.rst Thu Aug 2 17:33:11 2018 (r337139) @@ -2155,13 +2155,8 @@ Objective-C++ Language Features OpenMP Features =============== -Clang supports all OpenMP 3.1 directives and clauses. In addition, some -features of OpenMP 4.0 are supported. For example, ``#pragma omp simd``, -``#pragma omp for simd``, ``#pragma omp parallel for simd`` directives, extended -set of atomic constructs, ``proc_bind`` clause for all parallel-based -directives, ``depend`` clause for ``#pragma omp task`` directive (except for -array sections), ``#pragma omp cancel`` and ``#pragma omp cancellation point`` -directives, and ``#pragma omp taskgroup`` directive. +Clang supports all OpenMP 4.5 directives and clauses. See :doc:`OpenMPSupport` +for additional details. Use `-fopenmp` to enable OpenMP. Support for OpenMP can be disabled with `-fno-openmp`. @@ -2783,6 +2778,7 @@ Execute ``clang-cl /?`` to see a list of supported opt /Brepro Emit an object file which can be reproduced over time /C Don't discard comments when preprocessing /c Compile only + /d1PP Retain macro definitions in /E mode /d1reportAllClassLayout Dump record layout information /diagnostics:caret Enable caret and column diagnostics (on by default) /diagnostics:classic Disable column and caret diagnostics @@ -2816,6 +2812,7 @@ Execute ``clang-cl /?`` to see a list of supported opt /GS- Disable buffer security check /GS Enable buffer security check /Gs Set stack probe size + /guard: Enable Control Flow Guard with /guard:cf /Gv Set __vectorcall as a default calling convention /Gw- Don't put each data item in its own section /Gw Put each data item in its own section @@ -2871,6 +2868,7 @@ Execute ``clang-cl /?`` to see a list of supported opt /WX- Do not treat warnings as errors /WX Treat warnings as errors /w Disable all warnings + /X Don't add %INCLUDE% to the include search path /Y- Disable precompiled headers, overrides /Yc and /Yu /Yc Generate a pch file for all code up to and including /Yu Load a pch file and use it instead of all code up to and including @@ -2894,8 +2892,15 @@ Execute ``clang-cl /?`` to see a list of supported opt OPTIONS: -### Print (but do not run) the commands to run for this compilation --analyze Run the static analyzer + -faddrsig Emit an address-significance table -fansi-escape-codes Use ANSI escape codes for diagnostics + -fblocks Enable the 'blocks' language feature + -fcf-protection= Instrument control-flow architecture protection. Options: return, branch, full, none. + -fcf-protection Enable cf-protection in 'full' mode -fcolor-diagnostics Use colors in diagnostics + -fcomplete-member-pointers + Require member pointer base types to be complete if they would be significant under the Microsoft ABI + -fcoverage-mapping Generate coverage mapping to enable code coverage analysis -fdebug-macro Emit macro debug information -fdelayed-template-parsing Parse templated function definitions at the end of the translation unit @@ -2905,6 +2910,7 @@ Execute ``clang-cl /?`` to see a list of supported opt Print fix-its in machine parseable form -flto= Set LTO mode to either 'full' or 'thin' -flto Enable LTO in 'full' mode + -fmerge-all-constants Allow merging of constants -fms-compatibility-version= Dot-separated value representing the Microsoft compiler version number to report in _MSC_VER (0 = don't define it (default)) @@ -2912,9 +2918,17 @@ Execute ``clang-cl /?`` to see a list of supported opt -fms-extensions Accept some non-standard constructs supported by the Microsoft compiler -fmsc-version= Microsoft compiler version number to report in _MSC_VER (0 = don't define it (default)) + -fno-addrsig Don't emit an address-significance table + -fno-builtin- Disable implicit builtin knowledge of a specific function + -fno-builtin Disable implicit builtin knowledge of functions + -fno-complete-member-pointers + Do not require member pointer base types to be complete if they would be significant under the Microsoft ABI + -fno-coverage-mapping Disable code coverage analysis -fno-debug-macro Do not emit macro debug information -fno-delayed-template-parsing Disable delayed template parsing + -fno-sanitize-address-poison-class-member-array-new-cookie + Disable poisoning array cookies when using class member operator new[] in AddressSanitizer -fno-sanitize-address-use-after-scope Disable use-after-scope detection in AddressSanitizer -fno-sanitize-blacklist Don't use blacklist file for sanitizers @@ -2950,6 +2964,8 @@ Execute ``clang-cl /?`` to see a list of supported opt Level of field padding for AddressSanitizer -fsanitize-address-globals-dead-stripping Enable linker dead stripping of globals in AddressSanitizer + -fsanitize-address-poison-class-member-array-new-cookie + Enable poisoning array cookies when using class member operator new[] in AddressSanitizer -fsanitize-address-use-after-scope Enable use-after-scope detection in AddressSanitizer -fsanitize-blacklist= Modified: vendor/clang/dist/include/clang-c/Index.h ============================================================================== --- vendor/clang/dist/include/clang-c/Index.h Thu Aug 2 17:32:55 2018 (r337138) +++ vendor/clang/dist/include/clang-c/Index.h Thu Aug 2 17:33:11 2018 (r337139) @@ -160,7 +160,7 @@ typedef struct CXVersion { int Major; /** * The minor version number, e.g., the '7' in '10.7.3'. This value - * will be negative if no minor version number was provided, e.g., for + * will be negative if no minor version number was provided, e.g., for * version '10'. */ int Minor; @@ -387,7 +387,7 @@ CINDEX_LINKAGE int clang_getFileUniqueID(CXFile file, * multiple inclusions, either with the conventional * \#ifndef/\#define/\#endif macro guards or with \#pragma once. */ -CINDEX_LINKAGE unsigned +CINDEX_LINKAGE unsigned clang_isFileMultipleIncludeGuarded(CXTranslationUnit tu, CXFile file); /** @@ -786,7 +786,7 @@ typedef void *CXDiagnostic; * A group of CXDiagnostics. */ typedef void *CXDiagnosticSet; - + /** * Determine the number of diagnostics in a CXDiagnosticSet. */ @@ -802,7 +802,7 @@ CINDEX_LINKAGE unsigned clang_getNumDiagnosticsInSet(C * via a call to \c clang_disposeDiagnostic(). */ CINDEX_LINKAGE CXDiagnostic clang_getDiagnosticInSet(CXDiagnosticSet Diags, - unsigned Index); + unsigned Index); /** * Describes the kind of error that occurred (if any) in a call to @@ -813,26 +813,26 @@ enum CXLoadDiag_Error { * Indicates that no error occurred. */ CXLoadDiag_None = 0, - + /** * Indicates that an unknown error occurred while attempting to * deserialize diagnostics. */ CXLoadDiag_Unknown = 1, - + /** * Indicates that the file containing the serialized diagnostics * could not be opened. */ CXLoadDiag_CannotLoad = 2, - + /** * Indicates that the serialized diagnostics file is invalid or * corrupt. */ CXLoadDiag_InvalidFile = 3 }; - + /** * Deserialize a set of diagnostics from a Clang diagnostics bitcode * file. @@ -856,7 +856,7 @@ CINDEX_LINKAGE CXDiagnosticSet clang_loadDiagnostics(c CINDEX_LINKAGE void clang_disposeDiagnosticSet(CXDiagnosticSet Diags); /** - * Retrieve the child diagnostics of a CXDiagnostic. + * Retrieve the child diagnostics of a CXDiagnostic. * * This CXDiagnosticSet does not need to be released by * clang_disposeDiagnosticSet. @@ -888,7 +888,7 @@ CINDEX_LINKAGE CXDiagnostic clang_getDiagnostic(CXTran * \param Unit the translation unit to query. */ CINDEX_LINKAGE CXDiagnosticSet - clang_getDiagnosticSetFromTU(CXTranslationUnit Unit); + clang_getDiagnosticSetFromTU(CXTranslationUnit Unit); /** * Destroy a diagnostic. @@ -934,7 +934,7 @@ enum CXDiagnosticDisplayOptions { * \c -fdiagnostics-print-source-range-info. */ CXDiagnostic_DisplaySourceRanges = 0x04, - + /** * Display the option name associated with this diagnostic, if any. * @@ -943,12 +943,12 @@ enum CXDiagnosticDisplayOptions { * \c -fdiagnostics-show-option. */ CXDiagnostic_DisplayOption = 0x08, - + /** * Display the category number associated with this diagnostic, if any. * * The category number is displayed within brackets after the diagnostic text. - * This option corresponds to the clang flag + * This option corresponds to the clang flag * \c -fdiagnostics-show-category=id. */ CXDiagnostic_DisplayCategoryId = 0x10, @@ -957,7 +957,7 @@ enum CXDiagnosticDisplayOptions { * Display the category name associated with this diagnostic, if any. * * The category name is displayed within brackets after the diagnostic text. - * This option corresponds to the clang flag + * This option corresponds to the clang flag * \c -fdiagnostics-show-category=name. */ CXDiagnostic_DisplayCategoryName = 0x20 @@ -1019,7 +1019,7 @@ CINDEX_LINKAGE CXString clang_getDiagnosticSpelling(CX * diagnostic (if any). * * \returns A string that contains the command-line option used to enable this - * warning, such as "-Wconversion" or "-pedantic". + * warning, such as "-Wconversion" or "-pedantic". */ CINDEX_LINKAGE CXString clang_getDiagnosticOption(CXDiagnostic Diag, CXString *Disable); @@ -1028,7 +1028,7 @@ CINDEX_LINKAGE CXString clang_getDiagnosticOption(CXDi * Retrieve the category number for this diagnostic. * * Diagnostics can be categorized into groups along with other, related - * diagnostics (e.g., diagnostics under the same warning flag). This routine + * diagnostics (e.g., diagnostics under the same warning flag). This routine * retrieves the category number for the given diagnostic. * * \returns The number of the category that contains this diagnostic, or zero @@ -1041,7 +1041,7 @@ CINDEX_LINKAGE unsigned clang_getDiagnosticCategory(CX * is now deprecated. Use clang_getDiagnosticCategoryText() * instead. * - * \param Category A diagnostic category number, as returned by + * \param Category A diagnostic category number, as returned by * \c clang_getDiagnosticCategory(). * * \returns The name of the given diagnostic category. @@ -1055,7 +1055,7 @@ CXString clang_getDiagnosticCategoryName(unsigned Cate * \returns The text of the given diagnostic category. */ CINDEX_LINKAGE CXString clang_getDiagnosticCategoryText(CXDiagnostic); - + /** * Determine the number of source ranges associated with the given * diagnostic. @@ -1242,9 +1242,9 @@ enum CXTranslationUnit_Flags { * intent of producing a precompiled header. */ CXTranslationUnit_Incomplete = 0x02, - + /** - * Used to indicate that the translation unit should be built with an + * Used to indicate that the translation unit should be built with an * implicit precompiled header for the preamble. * * An implicit precompiled header is used as an optimization when a @@ -1258,7 +1258,7 @@ enum CXTranslationUnit_Flags { * precompiled header to improve parsing performance. */ CXTranslationUnit_PrecompiledPreamble = 0x04, - + /** * Used to indicate that the translation unit should cache some * code-completion results with each reparse of the source file. @@ -1343,7 +1343,7 @@ enum CXTranslationUnit_Flags { * to indicate that the translation unit is likely to be reparsed many times, * either explicitly (via \c clang_reparseTranslationUnit()) or implicitly * (e.g., by code completion (\c clang_codeCompletionAt())). The returned flag - * set contains an unspecified set of optimizations (e.g., the precompiled + * set contains an unspecified set of optimizations (e.g., the precompiled * preamble) geared toward improving the performance of these routines. The * set of optimizations enabled may change from one version to the next. */ @@ -1374,7 +1374,7 @@ clang_parseTranslationUnit(CXIndex CIdx, * command-line arguments so that the compilation can be configured in the same * way that the compiler is configured on the command line. * - * \param CIdx The index object with which the translation unit will be + * \param CIdx The index object with which the translation unit will be * associated. * * \param source_filename The name of the source file to load, or NULL if the @@ -1383,7 +1383,7 @@ clang_parseTranslationUnit(CXIndex CIdx, * \param command_line_args The command-line arguments that would be * passed to the \c clang executable if it were being invoked out-of-process. * These command-line options will be parsed and will affect how the translation - * unit is parsed. Note that the following options are ignored: '-c', + * unit is parsed. Note that the following options are ignored: '-c', * '-emit-ast', '-fsyntax-only' (which is the default), and '-o \'. * * \param num_command_line_args The number of command-line arguments in @@ -1463,32 +1463,32 @@ enum CXSaveError { * Indicates that no error occurred while saving a translation unit. */ CXSaveError_None = 0, - + /** * Indicates that an unknown error occurred while attempting to save * the file. * - * This error typically indicates that file I/O failed when attempting to + * This error typically indicates that file I/O failed when attempting to * write the file. */ CXSaveError_Unknown = 1, - + /** * Indicates that errors during translation prevented this attempt * to save the translation unit. - * + * * Errors that prevent the translation unit from being saved can be * extracted using \c clang_getNumDiagnostics() and \c clang_getDiagnostic(). */ CXSaveError_TranslationErrors = 2, - + /** * Indicates that the translation unit to be saved was somehow * invalid (e.g., NULL). */ CXSaveError_InvalidTU = 3 }; - + /** * Saves a translation unit into a serialized representation of * that translation unit on disk. @@ -1509,7 +1509,7 @@ enum CXSaveError { * CXSaveTranslationUnit_XXX flags. * * \returns A value that will match one of the enumerators of the CXSaveError - * enumeration. Zero (CXSaveError_None) indicates that the translation unit was + * enumeration. Zero (CXSaveError_None) indicates that the translation unit was * saved successfully, while a non-zero value indicates that a problem occurred. */ CINDEX_LINKAGE int clang_saveTranslationUnit(CXTranslationUnit TU, @@ -1543,7 +1543,7 @@ enum CXReparse_Flags { */ CXReparse_None = 0x0 }; - + /** * Returns the set of flags that is suitable for reparsing a translation * unit. @@ -1551,7 +1551,7 @@ enum CXReparse_Flags { * The set of flags returned provide options for * \c clang_reparseTranslationUnit() by default. The returned flag * set contains an unspecified set of optimizations geared toward common uses - * of reparsing. The set of optimizations enabled may change from one version + * of reparsing. The set of optimizations enabled may change from one version * to the next. */ CINDEX_LINKAGE unsigned clang_defaultReparseOptions(CXTranslationUnit TU); @@ -1563,17 +1563,17 @@ CINDEX_LINKAGE unsigned clang_defaultReparseOptions(CX * created the given translation unit, for example because those source files * have changed (either on disk or as passed via \p unsaved_files). The * source code will be reparsed with the same command-line options as it - * was originally parsed. + * was originally parsed. * * Reparsing a translation unit invalidates all cursors and source locations * that refer into that translation unit. This makes reparsing a translation * unit semantically equivalent to destroying the translation unit and then * creating a new translation unit with the same command-line arguments. - * However, it may be more efficient to reparse a translation + * However, it may be more efficient to reparse a translation * unit using this routine. * * \param TU The translation unit whose contents will be re-parsed. The - * translation unit must originally have been built with + * translation unit must originally have been built with * \c clang_createTranslationUnitFromSourceFile(). * * \param num_unsaved_files The number of unsaved file entries in \p @@ -1584,7 +1584,7 @@ CINDEX_LINKAGE unsigned clang_defaultReparseOptions(CX * those files. The contents and name of these files (as specified by * CXUnsavedFile) are copied when necessary, so the client only needs to * guarantee their validity until the call to this function returns. - * + * * \param options A bitset of options composed of the flags in CXReparse_Flags. * The function \c clang_defaultReparseOptions() produces a default set of * options recommended for most uses, based on the translation unit. @@ -1612,8 +1612,8 @@ enum CXTUResourceUsageKind { CXTUResourceUsage_AST_SideTables = 6, CXTUResourceUsage_SourceManager_Membuffer_Malloc = 7, CXTUResourceUsage_SourceManager_Membuffer_MMap = 8, - CXTUResourceUsage_ExternalASTSource_Membuffer_Malloc = 9, - CXTUResourceUsage_ExternalASTSource_Membuffer_MMap = 10, + CXTUResourceUsage_ExternalASTSource_Membuffer_Malloc = 9, + CXTUResourceUsage_ExternalASTSource_Membuffer_MMap = 10, CXTUResourceUsage_Preprocessor = 11, CXTUResourceUsage_PreprocessingRecord = 12, CXTUResourceUsage_SourceManager_DataStructures = 13, @@ -1635,8 +1635,8 @@ const char *clang_getTUResourceUsageName(enum CXTUReso typedef struct CXTUResourceUsageEntry { /* The memory usage category. */ - enum CXTUResourceUsageKind kind; - /* Amount of resources used. + enum CXTUResourceUsageKind kind; + /* Amount of resources used. The units will depend on the resource kind. */ unsigned long amount; } CXTUResourceUsageEntry; @@ -1819,7 +1819,7 @@ enum CXCursorKind { */ CXCursor_TypeRef = 43, CXCursor_CXXBaseSpecifier = 44, - /** + /** * A reference to a class template, function template, template * template parameter, or class template partial specialization. */ @@ -1829,14 +1829,14 @@ enum CXCursorKind { */ CXCursor_NamespaceRef = 46, /** - * A reference to a member of a struct, union, or class that occurs in + * A reference to a member of a struct, union, or class that occurs in * some non-expression context, e.g., a designated initializer. */ CXCursor_MemberRef = 47, /** * A reference to a labeled statement. * - * This cursor kind is used to describe the jump to "start_over" in the + * This cursor kind is used to describe the jump to "start_over" in the * goto statement in the following example: * * \code @@ -1849,7 +1849,7 @@ enum CXCursorKind { * A label reference cursor refers to a label statement. */ CXCursor_LabelRef = 48, - + /** * A reference to a set of overloaded functions or function templates * that has not yet been resolved to a specific function or function template. @@ -1882,18 +1882,18 @@ enum CXCursorKind { * argument-dependent lookup (e.g., the "swap" function at the end of the * example). * - * The functions \c clang_getNumOverloadedDecls() and + * The functions \c clang_getNumOverloadedDecls() and * \c clang_getOverloadedDecl() can be used to retrieve the definitions * referenced by this cursor. */ CXCursor_OverloadedDeclRef = 49, - + /** - * A reference to a variable that occurs in some non-expression + * A reference to a variable that occurs in some non-expression * context, e.g., a C++ lambda capture list. */ CXCursor_VariableRef = 50, - + CXCursor_LastRef = CXCursor_VariableRef, /* Error conditions */ @@ -2153,7 +2153,7 @@ enum CXCursorKind { * \endcode */ CXCursor_LambdaExpr = 144, - + /** Objective-c Boolean Literal. */ CXCursor_ObjCBoolLiteralExpr = 145, @@ -2189,10 +2189,10 @@ enum CXCursorKind { * reported. */ CXCursor_UnexposedStmt = 200, - - /** A labelled statement in a function. + + /** A labelled statement in a function. * - * This cursor kind is used to describe the "start_over:" label statement in + * This cursor kind is used to describe the "start_over:" label statement in * the following example: * * \code @@ -2650,7 +2650,7 @@ CINDEX_LINKAGE int clang_Cursor_isNull(CXCursor cursor * Compute a hash value for the given cursor. */ CINDEX_LINKAGE unsigned clang_hashCursor(CXCursor); - + /** * Retrieve the kind of the given cursor. */ @@ -2718,7 +2718,7 @@ CINDEX_LINKAGE unsigned clang_isTranslationUnit(enum C * element, such as a preprocessor directive or macro instantiation. */ CINDEX_LINKAGE unsigned clang_isPreprocessing(enum CXCursorKind); - + /*** * Determine whether the given cursor represents a currently * unexposed piece of the AST (e.g., CXCursor_UnexposedStmt). @@ -2785,7 +2785,7 @@ CINDEX_LINKAGE enum CXVisibilityKind clang_getCursorVi * * \returns The availability of the cursor. */ -CINDEX_LINKAGE enum CXAvailabilityKind +CINDEX_LINKAGE enum CXAvailabilityKind clang_getCursorAvailability(CXCursor cursor); /** @@ -2831,10 +2831,10 @@ typedef struct CXPlatformAvailability { * * \param cursor The cursor to query. * - * \param always_deprecated If non-NULL, will be set to indicate whether the + * \param always_deprecated If non-NULL, will be set to indicate whether the * entity is deprecated on all platforms. * - * \param deprecated_message If non-NULL, will be set to the message text + * \param deprecated_message If non-NULL, will be set to the message text * provided along with the unconditional deprecation of this entity. The client * is responsible for deallocating this string. * @@ -2842,7 +2842,7 @@ typedef struct CXPlatformAvailability { * entity is unavailable on all platforms. * * \param unavailable_message If non-NULL, will be set to the message text - * provided along with the unconditional unavailability of this entity. The + * provided along with the unconditional unavailability of this entity. The * client is responsible for deallocating this string. * * \param availability If non-NULL, an array of CXPlatformAvailability instances @@ -2850,15 +2850,15 @@ typedef struct CXPlatformAvailability { * the number of platforms for which availability information is available (as * returned by this function) or \c availability_size, whichever is smaller. * - * \param availability_size The number of elements available in the + * \param availability_size The number of elements available in the * \c availability array. * * \returns The number of platforms (N) for which availability information is * available (which is unrelated to \c availability_size). * - * Note that the client is responsible for calling - * \c clang_disposeCXPlatformAvailability to free each of the - * platform-availability structures returned. There are + * Note that the client is responsible for calling + * \c clang_disposeCXPlatformAvailability to free each of the + * platform-availability structures returned. There are * \c min(N, availability_size) such structures. */ CINDEX_LINKAGE int @@ -2875,7 +2875,7 @@ clang_getCursorPlatformAvailability(CXCursor cursor, */ CINDEX_LINKAGE void clang_disposeCXPlatformAvailability(CXPlatformAvailability *availability); - + /** * Describe the "language" of the entity referred to by a cursor. */ @@ -2948,7 +2948,7 @@ CINDEX_LINKAGE unsigned clang_CXCursorSet_insert(CXCur * * The semantic parent of a cursor is the cursor that semantically contains * the given \p cursor. For many declarations, the lexical and semantic parents - * are equivalent (the lexical parent is returned by + * are equivalent (the lexical parent is returned by * \c clang_getCursorLexicalParent()). They diverge when declarations or * definitions are provided out-of-line. For example: * @@ -2983,7 +2983,7 @@ CINDEX_LINKAGE CXCursor clang_getCursorSemanticParent( * * The lexical parent of a cursor is the cursor in which the given \p cursor * was actually written. For many declarations, the lexical and semantic parents - * are equivalent (the semantic parent is returned by + * are equivalent (the semantic parent is returned by * \c clang_getCursorSemanticParent()). They diverge when declarations or * definitions are provided out-of-line. For example: * @@ -3046,18 +3046,18 @@ CINDEX_LINKAGE CXCursor clang_getCursorLexicalParent(C * \param cursor A cursor representing an Objective-C or C++ * method. This routine will compute the set of methods that this * method overrides. - * + * * \param overridden A pointer whose pointee will be replaced with a * pointer to an array of cursors, representing the set of overridden * methods. If there are no overridden methods, the pointee will be - * set to NULL. The pointee must be freed via a call to + * set to NULL. The pointee must be freed via a call to * \c clang_disposeOverriddenCursors(). * * \param num_overridden A pointer to the number of overridden * functions, will be set to the number of overridden functions in the * array pointed to by \p overridden. */ -CINDEX_LINKAGE void clang_getOverriddenCursors(CXCursor cursor, +CINDEX_LINKAGE void clang_getOverriddenCursors(CXCursor cursor, CXCursor **overridden, unsigned *num_overridden); @@ -3072,7 +3072,7 @@ CINDEX_LINKAGE void clang_disposeOverriddenCursors(CXC * cursor. */ CINDEX_LINKAGE CXFile clang_getIncludedFile(CXCursor cursor); - + /** * @} */ @@ -3133,7 +3133,7 @@ CINDEX_LINKAGE CXSourceRange clang_getCursorExtent(CXC /** * @} */ - + /** * \defgroup CINDEX_TYPES Type information for CXCursors * @@ -3582,7 +3582,7 @@ CINDEX_LINKAGE CXString clang_getDeclObjCTypeEncoding( /** * Returns the Objective-C type encoding for the specified CXType. */ -CINDEX_LINKAGE CXString clang_Type_getObjCEncoding(CXType type); +CINDEX_LINKAGE CXString clang_Type_getObjCEncoding(CXType type); /** * Retrieve the spelling of a given CXTypeKind. @@ -3842,7 +3842,7 @@ CINDEX_LINKAGE unsigned clang_Cursor_isBitField(CXCurs * CX_CXXBaseSpecifier is virtual. */ CINDEX_LINKAGE unsigned clang_isVirtualBase(CXCursor); - + /** * Represents the C++ access control level to a base class for a * cursor with kind CX_CXXBaseSpecifier. @@ -3887,7 +3887,7 @@ enum CX_StorageClass { CINDEX_LINKAGE enum CX_StorageClass clang_Cursor_getStorageClass(CXCursor); /** - * Determine the number of overloaded declarations referenced by a + * Determine the number of overloaded declarations referenced by a * \c CXCursor_OverloadedDeclRef cursor. * * \param cursor The cursor whose overloaded declarations are being queried. @@ -3906,18 +3906,18 @@ CINDEX_LINKAGE unsigned clang_getNumOverloadedDecls(CX * \param index The zero-based index into the set of overloaded declarations in * the cursor. * - * \returns A cursor representing the declaration referenced by the given - * \c cursor at the specified \c index. If the cursor does not have an + * \returns A cursor representing the declaration referenced by the given + * \c cursor at the specified \c index. If the cursor does not have an * associated set of overloaded declarations, or if the index is out of bounds, * returns \c clang_getNullCursor(); */ -CINDEX_LINKAGE CXCursor clang_getOverloadedDecl(CXCursor cursor, +CINDEX_LINKAGE CXCursor clang_getOverloadedDecl(CXCursor cursor, unsigned index); - + /** * @} */ - + /** * \defgroup CINDEX_ATTRIBUTES Information for attributes * @@ -4021,7 +4021,7 @@ CINDEX_LINKAGE unsigned clang_visitChildren(CXCursor p * The visitor should return one of the \c CXChildVisitResult values * to direct clang_visitChildrenWithBlock(). */ -typedef enum CXChildVisitResult +typedef enum CXChildVisitResult (^CXCursorVisitorBlock)(CXCursor cursor, CXCursor parent); /** @@ -4109,10 +4109,10 @@ CINDEX_LINKAGE CXString clang_getCursorSpelling(CXCurs * Most of the times there is only one range for the complete spelling but for * Objective-C methods and Objective-C message expressions, there are multiple * pieces for each selector identifier. - * + * * \param pieceIndex the index of the spelling name piece. If this is greater * than the actual number of pieces, it will return a NULL (invalid) range. - * + * * \param options Reserved. */ CINDEX_LINKAGE CXSourceRange clang_Cursor_getSpellingNameRange(CXCursor, @@ -4206,11 +4206,11 @@ CINDEX_LINKAGE CXString clang_getCursorPrettyPrinted(C * Retrieve the display name for the entity referenced by this cursor. * * The display name contains extra information that helps identify the cursor, - * such as the parameters of a function or template or the arguments of a + * such as the parameters of a function or template or the arguments of a * class template specialization. */ CINDEX_LINKAGE CXString clang_getCursorDisplayName(CXCursor); - + /** For a cursor that is a reference, retrieve a cursor representing the * entity that it references. * @@ -4274,10 +4274,10 @@ CINDEX_LINKAGE unsigned clang_isCursorDefinition(CXCur * }; * \endcode * - * The declarations and the definition of \c X are represented by three - * different cursors, all of which are declarations of the same underlying + * The declarations and the definition of \c X are represented by three + * different cursors, all of which are declarations of the same underlying * entity. One of these cursor is considered the "canonical" cursor, which - * is effectively the representative for the underlying entity. One can + * is effectively the representative for the underlying entity. One can * determine if two cursors are declarations of the same underlying entity by * comparing their canonical cursors. * @@ -4301,11 +4301,11 @@ CINDEX_LINKAGE int clang_Cursor_getObjCSelectorIndex(C /** * Given a cursor pointing to a C++ method call or an Objective-C * message, returns non-zero if the method/message is "dynamic", meaning: - * + * * For a C++ method: the call is virtual. * For an Objective-C message: the receiver is an object instance, not 'super' * or a specific class. - * + * * If the method/message is "static" or the cursor does not point to a * method/message, it will return zero. */ @@ -4575,7 +4575,7 @@ CINDEX_LINKAGE unsigned clang_CXXMethod_isDefaulted(CX CINDEX_LINKAGE unsigned clang_CXXMethod_isPureVirtual(CXCursor C); /** - * Determine if a C++ member function or member function template is + * Determine if a C++ member function or member function template is * declared 'static'. */ CINDEX_LINKAGE unsigned clang_CXXMethod_isStatic(CXCursor C); @@ -4622,16 +4622,16 @@ CINDEX_LINKAGE unsigned clang_CXXMethod_isConst(CXCurs * \c CXCursor_NoDeclFound. */ CINDEX_LINKAGE enum CXCursorKind clang_getTemplateCursorKind(CXCursor C); - + /** * Given a cursor that may represent a specialization or instantiation * of a template, retrieve the cursor that represents the template that it * specializes or from which it was instantiated. * - * This routine determines the template involved both for explicit + * This routine determines the template involved both for explicit * specializations of templates and for implicit instantiations of the template, * both of which are referred to as "specializations". For a class template - * specialization (e.g., \c std::vector), this routine will return + * specialization (e.g., \c std::vector), this routine will return * either the primary template (\c std::vector) or, if the specialization was * instantiated from a class template partial specialization, the class template * partial specialization. For a class template partial specialization and a @@ -4639,7 +4639,7 @@ CINDEX_LINKAGE enum CXCursorKind clang_getTemplateCurs * this routine will return the specialized template. * * For members of a class template (e.g., member functions, member classes, or - * static data members), returns the specialized or instantiated member. + * static data members), returns the specialized or instantiated member. * Although not strictly "templates" in the C++ language, members of class * templates have the same notions of specializations and instantiations that * templates do, so this routine treats them similarly. @@ -4647,7 +4647,7 @@ CINDEX_LINKAGE enum CXCursorKind clang_getTemplateCurs * \param C A cursor that may be a specialization of a template or a member * of a template. * - * \returns If the given cursor is a specialization or instantiation of a + * \returns If the given cursor is a specialization or instantiation of a * template or a member thereof, the template or member that it specializes or * from which it was instantiated. Otherwise, returns a NULL cursor. */ @@ -4659,11 +4659,11 @@ CINDEX_LINKAGE CXCursor clang_getSpecializedCursorTemp * * \param C A cursor pointing to a member reference, a declaration reference, or * an operator call. - * \param NameFlags A bitset with three independent flags: + * \param NameFlags A bitset with three independent flags: * CXNameRange_WantQualifier, CXNameRange_WantTemplateArgs, and * CXNameRange_WantSinglePiece. - * \param PieceIndex For contiguous names or when passing the flag - * CXNameRange_WantSinglePiece, only one piece with index 0 is + * \param PieceIndex For contiguous names or when passing the flag + * CXNameRange_WantSinglePiece, only one piece with index 0 is * available. When the CXNameRange_WantSinglePiece flag is not passed for a * non-contiguous names, this index can be used to retrieve the individual * pieces of the name. See also CXNameRange_WantSinglePiece. @@ -4672,7 +4672,7 @@ CINDEX_LINKAGE CXCursor clang_getSpecializedCursorTemp * name, or if the PieceIndex is out-of-range, a null-cursor will be returned. */ CINDEX_LINKAGE CXSourceRange clang_getCursorReferenceNameRange(CXCursor C, - unsigned NameFlags, + unsigned NameFlags, unsigned PieceIndex); enum CXNameRefFlags { @@ -4681,7 +4681,7 @@ enum CXNameRefFlags { * range. */ CXNameRange_WantQualifier = 0x1, - + /** * Include the explicit template arguments, e.g. \ in x.f, * in the range. @@ -4700,7 +4700,7 @@ enum CXNameRefFlags { */ CXNameRange_WantSinglePiece = 0x4 }; - + /** * @} */ @@ -5153,7 +5153,7 @@ clang_getNumCompletionChunks(CXCompletionString comple /** * Determine the priority of this code completion. * - * The priority of a code completion indicates how likely it is that this + * The priority of a code completion indicates how likely it is that this * particular completion is the completion that the user will select. The * priority is selected by various internal heuristics. * @@ -5164,7 +5164,7 @@ clang_getNumCompletionChunks(CXCompletionString comple */ CINDEX_LINKAGE unsigned clang_getCompletionPriority(CXCompletionString completion_string); - + /** * Determine the availability of the entity that this code-completion *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:17 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 550FB106D6E7; Thu, 2 Aug 2018 17:33:17 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F16C97233D; Thu, 2 Aug 2018 17:33:16 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A155C14D30; Thu, 2 Aug 2018 17:33:16 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXGdw029371; Thu, 2 Aug 2018 17:33:16 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXGwx029370; Thu, 2 Aug 2018 17:33:16 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXGwx029370@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337140 - vendor/clang/clang-trunk-r338536 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/clang/clang-trunk-r338536 X-SVN-Commit-Revision: 337140 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:17 -0000 Author: dim Date: Thu Aug 2 17:33:16 2018 New Revision: 337140 URL: https://svnweb.freebsd.org/changeset/base/337140 Log: Tag clang trunk r338536. Added: vendor/clang/clang-trunk-r338536/ - copied from r337139, vendor/clang/dist/ From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47096106D753; Thu, 2 Aug 2018 17:33:29 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 57BF572505; Thu, 2 Aug 2018 17:33:27 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0654114D32; Thu, 2 Aug 2018 17:33:27 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXQww029457; Thu, 2 Aug 2018 17:33:26 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXJNY029420; Thu, 2 Aug 2018 17:33:19 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXJNY029420@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337141 - in vendor/compiler-rt/dist: lib/asan lib/asan/tests lib/builtins lib/profile lib/sanitizer_common lib/ubsan lib/ubsan_minimal lib/xray lib/xray/tests/unit test/asan/TestCases ... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/compiler-rt/dist: lib/asan lib/asan/tests lib/builtins lib/profile lib/sanitizer_common lib/ubsan lib/ubsan_minimal lib/xray lib/xray/tests/unit test/asan/TestCases test/fuzzer test/profile ... X-SVN-Commit-Revision: 337141 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:29 -0000 Author: dim Date: Thu Aug 2 17:33:19 2018 New Revision: 337141 URL: https://svnweb.freebsd.org/changeset/base/337141 Log: Vendor import of compiler-rt trunk r338536: https://llvm.org/svn/llvm-project/compiler-rt/trunk@338536 Added: vendor/compiler-rt/dist/test/fuzzer/ImplicitIntegerTruncationTest.cpp (contents, props changed) vendor/compiler-rt/dist/test/fuzzer/fuzzer-implicit-integer-truncation.test vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-fork.c (contents, props changed) vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-fork.c.gcov vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch1.c (contents, props changed) vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch1.c.gcov vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch2.c (contents, props changed) vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch2.c.gcov vendor/compiler-rt/dist/test/profile/Posix/ vendor/compiler-rt/dist/test/profile/Posix/instrprof-gcov-fork.test vendor/compiler-rt/dist/test/profile/instrprof-gcov-switch.test vendor/compiler-rt/dist/test/profile/instrprof-set-dir-mode.c (contents, props changed) vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitCast/ vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/ vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation-blacklist.c (contents, props changed) vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation-summary.cpp (contents, props changed) vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation.c (contents, props changed) vendor/compiler-rt/dist/test/ubsan_minimal/TestCases/implicit-integer-truncation.c (contents, props changed) Modified: vendor/compiler-rt/dist/lib/asan/asan_mapping.h vendor/compiler-rt/dist/lib/asan/tests/asan_test.cc vendor/compiler-rt/dist/lib/builtins/CMakeLists.txt vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.c vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.h vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_mutex.h vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_win.cc vendor/compiler-rt/dist/lib/ubsan/ubsan_checks.inc vendor/compiler-rt/dist/lib/ubsan/ubsan_diag.h vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.cc vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.h vendor/compiler-rt/dist/lib/ubsan/ubsan_interface.inc vendor/compiler-rt/dist/lib/ubsan_minimal/ubsan_minimal_handlers.cc vendor/compiler-rt/dist/lib/xray/tests/unit/profile_collector_test.cc vendor/compiler-rt/dist/lib/xray/xray_buffer_queue.cc vendor/compiler-rt/dist/lib/xray/xray_profile_collector.cc vendor/compiler-rt/dist/lib/xray/xray_profiling.cc vendor/compiler-rt/dist/test/asan/TestCases/intercept-rethrow-exception.cc vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-multi-threaded.cc vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-single-threaded.cc Modified: vendor/compiler-rt/dist/lib/asan/asan_mapping.h ============================================================================== --- vendor/compiler-rt/dist/lib/asan/asan_mapping.h Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/asan/asan_mapping.h Thu Aug 2 17:33:19 2018 (r337141) @@ -122,7 +122,7 @@ // || `[0x400000000000, 0x47ffffffffff]` || LowShadow || // || `[0x000000000000, 0x3fffffffffff]` || LowMem || // -// Shadow mapping on NerBSD/i386 with SHADOW_OFFSET == 0x40000000: +// Shadow mapping on NetBSD/i386 with SHADOW_OFFSET == 0x40000000: // || `[0x60000000, 0xfffff000]` || HighMem || // || `[0x4c000000, 0x5fffffff]` || HighShadow || // || `[0x48000000, 0x4bffffff]` || ShadowGap || Modified: vendor/compiler-rt/dist/lib/asan/tests/asan_test.cc ============================================================================== --- vendor/compiler-rt/dist/lib/asan/tests/asan_test.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/asan/tests/asan_test.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -963,7 +963,7 @@ TEST(AddressSanitizer, ThreadNamesTest) { #if ASAN_NEEDS_SEGV TEST(AddressSanitizer, ShadowGapTest) { #if SANITIZER_WORDSIZE == 32 - char *addr = (char*)0x22000000; + char *addr = (char*)0x23000000; #else # if defined(__powerpc64__) char *addr = (char*)0x024000800000; Modified: vendor/compiler-rt/dist/lib/builtins/CMakeLists.txt ============================================================================== --- vendor/compiler-rt/dist/lib/builtins/CMakeLists.txt Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/builtins/CMakeLists.txt Thu Aug 2 17:33:19 2018 (r337141) @@ -567,6 +567,15 @@ else () set(_arch "arm") endif() + # For ARM archs, exclude any VFP builtins if VFP is not supported + if (${arch} MATCHES "^(arm|armhf|armv7|armv7s|armv7k|armv7m|armv7em)$") + string(REPLACE ";" " " _TARGET_${arch}_CFLAGS "${TARGET_${arch}_CFLAGS}") + check_compile_definition(__VFP_FP__ "${CMAKE_C_FLAGS} ${_TARGET_${arch}_CFLAGS}" COMPILER_RT_HAS_${arch}_VFP) + if(NOT COMPILER_RT_HAS_${arch}_VFP) + list(REMOVE_ITEM ${arch}_SOURCES ${arm_Thumb1_VFPv2_SOURCES} ${arm_Thumb1_SjLj_EH_SOURCES}) + endif() + endif() + # Filter out generic versions of routines that are re-implemented in # architecture specific manner. This prevents multiple definitions of the # same symbols, making the symbol selection non-deterministic. Modified: vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.c ============================================================================== --- vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.c Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.c Thu Aug 2 17:33:19 2018 (r337141) @@ -35,6 +35,8 @@ #include "InstrProfiling.h" #include "InstrProfilingUtil.h" +COMPILER_RT_WEAK unsigned lprofDirMode = 0755; + COMPILER_RT_VISIBILITY void __llvm_profile_recursive_mkdir(char *path) { int i; @@ -47,11 +49,18 @@ void __llvm_profile_recursive_mkdir(char *path) { #ifdef _WIN32 _mkdir(path); #else - mkdir(path, 0755); /* Some of these will fail, ignore it. */ + /* Some of these will fail, ignore it. */ + mkdir(path, __llvm_profile_get_dir_mode()); #endif path[i] = save; } } + +COMPILER_RT_VISIBILITY +void __llvm_profile_set_dir_mode(unsigned Mode) { lprofDirMode = Mode; } + +COMPILER_RT_VISIBILITY +unsigned __llvm_profile_get_dir_mode(void) { return lprofDirMode; } #if COMPILER_RT_HAS_ATOMICS != 1 COMPILER_RT_VISIBILITY Modified: vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.h ============================================================================== --- vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.h Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/profile/InstrProfilingUtil.h Thu Aug 2 17:33:19 2018 (r337141) @@ -16,6 +16,12 @@ /*! \brief Create a directory tree. */ void __llvm_profile_recursive_mkdir(char *Pathname); +/*! Set the mode used when creating profile directories. */ +void __llvm_profile_set_dir_mode(unsigned Mode); + +/*! Return the directory creation mode. */ +unsigned __llvm_profile_get_dir_mode(void); + int lprofLockFd(int fd); int lprofUnlockFd(int fd); Modified: vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_mutex.h ============================================================================== --- vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_mutex.h Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_mutex.h Thu Aug 2 17:33:19 2018 (r337141) @@ -73,13 +73,8 @@ class SpinMutex : public StaticSpinMutex { class BlockingMutex { public: -#if SANITIZER_WINDOWS - // Windows does not currently support LinkerInitialized - explicit BlockingMutex(LinkerInitialized); -#else explicit constexpr BlockingMutex(LinkerInitialized) - : opaque_storage_ {0, }, owner_(0) {} -#endif + : opaque_storage_ {0, }, owner_ {0} {} BlockingMutex(); void Lock(); void Unlock(); Modified: vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_win.cc ============================================================================== --- vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_win.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/sanitizer_common/sanitizer_win.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -767,43 +767,22 @@ void *internal_start_thread(void (*func)(void *arg), v void internal_join_thread(void *th) { } // ---------------------- BlockingMutex ---------------- {{{1 -const uptr LOCK_UNINITIALIZED = 0; -const uptr LOCK_READY = (uptr)-1; -BlockingMutex::BlockingMutex(LinkerInitialized li) { - // FIXME: see comments in BlockingMutex::Lock() for the details. - CHECK(li == LINKER_INITIALIZED || owner_ == LOCK_UNINITIALIZED); - - CHECK(sizeof(CRITICAL_SECTION) <= sizeof(opaque_storage_)); - InitializeCriticalSection((LPCRITICAL_SECTION)opaque_storage_); - owner_ = LOCK_READY; -} - BlockingMutex::BlockingMutex() { - CHECK(sizeof(CRITICAL_SECTION) <= sizeof(opaque_storage_)); - InitializeCriticalSection((LPCRITICAL_SECTION)opaque_storage_); - owner_ = LOCK_READY; + CHECK(sizeof(SRWLOCK) <= sizeof(opaque_storage_)); + internal_memset(this, 0, sizeof(*this)); } void BlockingMutex::Lock() { - if (owner_ == LOCK_UNINITIALIZED) { - // FIXME: hm, global BlockingMutex objects are not initialized?!? - // This might be a side effect of the clang+cl+link Frankenbuild... - new(this) BlockingMutex((LinkerInitialized)(LINKER_INITIALIZED + 1)); - - // FIXME: If it turns out the linker doesn't invoke our - // constructors, we should probably manually Lock/Unlock all the global - // locks while we're starting in one thread to avoid double-init races. - } - EnterCriticalSection((LPCRITICAL_SECTION)opaque_storage_); - CHECK_EQ(owner_, LOCK_READY); + AcquireSRWLockExclusive((PSRWLOCK)opaque_storage_); + CHECK_EQ(owner_, 0); owner_ = GetThreadSelf(); } void BlockingMutex::Unlock() { - CHECK_EQ(owner_, GetThreadSelf()); - owner_ = LOCK_READY; - LeaveCriticalSection((LPCRITICAL_SECTION)opaque_storage_); + CheckLocked(); + owner_ = 0; + ReleaseSRWLockExclusive((PSRWLOCK)opaque_storage_); } void BlockingMutex::CheckLocked() { Modified: vendor/compiler-rt/dist/lib/ubsan/ubsan_checks.inc ============================================================================== --- vendor/compiler-rt/dist/lib/ubsan/ubsan_checks.inc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/ubsan/ubsan_checks.inc Thu Aug 2 17:33:19 2018 (r337141) @@ -30,6 +30,8 @@ UBSAN_CHECK(IntegerDivideByZero, "integer-divide-by-ze "integer-divide-by-zero") UBSAN_CHECK(FloatDivideByZero, "float-divide-by-zero", "float-divide-by-zero") UBSAN_CHECK(InvalidBuiltin, "invalid-builtin-use", "invalid-builtin-use") +UBSAN_CHECK(ImplicitIntegerTruncation, "implicit-integer-truncation", + "implicit-integer-truncation") UBSAN_CHECK(InvalidShiftBase, "invalid-shift-base", "shift-base") UBSAN_CHECK(InvalidShiftExponent, "invalid-shift-exponent", "shift-exponent") UBSAN_CHECK(OutOfBoundsIndex, "out-of-bounds-index", "bounds") Modified: vendor/compiler-rt/dist/lib/ubsan/ubsan_diag.h ============================================================================== --- vendor/compiler-rt/dist/lib/ubsan/ubsan_diag.h Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/ubsan/ubsan_diag.h Thu Aug 2 17:33:19 2018 (r337141) @@ -178,7 +178,7 @@ class Diag { (public) }; private: - static const unsigned MaxArgs = 5; + static const unsigned MaxArgs = 8; static const unsigned MaxRanges = 1; /// The arguments which have been added to this diagnostic so far. Modified: vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.cc ============================================================================== --- vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -451,6 +451,49 @@ void __ubsan::__ubsan_handle_load_invalid_value_abort( Die(); } +static void handleImplicitConversion(ImplicitConversionData *Data, + ReportOptions Opts, ValueHandle Src, + ValueHandle Dst) { + SourceLocation Loc = Data->Loc.acquire(); + ErrorType ET = ErrorType::GenericUB; + + switch (Data->Kind) { + case ICCK_IntegerTruncation: + ET = ErrorType::ImplicitIntegerTruncation; + break; + } + + if (ignoreReport(Loc, Opts, ET)) + return; + + const TypeDescriptor &SrcTy = Data->FromType; + const TypeDescriptor &DstTy = Data->ToType; + + ScopedReport R(Opts, Loc, ET); + + // FIXME: is it possible to dump the values as hex with fixed width? + + Diag(Loc, DL_Error, ET, + "implicit conversion from type %0 of value %1 (%2-bit, %3signed) to " + "type %4 changed the value to %5 (%6-bit, %7signed)") + << SrcTy << Value(SrcTy, Src) << SrcTy.getIntegerBitWidth() + << (SrcTy.isSignedIntegerTy() ? "" : "un") << DstTy << Value(DstTy, Dst) + << DstTy.getIntegerBitWidth() << (DstTy.isSignedIntegerTy() ? "" : "un"); +} + +void __ubsan::__ubsan_handle_implicit_conversion(ImplicitConversionData *Data, + ValueHandle Src, + ValueHandle Dst) { + GET_REPORT_OPTIONS(false); + handleImplicitConversion(Data, Opts, Src, Dst); +} +void __ubsan::__ubsan_handle_implicit_conversion_abort( + ImplicitConversionData *Data, ValueHandle Src, ValueHandle Dst) { + GET_REPORT_OPTIONS(true); + handleImplicitConversion(Data, Opts, Src, Dst); + Die(); +} + static void handleInvalidBuiltin(InvalidBuiltinData *Data, ReportOptions Opts) { SourceLocation Loc = Data->Loc.acquire(); ErrorType ET = ErrorType::InvalidBuiltin; Modified: vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.h ============================================================================== --- vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.h Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/ubsan/ubsan_handlers.h Thu Aug 2 17:33:19 2018 (r337141) @@ -122,6 +122,23 @@ struct InvalidValueData { /// \brief Handle a load of an invalid value for the type. RECOVERABLE(load_invalid_value, InvalidValueData *Data, ValueHandle Val) +/// Known implicit conversion check kinds. +/// Keep in sync with the enum of the same name in CGExprScalar.cpp +enum ImplicitConversionCheckKind : unsigned char { + ICCK_IntegerTruncation = 0, +}; + +struct ImplicitConversionData { + SourceLocation Loc; + const TypeDescriptor &FromType; + const TypeDescriptor &ToType; + /* ImplicitConversionCheckKind */ unsigned char Kind; +}; + +/// \brief Implict conversion that changed the value. +RECOVERABLE(implicit_conversion, ImplicitConversionData *Data, ValueHandle Src, + ValueHandle Dst) + /// Known builtin check kinds. /// Keep in sync with the enum of the same name in CodeGenFunction.h enum BuiltinCheckKind : unsigned char { Modified: vendor/compiler-rt/dist/lib/ubsan/ubsan_interface.inc ============================================================================== --- vendor/compiler-rt/dist/lib/ubsan/ubsan_interface.inc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/ubsan/ubsan_interface.inc Thu Aug 2 17:33:19 2018 (r337141) @@ -22,6 +22,8 @@ INTERFACE_FUNCTION(__ubsan_handle_float_cast_overflow) INTERFACE_FUNCTION(__ubsan_handle_float_cast_overflow_abort) INTERFACE_FUNCTION(__ubsan_handle_function_type_mismatch) INTERFACE_FUNCTION(__ubsan_handle_function_type_mismatch_abort) +INTERFACE_FUNCTION(__ubsan_handle_implicit_conversion) +INTERFACE_FUNCTION(__ubsan_handle_implicit_conversion_abort) INTERFACE_FUNCTION(__ubsan_handle_invalid_builtin) INTERFACE_FUNCTION(__ubsan_handle_invalid_builtin_abort) INTERFACE_FUNCTION(__ubsan_handle_load_invalid_value) Modified: vendor/compiler-rt/dist/lib/ubsan_minimal/ubsan_minimal_handlers.cc ============================================================================== --- vendor/compiler-rt/dist/lib/ubsan_minimal/ubsan_minimal_handlers.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/ubsan_minimal/ubsan_minimal_handlers.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -109,6 +109,7 @@ HANDLER(float_cast_overflow, "float-cast-overflow") HANDLER(load_invalid_value, "load-invalid-value") HANDLER(invalid_builtin, "invalid-builtin") HANDLER(function_type_mismatch, "function-type-mismatch") +HANDLER(implicit_conversion, "implicit-conversion") HANDLER(nonnull_arg, "nonnull-arg") HANDLER(nonnull_return, "nonnull-return") HANDLER(nullability_arg, "nullability-arg") Modified: vendor/compiler-rt/dist/lib/xray/tests/unit/profile_collector_test.cc ============================================================================== --- vendor/compiler-rt/dist/lib/xray/tests/unit/profile_collector_test.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/xray/tests/unit/profile_collector_test.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -15,6 +15,8 @@ #include "xray_profile_collector.h" #include "xray_profiling_flags.h" #include +#include +#include #include #include #include @@ -24,6 +26,29 @@ namespace { static constexpr auto kHeaderSize = 16u; +constexpr uptr ExpectedProfilingVersion = 0x20180424; + +struct ExpectedProfilingFileHeader { + const u64 MagicBytes = 0x7872617970726f66; // Identifier for XRay profiling + // files 'xrayprof' in hex. + const u64 Version = ExpectedProfilingVersion; + u64 Timestamp = 0; + u64 PID = 0; +}; + +void ValidateFileHeaderBlock(XRayBuffer B) { + ASSERT_NE(static_cast(B.Data), nullptr); + ASSERT_EQ(B.Size, sizeof(ExpectedProfilingFileHeader)); + typename std::aligned_storage::type + FileHeaderStorage; + ExpectedProfilingFileHeader ExpectedHeader; + std::memcpy(&FileHeaderStorage, B.Data, B.Size); + auto &FileHeader = + *reinterpret_cast(&FileHeaderStorage); + ASSERT_EQ(ExpectedHeader.MagicBytes, FileHeader.MagicBytes); + ASSERT_EQ(ExpectedHeader.Version, FileHeader.Version); +} + void ValidateBlock(XRayBuffer B) { profilingFlags()->setDefaults(); ASSERT_NE(static_cast(B.Data), nullptr); @@ -107,9 +132,13 @@ TEST(profileCollectorServiceTest, PostSerializeCollect // Then we serialize the data. profileCollectorService::serialize(); - // Then we go through a single buffer to see whether we're getting the data we - // expect. + // Then we go through two buffers to see whether we're getting the data we + // expect. The first block must always be as large as a file header, which + // will have a fixed size. auto B = profileCollectorService::nextBuffer({nullptr, 0}); + ValidateFileHeaderBlock(B); + + B = profileCollectorService::nextBuffer(B); ValidateBlock(B); u32 BlockSize; u32 BlockNum; @@ -169,6 +198,9 @@ TEST(profileCollectorServiceTest, PostSerializeCollect // Ensure that we see two buffers. auto B = profileCollectorService::nextBuffer({nullptr, 0}); + ValidateFileHeaderBlock(B); + + B = profileCollectorService::nextBuffer(B); ValidateBlock(B); B = profileCollectorService::nextBuffer(B); Modified: vendor/compiler-rt/dist/lib/xray/xray_buffer_queue.cc ============================================================================== --- vendor/compiler-rt/dist/lib/xray/xray_buffer_queue.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/xray/xray_buffer_queue.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -13,17 +13,50 @@ // //===----------------------------------------------------------------------===// #include "xray_buffer_queue.h" -#include "sanitizer_common/sanitizer_allocator_internal.h" #include "sanitizer_common/sanitizer_common.h" #include "sanitizer_common/sanitizer_libc.h" +#include "sanitizer_common/sanitizer_posix.h" #include +#include +#ifndef MAP_NORESERVE +// no-op on NetBSD (at least), unsupported flag on FreeBSD +#define MAP_NORESERVE 0 +#endif + using namespace __xray; using namespace __sanitizer; +template static T *allocRaw(size_t N) { + // TODO: Report errors? + // We use MAP_NORESERVE on platforms where it's supported to ensure that the + // pages we're allocating for XRay never end up in pages that can be swapped + // in/out. We're doing this because for FDR mode, we want to ensure that + // writes to the buffers stay resident in memory to prevent XRay itself from + // causing swapping/thrashing. + // + // In the case when XRay pages cannot be swapped in/out or there's not enough + // RAM to back these pages, we're willing to cause a segmentation fault + // instead of introducing latency in the measurement. We assume here that + // there are enough pages that are swappable in/out outside of the buffers + // being used by FDR mode (which are bounded and configurable anyway) to allow + // us to keep using always-resident memory. + // + // TODO: Make this configurable? + void *A = reinterpret_cast( + internal_mmap(NULL, N * sizeof(T), PROT_WRITE | PROT_READ, + MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE, -1, 0)); + return (A == MAP_FAILED) ? nullptr : reinterpret_cast(A); +} + +template static void deallocRaw(T *ptr, size_t N) { + // TODO: Report errors? + if (ptr != nullptr) + internal_munmap(ptr, N); +} + template static T *initArray(size_t N) { - auto A = reinterpret_cast( - InternalAlloc(N * sizeof(T), nullptr, kCacheLineSize)); + auto A = allocRaw(N); if (A != nullptr) while (N > 0) new (A + (--N)) T(); @@ -42,19 +75,19 @@ BufferQueue::BufferQueue(size_t B, size_t N, bool &Suc // Clean up the buffers we've already allocated. for (auto B = Buffers, E = Buffers + BufferCount; B != E; ++B) B->~BufferRep(); - InternalFree(Buffers); + deallocRaw(Buffers, N); Success = false; return; }; for (size_t i = 0; i < N; ++i) { auto &T = Buffers[i]; - void *Tmp = InternalAlloc(BufferSize, nullptr, 64); + void *Tmp = allocRaw(BufferSize); if (Tmp == nullptr) { Success = false; return; } - void *Extents = InternalAlloc(sizeof(BufferExtents), nullptr, 64); + auto *Extents = allocRaw(1); if (Extents == nullptr) { Success = false; return; @@ -62,7 +95,7 @@ BufferQueue::BufferQueue(size_t B, size_t N, bool &Suc auto &Buf = T.Buff; Buf.Data = Tmp; Buf.Size = B; - Buf.Extents = reinterpret_cast(Extents); + Buf.Extents = Extents; OwnedBuffers[i] = Tmp; } Success = true; @@ -128,11 +161,11 @@ BufferQueue::~BufferQueue() { for (auto I = Buffers, E = Buffers + BufferCount; I != E; ++I) { auto &T = *I; auto &Buf = T.Buff; - InternalFree(Buf.Data); - InternalFree(Buf.Extents); + deallocRaw(Buf.Data, Buf.Size); + deallocRaw(Buf.Extents, 1); } for (auto B = Buffers, E = Buffers + BufferCount; B != E; ++B) B->~BufferRep(); - InternalFree(Buffers); - InternalFree(OwnedBuffers); + deallocRaw(Buffers, BufferCount); + deallocRaw(OwnedBuffers, BufferCount); } Modified: vendor/compiler-rt/dist/lib/xray/xray_profile_collector.cc ============================================================================== --- vendor/compiler-rt/dist/lib/xray/xray_profile_collector.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/xray/xray_profile_collector.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -37,6 +37,19 @@ struct ProfileBuffer { size_t Size; }; +// Current version of the profile format. +constexpr u64 XRayProfilingVersion = 0x20180424; + +// Identifier for XRay profiling files 'xrayprof' in hex. +constexpr u64 XRayMagicBytes = 0x7872617970726f66; + +struct XRayProfilingFileHeader { + const u64 MagicBytes = XRayMagicBytes; + const u64 Version = XRayProfilingVersion; + u64 Timestamp = 0; // System time in nanoseconds. + u64 PID = 0; // Process ID. +}; + struct BlockHeader { u32 BlockSize; u32 BlockNum; @@ -302,7 +315,22 @@ XRayBuffer nextBuffer(XRayBuffer B) { if (ProfileBuffers == nullptr || ProfileBuffers->Size() == 0) return {nullptr, 0}; - if (B.Data == nullptr) + static pthread_once_t Once = PTHREAD_ONCE_INIT; + static typename std::aligned_storage::type + FileHeaderStorage; + pthread_once(&Once, + +[] { new (&FileHeaderStorage) XRayProfilingFileHeader{}; }); + + if (UNLIKELY(B.Data == nullptr)) { + // The first buffer should always contain the file header information. + auto &FileHeader = + *reinterpret_cast(&FileHeaderStorage); + FileHeader.Timestamp = NanoTime(); + FileHeader.PID = internal_getpid(); + return {&FileHeaderStorage, sizeof(XRayProfilingFileHeader)}; + } + + if (UNLIKELY(B.Data == &FileHeaderStorage)) return {(*ProfileBuffers)[0].Data, (*ProfileBuffers)[0].Size}; BlockHeader Header; Modified: vendor/compiler-rt/dist/lib/xray/xray_profiling.cc ============================================================================== --- vendor/compiler-rt/dist/lib/xray/xray_profiling.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/lib/xray/xray_profiling.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -32,16 +32,6 @@ namespace __xray { namespace { -constexpr uptr XRayProfilingVersion = 0x20180424; - -struct XRayProfilingFileHeader { - const u64 MagicBytes = 0x7872617970726f66; // Identifier for XRay profiling - // files 'xrayprof' in hex. - const uptr Version = XRayProfilingVersion; - uptr Timestamp = 0; // System time in nanoseconds. - uptr PID = 0; // Process ID. -}; - atomic_sint32_t ProfilerLogFlushStatus = { XRayLogFlushStatus::XRAY_LOG_NOT_FLUSHING}; @@ -144,14 +134,7 @@ XRayLogFlushStatus profilingFlush() XRAY_NEVER_INSTRUM if (Verbosity()) Report("profiling: Failed to flush to file, dropping data.\n"); } else { - XRayProfilingFileHeader Header; - Header.Timestamp = NanoTime(); - Header.PID = internal_getpid(); - retryingWriteAll(Fd, reinterpret_cast(&Header), - reinterpret_cast(&Header) + - sizeof(Header)); - - // Now for each of the threads, write out the profile data as we would + // Now for each of the buffers, write out the profile data as we would // see it in memory, verbatim. while (B.Data != nullptr && B.Size != 0) { retryingWriteAll(Fd, reinterpret_cast(B.Data), Modified: vendor/compiler-rt/dist/test/asan/TestCases/intercept-rethrow-exception.cc ============================================================================== --- vendor/compiler-rt/dist/test/asan/TestCases/intercept-rethrow-exception.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/test/asan/TestCases/intercept-rethrow-exception.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -1,7 +1,7 @@ // Regression test for // https://bugs.llvm.org/show_bug.cgi?id=32434 -// RUN: %clangxx_asan -O0 %s -o %t +// RUN: %clangxx_asan -fexceptions -O0 %s -o %t // RUN: %run %t #include Added: vendor/compiler-rt/dist/test/fuzzer/ImplicitIntegerTruncationTest.cpp ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/fuzzer/ImplicitIntegerTruncationTest.cpp Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,27 @@ +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. + +// Test for signed-integer-overflow. +#include +#include +#include +#include +#include +#include + +static volatile int Sink; +static unsigned char Large = UINT8_MAX; + +extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { + assert(Data); + if (Size > 0 && Data[0] == 'H') { + Sink = 1; + if (Size > 1 && Data[1] == 'i') { + Sink = 2; + if (Size > 2 && Data[2] == '!') { + Large = Large + 1; // 'char overflow'. + } + } + } + return 0; +} Added: vendor/compiler-rt/dist/test/fuzzer/fuzzer-implicit-integer-truncation.test ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/fuzzer/fuzzer-implicit-integer-truncation.test Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,5 @@ +RUN: rm -f %t-ImplicitIntegerTruncationTest-Ubsan +RUN: %cpp_compiler -fsanitize=implicit-integer-truncation -fno-sanitize-recover=all %S/ImplicitIntegerTruncationTest.cpp -o %t-ImplicitIntegerTruncationTest-Ubsan +RUN: not %run %t-ImplicitIntegerTruncationTest-Ubsan 2>&1 | FileCheck %s +CHECK: runtime error: implicit conversion from type 'int' of value 256 (32-bit, signed) to type 'unsigned char' changed the value to 0 (8-bit, unsigned) +CHECK: Test unit written to ./crash- Added: vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-fork.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-fork.c Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,15 @@ +#include + +void func1() {} +void func2() {} + +int main(void) +{ + func1(); + + fork(); + + func2(); + + return 0; +} Added: vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-fork.c.gcov ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-fork.c.gcov Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,23 @@ +// CHECK: -: 0:Source:{{.*}}Inputs/instrprof-gcov-fork.c +// CHECK-NEXT: -: 0:Graph:instrprof-gcov-fork.gcno +// CHECK-NEXT: -: 0:Data:instrprof-gcov-fork.gcda +// CHECK-NEXT: -: 0:Runs:1 +// CHECK-NEXT: -: 0:Programs:1 +// CHECK-NEXT: -: 1:#include +// CHECK-NEXT: -: 2: +// CHECK-NEXT:function func1 called 1 returned 100% blocks executed 100% +// CHECK-NEXT: 1: 3:void func1() {} +// CHECK-NEXT:function func2 called 2 returned 100% blocks executed 100% +// CHECK-NEXT: 2: 4:void func2() {} +// CHECK-NEXT: -: 5: +// CHECK-NEXT:function main called 1 returned 100% blocks executed 100% +// CHECK-NEXT: -: 6:int main(void) +// CHECK-NEXT: -: 7:{ +// CHECK-NEXT: 1: 8: func1(); +// CHECK-NEXT: -: 9: +// CHECK-NEXT: 1: 10: fork(); +// CHECK-NEXT: -: 11: +// CHECK-NEXT: 2: 12: func2(); +// CHECK-NEXT: -: 13: +// CHECK-NEXT: 2: 14: return 0; +// CHECK-NEXT: -: 15:} Added: vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch1.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch1.c Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,18 @@ +int main(void) +{ + int i = 22; + + switch (i) { + case 7: + break; + + case 22: + i = 7; + break; + + case 42: + break; + } + + return 0; +} Added: vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch1.c.gcov ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch1.c.gcov Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,23 @@ +// CHECK: -: 0:Source:{{.*}}Inputs/instrprof-gcov-switch1.c +// CHECK-NEXT: -: 0:Graph:instrprof-gcov-switch1.gcno +// CHECK-NEXT: -: 0:Data:instrprof-gcov-switch1.gcda +// CHECK-NEXT: -: 0:Runs:1 +// CHECK-NEXT: -: 0:Programs:1 +// CHECK-NEXT: -: 1:int main(void) +// CHECK-NEXT: -: 2:{ +// CHECK-NEXT: 2: 3: int i = 22; +// CHECK-NEXT: -: 4: +// CHECK-NEXT: 2: 5: switch (i) { +// CHECK-NEXT: -: 6: case 7: +// CHECK-NEXT: #####: 7: break; +// CHECK-NEXT: -: 8: +// CHECK-NEXT: -: 9: case 22: +// CHECK-NEXT: 1: 10: i = 7; +// CHECK-NEXT: 1: 11: break; +// CHECK-NEXT: -: 12: +// CHECK-NEXT: -: 13: case 42: +// CHECK-NEXT: #####: 14: break; +// CHECK-NEXT: -: 15: } +// CHECK-NEXT: -: 16: +// CHECK-NEXT: 1: 17: return 0; +// CHECK-NEXT: -: 18:} Added: vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch2.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch2.c Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,18 @@ +int main(void) +{ + int i = 22; + + switch (i) { + case 7: + break; + + case 22: + i = 7; + + case 42: + i = 22; + break; + } + + return 0; +} Added: vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch2.c.gcov ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/Inputs/instrprof-gcov-switch2.c.gcov Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,23 @@ +// CHECK: -: 0:Source:{{.*}}Inputs/instrprof-gcov-switch2.c +// CHECK-NEXT: -: 0:Graph:instrprof-gcov-switch2.gcno +// CHECK-NEXT: -: 0:Data:instrprof-gcov-switch2.gcda +// CHECK-NEXT: -: 0:Runs:1 +// CHECK-NEXT: -: 0:Programs:1 +// CHECK-NEXT: -: 1:int main(void) +// CHECK-NEXT: -: 2:{ +// CHECK-NEXT: 3: 3: int i = 22; +// CHECK-NEXT: -: 4: +// CHECK-NEXT: 3: 5: switch (i) { +// CHECK-NEXT: -: 6: case 7: +// CHECK-NEXT: #####: 7: break; +// CHECK-NEXT: -: 8: +// CHECK-NEXT: -: 9: case 22: +// CHECK-NEXT: 1: 10: i = 7; +// CHECK-NEXT: -: 11: +// CHECK-NEXT: -: 12: case 42: +// CHECK-NEXT: 1: 13: i = 22; +// CHECK-NEXT: 1: 14: break; +// CHECK-NEXT: -: 15: } +// CHECK-NEXT: -: 16: +// CHECK-NEXT: 1: 17: return 0; +// CHECK-NEXT: -: 18:} Added: vendor/compiler-rt/dist/test/profile/Posix/instrprof-gcov-fork.test ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/Posix/instrprof-gcov-fork.test Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,12 @@ +XFAIL: * + +RUN: mkdir -p %t.d +RUN: cd %t.d + +RUN: %clang --coverage -o %t %S/../Inputs/instrprof-gcov-fork.c +RUN: test -f instrprof-gcov-fork.gcno + +RUN: rm -f instrprof-gcov-fork.gcda +RUN: %run %t +RUN: llvm-cov gcov -b -c instrprof-gcov-fork.gcda +RUN: FileCheck --match-full-lines --strict-whitespace --input-file instrprof-gcov-fork.c.gcov %S/../Inputs/instrprof-gcov-fork.c.gcov Added: vendor/compiler-rt/dist/test/profile/instrprof-gcov-switch.test ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/instrprof-gcov-switch.test Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,16 @@ +RUN: mkdir -p %t.d +RUN: cd %t.d + +RUN: %clang --coverage -o %t %S/Inputs/instrprof-gcov-switch1.c +RUN: test -f instrprof-gcov-switch1.gcno +RUN: rm -f instrprof-gcov-switch1.gcda +RUN: %run %t +RUN: llvm-cov gcov instrprof-gcov-switch1.gcda +RUN: FileCheck --match-full-lines --strict-whitespace --input-file instrprof-gcov-switch1.c.gcov %S/Inputs/instrprof-gcov-switch1.c.gcov + +RUN: %clang --coverage -o %t %S/Inputs/instrprof-gcov-switch2.c +RUN: test -f instrprof-gcov-switch2.gcno +RUN: rm -f instrprof-gcov-switch2.gcda +RUN: %run %t +RUN: llvm-cov gcov instrprof-gcov-switch2.gcda +RUN: FileCheck --match-full-lines --strict-whitespace --input-file instrprof-gcov-switch2.c.gcov %S/Inputs/instrprof-gcov-switch2.c.gcov Added: vendor/compiler-rt/dist/test/profile/instrprof-set-dir-mode.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/profile/instrprof-set-dir-mode.c Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,48 @@ +// UNSUPPORTED: windows +// RUN: %clang_pgogen -o %t.bin %s -DTESTPATH=\"%t.dir\" +// RUN: rm -rf %t.dir +// RUN: %run %t.bin + +#include +#include +#include +#include + +void __llvm_profile_set_dir_mode(unsigned Mode); +unsigned __llvm_profile_get_dir_mode(void); +void __llvm_profile_recursive_mkdir(char *Path); + +static int test(unsigned Mode, const char *TestDir) { + int Ret = 0; + + /* Create a dir and set the mode accordingly. */ + char *Dir = strdup(TestDir); + if (!Dir) + return -1; + __llvm_profile_set_dir_mode(Mode); + __llvm_profile_recursive_mkdir(Dir); + + if (Mode != __llvm_profile_get_dir_mode()) + Ret = -1; + else { + const unsigned Expected = ~umask(0) & Mode; + struct stat DirSt; + if (stat(Dir, &DirSt) == -1) + Ret = -1; + else if (DirSt.st_mode != Expected) { + printf("Modes do not match: Expected %o but found %o (%s)\n", Expected, + DirSt.st_mode, Dir); + Ret = -1; + } + } + + free(Dir); + return Ret; +} + +int main(void) { + if (test(S_IFDIR | 0777, TESTPATH "/foo/bar/baz/") || + test(S_IFDIR | 0666, TESTPATH "/foo/bar/qux/")) + return -1; + return 0; +} Added: vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation-blacklist.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation-blacklist.c Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,20 @@ +// FIXME: https://code.google.com/p/address-sanitizer/issues/detail?id=316 +// I'm not sure this is actually *that* issue, but this seems oddly similar to the other XFAIL'ed cases. +// XFAIL: android +// UNSUPPORTED: ios + +// RUN: rm -f %tmp +// RUN: echo "[implicit-integer-truncation]" >> %tmp +// RUN: echo "fun:*implicitTruncation*" >> %tmp +// RUN: %clang -fsanitize=implicit-integer-truncation -fno-sanitize-recover=implicit-integer-truncation -fsanitize-blacklist=%tmp -O0 %s -o %t && not %run %t 2>&1 +// RUN: %clang -fsanitize=implicit-integer-truncation -fno-sanitize-recover=implicit-integer-truncation -fsanitize-blacklist=%tmp -O1 %s -o %t && not %run %t 2>&1 +// RUN: %clang -fsanitize=implicit-integer-truncation -fno-sanitize-recover=implicit-integer-truncation -fsanitize-blacklist=%tmp -O2 %s -o %t && not %run %t 2>&1 +// RUN: %clang -fsanitize=implicit-integer-truncation -fno-sanitize-recover=implicit-integer-truncation -fsanitize-blacklist=%tmp -O3 %s -o %t && not %run %t 2>&1 + +unsigned char implicitTruncation(unsigned int argc) { + return argc; // BOOM +} + +int main(int argc, char **argv) { + return implicitTruncation(~0U); +} Added: vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation-summary.cpp ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation-summary.cpp Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,13 @@ +// RUN: %clangxx -fsanitize=implicit-integer-truncation %s -o %t +// RUN: %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-NOTYPE +// RUN: %env_ubsan_opts=report_error_type=1 %run %t 2>&1 | FileCheck %s --check-prefix=CHECK-TYPE +// REQUIRES: !ubsan-standalone && !ubsan-standalone-static + +#include + +int main() { + uint8_t t0 = (~(uint32_t(0))); + // CHECK-NOTYPE: SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior {{.*}}summary.cpp:[[@LINE-1]]:16 + // CHECK-TYPE: SUMMARY: UndefinedBehaviorSanitizer: implicit-integer-truncation {{.*}}summary.cpp:[[@LINE-2]]:16 + return 0; +} Added: vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/ubsan/TestCases/ImplicitConversion/integer-truncation.c Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,63 @@ +// RUN: %clang -x c -fsanitize=implicit-integer-truncation %s -o %t && %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK +// RUN: %clangxx -x c++ -fsanitize=implicit-integer-truncation %s -o %t && %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK + +#include + +#if !defined(__cplusplus) +#define bool _Bool +#endif + +int main() { +// CHECK-NOT: integer-truncation.c + + // Negative tests. Even if they produce unexpected results, this sanitizer does not care. + int8_t n0 = (~((uint32_t)0)); // ~0 -> -1, but do not warn. + uint8_t n2 = 128; + uint8_t n3 = 255; + // Bools do not count + bool b0 = (~((uint32_t)0)); + bool b1 = 255; + + // Explicit and-ing of bits will silence it. + uint8_t nc0 = (~((uint32_t)0)) & 255; + + // Explicit casts + uint8_t i0 = (uint8_t)(~((uint32_t)0)); + +#if defined(__cplusplus) + uint8_t i1 = uint8_t(~(uint32_t(0))); + uint8_t i2 = static_cast(~(uint32_t(0))); +#endif + + // Positive tests. + + uint8_t t_b0 = (~((uint16_t)(0))); +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:18: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'uint8_t' (aka 'unsigned char') changed the value to 255 (8-bit, unsigned) + + uint8_t t_b1 = (~((uint32_t)0)); +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:18: runtime error: implicit conversion from type 'uint32_t' (aka 'unsigned int') of value 4294967295 (32-bit, unsigned) to type 'uint8_t' (aka 'unsigned char') changed the value to 255 (8-bit, unsigned) + uint16_t t_b2 = (~((uint32_t)0)); +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:19: runtime error: implicit conversion from type 'uint32_t' (aka 'unsigned int') of value 4294967295 (32-bit, unsigned) to type 'uint16_t' (aka 'unsigned short') changed the value to 65535 (16-bit, unsigned) + + uint8_t t_b3 = ~((uint64_t)0); +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:18: runtime error: implicit conversion from type 'uint64_t' (aka 'unsigned long{{[^']*}}') of value 18446744073709551615 (64-bit, unsigned) to type 'uint8_t' (aka 'unsigned char') changed the value to 255 (8-bit, unsigned) + uint16_t t_b4 = ~((uint64_t)0); +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:19: runtime error: implicit conversion from type 'uint64_t' (aka 'unsigned long{{[^']*}}') of value 18446744073709551615 (64-bit, unsigned) to type 'uint16_t' (aka 'unsigned short') changed the value to 65535 (16-bit, unsigned) + uint32_t t_b5 = ~((uint64_t)0); +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:19: runtime error: implicit conversion from type 'uint64_t' (aka 'unsigned long{{[^']*}}') of value 18446744073709551615 (64-bit, unsigned) to type 'uint32_t' (aka 'unsigned int') changed the value to 4294967295 (32-bit, unsigned) + + int8_t t1 = 255; +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:15: runtime error: implicit conversion from type 'int' of value 255 (32-bit, signed) to type 'int8_t' (aka 'signed char') changed the value to -1 (8-bit, signed) + uint8_t t2 = 256; +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:16: runtime error: implicit conversion from type 'int' of value 256 (32-bit, signed) to type 'uint8_t' (aka 'unsigned char') changed the value to 0 (8-bit, unsigned) + int8_t t3 = 256; +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:15: runtime error: implicit conversion from type 'int' of value 256 (32-bit, signed) to type 'int8_t' (aka 'signed char') changed the value to 0 (8-bit, signed) + uint8_t t4 = 257; +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:16: runtime error: implicit conversion from type 'int' of value 257 (32-bit, signed) to type 'uint8_t' (aka 'unsigned char') changed the value to 1 (8-bit, unsigned) + int8_t t5 = 257; +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:15: runtime error: implicit conversion from type 'int' of value 257 (32-bit, signed) to type 'int8_t' (aka 'signed char') changed the value to 1 (8-bit, signed) + int8_t t6 = 128; +// CHECK: {{.*}}integer-truncation.c:[[@LINE-1]]:15: runtime error: implicit conversion from type 'int' of value 128 (32-bit, signed) to type 'int8_t' (aka 'signed char') changed the value to -128 (8-bit, signed) + + return 0; +} Added: vendor/compiler-rt/dist/test/ubsan_minimal/TestCases/implicit-integer-truncation.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/compiler-rt/dist/test/ubsan_minimal/TestCases/implicit-integer-truncation.c Thu Aug 2 17:33:19 2018 (r337141) @@ -0,0 +1,24 @@ +// RUN: %clang -fsanitize=implicit-integer-truncation %s -o %t && %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK + +#include + +int main() { +// CHECK-NOT: integer-truncation.c + + // Negative tests. Even if they produce unexpected results, this sanitizer does not care. + int8_t n0 = (~((uint32_t)(0))); // ~0 -> -1, but do not warn. + uint8_t n2 = 128; + uint8_t n3 = 255; + // Bools do not count + _Bool b0 = (~((uint32_t)(0))); + _Bool b1 = 255; + + // Explicit and-ing of bits will silence it. + uint8_t nc0 = ((~((uint32_t)(0))) & 255); + + // Positive tests. + uint8_t t0 = (~((uint32_t)(0))); +// CHECK: implicit-conversion + + return 0; +} Modified: vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-multi-threaded.cc ============================================================================== --- vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-multi-threaded.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-multi-threaded.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -51,7 +51,8 @@ volatile int buffer_counter = 0; assert(__xray_log_finalize() == XRayLogInitStatus::XRAY_LOG_FINALIZED); assert(__xray_log_process_buffers(process_buffer) == XRayLogFlushStatus::XRAY_LOG_FLUSHED); - // We're running three threds, so we expect three buffers. - assert(buffer_counter == 3); + // We're running three threads, so we expect four buffers (including the file + // header buffer). + assert(buffer_counter == 4); assert(__xray_log_flushLog() == XRayLogFlushStatus::XRAY_LOG_FLUSHED); } Modified: vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-single-threaded.cc ============================================================================== --- vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-single-threaded.cc Thu Aug 2 17:33:16 2018 (r337140) +++ vendor/compiler-rt/dist/test/xray/TestCases/Posix/profiling-single-threaded.cc Thu Aug 2 17:33:19 2018 (r337141) @@ -47,7 +47,10 @@ volatile int buffer_counter = 0; f0(); assert(__xray_log_process_buffers(process_buffer) == XRayLogFlushStatus::XRAY_LOG_FLUSHED); - assert(buffer_counter == 1); + // There's always at least one buffer, containing the profile file header. We + // assert that we have two, to indicate that we're expecting exactly one + // thread's worth of data. + assert(buffer_counter == 2); assert(__xray_log_flushLog() == XRayLogFlushStatus::XRAY_LOG_FLUSHED); // Let's reset the counter. @@ -60,6 +63,6 @@ volatile int buffer_counter = 0; f0(); assert(__xray_log_process_buffers(process_buffer) == XRayLogFlushStatus::XRAY_LOG_FLUSHED); - assert(buffer_counter == 1); + assert(buffer_counter == 2); assert(__xray_log_flushLog() == XRayLogFlushStatus::XRAY_LOG_FLUSHED); } From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CECC2106D760; Thu, 2 Aug 2018 17:33:30 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 825D672521; Thu, 2 Aug 2018 17:33:30 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 484E014D33; Thu, 2 Aug 2018 17:33:30 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXUbr029506; Thu, 2 Aug 2018 17:33:30 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXUB1029505; Thu, 2 Aug 2018 17:33:30 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXUB1029505@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337142 - vendor/compiler-rt/compiler-rt-trunk-r338536 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/compiler-rt/compiler-rt-trunk-r338536 X-SVN-Commit-Revision: 337142 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:31 -0000 Author: dim Date: Thu Aug 2 17:33:29 2018 New Revision: 337142 URL: https://svnweb.freebsd.org/changeset/base/337142 Log: Tag compiler-rt trunk r338536. Added: vendor/compiler-rt/compiler-rt-trunk-r338536/ - copied from r337141, vendor/compiler-rt/dist/ From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71045106D7AE; Thu, 2 Aug 2018 17:33:38 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7B68F7265F; Thu, 2 Aug 2018 17:33:36 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2685814D34; Thu, 2 Aug 2018 17:33:36 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXaiQ029565; Thu, 2 Aug 2018 17:33:36 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXY3u029552; Thu, 2 Aug 2018 17:33:34 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXY3u029552@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337143 - in vendor/libc++/dist: include include/experimental include/support/newlib lib/abi src test/libcxx test/libcxx/language.support test/std/containers/associative/map/map.modifie... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/libc++/dist: include include/experimental include/support/newlib lib/abi src test/libcxx test/libcxx/language.support test/std/containers/associative/map/map.modifiers test/std/containers/as... X-SVN-Commit-Revision: 337143 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:38 -0000 Author: dim Date: Thu Aug 2 17:33:33 2018 New Revision: 337143 URL: https://svnweb.freebsd.org/changeset/base/337143 Log: Vendor import of libc++ trunk r338536: https://llvm.org/svn/llvm-project/libcxx/trunk@338536 Added: vendor/libc++/dist/include/__node_handle vendor/libc++/dist/include/charconv vendor/libc++/dist/src/charconv.cpp (contents, props changed) vendor/libc++/dist/test/libcxx/language.support/has_c11_features.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/map/map.modifiers/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/map/map.modifiers/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/map/map.modifiers/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/map/map.modifiers/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multimap/multimap.modifiers/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multimap/multimap.modifiers/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multimap/multimap.modifiers/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multimap/multimap.modifiers/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multiset/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multiset/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multiset/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/multiset/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/set/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/set/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/set/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/associative/set/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/container.node/ vendor/libc++/dist/test/std/containers/container.node/node_handle.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.map/unord.map.modifiers/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.map/unord.map.modifiers/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.map/unord.map.modifiers/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.map/unord.map.modifiers/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multimap/unord.multimap.modifiers/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multimap/unord.multimap.modifiers/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multimap/unord.multimap.modifiers/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multimap/unord.multimap.modifiers/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multiset/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multiset/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multiset/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.multiset/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.set/extract_iterator.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.set/extract_key.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.set/insert_node_type.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/containers/unord/unord.set/insert_node_type_hint.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/experimental/simd/simd.abi/ vendor/libc++/dist/test/std/experimental/simd/simd.abi/vector_extension.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/experimental/simd/simd.access/ vendor/libc++/dist/test/std/experimental/simd/simd.access/default.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/experimental/simd/simd.cons/default.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/experimental/simd/simd.cons/load.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/experimental/simd/simd.mem/ vendor/libc++/dist/test/std/experimental/simd/simd.mem/load.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/experimental/simd/simd.mem/store.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/utilities/charconv/ vendor/libc++/dist/test/std/utilities/charconv/charconv.from.chars/ vendor/libc++/dist/test/std/utilities/charconv/charconv.from.chars/integral.bool.fail.cpp (contents, props changed) vendor/libc++/dist/test/std/utilities/charconv/charconv.from.chars/integral.pass.cpp (contents, props changed) vendor/libc++/dist/test/std/utilities/charconv/charconv.to.chars/ vendor/libc++/dist/test/std/utilities/charconv/charconv.to.chars/integral.bool.fail.cpp (contents, props changed) vendor/libc++/dist/test/std/utilities/charconv/charconv.to.chars/integral.pass.cpp (contents, props changed) vendor/libc++/dist/test/support/charconv_test_helpers.h (contents, props changed) Modified: vendor/libc++/dist/include/CMakeLists.txt vendor/libc++/dist/include/__config vendor/libc++/dist/include/__functional_base vendor/libc++/dist/include/__hash_table vendor/libc++/dist/include/__mutex_base vendor/libc++/dist/include/__tree vendor/libc++/dist/include/cfloat vendor/libc++/dist/include/ctime vendor/libc++/dist/include/experimental/__config vendor/libc++/dist/include/experimental/simd vendor/libc++/dist/include/float.h vendor/libc++/dist/include/functional vendor/libc++/dist/include/map vendor/libc++/dist/include/memory vendor/libc++/dist/include/module.modulemap vendor/libc++/dist/include/new vendor/libc++/dist/include/set vendor/libc++/dist/include/shared_mutex vendor/libc++/dist/include/support/newlib/xlocale.h vendor/libc++/dist/include/system_error vendor/libc++/dist/include/unordered_map vendor/libc++/dist/include/unordered_set vendor/libc++/dist/include/utility vendor/libc++/dist/lib/abi/x86_64-unknown-linux-gnu.v1.abilist vendor/libc++/dist/src/bind.cpp vendor/libc++/dist/src/future.cpp vendor/libc++/dist/src/memory.cpp vendor/libc++/dist/src/mutex.cpp vendor/libc++/dist/src/new.cpp vendor/libc++/dist/src/shared_mutex.cpp vendor/libc++/dist/src/system_error.cpp vendor/libc++/dist/src/utility.cpp vendor/libc++/dist/test/libcxx/double_include.sh.cpp vendor/libc++/dist/test/std/depr/depr.c.headers/float_h.pass.cpp vendor/libc++/dist/test/std/depr/depr.c.headers/stdlib_h.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.casts/simd_cast.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.casts/static_simd_cast.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.cons/broadcast.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.cons/generator.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.traits/abi_for_size.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.traits/is_abi_tag.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.traits/is_simd.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.traits/is_simd_flag_type.pass.cpp vendor/libc++/dist/test/std/experimental/simd/simd.traits/is_simd_mask.pass.cpp vendor/libc++/dist/test/std/language.support/support.limits/c.limits/cfloat.pass.cpp vendor/libc++/dist/test/std/language.support/support.runtime/cstdlib.pass.cpp vendor/libc++/dist/test/std/language.support/support.runtime/ctime.pass.cpp vendor/libc++/dist/test/std/utilities/time/date.time/ctime.pass.cpp vendor/libc++/dist/test/support/Counter.h vendor/libc++/dist/test/support/test_macros.h vendor/libc++/dist/www/cxx1z_status.html Modified: vendor/libc++/dist/include/CMakeLists.txt ============================================================================== --- vendor/libc++/dist/include/CMakeLists.txt Thu Aug 2 17:33:29 2018 (r337142) +++ vendor/libc++/dist/include/CMakeLists.txt Thu Aug 2 17:33:33 2018 (r337143) @@ -11,6 +11,7 @@ set(files __libcpp_version __locale __mutex_base + __node_handle __nullptr __split_buffer __sso_allocator @@ -31,6 +32,7 @@ set(files cerrno cfenv cfloat + charconv chrono cinttypes ciso646 Modified: vendor/libc++/dist/include/__config ============================================================================== --- vendor/libc++/dist/include/__config Thu Aug 2 17:33:29 2018 (r337142) +++ vendor/libc++/dist/include/__config Thu Aug 2 17:33:33 2018 (r337143) @@ -510,7 +510,7 @@ namespace std { #define _LIBCPP_HAS_IS_BASE_OF #endif -#if !__EXCEPTIONS +#if !__EXCEPTIONS && !defined(_LIBCPP_NO_EXCEPTIONS) #define _LIBCPP_NO_EXCEPTIONS #endif @@ -620,6 +620,8 @@ namespace std { #define _LIBCPP_ALWAYS_INLINE __forceinline +#define _LIBCPP_HAS_NO_VECTOR_EXTENSION + #elif defined(_LIBCPP_COMPILER_IBM) #define _ALIGNAS(x) __attribute__((__aligned__(x))) @@ -651,6 +653,8 @@ namespace std { #define _LIBCPP_HAS_NO_ASAN #define _LIBCPP_ALWAYS_INLINE __attribute__ ((__always_inline__)) + +#define _LIBCPP_HAS_NO_VECTOR_EXTENSION #endif // _LIBCPP_COMPILER_[CLANG|GCC|MSVC|IBM] Modified: vendor/libc++/dist/include/__functional_base ============================================================================== --- vendor/libc++/dist/include/__functional_base Thu Aug 2 17:33:29 2018 (r337142) +++ vendor/libc++/dist/include/__functional_base Thu Aug 2 17:33:33 2018 (r337143) @@ -561,7 +561,7 @@ struct __is_transparent<_Tp, _Up, struct _LIBCPP_TEMPLATE_VIS allocator_arg_t { }; -#if defined(_LIBCPP_CXX03_LANG) || defined(_LIBCPP_BUILDING_MEMORY) +#if defined(_LIBCPP_CXX03_LANG) || defined(_LIBCPP_BUILDING_LIBRARY) extern const allocator_arg_t allocator_arg; #else /* _LIBCPP_INLINE_VAR */ constexpr allocator_arg_t allocator_arg = allocator_arg_t(); Modified: vendor/libc++/dist/include/__hash_table ============================================================================== --- vendor/libc++/dist/include/__hash_table Thu Aug 2 17:33:29 2018 (r337142) +++ vendor/libc++/dist/include/__hash_table Thu Aug 2 17:33:33 2018 (r337143) @@ -859,7 +859,18 @@ class __hash_node_destructor (public) template friend class __hash_map_node_destructor; }; +#if _LIBCPP_STD_VER > 14 +template +struct __generic_container_node_destructor; +template +struct __generic_container_node_destructor<__hash_node<_Tp, _VoidPtr>, _Alloc> + : __hash_node_destructor<_Alloc> +{ + using __hash_node_destructor<_Alloc>::__hash_node_destructor; +}; +#endif + #ifndef _LIBCPP_CXX03_LANG template struct __diagnose_hash_table_helper { @@ -1151,6 +1162,30 @@ class __hash_table (public) return __emplace_unique_key_args(_NodeTypes::__get_key(__x), __x); } +#if _LIBCPP_STD_VER > 14 + template + _LIBCPP_INLINE_VISIBILITY + _InsertReturnType __node_handle_insert_unique(_NodeHandle&& __nh); + template + _LIBCPP_INLINE_VISIBILITY + iterator __node_handle_insert_unique(const_iterator __hint, + _NodeHandle&& __nh); + + template + _LIBCPP_INLINE_VISIBILITY + iterator __node_handle_insert_multi(_NodeHandle&& __nh); + template + _LIBCPP_INLINE_VISIBILITY + iterator __node_handle_insert_multi(const_iterator __hint, _NodeHandle&& __nh); + + template + _LIBCPP_INLINE_VISIBILITY + _NodeHandle __node_handle_extract(key_type const& __key); + template + _LIBCPP_INLINE_VISIBILITY + _NodeHandle __node_handle_extract(const_iterator __it); +#endif + void clear() _NOEXCEPT; void rehash(size_type __n); _LIBCPP_INLINE_VISIBILITY void reserve(size_type __n) @@ -2125,6 +2160,91 @@ __hash_table<_Tp, _Hash, _Equal, _Alloc>::__insert_mul } #endif // _LIBCPP_CXX03_LANG + +#if _LIBCPP_STD_VER > 14 +template +template +_LIBCPP_INLINE_VISIBILITY +_InsertReturnType +__hash_table<_Tp, _Hash, _Equal, _Alloc>::__node_handle_insert_unique( + _NodeHandle&& __nh) +{ + if (__nh.empty()) + return _InsertReturnType{end(), false, _NodeHandle()}; + pair __result = __node_insert_unique(__nh.__ptr_); + if (__result.second) + __nh.__release(); + return _InsertReturnType{__result.first, __result.second, _VSTD::move(__nh)}; +} + +template +template +_LIBCPP_INLINE_VISIBILITY +typename __hash_table<_Tp, _Hash, _Equal, _Alloc>::iterator +__hash_table<_Tp, _Hash, _Equal, _Alloc>::__node_handle_insert_unique( + const_iterator, _NodeHandle&& __nh) +{ + if (__nh.empty()) + return end(); + pair __result = __node_insert_unique(__nh.__ptr_); + if (__result.second) + __nh.__release(); + return __result.first; +} + +template +template +_LIBCPP_INLINE_VISIBILITY +_NodeHandle +__hash_table<_Tp, _Hash, _Equal, _Alloc>::__node_handle_extract( + key_type const& __key) +{ + iterator __i = find(__key); + if (__i == end()) + return _NodeHandle(); + return __node_handle_extract<_NodeHandle>(__i); +} + +template +template +_LIBCPP_INLINE_VISIBILITY +_NodeHandle +__hash_table<_Tp, _Hash, _Equal, _Alloc>::__node_handle_extract( + const_iterator __p) +{ + allocator_type __alloc(__node_alloc()); + return _NodeHandle(remove(__p).release(), __alloc); +} + +template +template +_LIBCPP_INLINE_VISIBILITY +typename __hash_table<_Tp, _Hash, _Equal, _Alloc>::iterator +__hash_table<_Tp, _Hash, _Equal, _Alloc>::__node_handle_insert_multi( + _NodeHandle&& __nh) +{ + if (__nh.empty()) + return end(); + iterator __result = __node_insert_multi(__nh.__ptr_); + __nh.__release(); + return __result; +} + +template +template +_LIBCPP_INLINE_VISIBILITY +typename __hash_table<_Tp, _Hash, _Equal, _Alloc>::iterator +__hash_table<_Tp, _Hash, _Equal, _Alloc>::__node_handle_insert_multi( + const_iterator __hint, _NodeHandle&& __nh) +{ + if (__nh.empty()) + return end(); + iterator __result = __node_insert_multi(__hint, __nh.__ptr_); + __nh.__release(); + return __result; +} + +#endif // _LIBCPP_STD_VER > 14 template void Modified: vendor/libc++/dist/include/__mutex_base ============================================================================== --- vendor/libc++/dist/include/__mutex_base Thu Aug 2 17:33:29 2018 (r337142) +++ vendor/libc++/dist/include/__mutex_base Thu Aug 2 17:33:33 2018 (r337143) @@ -74,7 +74,7 @@ struct _LIBCPP_TYPE_VIS defer_lock_t {}; struct _LIBCPP_TYPE_VIS try_to_lock_t {}; struct _LIBCPP_TYPE_VIS adopt_lock_t {}; -#if defined(_LIBCPP_CXX03_LANG) || defined(_LIBCPP_BUILDING_MUTEX) +#if defined(_LIBCPP_CXX03_LANG) || defined(_LIBCPP_BUILDING_LIBRARY) extern const defer_lock_t defer_lock; extern const try_to_lock_t try_to_lock; Added: vendor/libc++/dist/include/__node_handle ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libc++/dist/include/__node_handle Thu Aug 2 17:33:33 2018 (r337143) @@ -0,0 +1,212 @@ +// -*- C++ -*- +//===----------------------------------------------------------------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is dual licensed under the MIT and the University of Illinois Open +// Source Licenses. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// + +#ifndef _LIBCPP___NODE_HANDLE +#define _LIBCPP___NODE_HANDLE + +#include <__config> +#include +#include + +#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER) +#pragma GCC system_header +#endif + +_LIBCPP_PUSH_MACROS +#include <__undef_macros> + +_LIBCPP_BEGIN_NAMESPACE_STD + +#if _LIBCPP_STD_VER > 14 + +#define __cpp_lib_node_extract 201606L + +// Specialized in __tree & __hash_table for their _NodeType. +template +struct __generic_container_node_destructor; + +template class _MapOrSetSpecifics> +class _LIBCPP_TEMPLATE_VIS __basic_node_handle + : public _MapOrSetSpecifics< + _NodeType, + __basic_node_handle<_NodeType, _Alloc, _MapOrSetSpecifics>> +{ + template + friend class __tree; + template + friend class __hash_table; + friend struct _MapOrSetSpecifics< + _NodeType, __basic_node_handle<_NodeType, _Alloc, _MapOrSetSpecifics>>; + + typedef allocator_traits<_Alloc> __alloc_traits; + typedef typename __rebind_pointer::type + __node_pointer_type; + +public: + typedef _Alloc allocator_type; + +private: + __node_pointer_type __ptr_ = nullptr; + optional __alloc_; + + _LIBCPP_INLINE_VISIBILITY + void __release() + { + __ptr_ = nullptr; + __alloc_ = _VSTD::nullopt; + } + + _LIBCPP_INLINE_VISIBILITY + void __destroy_node_pointer() + { + if (__ptr_ != nullptr) + { + typedef typename __allocator_traits_rebind< + allocator_type, _NodeType>::type __node_alloc_type; + __node_alloc_type __alloc(*__alloc_); + __generic_container_node_destructor<_NodeType, __node_alloc_type>( + __alloc, true)(__ptr_); + __ptr_ = nullptr; + } + } + + _LIBCPP_INLINE_VISIBILITY + __basic_node_handle(__node_pointer_type __ptr, + allocator_type const& __alloc) + : __ptr_(__ptr), __alloc_(__alloc) + { + } + +public: + _LIBCPP_INLINE_VISIBILITY + __basic_node_handle() = default; + + _LIBCPP_INLINE_VISIBILITY + __basic_node_handle(__basic_node_handle&& __other) noexcept + : __ptr_(__other.__ptr_), + __alloc_(_VSTD::move(__other.__alloc_)) + { + __other.__ptr_ = nullptr; + __other.__alloc_ = _VSTD::nullopt; + } + + _LIBCPP_INLINE_VISIBILITY + __basic_node_handle& operator=(__basic_node_handle&& __other) + { + _LIBCPP_ASSERT( + __alloc_ == _VSTD::nullopt || + __alloc_traits::propagate_on_container_move_assignment::value || + __alloc_ == __other.__alloc_, + "node_type with incompatible allocator passed to " + "node_type::operator=(node_type&&)"); + + __destroy_node_pointer(); + __ptr_ = __other.__ptr_; + + if (__alloc_traits::propagate_on_container_move_assignment::value || + __alloc_ == _VSTD::nullopt) + __alloc_ = _VSTD::move(__other.__alloc_); + + __other.__ptr_ = nullptr; + __other.__alloc_ = _VSTD::nullopt; + + return *this; + } + + _LIBCPP_INLINE_VISIBILITY + allocator_type get_allocator() const { return *__alloc_; } + + _LIBCPP_INLINE_VISIBILITY + explicit operator bool() const { return __ptr_ != nullptr; } + + _LIBCPP_NODISCARD_AFTER_CXX17 _LIBCPP_INLINE_VISIBILITY + bool empty() const { return __ptr_ == nullptr; } + + _LIBCPP_INLINE_VISIBILITY + void swap(__basic_node_handle& __other) noexcept( + __alloc_traits::propagate_on_container_swap::value || + __alloc_traits::is_always_equal::value) + { + using _VSTD::swap; + swap(__ptr_, __other.__ptr_); + if (__alloc_traits::propagate_on_container_swap::value || + __alloc_ == _VSTD::nullopt || __other.__alloc_ == _VSTD::nullopt) + swap(__alloc_, __other.__alloc_); + } + + _LIBCPP_INLINE_VISIBILITY + friend void swap(__basic_node_handle& __a, __basic_node_handle& __b) + noexcept(noexcept(__a.swap(__b))) { __a.swap(__b); } + + _LIBCPP_INLINE_VISIBILITY + ~__basic_node_handle() + { + __destroy_node_pointer(); + } +}; + +template +struct __set_node_handle_specifics +{ + typedef typename _NodeType::__node_value_type value_type; + + _LIBCPP_INLINE_VISIBILITY + value_type& value() const + { + return static_cast<_Derived const*>(this)->__ptr_->__value_; + } +}; + +template +struct __map_node_handle_specifics +{ + typedef typename _NodeType::__node_value_type::key_type key_type; + typedef typename _NodeType::__node_value_type::mapped_type mapped_type; + + _LIBCPP_INLINE_VISIBILITY + key_type& key() const + { + return static_cast<_Derived const*>(this)-> + __ptr_->__value_.__ref().first; + } + + _LIBCPP_INLINE_VISIBILITY + mapped_type& mapped() const + { + return static_cast<_Derived const*>(this)-> + __ptr_->__value_.__ref().second; + } +}; + +template +using __set_node_handle = + __basic_node_handle< _NodeType, _Alloc, __set_node_handle_specifics>; + +template +using __map_node_handle = + __basic_node_handle< _NodeType, _Alloc, __map_node_handle_specifics>; + +template +_LIBCPP_TEMPLATE_VIS +struct __insert_return_type +{ + _Iterator position; + bool inserted; + _NodeType node; +}; + +#endif // _LIBCPP_STD_VER > 14 + +_LIBCPP_END_NAMESPACE_STD +_LIBCPP_POP_MACROS + +#endif Modified: vendor/libc++/dist/include/__tree ============================================================================== --- vendor/libc++/dist/include/__tree Thu Aug 2 17:33:29 2018 (r337142) +++ vendor/libc++/dist/include/__tree Thu Aug 2 17:33:33 2018 (r337143) @@ -796,6 +796,16 @@ class __tree_node_destructor (public) template friend class __map_node_destructor; }; +#if _LIBCPP_STD_VER > 14 +template +struct __generic_container_node_destructor; +template +struct __generic_container_node_destructor<__tree_node<_Tp, _VoidPtr>, _Alloc> + : __tree_node_destructor<_Alloc> +{ + using __tree_node_destructor<_Alloc>::__tree_node_destructor; +}; +#endif template class _LIBCPP_TEMPLATE_VIS __tree_iterator @@ -1338,6 +1348,33 @@ class __tree (public) iterator __node_insert_multi(__node_pointer __nd); iterator __node_insert_multi(const_iterator __p, __node_pointer __nd); + + _LIBCPP_INLINE_VISIBILITY iterator __remove_node_pointer(__node_pointer); + +#if _LIBCPP_STD_VER > 14 + template + _LIBCPP_INLINE_VISIBILITY + _InsertReturnType __node_handle_insert_unique(_NodeHandle&&); + template + _LIBCPP_INLINE_VISIBILITY + iterator __node_handle_insert_unique(const_iterator, _NodeHandle&&); + + template + _LIBCPP_INLINE_VISIBILITY + iterator __node_handle_insert_multi(_NodeHandle&&); + template + _LIBCPP_INLINE_VISIBILITY + iterator __node_handle_insert_multi(const_iterator, _NodeHandle&&); + + + template + _LIBCPP_INLINE_VISIBILITY + _NodeHandle __node_handle_extract(key_type const&); + template + _LIBCPP_INLINE_VISIBILITY + _NodeHandle __node_handle_extract(const_iterator); +#endif + iterator erase(const_iterator __p); iterator erase(const_iterator __f, const_iterator __l); template @@ -2347,17 +2384,138 @@ __tree<_Tp, _Compare, _Allocator>::__node_insert_multi template typename __tree<_Tp, _Compare, _Allocator>::iterator -__tree<_Tp, _Compare, _Allocator>::erase(const_iterator __p) +__tree<_Tp, _Compare, _Allocator>::__remove_node_pointer(__node_pointer __ptr) { - __node_pointer __np = __p.__get_np(); - iterator __r(__p.__ptr_); + iterator __r(__ptr); ++__r; - if (__begin_node() == __p.__ptr_) + if (__begin_node() == __ptr) __begin_node() = __r.__ptr_; --size(); - __node_allocator& __na = __node_alloc(); __tree_remove(__end_node()->__left_, - static_cast<__node_base_pointer>(__np)); + static_cast<__node_base_pointer>(__ptr)); + return __r; +} + +#if _LIBCPP_STD_VER > 14 +template +template +_LIBCPP_INLINE_VISIBILITY +_InsertReturnType +__tree<_Tp, _Compare, _Allocator>::__node_handle_insert_unique( + _NodeHandle&& __nh) +{ + if (__nh.empty()) + return _InsertReturnType{end(), false, _NodeHandle()}; + + __node_pointer __ptr = __nh.__ptr_; + __parent_pointer __parent; + __node_base_pointer& __child = __find_equal(__parent, + __ptr->__value_); + if (__child != nullptr) + return _InsertReturnType{ + iterator(static_cast<__node_pointer>(__child)), + false, _VSTD::move(__nh)}; + + __insert_node_at(__parent, __child, + static_cast<__node_base_pointer>(__ptr)); + __nh.__release(); + return _InsertReturnType{iterator(__ptr), true, _NodeHandle()}; +} + +template +template +_LIBCPP_INLINE_VISIBILITY +typename __tree<_Tp, _Compare, _Allocator>::iterator +__tree<_Tp, _Compare, _Allocator>::__node_handle_insert_unique( + const_iterator __hint, _NodeHandle&& __nh) +{ + if (__nh.empty()) + return end(); + + __node_pointer __ptr = __nh.__ptr_; + __parent_pointer __parent; + __node_base_pointer __dummy; + __node_base_pointer& __child = __find_equal(__hint, __parent, __dummy, + __ptr->__value_); + __node_pointer __r = static_cast<__node_pointer>(__child); + if (__child == nullptr) + { + __insert_node_at(__parent, __child, + static_cast<__node_base_pointer>(__ptr)); + __r = __ptr; + __nh.__release(); + } + return iterator(__r); +} + +template +template +_LIBCPP_INLINE_VISIBILITY +_NodeHandle +__tree<_Tp, _Compare, _Allocator>::__node_handle_extract(key_type const& __key) +{ + iterator __it = find(__key); + if (__it == end()) + return _NodeHandle(); + return __node_handle_extract<_NodeHandle>(__it); +} + +template +template +_LIBCPP_INLINE_VISIBILITY +_NodeHandle +__tree<_Tp, _Compare, _Allocator>::__node_handle_extract(const_iterator __p) +{ + __node_pointer __np = __p.__get_np(); + __remove_node_pointer(__np); + return _NodeHandle(__np, __alloc()); +} + +template +template +_LIBCPP_INLINE_VISIBILITY +typename __tree<_Tp, _Compare, _Allocator>::iterator +__tree<_Tp, _Compare, _Allocator>::__node_handle_insert_multi(_NodeHandle&& __nh) +{ + if (__nh.empty()) + return end(); + __node_pointer __ptr = __nh.__ptr_; + __parent_pointer __parent; + __node_base_pointer& __child = __find_leaf_high( + __parent, _NodeTypes::__get_key(__ptr->__value_)); + __insert_node_at(__parent, __child, static_cast<__node_base_pointer>(__ptr)); + __nh.__release(); + return iterator(__ptr); +} + +template +template +_LIBCPP_INLINE_VISIBILITY +typename __tree<_Tp, _Compare, _Allocator>::iterator +__tree<_Tp, _Compare, _Allocator>::__node_handle_insert_multi( + const_iterator __hint, _NodeHandle&& __nh) +{ + if (__nh.empty()) + return end(); + + __node_pointer __ptr = __nh.__ptr_; + __parent_pointer __parent; + __node_base_pointer& __child = __find_leaf(__hint, __parent, + _NodeTypes::__get_key(__ptr->__value_)); + __insert_node_at(__parent, __child, static_cast<__node_base_pointer>(__ptr)); + __nh.__release(); + return iterator(__ptr); +} + +#endif // _LIBCPP_STD_VER > 14 + +template +typename __tree<_Tp, _Compare, _Allocator>::iterator +__tree<_Tp, _Compare, _Allocator>::erase(const_iterator __p) +{ + __node_pointer __np = __p.__get_np(); + iterator __r = __remove_node_pointer(__np); + __node_allocator& __na = __node_alloc(); __node_traits::destroy(__na, _NodeTypes::__get_ptr( const_cast<__node_value_type&>(*__p))); __node_traits::deallocate(__na, __np, 1); Modified: vendor/libc++/dist/include/cfloat ============================================================================== --- vendor/libc++/dist/include/cfloat Thu Aug 2 17:33:29 2018 (r337142) +++ vendor/libc++/dist/include/cfloat Thu Aug 2 17:33:33 2018 (r337143) @@ -20,11 +20,18 @@ Macros: FLT_EVAL_METHOD // C99 FLT_RADIX + FLT_HAS_SUBNORM // C11 + DBL_HAS_SUBNORM // C11 + LDBL_HAS_SUBNORM // C11 + FLT_MANT_DIG DBL_MANT_DIG LDBL_MANT_DIG DECIMAL_DIG // C99 + FLT_DECIMAL_DIG // C11 + DBL_DECIMAL_DIG // C11 + LDBL_DECIMAL_DIG // C11 FLT_DIG DBL_DIG @@ -58,6 +65,9 @@ Macros: DBL_MIN LDBL_MIN + FLT_TRUE_MIN // C11 + DBL_TRUE_MIN // C11 + LDBL_TRUE_MIN // C11 */ #include <__config> Added: vendor/libc++/dist/include/charconv ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/libc++/dist/include/charconv Thu Aug 2 17:33:33 2018 (r337143) @@ -0,0 +1,610 @@ +// -*- C++ -*- +//===------------------------------ charconv ------------------------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is dual licensed under the MIT and the University of Illinois Open +// Source Licenses. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// + +#ifndef _LIBCPP_CHARCONV +#define _LIBCPP_CHARCONV + +/* + charconv synopsis + +namespace std { + + // floating-point format for primitive numerical conversion + enum class chars_format { + scientific = unspecified, + fixed = unspecified, + hex = unspecified, + general = fixed | scientific + }; + + // 23.20.2, primitive numerical output conversion + struct to_chars_result { + char* ptr; + errc ec; + }; + + to_chars_result to_chars(char* first, char* last, see below value, + int base = 10); + + to_chars_result to_chars(char* first, char* last, float value); + to_chars_result to_chars(char* first, char* last, double value); + to_chars_result to_chars(char* first, char* last, long double value); + + to_chars_result to_chars(char* first, char* last, float value, + chars_format fmt); + to_chars_result to_chars(char* first, char* last, double value, + chars_format fmt); + to_chars_result to_chars(char* first, char* last, long double value, + chars_format fmt); + + to_chars_result to_chars(char* first, char* last, float value, + chars_format fmt, int precision); + to_chars_result to_chars(char* first, char* last, double value, + chars_format fmt, int precision); + to_chars_result to_chars(char* first, char* last, long double value, + chars_format fmt, int precision); + + // 23.20.3, primitive numerical input conversion + struct from_chars_result { + const char* ptr; + errc ec; + }; + + from_chars_result from_chars(const char* first, const char* last, + see below& value, int base = 10); + + from_chars_result from_chars(const char* first, const char* last, + float& value, + chars_format fmt = chars_format::general); + from_chars_result from_chars(const char* first, const char* last, + double& value, + chars_format fmt = chars_format::general); + from_chars_result from_chars(const char* first, const char* last, + long double& value, + chars_format fmt = chars_format::general); + +} // namespace std + +*/ + +#include <__errc> +#include +#include +#include +#include +#include + +#include <__debug> + +#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER) +#pragma GCC system_header +#endif + +_LIBCPP_BEGIN_NAMESPACE_STD + +#if _LIBCPP_STD_VER > 11 + +enum class _LIBCPP_ENUM_VIS chars_format +{ + scientific = 0x1, + fixed = 0x2, + hex = 0x4, + general = fixed | scientific +}; + +struct _LIBCPP_TYPE_VIS to_chars_result +{ + char* ptr; + errc ec; +}; + +struct _LIBCPP_TYPE_VIS from_chars_result +{ + const char* ptr; + errc ec; +}; + +void to_chars(char*, char*, bool, int = 10) = delete; +void from_chars(const char*, const char*, bool, int = 10) = delete; + +namespace __itoa +{ + +static constexpr uint64_t __pow10_64[] = { + UINT64_C(0), + UINT64_C(10), + UINT64_C(100), + UINT64_C(1000), + UINT64_C(10000), + UINT64_C(100000), + UINT64_C(1000000), + UINT64_C(10000000), + UINT64_C(100000000), + UINT64_C(1000000000), + UINT64_C(10000000000), + UINT64_C(100000000000), + UINT64_C(1000000000000), + UINT64_C(10000000000000), + UINT64_C(100000000000000), + UINT64_C(1000000000000000), + UINT64_C(10000000000000000), + UINT64_C(100000000000000000), + UINT64_C(1000000000000000000), + UINT64_C(10000000000000000000), +}; + +static constexpr uint32_t __pow10_32[] = { + UINT32_C(0), UINT32_C(10), UINT32_C(100), + UINT32_C(1000), UINT32_C(10000), UINT32_C(100000), + UINT32_C(1000000), UINT32_C(10000000), UINT32_C(100000000), + UINT32_C(1000000000), +}; + +_LIBCPP_FUNC_VIS char* __u64toa(uint64_t __value, char* __buffer); +_LIBCPP_FUNC_VIS char* __u32toa(uint32_t __value, char* __buffer); + +template +struct _LIBCPP_HIDDEN __traits_base +{ + using type = uint64_t; + +#if !defined(_LIBCPP_COMPILER_MSVC) + static _LIBCPP_INLINE_VISIBILITY int __width(_Tp __v) + { + auto __t = (64 - __builtin_clzll(__v | 1)) * 1233 >> 12; + return __t - (__v < __pow10_64[__t]) + 1; + } +#endif + + static _LIBCPP_INLINE_VISIBILITY char* __convert(_Tp __v, char* __p) + { + return __u64toa(__v, __p); + } + + static _LIBCPP_INLINE_VISIBILITY auto& __pow() { return __pow10_64; } +}; + +template +struct _LIBCPP_HIDDEN + __traits_base<_Tp, decltype(void(uint32_t{declval<_Tp>()}))> +{ + using type = uint32_t; + +#if !defined(_LIBCPP_COMPILER_MSVC) + static _LIBCPP_INLINE_VISIBILITY int __width(_Tp __v) + { + auto __t = (32 - __builtin_clz(__v | 1)) * 1233 >> 12; + return __t - (__v < __pow10_32[__t]) + 1; + } +#endif + + static _LIBCPP_INLINE_VISIBILITY char* __convert(_Tp __v, char* __p) + { + return __u32toa(__v, __p); + } + + static _LIBCPP_INLINE_VISIBILITY auto& __pow() { return __pow10_32; } +}; + +template +inline _LIBCPP_INLINE_VISIBILITY bool +__mul_overflowed(unsigned char __a, _Tp __b, unsigned char& __r) +{ + auto __c = __a * __b; + __r = __c; + return __c > (numeric_limits::max)(); +} + +template +inline _LIBCPP_INLINE_VISIBILITY bool +__mul_overflowed(unsigned short __a, _Tp __b, unsigned short& __r) +{ + auto __c = __a * __b; + __r = __c; + return __c > (numeric_limits::max)(); +} + +template +inline _LIBCPP_INLINE_VISIBILITY bool +__mul_overflowed(_Tp __a, _Tp __b, _Tp& __r) +{ + static_assert(is_unsigned<_Tp>::value, ""); +#if !defined(_LIBCPP_COMPILER_MSVC) + return __builtin_mul_overflow(__a, __b, &__r); +#else + bool __did = __b && ((numeric_limits<_Tp>::max)() / __b) < __a; + __r = __a * __b; + return __did; +#endif +} + +template +inline _LIBCPP_INLINE_VISIBILITY bool +__mul_overflowed(_Tp __a, _Up __b, _Tp& __r) +{ + return __mul_overflowed(__a, static_cast<_Tp>(__b), __r); +} + +template +struct _LIBCPP_HIDDEN __traits : __traits_base<_Tp> +{ + static constexpr int digits = numeric_limits<_Tp>::digits10 + 1; + using __traits_base<_Tp>::__pow; + using typename __traits_base<_Tp>::type; + + // precondition: at least one non-zero character available + static _LIBCPP_INLINE_VISIBILITY char const* + __read(char const* __p, char const* __ep, type& __a, type& __b) + { + type __cprod[digits]; + int __j = digits - 1; + int __i = digits; + do + { + if (!('0' <= *__p && *__p <= '9')) + break; + __cprod[--__i] = *__p++ - '0'; + } while (__p != __ep && __i != 0); + + __a = __inner_product(__cprod + __i + 1, __cprod + __j, __pow() + 1, + __cprod[__i]); + if (__mul_overflowed(__cprod[__j], __pow()[__j - __i], __b)) + --__p; + return __p; + } + + template + static _LIBCPP_INLINE_VISIBILITY _Up + __inner_product(_It1 __first1, _It1 __last1, _It2 __first2, _Up __init) + { + for (; __first1 < __last1; ++__first1, ++__first2) + __init = __init + *__first1 * *__first2; + return __init; + } +}; + +} // namespace __itoa + +template +inline _LIBCPP_INLINE_VISIBILITY _Tp +__complement(_Tp __x) +{ + static_assert(is_unsigned<_Tp>::value, "cast to unsigned first"); + return _Tp(~__x + 1); +} + +template +inline _LIBCPP_INLINE_VISIBILITY auto +__to_unsigned(_Tp __x) +{ + return static_cast>(__x); +} + +template +inline _LIBCPP_INLINE_VISIBILITY to_chars_result +__to_chars_itoa(char* __first, char* __last, _Tp __value, true_type) +{ + auto __x = __to_unsigned(__value); + if (__value < 0 && __first != __last) + { + *__first++ = '-'; + __x = __complement(__x); + } + + return __to_chars_itoa(__first, __last, __x, false_type()); +} + +template +inline _LIBCPP_INLINE_VISIBILITY to_chars_result +__to_chars_itoa(char* __first, char* __last, _Tp __value, false_type) +{ + using __tx = __itoa::__traits<_Tp>; + auto __diff = __last - __first; + +#if !defined(_LIBCPP_COMPILER_MSVC) + if (__tx::digits <= __diff || __tx::__width(__value) <= __diff) + return {__tx::__convert(__value, __first), {}}; + else + return {__last, errc::value_too_large}; +#else + if (__tx::digits <= __diff) + return {__tx::__convert(__value, __first), {}}; + else + { + char __buf[__tx::digits]; + auto __p = __tx::__convert(__value, __buf); + auto __len = __p - __buf; + if (__len <= __diff) + { + memcpy(__first, __buf, __len); + return {__first + __len, {}}; + } + else + return {__last, errc::value_too_large}; + } +#endif +} + +template +inline _LIBCPP_INLINE_VISIBILITY to_chars_result +__to_chars_integral(char* __first, char* __last, _Tp __value, int __base, + true_type) +{ + auto __x = __to_unsigned(__value); *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CE83106D7B7; Thu, 2 Aug 2018 17:33:42 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18AFA726F1; Thu, 2 Aug 2018 17:33:39 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6449B14D35; Thu, 2 Aug 2018 17:33:39 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXdHl029611; Thu, 2 Aug 2018 17:33:39 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXdHN029610; Thu, 2 Aug 2018 17:33:39 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXdHN029610@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337144 - vendor/libc++/libc++-trunk-r338536 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/libc++/libc++-trunk-r338536 X-SVN-Commit-Revision: 337144 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:42 -0000 Author: dim Date: Thu Aug 2 17:33:38 2018 New Revision: 337144 URL: https://svnweb.freebsd.org/changeset/base/337144 Log: Tag libc++ trunk r338536. Added: vendor/libc++/libc++-trunk-r338536/ - copied from r337143, vendor/libc++/dist/ From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6EEF5106D805; Thu, 2 Aug 2018 17:33:49 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04A8072818; Thu, 2 Aug 2018 17:33:47 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5A3E214D37; Thu, 2 Aug 2018 17:33:47 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXlpg029685; Thu, 2 Aug 2018 17:33:47 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXgga029660; Thu, 2 Aug 2018 17:33:42 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXgga029660@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337145 - in vendor/lld/dist: COFF ELF ELF/Arch docs test/ELF test/ELF/Inputs test/ELF/linkerscript test/ELF/lto test/ELF/lto/Inputs X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/lld/dist: COFF ELF ELF/Arch docs test/ELF test/ELF/Inputs test/ELF/linkerscript test/ELF/lto test/ELF/lto/Inputs X-SVN-Commit-Revision: 337145 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:50 -0000 Author: dim Date: Thu Aug 2 17:33:42 2018 New Revision: 337145 URL: https://svnweb.freebsd.org/changeset/base/337145 Log: Vendor import of lld trunk r338536: https://llvm.org/svn/llvm-project/lld/trunk@338536 Added: vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-base.s (contents, props changed) vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-compat.s (contents, props changed) vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-toolchain.s (contents, props changed) vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-vfp.s (contents, props changed) vendor/lld/dist/test/ELF/Inputs/icf-absolute2.s (contents, props changed) vendor/lld/dist/test/ELF/arm-tag-vfp-args-errs.s (contents, props changed) vendor/lld/dist/test/ELF/arm-tag-vfp-args-illegal.s (contents, props changed) vendor/lld/dist/test/ELF/arm-tag-vfp-args.s (contents, props changed) vendor/lld/dist/test/ELF/execute-only-mixed-data.s (contents, props changed) vendor/lld/dist/test/ELF/execute-only.s (contents, props changed) vendor/lld/dist/test/ELF/icf-absolute2.s (contents, props changed) vendor/lld/dist/test/ELF/icf17.s (contents, props changed) vendor/lld/dist/test/ELF/lto/Inputs/libcall-archive.ll vendor/lld/dist/test/ELF/lto/libcall-archive.ll Modified: vendor/lld/dist/COFF/ICF.cpp vendor/lld/dist/ELF/Arch/ARM.cpp vendor/lld/dist/ELF/Arch/Hexagon.cpp vendor/lld/dist/ELF/Config.h vendor/lld/dist/ELF/Driver.cpp vendor/lld/dist/ELF/ICF.cpp vendor/lld/dist/ELF/InputFiles.cpp vendor/lld/dist/ELF/InputSection.cpp vendor/lld/dist/ELF/Options.td vendor/lld/dist/ELF/SyntheticSections.cpp vendor/lld/dist/ELF/SyntheticSections.h vendor/lld/dist/ELF/Writer.cpp vendor/lld/dist/docs/index.rst vendor/lld/dist/docs/ld.lld.1 vendor/lld/dist/docs/windows_support.rst vendor/lld/dist/test/ELF/arm-eabi-version.s vendor/lld/dist/test/ELF/hexagon.s vendor/lld/dist/test/ELF/icf-safe.s vendor/lld/dist/test/ELF/linkerscript/orphan-report.s vendor/lld/dist/test/ELF/oformat-binary.s vendor/lld/dist/test/ELF/relocatable-many-sections.s Modified: vendor/lld/dist/COFF/ICF.cpp ============================================================================== --- vendor/lld/dist/COFF/ICF.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/COFF/ICF.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -27,6 +27,7 @@ #include "llvm/Support/Debug.h" #include "llvm/Support/Parallel.h" #include "llvm/Support/raw_ostream.h" +#include "llvm/Support/xxhash.h" #include #include #include @@ -65,13 +66,6 @@ class ICF { (private) std::atomic Repeat = {false}; }; -// Returns a hash value for S. -uint32_t ICF::getHash(SectionChunk *C) { - return hash_combine(C->getOutputCharacteristics(), C->SectionName, - C->Relocs.size(), uint32_t(C->Header->SizeOfRawData), - C->Checksum, C->getContents()); -} - // Returns true if section S is subject of ICF. // // Microsoft's documentation @@ -265,7 +259,7 @@ void ICF::run(ArrayRef Vec) { // Initially, we use hash values to partition sections. for_each(parallel::par, Chunks.begin(), Chunks.end(), [&](SectionChunk *SC) { // Set MSB to 1 to avoid collisions with non-hash classs. - SC->Class[0] = getHash(SC) | (1 << 31); + SC->Class[0] = xxHash64(SC->getContents()) | (1 << 31); }); // From now on, sections in Chunks are ordered so that sections in Modified: vendor/lld/dist/ELF/Arch/ARM.cpp ============================================================================== --- vendor/lld/dist/ELF/Arch/ARM.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/Arch/ARM.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -97,10 +97,19 @@ ARM::ARM() { } uint32_t ARM::calcEFlags() const { + // The ABIFloatType is used by loaders to detect the floating point calling + // convention. + uint32_t ABIFloatType = 0; + if (Config->ARMVFPArgs == ARMVFPArgKind::Base || + Config->ARMVFPArgs == ARMVFPArgKind::Default) + ABIFloatType = EF_ARM_ABI_FLOAT_SOFT; + else if (Config->ARMVFPArgs == ARMVFPArgKind::VFP) + ABIFloatType = EF_ARM_ABI_FLOAT_HARD; + // We don't currently use any features incompatible with EF_ARM_EABI_VER5, // but we don't have any firm guarantees of conformance. Linux AArch64 // kernels (as of 2016) require an EABI version to be set. - return EF_ARM_EABI_VER5; + return EF_ARM_EABI_VER5 | ABIFloatType; } RelExpr ARM::getRelExpr(RelType Type, const Symbol &S, Modified: vendor/lld/dist/ELF/Arch/Hexagon.cpp ============================================================================== --- vendor/lld/dist/ELF/Arch/Hexagon.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/Arch/Hexagon.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -70,6 +70,12 @@ void Hexagon::relocateOne(uint8_t *Loc, RelType Type, switch (Type) { case R_HEX_NONE: break; + case R_HEX_12_X: + or32le(Loc, applyMask(0x000007e0, Val)); + break; + case R_HEX_32_6_X: + or32le(Loc, applyMask(0x0fff3fff, Val >> 6)); + break; case R_HEX_B15_PCREL: or32le(Loc, applyMask(0x00df20fe, Val >> 2)); break; Modified: vendor/lld/dist/ELF/Config.h ============================================================================== --- vendor/lld/dist/ELF/Config.h Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/Config.h Thu Aug 2 17:33:42 2018 (r337145) @@ -58,6 +58,9 @@ enum class SortSectionPolicy { Default, None, Alignmen // For --target2 enum class Target2Policy { Abs, Rel, GotRel }; +// For tracking ARM Float Argument PCS +enum class ARMVFPArgKind { Default, Base, VFP, ToolChain }; + struct SymbolVersion { llvm::StringRef Name; bool IsExternCpp; @@ -133,6 +136,7 @@ struct Configuration { bool EhFrameHdr; bool EmitRelocs; bool EnableNewDtags; + bool ExecuteOnly; bool ExportDynamic; bool FixCortexA53Errata843419; bool GcSections; @@ -195,6 +199,7 @@ struct Configuration { StripPolicy Strip; UnresolvedPolicy UnresolvedSymbols; Target2Policy Target2; + ARMVFPArgKind ARMVFPArgs = ARMVFPArgKind::Default; BuildIdKind BuildId = BuildIdKind::None; ELFKind EKind = ELFNoneKind; uint16_t DefaultSymbolVersion = llvm::ELF::VER_NDX_GLOBAL; Modified: vendor/lld/dist/ELF/Driver.cpp ============================================================================== --- vendor/lld/dist/ELF/Driver.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/Driver.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -302,6 +302,14 @@ static void checkOptions(opt::InputArgList &Args) { if (Config->Pie) error("-r and -pie may not be used together"); } + + if (Config->ExecuteOnly) { + if (Config->EMachine != EM_AARCH64) + error("-execute-only is only supported on AArch64 targets"); + + if (Config->SingleRoRx && !Script->HasSectionsCommand) + error("-execute-only and -no-rosegment cannot be used together"); + } } static const char *getReproduceOption(opt::InputArgList &Args) { @@ -493,6 +501,8 @@ static bool isOutputFormatBinary(opt::InputArgList &Ar StringRef S = Arg->getValue(); if (S == "binary") return true; + if (S.startswith("elf")) + return false; error("unknown --oformat value: " + S); } return false; @@ -747,6 +757,8 @@ void LinkerDriver::readConfigs(opt::InputArgList &Args Config->EnableNewDtags = Args.hasFlag(OPT_enable_new_dtags, OPT_disable_new_dtags, true); Config->Entry = Args.getLastArgValue(OPT_entry); + Config->ExecuteOnly = + Args.hasFlag(OPT_execute_only, OPT_no_execute_only, false); Config->ExportDynamic = Args.hasFlag(OPT_export_dynamic, OPT_no_export_dynamic, false); Config->FilterList = args::getStrings(Args, OPT_filter); @@ -1303,6 +1315,12 @@ static void findKeepUniqueSections(opt::InputArgList & } } +static const char *LibcallRoutineNames[] = { +#define HANDLE_LIBCALL(code, name) name, +#include "llvm/IR/RuntimeLibcalls.def" +#undef HANDLE_LIBCALL +}; + // Do actual linking. Note that when this function is called, // all linker scripts have already been parsed. template void LinkerDriver::link(opt::InputArgList &Args) { @@ -1369,10 +1387,20 @@ template void LinkerDriver::link(opt::Inp for (StringRef S : Config->Undefined) handleUndefined(S); - // If an entry symbol is in a static archive, pull out that file now - // to complete the symbol table. After this, no new names except a - // few linker-synthesized ones will be added to the symbol table. + // If an entry symbol is in a static archive, pull out that file now. handleUndefined(Config->Entry); + + // If any of our inputs are bitcode files, the LTO code generator may create + // references to certain library functions that might not be explicit in the + // bitcode file's symbol table. If any of those library functions are defined + // in a bitcode file in an archive member, we need to arrange to use LTO to + // compile those archive members by adding them to the link beforehand. + // + // With this the symbol table should be complete. After this, no new names + // except a few linker-synthesized ones will be added to the symbol table. + if (!BitcodeFiles.empty()) + for (const char *S : LibcallRoutineNames) + handleUndefined(S); // Return if there were name resolution errors. if (errorCount()) Modified: vendor/lld/dist/ELF/ICF.cpp ============================================================================== --- vendor/lld/dist/ELF/ICF.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/ICF.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -80,9 +80,10 @@ #include "SyntheticSections.h" #include "Writer.h" #include "lld/Common/Threads.h" -#include "llvm/ADT/Hashing.h" +#include "llvm/ADT/StringExtras.h" #include "llvm/BinaryFormat/ELF.h" #include "llvm/Object/ELF.h" +#include "llvm/Support/xxhash.h" #include #include @@ -155,12 +156,6 @@ template class ICF { (private) }; } -// Returns a hash value for S. Note that the information about -// relocation targets is not included in the hash value. -template static uint32_t getHash(InputSection *S) { - return hash_combine(S->Flags, S->getSize(), S->NumRelocations, S->Data); -} - // Returns true if section S is subject of ICF. static bool isEligible(InputSection *S) { if (!S->Live || S->KeepUnique || !(S->Flags & SHF_ALLOC)) @@ -441,7 +436,7 @@ template void ICF::run() { // Initially, we use hash values to partition sections. parallelForEach(Sections, [&](InputSection *S) { // Set MSB to 1 to avoid collisions with non-hash IDs. - S->Class[0] = getHash(S) | (1U << 31); + S->Class[0] = xxHash64(S->Data) | (1U << 31); }); // From now on, sections in Sections vector are ordered so that sections Modified: vendor/lld/dist/ELF/InputFiles.cpp ============================================================================== --- vendor/lld/dist/ELF/InputFiles.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/InputFiles.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -494,6 +494,46 @@ void ObjFile::initializeSections( } } +// For ARM only, to set the EF_ARM_ABI_FLOAT_SOFT or EF_ARM_ABI_FLOAT_HARD +// flag in the ELF Header we need to look at Tag_ABI_VFP_args to find out how +// the input objects have been compiled. +static void updateARMVFPArgs(const ARMAttributeParser &Attributes, + const InputFile *F) { + if (!Attributes.hasAttribute(ARMBuildAttrs::ABI_VFP_args)) + // If an ABI tag isn't present then it is implicitly given the value of 0 + // which maps to ARMBuildAttrs::BaseAAPCS. However many assembler files, + // including some in glibc that don't use FP args (and should have value 3) + // don't have the attribute so we do not consider an implicit value of 0 + // as a clash. + return; + + unsigned VFPArgs = Attributes.getAttributeValue(ARMBuildAttrs::ABI_VFP_args); + ARMVFPArgKind Arg; + switch (VFPArgs) { + case ARMBuildAttrs::BaseAAPCS: + Arg = ARMVFPArgKind::Base; + break; + case ARMBuildAttrs::HardFPAAPCS: + Arg = ARMVFPArgKind::VFP; + break; + case ARMBuildAttrs::ToolChainFPPCS: + // Tool chain specific convention that conforms to neither AAPCS variant. + Arg = ARMVFPArgKind::ToolChain; + break; + case ARMBuildAttrs::CompatibleFPAAPCS: + // Object compatible with all conventions. + return; + default: + error(toString(F) + ": unknown Tag_ABI_VFP_args value: " + Twine(VFPArgs)); + return; + } + // Follow ld.bfd and error if there is a mix of calling conventions. + if (Config->ARMVFPArgs != Arg && Config->ARMVFPArgs != ARMVFPArgKind::Default) + error(toString(F) + ": incompatible Tag_ABI_VFP_args"); + else + Config->ARMVFPArgs = Arg; +} + // The ARM support in lld makes some use of instructions that are not available // on all ARM architectures. Namely: // - Use of BLX instruction for interworking between ARM and Thumb state. @@ -573,6 +613,8 @@ InputSectionBase *ObjFile::createInputSection(co ArrayRef Contents = check(this->getObj().getSectionContents(&Sec)); Attributes.Parse(Contents, /*isLittle*/ Config->EKind == ELF32LEKind); updateSupportedARMFeatures(Attributes); + updateARMVFPArgs(Attributes, this); + // FIXME: Retain the first attribute section we see. The eglibc ARM // dynamic loaders require the presence of an attribute section for dlopen // to work. In a full implementation we would merge all attribute sections. Modified: vendor/lld/dist/ELF/InputSection.cpp ============================================================================== --- vendor/lld/dist/ELF/InputSection.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/InputSection.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -221,8 +221,8 @@ template Defined *InputSectionBase::getEnclosingFunction(uint64_t Offset) { for (Symbol *B : File->getSymbols()) if (Defined *D = dyn_cast(B)) - if (D->Section == this && D->Type == STT_FUNC && - D->Value <= Offset && Offset < D->Value + D->Size) + if (D->Section == this && D->Type == STT_FUNC && D->Value <= Offset && + Offset < D->Value + D->Size) return D; return nullptr; } @@ -671,7 +671,7 @@ static uint64_t getRelocTargetVA(const InputFile *File case R_TLSLD_GOT_FROM_END: return InX::Got->getTlsIndexOff() + A - InX::Got->getSize(); case R_TLSLD_GOT: - return InX::Got->getTlsIndexOff() + A; + return InX::Got->getTlsIndexOff() + A; case R_TLSLD_PC: return InX::Got->getTlsIndexVA() + A - P; } @@ -842,8 +842,7 @@ void InputSectionBase::relocateAlloc(uint8_t *Buf, uin // For each function-defining prologue, find any calls to __morestack, // and replace them with calls to __morestack_non_split. static void switchMorestackCallsToMorestackNonSplit( - llvm::DenseSet& Prologues, - std::vector& MorestackCalls) { + DenseSet &Prologues, std::vector &MorestackCalls) { // If the target adjusted a function's prologue, all calls to // __morestack inside that function should be switched to @@ -873,9 +872,8 @@ static void switchMorestackCallsToMorestackNonSplit( } } -static bool -enclosingPrologueAdjusted(uint64_t Offset, - const llvm::DenseSet &Prologues) { +static bool enclosingPrologueAdjusted(uint64_t Offset, + const DenseSet &Prologues) { for (Defined *F : Prologues) if (F->Value <= Offset && Offset < F->Value + F->Size) return true; @@ -891,7 +889,7 @@ void InputSectionBase::adjustSplitStackFunctionPrologu uint8_t *End) { if (!getFile()->SplitStack) return; - llvm::DenseSet AdjustedPrologues; + DenseSet AdjustedPrologues; std::vector MorestackCalls; for (Relocation &Rel : Relocations) { @@ -1071,8 +1069,7 @@ void MergeInputSection::splitNonStrings(ArrayRef Modified: vendor/lld/dist/ELF/Options.td ============================================================================== --- vendor/lld/dist/ELF/Options.td Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/Options.td Thu Aug 2 17:33:42 2018 (r337145) @@ -58,8 +58,8 @@ defm allow_multiple_definition: B<"allow-multiple-defi "Do not allow multiple definitions (default)">; defm apply_dynamic_relocs: B<"apply-dynamic-relocs", - "Apply dynamic relocations to place", - "Do not apply dynamic relocations to place">; + "Apply link-time values for dynamic relocations", + "Do not apply link-time values for dynamic relocations (default)">; defm as_needed: B<"as-needed", "Only set DT_NEEDED for shared libraries if used", @@ -130,6 +130,10 @@ def error_unresolved_symbols: F<"error-unresolved-symb HelpText<"Report unresolved symbols as errors">; defm exclude_libs: Eq<"exclude-libs", "Exclude static libraries from automatic export">; + +defm execute_only: B<"execute-only", + "Do not mark executable sections readable", + "Mark executable sections readable (default)">; defm export_dynamic: B<"export-dynamic", "Put symbols in the dynamic symbol table", Modified: vendor/lld/dist/ELF/SyntheticSections.cpp ============================================================================== --- vendor/lld/dist/ELF/SyntheticSections.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/SyntheticSections.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -335,7 +335,7 @@ void BuildIdSection::writeBuildId(ArrayRef Bu switch (Config->BuildId) { case BuildIdKind::Fast: computeHash(Buf, [](uint8_t *Dest, ArrayRef Arr) { - write64le(Dest, xxHash64(toStringRef(Arr))); + write64le(Dest, xxHash64(Arr)); }); break; case BuildIdKind::Md5: @@ -1935,6 +1935,23 @@ SymbolTableSection::SymbolTableSection(StringTab this->Entsize = sizeof(Elf_Sym); } +static BssSection *getCommonSec(Symbol *Sym) { + if (!Config->DefineCommon) + if (auto *D = dyn_cast(Sym)) + return dyn_cast_or_null(D->Section); + return nullptr; +} + +static uint32_t getSymSectionIndex(Symbol *Sym) { + if (getCommonSec(Sym)) + return SHN_COMMON; + if (!isa(Sym) || Sym->NeedsPltAddr) + return SHN_UNDEF; + if (const OutputSection *OS = Sym->getOutputSection()) + return OS->SectionIndex >= SHN_LORESERVE ? SHN_XINDEX : OS->SectionIndex; + return SHN_ABS; +} + // Write the internal symbol table contents to the output symbol table. template void SymbolTableSection::writeTo(uint8_t *Buf) { // The first entry is a null entry as per the ELF spec. @@ -1956,23 +1973,8 @@ template void SymbolTableSection::w } ESym->st_name = Ent.StrTabOffset; + ESym->st_shndx = getSymSectionIndex(Ent.Sym); - // Set a section index. - BssSection *CommonSec = nullptr; - if (!Config->DefineCommon) - if (auto *D = dyn_cast(Sym)) - CommonSec = dyn_cast_or_null(D->Section); - if (CommonSec) - ESym->st_shndx = SHN_COMMON; - else if (Sym->NeedsPltAddr) - ESym->st_shndx = SHN_UNDEF; - else if (const OutputSection *OutSec = Sym->getOutputSection()) - ESym->st_shndx = OutSec->SectionIndex; - else if (isa(Sym)) - ESym->st_shndx = SHN_ABS; - else - ESym->st_shndx = SHN_UNDEF; - // Copy symbol size if it is a defined symbol. st_size is not significant // for undefined symbols, so whether copying it or not is up to us if that's // the case. We'll leave it as zero because by not setting a value, we can @@ -1986,7 +1988,7 @@ template void SymbolTableSection::w // st_value is usually an address of a symbol, but that has a // special meaining for uninstantiated common symbols (this can // occur if -r is given). - if (CommonSec) + if (BssSection *CommonSec = getCommonSec(Ent.Sym)) ESym->st_value = CommonSec->Alignment; else ESym->st_value = Sym->getVA(); @@ -2026,6 +2028,44 @@ template void SymbolTableSection::w } } +SymtabShndxSection::SymtabShndxSection() + : SyntheticSection(0, SHT_SYMTAB_SHNDX, 4, ".symtab_shndxr") { + this->Entsize = 4; +} + +void SymtabShndxSection::writeTo(uint8_t *Buf) { + // We write an array of 32 bit values, where each value has 1:1 association + // with an entry in .symtab. If the corresponding entry contains SHN_XINDEX, + // we need to write actual index, otherwise, we must write SHN_UNDEF(0). + Buf += 4; // Ignore .symtab[0] entry. + for (const SymbolTableEntry &Entry : InX::SymTab->getSymbols()) { + if (getSymSectionIndex(Entry.Sym) == SHN_XINDEX) + write32(Buf, Entry.Sym->getOutputSection()->SectionIndex); + Buf += 4; + } +} + +bool SymtabShndxSection::empty() const { + // SHT_SYMTAB can hold symbols with section indices values up to + // SHN_LORESERVE. If we need more, we want to use extension SHT_SYMTAB_SHNDX + // section. Problem is that we reveal the final section indices a bit too + // late, and we do not know them here. For simplicity, we just always create + // a .symtab_shndxr section when the amount of output sections is huge. + size_t Size = 0; + for (BaseCommand *Base : Script->SectionCommands) + if (isa(Base)) + ++Size; + return Size < SHN_LORESERVE; +} + +void SymtabShndxSection::finalizeContents() { + getParent()->Link = InX::SymTab->getParent()->SectionIndex; +} + +size_t SymtabShndxSection::getSize() const { + return InX::SymTab->getNumSymbols() * 4; +} + // .hash and .gnu.hash sections contain on-disk hash tables that map // symbol names to their dynamic symbol table indices. Their purpose // is to help the dynamic linker resolve symbols quickly. If ELF files @@ -3025,6 +3065,7 @@ RelocationBaseSection *InX::RelaIplt; StringTableSection *InX::ShStrTab; StringTableSection *InX::StrTab; SymbolTableBaseSection *InX::SymTab; +SymtabShndxSection *InX::SymTabShndx; template GdbIndexSection *GdbIndexSection::create(); template GdbIndexSection *GdbIndexSection::create(); Modified: vendor/lld/dist/ELF/SyntheticSections.h ============================================================================== --- vendor/lld/dist/ELF/SyntheticSections.h Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/SyntheticSections.h Thu Aug 2 17:33:42 2018 (r337145) @@ -588,6 +588,16 @@ class SymbolTableSection final : public SymbolTableBas void writeTo(uint8_t *Buf) override; }; +class SymtabShndxSection final : public SyntheticSection { +public: + SymtabShndxSection(); + + void writeTo(uint8_t *Buf) override; + size_t getSize() const override; + bool empty() const override; + void finalizeContents() override; +}; + // Outputs GNU Hash section. For detailed explanation see: // https://blogs.oracle.com/ali/entry/gnu_hash_elf_sections class GnuHashTableSection final : public SyntheticSection { @@ -992,6 +1002,7 @@ struct InX { static StringTableSection *ShStrTab; static StringTableSection *StrTab; static SymbolTableBaseSection *SymTab; + static SymtabShndxSection* SymTabShndx; }; template struct In { Modified: vendor/lld/dist/ELF/Writer.cpp ============================================================================== --- vendor/lld/dist/ELF/Writer.cpp Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/ELF/Writer.cpp Thu Aug 2 17:33:42 2018 (r337145) @@ -287,6 +287,7 @@ template static void createSyntheticSecti if (Config->Strip != StripPolicy::All) { InX::StrTab = make(".strtab", false); InX::SymTab = make>(*InX::StrTab); + InX::SymTabShndx = make(); } if (Config->BuildId != BuildIdKind::None) { @@ -409,6 +410,8 @@ template static void createSyntheticSecti if (InX::SymTab) Add(InX::SymTab); + if (InX::SymTabShndx) + Add(InX::SymTabShndx); Add(InX::ShStrTab); if (InX::StrTab) Add(InX::StrTab); @@ -518,7 +521,6 @@ static bool shouldKeepInSymtab(SectionBase *Sec, Strin if (B.isSection()) return false; - if (Config->Discard == DiscardPolicy::None) return true; @@ -1605,6 +1607,15 @@ template void Writer::finalizeSecti if (auto *Sec = dyn_cast(Base)) OutputSections.push_back(Sec); + // Ensure data sections are not mixed with executable sections when + // -execute-only is used. + if (Config->ExecuteOnly) + for (OutputSection *OS : OutputSections) + if (OS->Flags & SHF_EXECINSTR) + for (InputSection *IS : getInputSections(OS)) + if (!(IS->Flags & SHF_EXECINSTR)) + error("-execute-only does not support intermingling data and code"); + // Prefer command line supplied address over other constraints. for (OutputSection *Sec : OutputSections) { auto I = Config->SectionStartMap.find(Sec->Name); @@ -1639,12 +1650,13 @@ template void Writer::finalizeSecti // Dynamic section must be the last one in this list and dynamic // symbol table section (DynSymTab) must be the first one. applySynthetic( - {InX::DynSymTab, InX::Bss, InX::BssRelRo, InX::GnuHashTab, - InX::HashTab, InX::SymTab, InX::ShStrTab, InX::StrTab, - In::VerDef, InX::DynStrTab, InX::Got, InX::MipsGot, - InX::IgotPlt, InX::GotPlt, InX::RelaDyn, InX::RelrDyn, - InX::RelaIplt, InX::RelaPlt, InX::Plt, InX::Iplt, - InX::EhFrameHdr, In::VerSym, In::VerNeed, InX::Dynamic}, + {InX::DynSymTab, InX::Bss, InX::BssRelRo, InX::GnuHashTab, + InX::HashTab, InX::SymTab, InX::SymTabShndx, InX::ShStrTab, + InX::StrTab, In::VerDef, InX::DynStrTab, InX::Got, + InX::MipsGot, InX::IgotPlt, InX::GotPlt, InX::RelaDyn, + InX::RelrDyn, InX::RelaIplt, InX::RelaPlt, InX::Plt, + InX::Iplt, InX::EhFrameHdr, In::VerSym, In::VerNeed, + InX::Dynamic}, [](SyntheticSection *SS) { SS->finalizeContents(); }); if (!Script->HasSectionsCommand && !Config->Relocatable) @@ -1763,6 +1775,8 @@ static bool needsPtLoad(OutputSection *Sec) { static uint64_t computeFlags(uint64_t Flags) { if (Config->Omagic) return PF_R | PF_W | PF_X; + if (Config->ExecuteOnly && (Flags & PF_X)) + return Flags & ~PF_R; if (Config->SingleRoRx && !(Flags & PF_W)) return Flags | PF_X; return Flags; Modified: vendor/lld/dist/docs/index.rst ============================================================================== --- vendor/lld/dist/docs/index.rst Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/docs/index.rst Thu Aug 2 17:33:42 2018 (r337145) @@ -8,7 +8,7 @@ features that are useful for toolchain developers. The linker supports ELF (Unix), PE/COFF (Windows), Mach-O (macOS) and WebAssembly in descending order of completeness. Internally, LLD consists of several different linkers. The ELF port is the one that will be described in -this document. The PE/COFF port is almost complete except the lack of the +this document. The PE/COFF port is complete, including Windows debug info (PDB) support. The WebAssembly port is still a work in progress (See :doc:`WebAssembly`). The Mach-O port is built based on a different architecture than the others. For the details about Mach-O, please Modified: vendor/lld/dist/docs/ld.lld.1 ============================================================================== --- vendor/lld/dist/docs/ld.lld.1 Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/docs/ld.lld.1 Thu Aug 2 17:33:42 2018 (r337145) @@ -3,7 +3,7 @@ .\" .\" This man page documents only lld's ELF linking support, obtained originally .\" from FreeBSD. -.Dd April 28, 2018 +.Dd July 30, 2018 .Dt LD.LLD 1 .Os .Sh NAME @@ -30,6 +30,8 @@ These options are available: .It Fl -allow-multiple-definition Do not error if a symbol is defined multiple times. The first definition will be used. +.It Fl -apply-dynamic-relocs +Apply link-time values for dynamic relocations. .It Fl -as-needed Only set .Dv DT_NEEDED @@ -38,14 +40,14 @@ for shared libraries if used. Set the .Dv DT_AUXILIARY field to the specified name. -.It Fl -Bdynamic +.It Fl -Bdynamic , Fl -dy Link against shared libraries. -.It Fl -Bstatic +.It Fl -Bstatic , Fl -static , Fl -dn Do not link against shared libraries. -.It Fl -Bsymbolic-functions -Bind defined function symbols locally. .It Fl -Bsymbolic Bind defined symbols locally. +.It Fl -Bsymbolic-functions +Bind defined function symbols locally. .It Fl -build-id Ns = Ns Ar value Generate a build ID note. .Ar value @@ -93,7 +95,9 @@ may be .Cm none or .Cm zlib . -.It Fl -define-common +.It Fl -cref +Output cross reference table. +.It Fl -define-common , Fl d Assign space to common symbols. .It Fl -defsym Ns = Ns Ar symbol Ns = Ns Ar expression Define a symbol alias. @@ -107,9 +111,9 @@ or Demangle symbol names. .It Fl -disable-new-dtags Disable new dynamic tags. -.It Fl -discard-all +.It Fl -discard-all , Fl x Delete all local symbols. -.It Fl -discard-locals +.It Fl -discard-locals , Fl X Delete temporary local symbols. .It Fl -discard-none Keep all symbols in the symbol table. @@ -126,7 +130,7 @@ Request creation of section and .Dv PT_GNU_EH_FRAME segment header. -.It Fl -emit-relocs +.It Fl -emit-relocs , Fl q Generate relocations in the output. .It Fl -enable-new-dtags Enable new dynamic tags. @@ -142,21 +146,21 @@ A value of zero indicates that there is no limit. Report unresolved symbols as errors. .It Fl -exclude-libs Ns = Ns Ar value Exclude static libraries from automatic export. +.It Fl -export-dynamic , Fl E +Put symbols in the dynamic symbol table. .It Fl -export-dynamic-symbol Ns = Ns Ar symbol Include .Ar symbol in the dynamic symbol table. -.It Fl -export-dynamic -Put symbols in the dynamic symbol table. .It Fl -fatal-warnings Treat warnings as errors. -.It Fl -filter Ns = Ns Ar value +.It Fl -filter Ns = Ns Ar value , Fl F Ar value Set the .Dv DT_FILTER field to the specified value. .It Fl -fini Ns = Ns Ar symbol Specify a finalizer function. -.It Fl -format Ns = Ns Ar input-format +.It Fl -format Ns = Ns Ar input-format , Fl b Ar input-format Specify the format of the inputs following this option. .Ar input-format may be one of @@ -196,6 +200,14 @@ Set the base address to .Ar value . .It Fl -init Ns = Ns Ar symbol Specify an initializer function. +.It Fl -keep-unique Ns = Ns Ar symbol +Do not fold +.Ar symbol +during ICF. +.It Fl l Ar libName, Fl -library Ns = Ns Ar libName +Root name of library to use. +.It Fl L Ar dir , Fl -library-path Ns = Ns Ar dir +Add a directory to the library search path. .It Fl -lto-aa-pipeline Ns = Ns Ar value AA pipeline to run during LTO. Used in conjunction with @@ -206,15 +218,11 @@ Passes to run during LTO. Optimization level for LTO. .It Fl -lto-partitions Ns = Ns Ar value Number of LTO codegen partitions. -.It Fl L Ar dir -Add a directory to the library search path. -.It Fl l Ar libName -Root name of library to use. -.It Fl -Map Ns = Ns Ar file -Print a link map to -.Ar file . .It Fl m Ar value Set target emulation. +.It Fl -Map Ns = Ns Ar file , Fl M Ar file +Print a link map to +.Ar file . .It Fl -no-as-needed Always set .Dv DT_NEEDED @@ -243,26 +251,18 @@ Report version scripts that refer undefined symbols. Report unresolved symbols even if the linker is creating a shared library. .It Fl -no-whole-archive Restores the default behavior of loading archive members. -.It Fl -noinhibit-exec -Retain the executable output file whenever it is still usable. .It Fl -no-pie Do not create a position independent executable. +.It Fl -noinhibit-exec +Retain the executable output file whenever it is still usable. .It Fl -nostdlib Only search directories specified on the command line. -.It Fl -oformat Ns = Ns Ar format -Specify the format for the output object file. -The only supported -.Ar format -is -.Cm binary , -which produces output with no ELF header. -.It Fl -omagic -Set the text and data sections to be readable and writable. -.It Fl -opt-remarks-filename Ar file -Write optimization remarks in YAML format to -.Ar file . -.It Fl -opt-remarks-with-hotness -Include hotness information in the optimization remarks file. +.It Fl o Ar path +Write the output executable, library, or object to +.Ar path . +If not specified, +.Dv a.out +is used as a default. .It Fl O Ns Ar value Optimize output file size. .Ar value @@ -279,12 +279,20 @@ Enable string tail merging. .Pp .Fl O Ns Cm 1 is the default. -.It Fl o Ar path -Write the output executable, library, or object to -.Ar path . -If not specified, -.Dv a.out -is used as a default. +.It Fl -oformat Ns = Ns Ar format +Specify the format for the output object file. +The only supported +.Ar format +is +.Cm binary , +which produces output with no ELF header. +.It Fl -omagic , Fl N +Set the text and data sections to be readable and writable. +.It Fl -opt-remarks-filename Ar file +Write optimization remarks in YAML format to +.Ar file . +.It Fl -opt-remarks-with-hotness +Include hotness information in the optimization remarks file. .It Fl -pie Create a position independent executable. .It Fl -print-gc-sections @@ -300,13 +308,13 @@ and .It Fl -pop-state Undo the effect of .Fl -push-state. -.It Fl -relocatable +.It Fl -relocatable , Fl r Create relocatable object file. .It Fl -reproduce Ns = Ns Ar value Dump linker invocation and input files for debugging. .It Fl -retain-symbols-file Ns = Ns Ar file Retain only the symbols listed in the file. -.It Fl -rpath Ns = Ns Ar value +.It Fl -rpath Ns = Ns Ar value , Fl R Ar value Add a .Dv DT_RUNPATH to the output. @@ -316,14 +324,14 @@ The supported values are .Cm windows and .Cm posix . -.It Fl -script Ns = Ns Ar file +.It Fl -script Ns = Ns Ar file , Fl T Ar file Read linker script from .Ar file . -.It Fl -section-start Ns = Ar section Ns = Ns Ar address +.It Fl -section-start Ns = Ns Ar section Ns = Ns Ar address Set address of section. -.It Fl -shared +.It Fl -shared , Fl -Bsharable Build a shared object. -.It Fl -soname Ns = Ns Ar value +.It Fl -soname Ns = Ns Ar value , Fl h Ar value Set .Dv DT_SONAME to @@ -333,9 +341,9 @@ Specifies sections sorting rule when linkerscript is u .It Fl -start-lib Start a grouping of objects that should be treated as if they were together in an archive. -.It Fl -strip-all +.It Fl -strip-all , Fl s Strip all symbols. -.It Fl -strip-debug +.It Fl -strip-debug , Fl S Strip debugging information. .It Fl -symbol-ordering-file Ns = Ns Ar file Lay out sections in the order specified by @@ -376,6 +384,12 @@ Same as with .Li .data as the sectionname. +.It Fl -Ttext Ns = Ns Ar value +Same as +.Fl -section-start +with +.Li .text +as the sectionname. .It Fl -thinlto-cache-dir Ns = Ns Ar value Path to ThinLTO cached object file directory. .It Fl -thinlto-cache-policy Ns = Ns Ar value @@ -385,33 +399,27 @@ Number of ThinLTO jobs. .It Fl -threads Run the linker multi-threaded. This option is enabled by default. -.It Fl -trace-symbol Ns = Ns Ar symbol -Trace references to -.Ar symbol . .It Fl -trace Print the names of the input files. -.It Fl -Ttext Ns = Ns Ar value -Same as -.Fl -section-start -with -.Li .text -as the sectionname. -.It Fl -undefined Ns = Ns Ar symbol +.It Fl -trace-symbol Ns = Ns Ar symbol , Fl y Ar symbol +Trace references to +.Ar symbol . +.It Fl -undefined Ns = Ns Ar symbol , Fl u Ar symbol Force .Ar symbol to be an undefined symbol during linking. .It Fl -unresolved-symbols Ns = Ns Ar value Determine how to handle unresolved symbols. +.It Fl v +Display the version number and proceed with linking if object files are +specified. +.It Fl V , Fl -version +Display the version number and exit. .It Fl -verbose Verbose mode. .It Fl -version-script Ns = Ns Ar file Read version script from .Ar file . -.It Fl V , Fl -version -Display the version number and exit. -.It Fl v -Display the version number and proceed with linking if object files are -specified. .It Fl -warn-backrefs Warn about reverse or cyclic dependencies to or between static archives. This can be used to ensure linker invocation remains compatible with @@ -432,6 +440,10 @@ Make the main stack executable. Stack permissions are recorded in the .Dv PT_GNU_STACK segment. +.It Cm initfirst +Sets the +.Dv DF_1_INITFIRST +flag to indicate the module should be initialized first. .It Cm muldefs Do not error if a symbol is defined multiple times. The first definition will be used. Modified: vendor/lld/dist/docs/windows_support.rst ============================================================================== --- vendor/lld/dist/docs/windows_support.rst Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/docs/windows_support.rst Thu Aug 2 17:33:42 2018 (r337145) @@ -60,6 +60,13 @@ Debug info link.exe. However, LLD does not support /DEBUG:FASTLINK. +Downloading LLD +=============== + +The Windows version of LLD is included in the `pre-built binaries of LLVM's +releases `_ and in the `LLVM Snapshot +Builds `_. + Building LLD ============ Added: vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-base.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-base.s Thu Aug 2 17:33:42 2018 (r337145) @@ -0,0 +1,16 @@ + .arch armv7-a + .eabi_attribute 20, 1 + .eabi_attribute 21, 1 + .eabi_attribute 23, 3 + .eabi_attribute 24, 1 + .eabi_attribute 25, 1 + .eabi_attribute 26, 2 + .eabi_attribute 30, 6 + .eabi_attribute 34, 1 + .eabi_attribute 18, 4 + .eabi_attribute 28, 0 // Tag_ABI_VFP_args = 0 (AAPCS, Base variant) + + .syntax unified + .global f0 + .type f0, %function +f0: bx lr Added: vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-compat.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-compat.s Thu Aug 2 17:33:42 2018 (r337145) @@ -0,0 +1,16 @@ + .arch armv7-a + .eabi_attribute 20, 1 + .eabi_attribute 21, 1 + .eabi_attribute 23, 3 + .eabi_attribute 24, 1 + .eabi_attribute 25, 1 + .eabi_attribute 26, 2 + .eabi_attribute 30, 6 + .eabi_attribute 34, 1 + .eabi_attribute 18, 4 + .eabi_attribute 28, 3 // Tag_ABI_VFP_args = 3 (Compatible with all) + + .syntax unified + .global f3 + .type f3, %function +f3: bx lr Added: vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-toolchain.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-toolchain.s Thu Aug 2 17:33:42 2018 (r337145) @@ -0,0 +1,15 @@ + .arch armv7-a + .eabi_attribute 20, 1 + .eabi_attribute 21, 1 + .eabi_attribute 23, 3 + .eabi_attribute 24, 1 + .eabi_attribute 25, 1 + .eabi_attribute 26, 2 + .eabi_attribute 30, 6 + .eabi_attribute 34, 1 + .eabi_attribute 18, 4 + .eabi_attribute 28, 2 // Tag_ABI_VFP_args = 2 (Toolchain specific) + .syntax unified + .global f2 + .type f1, %function +f2: bx lr Added: vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-vfp.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist/test/ELF/Inputs/arm-vfp-arg-vfp.s Thu Aug 2 17:33:42 2018 (r337145) @@ -0,0 +1,15 @@ + .arch armv7-a + .eabi_attribute 20, 1 + .eabi_attribute 21, 1 + .eabi_attribute 23, 3 + .eabi_attribute 24, 1 + .eabi_attribute 25, 1 + .eabi_attribute 26, 2 + .eabi_attribute 30, 6 + .eabi_attribute 34, 1 + .eabi_attribute 18, 4 + .eabi_attribute 28, 1 // Tag_ABI_VFP_args = 1 (AAPCS, VFP variant) + .syntax unified + .global f1 + .type f1, %function +f1: bx lr Added: vendor/lld/dist/test/ELF/Inputs/icf-absolute2.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist/test/ELF/Inputs/icf-absolute2.s Thu Aug 2 17:33:42 2018 (r337145) @@ -0,0 +1,3 @@ +.globl a1, a2 +a1 = 1 +a2 = 2 Modified: vendor/lld/dist/test/ELF/arm-eabi-version.s ============================================================================== --- vendor/lld/dist/test/ELF/arm-eabi-version.s Thu Aug 2 17:33:38 2018 (r337144) +++ vendor/lld/dist/test/ELF/arm-eabi-version.s Thu Aug 2 17:33:42 2018 (r337145) @@ -9,6 +9,7 @@ _start: bx lr // CHECK: Flags [ +// CHECK-NEXT: 0x200 // CHECK-NEXT: 0x1000000 // CHECK-NEXT: 0x4000000 // CHECK-NEXT: ] Added: vendor/lld/dist/test/ELF/arm-tag-vfp-args-errs.s ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ vendor/lld/dist/test/ELF/arm-tag-vfp-args-errs.s Thu Aug 2 17:33:42 2018 (r337145) @@ -0,0 +1,29 @@ *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Thu Aug 2 17:33:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44B00106D825; Thu, 2 Aug 2018 17:33:54 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58BFB728B8; Thu, 2 Aug 2018 17:33:51 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AEAB714D38; Thu, 2 Aug 2018 17:33:50 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HXoD2029732; Thu, 2 Aug 2018 17:33:50 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXo1l029731; Thu, 2 Aug 2018 17:33:50 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXo1l029731@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337146 - vendor/lld/lld-trunk-r338536 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lld/lld-trunk-r338536 X-SVN-Commit-Revision: 337146 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:33:54 -0000 Author: dim Date: Thu Aug 2 17:33:50 2018 New Revision: 337146 URL: https://svnweb.freebsd.org/changeset/base/337146 Log: Tag lld trunk r338536. Added: vendor/lld/lld-trunk-r338536/ - copied from r337145, vendor/lld/dist/ From owner-svn-src-all@freebsd.org Thu Aug 2 17:34:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E64D106D887; Thu, 2 Aug 2018 17:34:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33DB472AF4; Thu, 2 Aug 2018 17:34:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EF90A14D3A; Thu, 2 Aug 2018 17:34:07 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HY7el029881; Thu, 2 Aug 2018 17:34:07 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HY7h4029880; Thu, 2 Aug 2018 17:34:07 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021734.w72HY7h4029880@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:34:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337148 - vendor/lldb/lldb-trunk-r338536 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lldb/lldb-trunk-r338536 X-SVN-Commit-Revision: 337148 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:34:08 -0000 Author: dim Date: Thu Aug 2 17:34:07 2018 New Revision: 337148 URL: https://svnweb.freebsd.org/changeset/base/337148 Log: Tag lldb trunk r338536. Added: vendor/lldb/lldb-trunk-r338536/ - copied from r337147, vendor/lldb/dist/ From owner-svn-src-all@freebsd.org Thu Aug 2 17:34:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29664106D87E; Thu, 2 Aug 2018 17:34:07 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 96A7072AE9; Thu, 2 Aug 2018 17:34:05 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B095C14D39; Thu, 2 Aug 2018 17:34:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72HY4Ts029832; Thu, 2 Aug 2018 17:34:04 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72HXtup029781; Thu, 2 Aug 2018 17:33:55 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808021733.w72HXtup029781@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Thu, 2 Aug 2018 17:33:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337147 - in vendor/lldb/dist: include/lldb/Utility packages/Python/lldbsuite/test packages/Python/lldbsuite/test/api/multiple-debuggers packages/Python/lldbsuite/test/api/multithreaded... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/lldb/dist: include/lldb/Utility packages/Python/lldbsuite/test packages/Python/lldbsuite/test/api/multiple-debuggers packages/Python/lldbsuite/test/api/multithreaded packages/Python/lldbsuit... X-SVN-Commit-Revision: 337147 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:34:07 -0000 Author: dim Date: Thu Aug 2 17:33:54 2018 New Revision: 337147 URL: https://svnweb.freebsd.org/changeset/base/337147 Log: Vendor import of lldb trunk r338536: https://llvm.org/svn/llvm-project/lldb/trunk@338536 Added: vendor/lldb/dist/unittests/Utility/StreamTest.cpp (contents, props changed) Modified: vendor/lldb/dist/include/lldb/Utility/CompletionRequest.h vendor/lldb/dist/include/lldb/Utility/Stream.h vendor/lldb/dist/packages/Python/lldbsuite/test/api/multiple-debuggers/TestMultipleDebuggers.py vendor/lldb/dist/packages/Python/lldbsuite/test/api/multithreaded/TestMultithreaded.py vendor/lldb/dist/packages/Python/lldbsuite/test/decorators.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-function/TestCallStdStringFunction.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-restarts/TestCallThatRestarts.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-throws/TestCallThatThrows.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/char/TestExprsChar.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/fixits/TestFixIts.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/issue_11588/Test11588.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/pr35310/TestExprsBug35310.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/save_jit_objects/TestSaveJITObjects.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/unwind_expression/TestUnwindExpression.py vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/xvalue/TestXValuePrinting.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/address_breakpoints/TestBadAddressBreakpoints.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_locations/TestBreakpointLocations.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_names/TestBreakpointNames.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/hardware_breakpoints/hardware_breakpoint_on_multiple_threads/TestHWBreakMultiThread.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/serialize/TestBreakpointSerialization.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/TestCompletion.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/main.cpp vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/bitset/TestDataFormatterLibcxxBitset.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/list/TestDataFormatterLibcxxList.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/multiset/TestDataFormatterLibcxxMultiSet.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/tuple/TestDataFormatterStdTuple.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/exec/TestExec.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame-language/TestGuessLanguage.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame_var/TestFrameVar.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestTargetXMLArch.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/gdbclientutils.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/history/TestHistoryRecall.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_unload/TestLoadUnload.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_using_paths/TestLoadUsingPaths.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/pre_run_dylibs/TestPreRunDylibs.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/register/intel_avx/TestZMMRegister.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/return-value/TestReturnValue.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/create_during_step/TestCreateDuringStep.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/num_threads/TestNumThreads.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/step_until/TestStepUntil.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/watchpoint/multi_watchpoint_slots/TestWatchpointMultipleSlots.py vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/watchpoint/watchpoint_disable/TestWatchpointDisable.py vendor/lldb/dist/packages/Python/lldbsuite/test/lang/c/find_struct_type/TestFindStructTypes.py vendor/lldb/dist/packages/Python/lldbsuite/test/lang/cpp/dynamic-value-same-basename/TestDynamicValueSameBase.py vendor/lldb/dist/packages/Python/lldbsuite/test/lang/cpp/incomplete-types/TestCppIncompleteTypes.py vendor/lldb/dist/packages/Python/lldbsuite/test/lang/cpp/llvm-style/TestLLVMStyle.py vendor/lldb/dist/packages/Python/lldbsuite/test/lang/cpp/scope/TestCppScope.py vendor/lldb/dist/packages/Python/lldbsuite/test/lang/cpp/trivial_abi/TestTrivialABI.py vendor/lldb/dist/packages/Python/lldbsuite/test/lldbdwarf.py vendor/lldb/dist/packages/Python/lldbsuite/test/lldbtest.py vendor/lldb/dist/packages/Python/lldbsuite/test/lldbutil.py vendor/lldb/dist/packages/Python/lldbsuite/test/macosx/find-app-in-bundle/TestFindAppInBundle.py vendor/lldb/dist/packages/Python/lldbsuite/test/macosx/find-dsym/bundle-with-dot-in-filename/TestBundleWithDotInFilename.py vendor/lldb/dist/packages/Python/lldbsuite/test/macosx/find-dsym/deep-bundle/TestDeepBundle.py vendor/lldb/dist/packages/Python/lldbsuite/test/macosx/thread-names/TestInterruptThreadNames.py vendor/lldb/dist/packages/Python/lldbsuite/test/python_api/name_lookup/TestNameLookup.py vendor/lldb/dist/packages/Python/lldbsuite/test/python_api/sbdata/TestSBData.py vendor/lldb/dist/packages/Python/lldbsuite/test/sample_test/TestSampleTest.py vendor/lldb/dist/packages/Python/lldbsuite/test/settings/TestSettings.py vendor/lldb/dist/packages/Python/lldbsuite/test/tools/lldb-mi/startup_options/TestMiStartupOptions.py vendor/lldb/dist/packages/Python/lldbsuite/test/tools/lldb-server/TestGdbRemote_qThreadStopInfo.py vendor/lldb/dist/packages/Python/lldbsuite/test/tools/lldb-server/TestLldbGdbServer.py vendor/lldb/dist/source/Commands/CommandCompletions.cpp vendor/lldb/dist/source/Commands/CommandObjectCommands.cpp vendor/lldb/dist/source/Commands/CommandObjectFrame.cpp vendor/lldb/dist/source/Commands/CommandObjectMultiword.cpp vendor/lldb/dist/source/Commands/CommandObjectPlatform.cpp vendor/lldb/dist/source/Commands/CommandObjectPlugin.cpp vendor/lldb/dist/source/Commands/CommandObjectProcess.cpp vendor/lldb/dist/source/Commands/CommandObjectSettings.cpp vendor/lldb/dist/source/Commands/CommandObjectTarget.cpp vendor/lldb/dist/source/Core/FormatEntity.cpp vendor/lldb/dist/source/Core/IOHandler.cpp vendor/lldb/dist/source/Interpreter/CommandInterpreter.cpp vendor/lldb/dist/source/Interpreter/CommandObject.cpp vendor/lldb/dist/source/Interpreter/CommandObjectRegexCommand.cpp vendor/lldb/dist/source/Interpreter/OptionValue.cpp vendor/lldb/dist/source/Interpreter/OptionValueArch.cpp vendor/lldb/dist/source/Interpreter/OptionValueBoolean.cpp vendor/lldb/dist/source/Interpreter/OptionValueEnumeration.cpp vendor/lldb/dist/source/Interpreter/OptionValueFileSpec.cpp vendor/lldb/dist/source/Interpreter/OptionValueUUID.cpp vendor/lldb/dist/source/Interpreter/Options.cpp vendor/lldb/dist/source/Plugins/Platform/Android/PlatformAndroid.cpp vendor/lldb/dist/source/Plugins/Platform/Windows/PlatformWindows.cpp vendor/lldb/dist/source/Plugins/SymbolFile/DWARF/DWARFCompileUnit.h vendor/lldb/dist/source/Plugins/SymbolFile/DWARF/DWARFUnit.h vendor/lldb/dist/source/Symbol/Variable.cpp vendor/lldb/dist/source/Utility/ArchSpec.cpp vendor/lldb/dist/source/Utility/CompletionRequest.cpp vendor/lldb/dist/source/Utility/Stream.cpp vendor/lldb/dist/unittests/Utility/CMakeLists.txt vendor/lldb/dist/unittests/Utility/CompletionRequestTest.cpp Modified: vendor/lldb/dist/include/lldb/Utility/CompletionRequest.h ============================================================================== --- vendor/lldb/dist/include/lldb/Utility/CompletionRequest.h Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/include/lldb/Utility/CompletionRequest.h Thu Aug 2 17:33:54 2018 (r337147) @@ -13,6 +13,7 @@ #include "lldb/Utility/Args.h" #include "lldb/Utility/StringList.h" #include "llvm/ADT/StringRef.h" +#include "llvm/ADT/StringSet.h" namespace lldb_private { @@ -77,9 +78,30 @@ class CompletionRequest { (public) void SetWordComplete(bool v) { m_word_complete = v; } - /// The array of matches returned. - StringList &GetMatches() { return *m_matches; } + /// Adds a possible completion string. If the completion was already + /// suggested before, it will not be added to the list of results. A copy of + /// the suggested completion is stored, so the given string can be free'd + /// afterwards. + /// + /// @param match The suggested completion. + void AddCompletion(llvm::StringRef completion) { + // Add the completion if we haven't seen the same value before. + if (m_match_set.insert(completion).second) + m_matches->AppendString(completion); + } + /// Adds multiple possible completion strings. + /// + /// \param completions The list of completions. + /// + /// @see AddCompletion + void AddCompletions(const StringList &completions) { + for (std::size_t i = 0; i < completions.GetSize(); ++i) + AddCompletion(completions.GetStringAtIndex(i)); + } + + std::size_t GetNumberOfMatches() const { return m_matches->GetSize(); } + llvm::StringRef GetCursorArgument() const { return GetParsedLine().GetArgumentAtIndex(GetCursorIndex()); } @@ -111,8 +133,15 @@ class CompletionRequest { (public) /// \btrue if this is a complete option value (a space will be inserted /// after the completion.) \bfalse otherwise. bool m_word_complete = false; - // We don't own the list. + + // Note: This list is kept private. This is by design to prevent that any + // completion depends on any already computed completion from another backend. + // Note: We don't own the list. It's owned by the creator of the + // CompletionRequest object. StringList *m_matches; + + /// List of added completions so far. Used to filter out duplicates. + llvm::StringSet<> m_match_set; }; } // namespace lldb_private Modified: vendor/lldb/dist/include/lldb/Utility/Stream.h ============================================================================== --- vendor/lldb/dist/include/lldb/Utility/Stream.h Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/include/lldb/Utility/Stream.h Thu Aug 2 17:33:54 2018 (r337147) @@ -524,8 +524,6 @@ class Stream { (public) //------------------------------------------------------------------ size_t PutULEB128(uint64_t uval); - static void UnitTest(Stream *s); - protected: //------------------------------------------------------------------ // Member variables Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/api/multiple-debuggers/TestMultipleDebuggers.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/api/multiple-debuggers/TestMultipleDebuggers.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/api/multiple-debuggers/TestMultipleDebuggers.py Thu Aug 2 17:33:54 2018 (r337147) @@ -18,7 +18,7 @@ class TestMultipleSimultaneousDebuggers(TestBase): mydir = TestBase.compute_mydir(__file__) - # This test case fails non-deterministically. + # This test case fails non-deterministically. @skipIfNoSBHeaders @expectedFailureAll(bugnumber="llvm.org/pr20282") def test_multiple_debuggers(self): Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/api/multithreaded/TestMultithreaded.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/api/multithreaded/TestMultithreaded.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/api/multithreaded/TestMultithreaded.py Thu Aug 2 17:33:54 2018 (r337147) @@ -16,7 +16,7 @@ import subprocess class SBBreakpointCallbackCase(TestBase): NO_DEBUG_INFO_TESTCASE = True - + def setUp(self): TestBase.setUp(self) self.generateSource('driver.cpp') Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/decorators.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/decorators.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/decorators.py Thu Aug 2 17:33:54 2018 (r337147) @@ -791,7 +791,7 @@ def skipIfLLVMTargetMissing(target): if targets.GetItemAtIndex(i).GetStringValue(99) == target: found = True break - + return unittest2.skipIf(not found, "requires " + target) # Call sysctl on darwin to see if a specified hardware feature is available on this machine. Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-function/TestCallStdStringFunction.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-function/TestCallStdStringFunction.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-function/TestCallStdStringFunction.py Thu Aug 2 17:33:54 2018 (r337147) @@ -46,7 +46,7 @@ class ExprCommandCallFunctionTestCase(TestBase): # Calling this function now succeeds, but we follow the typedef return type through to # const char *, and thus don't invoke the Summary formatter. - # clang's libstdc++ on ios arm64 inlines std::string::c_str() always; + # clang's libstdc++ on ios arm64 inlines std::string::c_str() always; # skip this part of the test. triple = self.dbg.GetSelectedPlatform().GetTriple() do_cstr_test = True Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-restarts/TestCallThatRestarts.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-restarts/TestCallThatRestarts.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-restarts/TestCallThatRestarts.py Thu Aug 2 17:33:54 2018 (r337147) @@ -48,7 +48,7 @@ class ExprCommandThatRestartsTestCase(TestBase): "Restored the zeroth frame correctly") def call_function(self): - (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, 'Stop here in main.', self.main_source_spec) # Make sure the SIGCHLD behavior is pass/no-stop/no-notify: Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-throws/TestCallThatThrows.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-throws/TestCallThatThrows.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/call-throws/TestCallThatThrows.py Thu Aug 2 17:33:54 2018 (r337147) @@ -37,7 +37,7 @@ class ExprCommandWithThrowTestCase(TestBase): def call_function(self): """Test calling function that throws.""" - (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, 'I am about to throw.', self.main_source_spec) options = lldb.SBExpressionOptions() Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/char/TestExprsChar.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/char/TestExprsChar.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/char/TestExprsChar.py Thu Aug 2 17:33:54 2018 (r337147) @@ -22,7 +22,7 @@ class ExprCharTestCase(TestBase): """These basic expression commands should work as expected.""" self.build(dictionary=dictionary) - (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, '// Break here', self.main_source_spec) frame = thread.GetFrameAtIndex(0) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/fixits/TestFixIts.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/fixits/TestFixIts.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/fixits/TestFixIts.py Thu Aug 2 17:33:54 2018 (r337147) @@ -37,7 +37,7 @@ class ExprCommandWithFixits(TestBase): def try_expressions(self): """Test calling expressions with errors that can be fixed by the FixIts.""" - (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, 'Stop here to evaluate expressions', self.main_source_spec) options = lldb.SBExpressionOptions() Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/issue_11588/Test11588.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/issue_11588/Test11588.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/issue_11588/Test11588.py Thu Aug 2 17:33:54 2018 (r337147) @@ -32,8 +32,8 @@ class Issue11581TestCase(TestBase): """valobj.AddressOf() should return correct values.""" self.build() - (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, - 'Set breakpoint here.', + (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + 'Set breakpoint here.', lldb.SBFileSpec("main.cpp", False)) self.runCmd("command script import --allow-reload s11588.py") self.runCmd( Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/pr35310/TestExprsBug35310.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/pr35310/TestExprsBug35310.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/pr35310/TestExprsBug35310.py Thu Aug 2 17:33:54 2018 (r337147) @@ -25,7 +25,7 @@ class ExprBug35310(TestBase): """ self.build() - (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, '// Break here', self.main_source_spec) frame = thread.GetFrameAtIndex(0) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/save_jit_objects/TestSaveJITObjects.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/save_jit_objects/TestSaveJITObjects.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/save_jit_objects/TestSaveJITObjects.py Thu Aug 2 17:33:54 2018 (r337147) @@ -16,7 +16,7 @@ class SaveJITObjectsTestCase(TestBase): def enumerateJITFiles(self): return [f for f in os.listdir(self.getBuildDir()) if f.startswith("jit")] - + def countJITFiles(self): return len(self.enumerateJITFiles()) @@ -31,7 +31,7 @@ class SaveJITObjectsTestCase(TestBase): os.chdir(self.getBuildDir()) src_file = "main.c" src_file_spec = lldb.SBFileSpec(src_file) - + (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint( self, "break", src_file_spec) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/unwind_expression/TestUnwindExpression.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/unwind_expression/TestUnwindExpression.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/unwind_expression/TestUnwindExpression.py Thu Aug 2 17:33:54 2018 (r337147) @@ -23,7 +23,7 @@ class UnwindFromExpressionTest(TestBase): def build_and_run_to_bkpt(self): self.build() - (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + (target, process, self.thread, bkpt) = lldbutil.run_to_source_breakpoint(self, "// Set a breakpoint here to get started", self.main_spec) # Next set a breakpoint in this function, set up Expression options to stop on Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/xvalue/TestXValuePrinting.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/xvalue/TestXValuePrinting.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/expression_command/xvalue/TestXValuePrinting.py Thu Aug 2 17:33:54 2018 (r337147) @@ -22,7 +22,7 @@ class ExprXValuePrintingTestCase(TestBase): """Printing an xvalue should work.""" self.build(dictionary=dictionary) - (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, '// Break here', self.main_source_spec) frame = thread.GetFrameAtIndex(0) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/address_breakpoints/TestBadAddressBreakpoints.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/address_breakpoints/TestBadAddressBreakpoints.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/address_breakpoints/TestBadAddressBreakpoints.py Thu Aug 2 17:33:54 2018 (r337147) @@ -31,7 +31,7 @@ class BadAddressBreakpointTestCase(TestBase): def address_breakpoints(self): """Test that breakpoints set on a bad address say they are bad.""" target, process, thread, bkpt = \ - lldbutil.run_to_source_breakpoint(self, + lldbutil.run_to_source_breakpoint(self, "Set a breakpoint here", lldb.SBFileSpec("main.c")) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/auto_continue/TestBreakpointAutoContinue.py Thu Aug 2 17:33:54 2018 (r337147) @@ -34,17 +34,17 @@ class BreakpointAutoContinue(TestBase): self.build() self.auto_continue_location() - def make_target_and_bkpt(self, additional_options=None, num_expected_loc=1, + def make_target_and_bkpt(self, additional_options=None, num_expected_loc=1, pattern="Set a breakpoint here"): exe = self.getBuildArtifact("a.out") self.target = self.dbg.CreateTarget(exe) self.assertTrue(self.target.IsValid(), "Target is not valid") - + extra_options_txt = "--auto-continue 1 " if additional_options: extra_options_txt += additional_options - bpno = lldbutil.run_break_set_by_source_regexp(self, pattern, - extra_options = extra_options_txt, + bpno = lldbutil.run_break_set_by_source_regexp(self, pattern, + extra_options = extra_options_txt, num_expected_locations = num_expected_loc) return bpno Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_locations/TestBreakpointLocations.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_locations/TestBreakpointLocations.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_locations/TestBreakpointLocations.py Thu Aug 2 17:33:54 2018 (r337147) @@ -46,12 +46,12 @@ class BreakpointLocationsTestCase(TestBase): self.assertTrue(target, "Target %s is not valid"%(exe)) # This should create a breakpoint with 3 locations. - + bkpt = target.BreakpointCreateByLocation("main.c", self.line) # The breakpoint list should show 3 locations. self.assertEqual(bkpt.GetNumLocations(), 3, "Wrong number of locations") - + self.expect( "breakpoint list -f", "Breakpoint locations shown correctly", @@ -62,7 +62,7 @@ class BreakpointLocationsTestCase(TestBase): "where = a.out`func_inlined .+unresolved, hit count = 0", "where = a.out`main .+\[inlined\].+unresolved, hit count = 0"]) - return bkpt + return bkpt def shadowed_bkpt_cond_test(self): """Test that options set on the breakpoint and location behave correctly.""" @@ -106,7 +106,7 @@ class BreakpointLocationsTestCase(TestBase): commands = ["AAAAAA", "BBBBBB", "CCCCCC"] str_list = lldb.SBStringList() str_list.AppendList(commands, len(commands)) - + bkpt.SetCommandLineCommands(str_list) cmd_list = lldb.SBStringList() bkpt.GetCommandLineCommands(cmd_list) @@ -123,7 +123,7 @@ class BreakpointLocationsTestCase(TestBase): bkpt.location[1].GetCommandLineCommands(loc_cmd_list) loc_list_size = loc_list.GetSize() - + # Check that the location has the right commands: self.assertEqual(loc_cmd_list.GetSize() , loc_list_size, "Added the right number of commands to location") for i in range(0,loc_list_size): Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_names/TestBreakpointNames.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_names/TestBreakpointNames.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/breakpoint_names/TestBreakpointNames.py Thu Aug 2 17:33:54 2018 (r337147) @@ -64,7 +64,7 @@ class BreakpointNames(TestBase): self.target = self.dbg.CreateTarget(exe) self.assertTrue(self.target, VALID_TARGET) self.main_file_spec = lldb.SBFileSpec(os.path.join(self.getSourceDir(), "main.c")) - + def check_name_in_target(self, bkpt_name): name_list = lldb.SBStringList() self.target.GetBreakpointNames(name_list) @@ -74,7 +74,7 @@ class BreakpointNames(TestBase): found_it = True break self.assertTrue(found_it, "Didn't find the name %s in the target's name list:"%(bkpt_name)) - + def setUp(self): # Call super's setUp(). TestBase.setUp(self) @@ -107,14 +107,14 @@ class BreakpointNames(TestBase): matches = bkpt.MatchesName(bkpt_name) self.assertTrue(matches, "We didn't match the name we just set") - + # Make sure we don't match irrelevant names: matches = bkpt.MatchesName("NotABreakpoint") self.assertTrue(not matches, "We matched a name we didn't set.") # Make sure the name is also in the target: self.check_name_in_target(bkpt_name) - + # Add another name, make sure that works too: bkpt.AddName(other_bkpt_name) @@ -132,7 +132,7 @@ class BreakpointNames(TestBase): bkpt.GetNames(name_list) num_names = name_list.GetSize() self.assertTrue(num_names == 1, "Name list has %d items, expected 1."%(num_names)) - + name = name_list.GetStringAtIndex(0) self.assertTrue(name == other_bkpt_name, "Remaining name was: %s expected %s."%(name, other_bkpt_name)) @@ -156,7 +156,7 @@ class BreakpointNames(TestBase): def do_check_using_names(self): """Use Python APIs to check names work in place of breakpoint ID's.""" - + bkpt = self.target.BreakpointCreateByLocation(self.main_file_spec, 10) bkpt_name = "ABreakpoint" other_bkpt_name= "_AnotherBreakpoint" @@ -255,12 +255,12 @@ class BreakpointNames(TestBase): bp_name.SetAutoContinue(new_auto_continue) self.assertEqual(bp_name.GetAutoContinue(), new_auto_continue, "Couldn't change auto-continue on the name") self.assertEqual(bkpt.GetAutoContinue(), new_auto_continue, "Option didn't propagate to the breakpoint.") - + # Now make this same breakpoint name - but from the command line - cmd_str = "breakpoint name configure %s -o %d -i %d -c '%s' -G %d -t %d -x %d -T '%s' -q '%s' -H '%s'"%(cl_bp_name_string, - self.is_one_shot, - self.ignore_count, - self.condition, + cmd_str = "breakpoint name configure %s -o %d -i %d -c '%s' -G %d -t %d -x %d -T '%s' -q '%s' -H '%s'"%(cl_bp_name_string, + self.is_one_shot, + self.ignore_count, + self.condition, self.auto_continue, self.tid, self.tidx, @@ -269,7 +269,7 @@ class BreakpointNames(TestBase): self.help_string) for cmd in self.cmd_list: cmd_str += " -C '%s'"%(cmd) - + self.runCmd(cmd_str, check=True) # Now look up this name again and check its options: cl_name = lldb.SBBreakpointName(self.target, cl_bp_name_string) @@ -280,14 +280,14 @@ class BreakpointNames(TestBase): new_help = "I do something even more interesting" cl_name.SetHelpString(new_help) self.assertEqual(new_help, cl_name.GetHelpString(), "SetHelpString didn't") - + # We should have three names now, make sure the target can list them: name_list = lldb.SBStringList() self.target.GetBreakpointNames(name_list) for name_string in [self.bp_name_string, other_bp_name_string, cl_bp_name_string]: self.assertTrue(name_string in name_list, "Didn't find %s in names"%(name_string)) - # Delete the name from the current target. Make sure that works and deletes the + # Delete the name from the current target. Make sure that works and deletes the # name from the breakpoint as well: self.target.DeleteBreakpointName(self.bp_name_string) name_list.Clear() @@ -305,7 +305,7 @@ class BreakpointNames(TestBase): self.assertTrue(self.bp_name_string not in name_list, "Didn't delete %s from the dummy target"%(self.bp_name_string)) # Also make sure the name got removed from breakpoints holding it: self.assertFalse(bkpt.MatchesName(self.bp_name_string), "Didn't remove the name from the breakpoint.") - + def check_permission_results(self, bp_name): self.assertEqual(bp_name.GetAllowDelete(), False, "Didn't set allow delete.") protected_bkpt = self.target.BreakpointCreateByLocation(self.main_file_spec, 10) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/hardware_breakpoints/hardware_breakpoint_on_multiple_threads/TestHWBreakMultiThread.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/hardware_breakpoints/hardware_breakpoint_on_multiple_threads/TestHWBreakMultiThread.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/hardware_breakpoints/hardware_breakpoint_on_multiple_threads/TestHWBreakMultiThread.py Thu Aug 2 17:33:54 2018 (r337147) @@ -25,7 +25,7 @@ class HardwareBreakpointMultiThreadTestCase(TestBase): @expectedFailureAndroid def test_hw_break_set_delete_multi_thread(self): self.build() - self.setTearDownCleanup() + self.setTearDownCleanup() self.break_multi_thread('delete') # LLDB supports hardware breakpoints for arm and aarch64 architectures. @@ -33,7 +33,7 @@ class HardwareBreakpointMultiThreadTestCase(TestBase): @expectedFailureAndroid def test_hw_break_set_disable_multi_thread(self): self.build() - self.setTearDownCleanup() + self.setTearDownCleanup() self.break_multi_thread('disable') def setUp(self): @@ -74,7 +74,7 @@ class HardwareBreakpointMultiThreadTestCase(TestBase): count = 0 while count < 2 : - + self.runCmd("process continue") # We should be stopped in hw_break_function Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/serialize/TestBreakpointSerialization.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/serialize/TestBreakpointSerialization.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/serialize/TestBreakpointSerialization.py Thu Aug 2 17:33:54 2018 (r337147) @@ -65,7 +65,7 @@ class BreakpointSerialization(TestBase): # Create the targets we are making breakpoints in and copying them to: self.orig_target = self.dbg.CreateTarget(exe) self.assertTrue(self.orig_target, VALID_TARGET) - + self.copy_target = self.dbg.CreateTarget(exe) self.assertTrue(self.copy_target, VALID_TARGET) @@ -91,7 +91,7 @@ class BreakpointSerialization(TestBase): num_source_bps = source_bps.GetSize() num_copy_bps = copy_bps.GetSize() self.assertTrue(num_source_bps == num_copy_bps, "Didn't get same number of input and output breakpoints - orig: %d copy: %d"%(num_source_bps, num_copy_bps)) - + for i in range(0, num_source_bps): source_bp = source_bps.GetBreakpointAtIndex(i) source_desc = lldb.SBStream() @@ -132,7 +132,7 @@ class BreakpointSerialization(TestBase): source_bps.Append(self.orig_target.BreakpointCreateByName("blubby", lldb.eFunctionNameTypeAuto, empty_module_list, empty_cu_list)) source_bps.Append(self.orig_target.BreakpointCreateByName("blubby", lldb.eFunctionNameTypeFull, empty_module_list,empty_cu_list)) source_bps.Append(self.orig_target.BreakpointCreateBySourceRegex("dont really care", blubby_file_spec)) - + # And some number greater than one: self.check_equivalence(source_bps) @@ -185,7 +185,7 @@ class BreakpointSerialization(TestBase): bkpt.SetOneShot(True) bkpt.SetThreadID(10) source_bps.Append(bkpt) - + # Make sure we get one right: self.check_equivalence(source_bps) source_bps.Clear() @@ -232,7 +232,7 @@ class BreakpointSerialization(TestBase): bkpt.SetThreadID(10) source_bps.Append(bkpt) all_bps.Append(bkpt) - + error = lldb.SBError() error = self.orig_target.BreakpointsWriteToFile(self.bkpts_file_spec, source_bps) self.assertTrue(error.Success(), "Failed writing breakpoints to file: %s."%(error.GetCString())) @@ -265,7 +265,7 @@ class BreakpointSerialization(TestBase): write_bps = lldb.SBBreakpointList(self.orig_target) bkpt.AddName(good_bkpt_name) write_bps.Append(bkpt) - + error = lldb.SBError() error = self.orig_target.BreakpointsWriteToFile(self.bkpts_file_spec, write_bps) self.assertTrue(error.Success(), "Failed writing breakpoints to file: %s."%(error.GetCString())) @@ -282,7 +282,3 @@ class BreakpointSerialization(TestBase): error = self.copy_target.BreakpointsCreateFromFile(self.bkpts_file_spec, names_list, copy_bps) self.assertTrue(error.Success(), "Failed reading breakpoints from file: %s"%(error.GetCString())) self.assertTrue(copy_bps.GetSize() == 1, "Found the matching breakpoint.") - - - - Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/breakpoint/step_over_breakpoint/TestStepOverBreakpoint.py Thu Aug 2 17:33:54 2018 (r337147) @@ -1,8 +1,8 @@ """ Test that breakpoints do not affect stepping. -Check for correct StopReason when stepping to the line with breakpoint +Check for correct StopReason when stepping to the line with breakpoint which chould be eStopReasonBreakpoint in general, -and eStopReasonPlanComplete when breakpoint's condition fails. +and eStopReasonPlanComplete when breakpoint's condition fails. """ from __future__ import print_function @@ -19,7 +19,7 @@ class StepOverBreakpointsTestCase(TestBase): def setUp(self): TestBase.setUp(self) - + self.build() exe = self.getBuildArtifact("a.out") src = lldb.SBFileSpec("main.cpp") @@ -32,7 +32,7 @@ class StepOverBreakpointsTestCase(TestBase): self.line1 = line_number('main.cpp', "breakpoint_1") self.line4 = line_number('main.cpp', "breakpoint_4") - self.breakpoint1 = self.target.BreakpointCreateByLocation(src, self.line1) + self.breakpoint1 = self.target.BreakpointCreateByLocation(src, self.line1) self.assertTrue( self.breakpoint1 and self.breakpoint1.GetNumLocations() == 1, VALID_BREAKPOINT) @@ -52,7 +52,7 @@ class StepOverBreakpointsTestCase(TestBase): self.thread = lldbutil.get_one_thread_stopped_at_breakpoint(self.process, self.breakpoint1) self.assertIsNotNone(self.thread, "Didn't stop at breakpoint 1.") - def test_step_instruction(self): + def test_step_instruction(self): # Count instructions between breakpoint_1 and breakpoint_4 contextList = self.target.FindFunctions('main', lldb.eFunctionNameTypeAuto) self.assertEquals(contextList.GetSize(), 1) @@ -89,7 +89,7 @@ class StepOverBreakpointsTestCase(TestBase): @skipIf(bugnumber="llvm.org/pr31972", hostoslist=["windows"]) def test_step_over(self): #lldb.DBG.EnableLog("lldb", ["step","breakpoint"]) - + self.thread.StepOver() # We should be stopped at the breakpoint_2 line with stop plan complete reason self.assertEquals(self.process.GetState(), lldb.eStateStopped) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/TestCompletion.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/TestCompletion.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/TestCompletion.py Thu Aug 2 17:33:54 2018 (r337147) @@ -39,6 +39,46 @@ class CommandLineCompletionTestCase(TestBase): self.complete_from_to('de', 'detach ') @skipIfFreeBSD # timing out on the FreeBSD buildbot + def test_frame_variable(self): + self.build() + self.main_source = "main.cpp" + self.main_source_spec = lldb.SBFileSpec(self.main_source) + self.dbg.CreateTarget(self.getBuildArtifact("a.out")) + + (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self, + '// Break here', self.main_source_spec) + self.assertEquals(process.GetState(), lldb.eStateStopped) + # FIXME: This pulls in the debug information to make the completions work, + # but the completions should also work without. + self.runCmd("frame variable fooo") + + self.complete_from_to('frame variable fo', + 'frame variable fooo') + self.complete_from_to('frame variable fooo.', + 'frame variable fooo.') + self.complete_from_to('frame variable fooo.dd', + 'frame variable fooo.dd') + + self.complete_from_to('frame variable ptr_fooo->', + 'frame variable ptr_fooo->') + self.complete_from_to('frame variable ptr_fooo->dd', + 'frame variable ptr_fooo->dd') + + self.complete_from_to('frame variable cont', + 'frame variable container') + self.complete_from_to('frame variable container.', + 'frame variable container.MemberVar') + self.complete_from_to('frame variable container.Mem', + 'frame variable container.MemberVar') + + self.complete_from_to('frame variable ptr_cont', + 'frame variable ptr_container') + self.complete_from_to('frame variable ptr_container->', + 'frame variable ptr_container->MemberVar') + self.complete_from_to('frame variable ptr_container->Mem', + 'frame variable ptr_container->MemberVar') + + @skipIfFreeBSD # timing out on the FreeBSD buildbot def test_process_attach_dash_dash_con(self): """Test that 'process attach --con' completes to 'process attach --continue '.""" self.complete_from_to( @@ -261,7 +301,7 @@ class CommandLineCompletionTestCase(TestBase): common_match = match_strings.GetStringAtIndex(0) if num_matches == 0: compare_string = str_input - else: + else: if common_match != None and len(common_match) > 0: compare_string = str_input + common_match else: @@ -273,8 +313,8 @@ class CommandLineCompletionTestCase(TestBase): if turn_off_re_match: self.expect( compare_string, msg=COMPLETION_MSG( - str_input, p), exe=False, substrs=[p]) + str_input, p, match_strings), exe=False, substrs=[p]) else: self.expect( compare_string, msg=COMPLETION_MSG( - str_input, p), exe=False, patterns=[p]) + str_input, p, match_strings), exe=False, patterns=[p]) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/main.cpp ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/main.cpp Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/completion/main.cpp Thu Aug 2 17:33:54 2018 (r337147) @@ -7,8 +7,15 @@ class Foo (public) } }; +struct Container { int MemberVar; }; + int main() { - Foo f; - f.Bar(1, 2); + Foo fooo; + Foo *ptr_fooo = &fooo; + fooo.Bar(1, 2); + + Container container; + Container *ptr_container = &container; + return container.MemberVar = 3; // Break here } Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/bitset/TestDataFormatterLibcxxBitset.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/bitset/TestDataFormatterLibcxxBitset.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/bitset/TestDataFormatterLibcxxBitset.py Thu Aug 2 17:33:54 2018 (r337147) @@ -49,7 +49,7 @@ class TestDataFormatterLibcxxBitset(TestBase): def test_ptr_and_ref(self): """Test that ref and ptr to std::bitset is displayed correctly""" self.build() - (_, process, _, bkpt) = lldbutil.run_to_source_breakpoint(self, + (_, process, _, bkpt) = lldbutil.run_to_source_breakpoint(self, 'Check ref and ptr', lldb.SBFileSpec("main.cpp", False)) @@ -57,6 +57,6 @@ class TestDataFormatterLibcxxBitset(TestBase): self.check("ptr", 13) lldbutil.continue_to_breakpoint(process, bkpt) - + self.check("ref", 200) self.check("ptr", 200) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/list/TestDataFormatterLibcxxList.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/list/TestDataFormatterLibcxxList.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/list/TestDataFormatterLibcxxList.py Thu Aug 2 17:33:54 2018 (r337147) @@ -90,7 +90,7 @@ class LibcxxListDataFormatterTestCase(TestBase): self.runCmd("n") # This gets up past the printf self.runCmd("n") # Now advance over the first push_back. - + self.expect("frame variable numbers_list", substrs=['list has 1 items', '[0] = ', Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/multiset/TestDataFormatterLibcxxMultiSet.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/multiset/TestDataFormatterLibcxxMultiSet.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/multiset/TestDataFormatterLibcxxMultiSet.py Thu Aug 2 17:33:54 2018 (r337147) @@ -134,7 +134,7 @@ class LibcxxMultiSetDataFormatterTestCase(TestBase): """Test that the data formatters work on ref and ptr.""" self.build() (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( - self, "Stop here to check by ref and ptr.", + self, "Stop here to check by ref and ptr.", lldb.SBFileSpec("main.cpp", False)) # The reference should print just like the value: self.check_ii("ref") Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py Thu Aug 2 17:33:54 2018 (r337147) @@ -130,7 +130,7 @@ class LibcxxSetDataFormatterTestCase(TestBase): """Test that the data formatters work on ref and ptr.""" self.build() (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( - self, "Stop here to check by ref and ptr.", + self, "Stop here to check by ref and ptr.", lldb.SBFileSpec("main.cpp", False)) # The reference should print just like the value: self.check_ii("ref") @@ -139,4 +139,4 @@ class LibcxxSetDataFormatterTestCase(TestBase): substrs=["ptr =", "size=7"]) self.expect("expr ptr", substrs=["size=7"]) - + Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libcxx/vector/TestDataFormatterLibcxxVector.py Thu Aug 2 17:33:54 2018 (r337147) @@ -192,7 +192,7 @@ class LibcxxVectorDataFormatterTestCase(TestBase): self.check_numbers("ref") # The pointer should just show the right number of elements: - + self.expect("frame variable ptr", substrs=['ptr =', ' size=7']) self.expect("p ptr", substrs=['$', 'size=7']) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/tuple/TestDataFormatterStdTuple.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/tuple/TestDataFormatterStdTuple.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-stl/libstdcpp/tuple/TestDataFormatterStdTuple.py Thu Aug 2 17:33:54 2018 (r337147) @@ -40,7 +40,7 @@ class StdTupleDataFormatterTestCase(TestBase): self.assertEqual('"foobar"', frame.GetValueForVariablePath("ts[0]").GetSummary()) self.assertFalse(frame.GetValueForVariablePath("ts[1]").IsValid()) - + self.assertEqual(1, frame.GetValueForVariablePath("tt[0]").GetValueAsUnsigned()) self.assertEqual('"baz"', frame.GetValueForVariablePath("tt[1]").GetSummary()) self.assertEqual(2, frame.GetValueForVariablePath("tt[2]").GetValueAsUnsigned()) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/exec/TestExec.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/exec/TestExec.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/exec/TestExec.py Thu Aug 2 17:33:54 2018 (r337147) @@ -96,7 +96,7 @@ class ExecTestCase(TestBase): if not skip_exec: self.assertTrue(process.GetState() == lldb.eStateStopped, "Process should be stopped at __dyld_start") - + threads = lldbutil.get_stopped_threads( process, lldb.eStopReasonExec) self.assertTrue( Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame-language/TestGuessLanguage.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame-language/TestGuessLanguage.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame-language/TestGuessLanguage.py Thu Aug 2 17:33:54 2018 (r337147) @@ -18,7 +18,7 @@ class TestFrameGuessLanguage(TestBase): mydir = TestBase.compute_mydir(__file__) - # If your test case doesn't stress debug info, the + # If your test case doesn't stress debug info, the # set this to true. That way it won't be run once for # each debug info format. NO_DEBUG_INFO_TESTCASE = True @@ -84,6 +84,6 @@ class TestFrameGuessLanguage(TestBase): self.check_language(thread, 0, c_frame_language) self.check_language(thread, 1, lldb.eLanguageTypeC_plus_plus) self.check_language(thread, 2, lldb.eLanguageTypeC_plus_plus) - - + + Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame_var/TestFrameVar.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame_var/TestFrameVar.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/frame_var/TestFrameVar.py Thu Aug 2 17:33:54 2018 (r337147) @@ -17,7 +17,7 @@ class TestFrameVar(TestBase): mydir = TestBase.compute_mydir(__file__) - # If your test case doesn't stress debug info, the + # If your test case doesn't stress debug info, the # set this to true. That way it won't be run once for # each debug info format. NO_DEBUG_INFO_TESTCASE = True @@ -67,7 +67,7 @@ class TestFrameVar(TestBase): frame = threads[0].GetFrameAtIndex(0) command_result = lldb.SBCommandReturnObject() interp = self.dbg.GetCommandInterpreter() - + # Just get args: result = interp.HandleCommand("frame var -l", command_result) self.assertEqual(result, lldb.eReturnStatusSuccessFinishResult, "frame var -a didn't succeed") @@ -85,7 +85,7 @@ class TestFrameVar(TestBase): self.assertTrue("argv" not in output, "Locals found argv") self.assertTrue("test_var" in output, "Locals didn't find test_var") self.assertTrue("g_var" not in output, "Locals found a global") - + # Get the file statics: result = interp.HandleCommand("frame var -l -a -g", command_result) self.assertEqual(result, lldb.eReturnStatusSuccessFinishResult, "frame var -a didn't succeed") @@ -94,6 +94,6 @@ class TestFrameVar(TestBase): self.assertTrue("argv" not in output, "Globals found argv") self.assertTrue("test_var" not in output, "Globals found test_var") self.assertTrue("g_var" in output, "Globals didn't find g_var") - - + + Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestTargetXMLArch.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestTargetXMLArch.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/TestTargetXMLArch.py Thu Aug 2 17:33:54 2018 (r337147) @@ -98,12 +98,12 @@ class TestTargetXMLArch(GDBRemoteTestBase): return "T05thread:00000001;06:9038d60f00700000;07:98b4062680ffffff;10:c0d7bf1b80ffffff;" def readRegister(self, register): - regs = {0x0: "00b0060000610000", - 0xa: "68fe471c80ffffff", - 0xc: "60574a1c80ffffff", - 0xd: "18f3042680ffffff", - 0xe: "be8a4d7142000000", - 0xf: "50df471c80ffffff", + regs = {0x0: "00b0060000610000", + 0xa: "68fe471c80ffffff", + 0xc: "60574a1c80ffffff", + 0xd: "18f3042680ffffff", + 0xe: "be8a4d7142000000", + 0xf: "50df471c80ffffff", 0x10: "c0d7bf1b80ffffff" } if register in regs: return regs[register] Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/gdbclientutils.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/gdbclientutils.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/gdb_remote_client/gdbclientutils.py Thu Aug 2 17:33:54 2018 (r337147) @@ -467,7 +467,7 @@ class GDBRemoteTestBase(TestBase): i = 0 j = 0 log = self.server.responder.packetLog - + while i < len(packets) and j < len(log): if log[j] == packets[i]: i += 1 Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/history/TestHistoryRecall.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/history/TestHistoryRecall.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/history/TestHistoryRecall.py Thu Aug 2 17:33:54 2018 (r337147) @@ -17,7 +17,7 @@ class TestHistoryRecall(TestBase): mydir = TestBase.compute_mydir(__file__) - # If your test case doesn't stress debug info, the + # If your test case doesn't stress debug info, the # set this to true. That way it won't be run once for # each debug info format. NO_DEBUG_INFO_TESTCASE = True @@ -35,7 +35,7 @@ class TestHistoryRecall(TestBase): result = lldb.SBCommandReturnObject() interp.HandleCommand("command history", result, True) interp.HandleCommand("platform list", result, True) - + interp.HandleCommand("!0", result, False) self.assertTrue(result.Succeeded(), "!0 command did not work: %s"%(result.GetError())) self.assertTrue("command history" in result.GetOutput(), "!0 didn't rerun command history") Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_unload/TestLoadUnload.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_unload/TestLoadUnload.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_unload/TestLoadUnload.py Thu Aug 2 17:33:54 2018 (r337147) @@ -20,7 +20,7 @@ class LoadUnloadTestCase(TestBase): mydir = TestBase.compute_mydir(__file__) NO_DEBUG_INFO_TESTCASE = True - + def setUp(self): # Call super's setUp(). TestBase.setUp(self) @@ -210,7 +210,7 @@ class LoadUnloadTestCase(TestBase): if not self.platformIsDarwin(): env_cmd_string += ":" + wd self.runCmd(env_cmd_string) - + # This time, the hidden library should be picked up. self.expect("run", substrs=["return", "12345"]) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_using_paths/TestLoadUsingPaths.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_using_paths/TestLoadUsingPaths.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/load_using_paths/TestLoadUsingPaths.py Thu Aug 2 17:33:54 2018 (r337147) @@ -20,7 +20,7 @@ class LoadUsingPathsTestCase(TestBase): mydir = TestBase.compute_mydir(__file__) NO_DEBUG_INFO_TESTCASE = True - + def setUp(self): # Call super's setUp(). TestBase.setUp(self) @@ -52,8 +52,8 @@ class LoadUsingPathsTestCase(TestBase): # The directory with the dynamic library we did not link to. path_dir = os.path.join(self.getBuildDir(), "hidden") - (target, process, thread, - _) = lldbutil.run_to_source_breakpoint(self, + (target, process, thread, + _) = lldbutil.run_to_source_breakpoint(self, "Break here to do the load using paths", lldb.SBFileSpec("main.cpp")) error = lldb.SBError() @@ -63,18 +63,18 @@ class LoadUsingPathsTestCase(TestBase): paths.AppendString(os.path.join(self.wd, "no_such_dir")) out_spec = lldb.SBFileSpec() - + # First try with no correct directories on the path, and make sure that doesn't blow up: token = process.LoadImageUsingPaths(lib_spec, paths, out_spec, error) self.assertEqual(token, lldb.LLDB_INVALID_IMAGE_TOKEN, "Only looked on the provided path.") - + # Now add the correct dir to the paths list and try again: paths.AppendString(self.hidden_dir) token = process.LoadImageUsingPaths(lib_spec, paths, out_spec, error) self.assertNotEqual(token, lldb.LLDB_INVALID_IMAGE_TOKEN, "Got a valid token") self.assertEqual(out_spec, lldb.SBFileSpec(self.hidden_lib), "Found the expected library") - + # Make sure this really is in the image list: loaded_module = target.FindModule(out_spec) @@ -89,11 +89,11 @@ class LoadUsingPathsTestCase(TestBase): # Make sure the token works to unload it: process.UnloadImage(token) - # Make sure this really is no longer in the image list: + # Make sure this really is no longer in the image list: loaded_module = target.FindModule(out_spec) self.assertFalse(loaded_module.IsValid(), "The unloaded module is no longer in the image list.") - + # Make sure a relative path also works: paths.Clear() paths.AppendString(os.path.join(self.wd, "no_such_dir")) @@ -107,7 +107,7 @@ class LoadUsingPathsTestCase(TestBase): self.assertEqual(out_spec, lldb.SBFileSpec(self.hidden_lib), "Found the expected library with relative path") process.UnloadImage(token) - + # Make sure the presence of an empty path doesn't mess anything up: paths.Clear() paths.AppendString("") @@ -122,9 +122,9 @@ class LoadUsingPathsTestCase(TestBase): self.assertEqual(out_spec, lldb.SBFileSpec(self.hidden_lib), "Found the expected library with included empty path") process.UnloadImage(token) - + # Finally, passing in an absolute path should work like the basename: # This should NOT work because we've taken hidden_dir off the paths: abs_spec = lldb.SBFileSpec(os.path.join(self.hidden_dir, self.lib_name)) @@ -140,4 +140,4 @@ class LoadUsingPathsTestCase(TestBase): self.assertNotEqual(token, lldb.LLDB_INVALID_IMAGE_TOKEN, "Got a valid token") self.assertEqual(out_spec, lldb.SBFileSpec(self.hidden_lib), "Found the expected library") - + Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/pre_run_dylibs/TestPreRunDylibs.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/pre_run_dylibs/TestPreRunDylibs.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/pre_run_dylibs/TestPreRunDylibs.py Thu Aug 2 17:33:54 2018 (r337147) @@ -35,4 +35,4 @@ class TestPreRunLibraries(TestBase): self.assertTrue(found_it, "Couldn't find unlikely_to_occur_name in loaded libraries.") - + Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/register/intel_avx/TestZMMRegister.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/register/intel_avx/TestZMMRegister.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/register/intel_avx/TestZMMRegister.py Thu Aug 2 17:33:54 2018 (r337147) @@ -81,7 +81,7 @@ class TestYMMRegister(TestBase): self.expect( "register read ymm" + str(i), substrs=[pattern]) - + self.expect("continue", PROCESS_STOPPED, substrs=['stopped']) # Check stop reason; Should be either signal SIGTRAP or EXC_BREAKPOINT Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/return-value/TestReturnValue.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/return-value/TestReturnValue.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/return-value/TestReturnValue.py Thu Aug 2 17:33:54 2018 (r337147) @@ -68,7 +68,7 @@ class ReturnValueTestCase(TestBase): # Run again and we will stop in inner_sint the second time outer_sint is called. # Then test stepping out two frames at once: - + thread_list = lldbutil.continue_to_breakpoint(self.process, inner_sint_bkpt) self.assertTrue(len(thread_list) == 1) thread = thread_list[0] Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/create_during_step/TestCreateDuringStep.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/create_during_step/TestCreateDuringStep.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/create_during_step/TestCreateDuringStep.py Thu Aug 2 17:33:54 2018 (r337147) @@ -90,7 +90,7 @@ class CreateDuringStepTestCase(TestBase): target = self.dbg.GetSelectedTarget() # This should create a breakpoint in the stepping thread. - self.bkpt = target.BreakpointCreateByLocation("main.cpp", self.breakpoint) + self.bkpt = target.BreakpointCreateByLocation("main.cpp", self.breakpoint) # Run the program. self.runCmd("run", RUN_SUCCEEDED) Modified: vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/num_threads/TestNumThreads.py ============================================================================== --- vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/num_threads/TestNumThreads.py Thu Aug 2 17:33:50 2018 (r337146) +++ vendor/lldb/dist/packages/Python/lldbsuite/test/functionalities/thread/num_threads/TestNumThreads.py Thu Aug 2 17:33:54 2018 (r337147) @@ -92,7 +92,7 @@ class NumberOfThreadsTestCase(TestBase): self.assertTrue( num_threads >= 10, 'Number of expected threads and actual threads do not match.') - + # Attempt to walk each of the thread's executing the thread3 function to # the same breakpoint. def is_thread3(thread): *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Thu Aug 2 17:52:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8119E106E05E; Thu, 2 Aug 2018 17:52:49 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2AEC773994; Thu, 2 Aug 2018 17:52:49 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id D6DB310B429; Thu, 2 Aug 2018 13:52:47 -0400 (EDT) Subject: Re: svn commit: r337127 - head/sys/riscv/riscv To: Konstantin Belousov References: <201808021213.w72CDqQV065424@repo.freebsd.org> <7842b7e4-d14e-805f-719e-b393fc80b74c@FreeBSD.org> <20180802155030.GA40119@kib.kiev.ua> Cc: Ruslan Bukin , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org From: John Baldwin Message-ID: <22f6c1e0-11c2-8421-2a27-6ba59ef7b6dc@FreeBSD.org> Date: Thu, 2 Aug 2018 10:52:46 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20180802155030.GA40119@kib.kiev.ua> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 02 Aug 2018 13:52:48 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 17:52:49 -0000 On 8/2/18 8:50 AM, Konstantin Belousov wrote: > On Thu, Aug 02, 2018 at 08:16:59AM -0700, John Baldwin wrote: >> On 8/2/18 5:13 AM, Ruslan Bukin wrote: >>> Author: br >>> Date: Thu Aug 2 12:13:52 2018 >>> New Revision: 337127 >>> URL: https://svnweb.freebsd.org/changeset/base/337127 >>> >>> Log: >>> Don't overwrite tp in set_mcontext(). >>> >>> This makes libthr/swapcontext_test:swapcontext1 happy. >>> >>> Sponsored by: DARPA, AFRL >> >> This seems a bit odd. Is get_mcontext() not setting gp_tp correctly, or is >> makecontext() not preserving it perhaps? >> >> In general I feel like we should be saving/restoring 'tp' in the trapframe >> on kernel entry/exit like other GPRs instead of in the pcb and then >> including it in 'struct reg'. This makes it possible for debuggers to >> handle TLS variables more sanely (and in general we need to fix the various >> struct reg in FreeBSD architectures to include TLS registers when possible, >> such as adding fs_base and gs_base on x86). It would be nice if RISC-V >> started off on the correct foot here. > > I wanted to do this on x86, but I gave up due to the ABI issues. > When FSGSBASE instructions are supported on amd64, we save base > on context switch and restore on return to usermode. For struct reg Linux has done the equivalent of adding fs_base/gs_base if that is the ABI issues. Cores are pretty easy as debuggers can use the note size to determine if fs_base/gs_base are present and old debuggers will just ignore the larger notes. However, PT_GETREGS and PT_SETREGS is more of a pain. Probably we should define a "register set" API for ptrace() which I've been kicking around in my head that includes a way to communicate the size of the register block. I think Linux uses 'data' for it's register set ptrace ops to specify the register size. I think I'd like to be able to use 'data' for the userland size and instead borrow N upper bits in the ptrace op as the register set identifier. Either that or the 'addr' would need to start pointing to an indirect structure when working with a register set. -- John Baldwin From owner-svn-src-all@freebsd.org Thu Aug 2 18:28:03 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88DEF106ECE4; Thu, 2 Aug 2018 18:28:03 +0000 (UTC) (envelope-from bwidawsk@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36A8075262; Thu, 2 Aug 2018 18:28:03 +0000 (UTC) (envelope-from bwidawsk@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1644C155D8; Thu, 2 Aug 2018 18:28:03 +0000 (UTC) (envelope-from bwidawsk@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72IS2U7055987; Thu, 2 Aug 2018 18:28:02 GMT (envelope-from bwidawsk@FreeBSD.org) Received: (from bwidawsk@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72IS2ts055984; Thu, 2 Aug 2018 18:28:02 GMT (envelope-from bwidawsk@FreeBSD.org) Message-Id: <201808021828.w72IS2ts055984@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bwidawsk set sender to bwidawsk@FreeBSD.org using -f From: Ben Widawsky Date: Thu, 2 Aug 2018 18:28:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337157 - in head: . tools/tools/git X-SVN-Group: head X-SVN-Commit-Author: bwidawsk X-SVN-Commit-Paths: in head: . tools/tools/git X-SVN-Commit-Revision: 337157 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 18:28:03 -0000 Author: bwidawsk Date: Thu Aug 2 18:28:02 2018 New Revision: 337157 URL: https://svnweb.freebsd.org/changeset/base/337157 Log: tools: Add a git-svn bootstrap script codified version of https://wiki.freebsd.org/GitWorkflow/GitSvn#Using_git-svn Approved by: emaste (mentor) Suggested by: Warner Losh (imp) Differential Revision: https://reviews.freebsd.org/D16528 Added: head/tools/tools/git/git-svn-init (contents, props changed) Modified: head/.gitattributes head/tools/tools/git/HOWTO Modified: head/.gitattributes ============================================================================== --- head/.gitattributes Thu Aug 2 18:24:03 2018 (r337156) +++ head/.gitattributes Thu Aug 2 18:28:02 2018 (r337157) @@ -3,3 +3,4 @@ *.cpp diff=cpp *.hpp diff=cpp *.py diff=python +. svn-properties=svn:executable=tools/tools/git/git-svn-init Modified: head/tools/tools/git/HOWTO ============================================================================== --- head/tools/tools/git/HOWTO Thu Aug 2 18:24:03 2018 (r337156) +++ head/tools/tools/git/HOWTO Thu Aug 2 18:28:02 2018 (r337157) @@ -157,3 +157,10 @@ and it will do its thing and leave the tree on the mas Your tree must be clean to start this, and while it tries to catch some failures, not all of them have been allowed for. + +IV. git-svn-init +git-svn-init is a script that initializes the right git-svn connection as +outlined in https://wiki.freebsd.org/GitWorkflow/GitSvn. It would be a precursor +to the script git-svn-rebase. The script contains help, but generally you can +run the script with no arguments and it will attempt to set up both src and +ports repositories. Added: head/tools/tools/git/git-svn-init ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/tools/tools/git/git-svn-init Thu Aug 2 18:28:02 2018 (r337157) @@ -0,0 +1,195 @@ +#!/bin/sh + +# $FreeBSD$ + +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright(c) 2018 Intel Corporation. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +# This is the codified version of what was/is on the wiki page for using git in +# your workflow. It sets up proper repositories, with the correct remotes. + +# Environment variables which can be overridden if desired. Not worth +# parameterizing them. +GIT_IN_PATH=$(which git) +GIT=${GIT-${GIT_IN_PATH}} + +GIT_PORTS_REPO=${GIT_PORTS_REPO-git://github.com/freebsd/freebsd-ports.git} +GIT_SVN_PORTS_ROOT_URI=${GIT_SVN_PORTS_ROOT_URI-svn.freebsd.org/ports} +GIT_SVN_PORTS_URI=${GIT_SVN_PORTS_URI-repo.freebsd.org/ports} + +GIT_SRC_REPO=${GIT_SRC_REPO-git://github.com/freebsd/freebsd.git} +GIT_SVN_SRC_ROOT_URI=${GIT_SVN_SRC_ROOT_URI-svn.freebsd.org/base} +GIT_SVN_SRC_URI=${GIT_SVN_SRC_URI-repo.freebsd.org/base} + +GIT_SVN_PORTS_PUSH_URI=$GIT_SVN_PORTS_URI +GIT_SVN_SRC_PUSH_URI=$GIT_SVN_SRC_URI + +usage() +{ + cat < Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93D54106EF43; Thu, 2 Aug 2018 18:37:03 +0000 (UTC) (envelope-from bwidawsk@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 406A375783; Thu, 2 Aug 2018 18:37:03 +0000 (UTC) (envelope-from bwidawsk@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1D8721579C; Thu, 2 Aug 2018 18:37:03 +0000 (UTC) (envelope-from bwidawsk@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Ib2jm061071; Thu, 2 Aug 2018 18:37:02 GMT (envelope-from bwidawsk@FreeBSD.org) Received: (from bwidawsk@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Ib2ZD061070; Thu, 2 Aug 2018 18:37:02 GMT (envelope-from bwidawsk@FreeBSD.org) Message-Id: <201808021837.w72Ib2ZD061070@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bwidawsk set sender to bwidawsk@FreeBSD.org using -f From: Ben Widawsky Date: Thu, 2 Aug 2018 18:37:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337158 - head X-SVN-Group: head X-SVN-Commit-Author: bwidawsk X-SVN-Commit-Paths: head X-SVN-Commit-Revision: 337158 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 18:37:03 -0000 Author: bwidawsk Date: Thu Aug 2 18:37:02 2018 New Revision: 337158 URL: https://svnweb.freebsd.org/changeset/base/337158 Log: Remove spuriously added svn properties Modified: head/.gitattributes Modified: head/.gitattributes ============================================================================== --- head/.gitattributes Thu Aug 2 18:28:02 2018 (r337157) +++ head/.gitattributes Thu Aug 2 18:37:02 2018 (r337158) @@ -3,4 +3,3 @@ *.cpp diff=cpp *.hpp diff=cpp *.py diff=python -. svn-properties=svn:executable=tools/tools/git/git-svn-init From owner-svn-src-all@freebsd.org Thu Aug 2 18:51:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D6C6104A53C; Thu, 2 Aug 2018 18:51:45 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B893875EF0; Thu, 2 Aug 2018 18:51:44 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 99E8B15A98; Thu, 2 Aug 2018 18:51:44 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Ipi5K071237; Thu, 2 Aug 2018 18:51:44 GMT (envelope-from gjb@FreeBSD.org) Received: (from gjb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72IpiB7071236; Thu, 2 Aug 2018 18:51:44 GMT (envelope-from gjb@FreeBSD.org) Message-Id: <201808021851.w72IpiB7071236@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: gjb set sender to gjb@FreeBSD.org using -f From: Glen Barber Date: Thu, 2 Aug 2018 18:51:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337159 - head/release X-SVN-Group: head X-SVN-Commit-Author: gjb X-SVN-Commit-Paths: head/release X-SVN-Commit-Revision: 337159 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 18:51:45 -0000 Author: gjb Date: Thu Aug 2 18:51:44 2018 New Revision: 337159 URL: https://svnweb.freebsd.org/changeset/base/337159 Log: Fix the ftp-stage target for arm embedded builds. The images were renamed from KERNCONF to BOARDNAME when specified, which would result in an image name of: 12.0-CURRENT-arm-armv7-GENERIC.img which would then be renamed to use the BOARDNAME for the SoC the image is targeted to use. BOARDNAME was specified for all images as of r336994, which now causes the ftp-stage target to fail, as the rename is no longer necessary. Sponsored by: The FreeBSD Foundation Modified: head/release/Makefile.mirrors Modified: head/release/Makefile.mirrors ============================================================================== --- head/release/Makefile.mirrors Thu Aug 2 18:37:02 2018 (r337158) +++ head/release/Makefile.mirrors Thu Aug 2 18:51:44 2018 (r337159) @@ -58,11 +58,6 @@ TLD?= ${FTPDIR}/releases .if defined(EMBEDDED) && !empty(EMBEDDED) . if ${TARGET:Marm*} != "" && (${TARGET_ARCH:Marm*} != "" || ${TARGET_ARCH} == "aarch64") -. if !defined(BOARDNAME) && empty(BOARDNAME) -BOARDNAME:= ${KERNCONF} -. else -OLDNAME:= ${KERNCONF} -. endif . if ${BRANCH} == "STABLE" || ${BRANCH} == "CURRENT" || ${BRANCH} == "PRERELEASE" || ${BRANCH:MALPHA*} != "" SNAPSHOT= 1 . endif @@ -91,13 +86,7 @@ iso-images-stage: cd ${RELEASEDIR} && rm -f CHECKSUM.* . for IMAGE in ${IMAGES} . if defined(EMBEDDED) && !empty(EMBEDDED) -. if defined(OLDNAME) && !empty(OLDNAME) - @# arm/armv6 IMX6 -> WANDBOARD, for example. cd ${RELEASEDIR} && \ - mv ${OSRELEASE}-${OLDNAME}.${IMAGE}.xz \ - ${OSRELEASE}-${BOARDNAME}.${IMAGE}.xz -. endif - cd ${RELEASEDIR} && \ mv ${OSRELEASE}-${BOARDNAME}.${IMAGE}.xz \ ${OSRELEASE}-${BOARDNAME}-${SNAP_SUFFIX}.${IMAGE}.xz cp -p ${RELEASEDIR}/${OSRELEASE}-${BOARDNAME}-${SNAP_SUFFIX}.${IMAGE}.xz \ @@ -150,12 +139,6 @@ iso-images-stage: .else # not snapshot . for IMAGE in ${IMAGES} . if defined(EMBEDDED) && !empty(EMBEDDED) -. if defined(OLDNAME) && !empty(OLDNAME) - @# arm/armv6 IMX6 -> WANDBOARD, for example. - cd ${RELEASEDIR} && \ - mv ${OSRELEASE}-${OLDNAME}.${IMAGE}.xz \ - ${OSRELEASE}-${BOARDNAME}.${IMAGE}.xz -. endif cp -p ${RELEASEDIR}/${OSRELEASE}-${BOARDNAME}.${IMAGE}.xz \ ${ISO_DIR}/${OSRELEASE}-${BOARDNAME}.${IMAGE}.xz cd ${TLD}/ISO-IMAGES/${REVISION} && \ From owner-svn-src-all@freebsd.org Thu Aug 2 18:55:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98915104A715; Thu, 2 Aug 2018 18:55:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49AB9762C2; Thu, 2 Aug 2018 18:55:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2B8BC15AD5; Thu, 2 Aug 2018 18:55:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72ItwM6071479; Thu, 2 Aug 2018 18:55:58 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72ItuFF071465; Thu, 2 Aug 2018 18:55:56 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808021855.w72ItuFF071465@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 18:55:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337160 - in head: cddl/contrib/opensolaris/lib/libzfs/common cddl/lib/libzfs cddl/lib/libzfs_core cddl/lib/libzpool cddl/sbin/zfs cddl/sbin/zpool cddl/usr.bin/zinject cddl/usr.bin/zstr... X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/lib/libzfs/common cddl/lib/libzfs cddl/lib/libzfs_core cddl/lib/libzpool cddl/sbin/zfs cddl/sbin/zpool cddl/usr.bin/zinject cddl/usr.bin/zstreamdump cddl/usr.bin/ztes... X-SVN-Commit-Revision: 337160 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 18:55:59 -0000 Author: mav Date: Thu Aug 2 18:55:55 2018 New Revision: 337160 URL: https://svnweb.freebsd.org/changeset/base/337160 Log: Do not blindly include illumos kernel headers instead of user-space. It is not needed now, and I doubt it much helped at all, creating more confusions then good. Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c head/cddl/lib/libzfs/Makefile head/cddl/lib/libzfs_core/Makefile head/cddl/lib/libzpool/Makefile head/cddl/sbin/zfs/Makefile head/cddl/sbin/zpool/Makefile head/cddl/usr.bin/zinject/Makefile head/cddl/usr.bin/zstreamdump/Makefile head/cddl/usr.bin/ztest/Makefile head/cddl/usr.sbin/zdb/Makefile head/cddl/usr.sbin/zfsd/Makefile.common head/cddl/usr.sbin/zhack/Makefile head/lib/libprocstat/zfs/Makefile head/usr.sbin/fstyp/Makefile Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 18:55:55 2018 (r337160) @@ -50,7 +50,9 @@ #include #include #include +#ifdef illumos #include +#endif #include #include Modified: head/cddl/lib/libzfs/Makefile ============================================================================== --- head/cddl/lib/libzfs/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/lib/libzfs/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -50,7 +50,6 @@ CFLAGS+= -I${SRCTOP}/cddl/compat/opensolaris/lib/libum CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libnvpair Modified: head/cddl/lib/libzfs_core/Makefile ============================================================================== --- head/cddl/lib/libzfs_core/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/lib/libzfs_core/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -25,7 +25,6 @@ CFLAGS+= -I${SRCTOP}/cddl/compat/opensolaris/lib/libum CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libnvpair Modified: head/cddl/lib/libzpool/Makefile ============================================================================== --- head/cddl/lib/libzpool/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/lib/libzpool/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -47,7 +47,6 @@ CFLAGS+= -I${SRCTOP}/sys/cddl/compat/opensolaris CFLAGS+= -I${SRCTOP}/cddl/compat/opensolaris/include CFLAGS+= -I${SRCTOP}/cddl/compat/opensolaris/lib/libumem CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs Modified: head/cddl/sbin/zfs/Makefile ============================================================================== --- head/cddl/sbin/zfs/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/sbin/zfs/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -19,7 +19,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libu CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libnvpair CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs LIBADD= jail nvpair uutil zfs_core zfs Modified: head/cddl/sbin/zpool/Makefile ============================================================================== --- head/cddl/sbin/zpool/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/sbin/zpool/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -23,7 +23,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libn CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/cmd/stat/common Modified: head/cddl/usr.bin/zinject/Makefile ============================================================================== --- head/cddl/usr.bin/zinject/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/usr.bin/zinject/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -15,7 +15,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libz CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libnvpair CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs/ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head Modified: head/cddl/usr.bin/zstreamdump/Makefile ============================================================================== --- head/cddl/usr.bin/zstreamdump/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/usr.bin/zstreamdump/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -13,7 +13,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libz CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libnvpair CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head Modified: head/cddl/usr.bin/ztest/Makefile ============================================================================== --- head/cddl/usr.bin/ztest/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/usr.bin/ztest/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -14,7 +14,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libn CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libcmdutils CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head Modified: head/cddl/usr.sbin/zdb/Makefile ============================================================================== --- head/cddl/usr.sbin/zdb/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/usr.sbin/zdb/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -20,7 +20,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libz CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head Modified: head/cddl/usr.sbin/zfsd/Makefile.common ============================================================================== --- head/cddl/usr.sbin/zfsd/Makefile.common Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/usr.sbin/zfsd/Makefile.common Thu Aug 2 18:55:55 2018 (r337160) @@ -28,7 +28,6 @@ INCFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/li INCFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs INCFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common INCFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs -INCFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS= -g -DNEED_SOLARIS_BOOLEAN ${INCFLAGS} Modified: head/cddl/usr.sbin/zhack/Makefile ============================================================================== --- head/cddl/usr.sbin/zhack/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/cddl/usr.sbin/zhack/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -18,7 +18,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libz CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head Modified: head/lib/libprocstat/zfs/Makefile ============================================================================== --- head/lib/libprocstat/zfs/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/lib/libprocstat/zfs/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -13,7 +13,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libz CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/common/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head CFLAGS+= -I${.CURDIR:H} CFLAGS+= -DNEED_SOLARIS_BOOLEAN Modified: head/usr.sbin/fstyp/Makefile ============================================================================== --- head/usr.sbin/fstyp/Makefile Thu Aug 2 18:51:44 2018 (r337159) +++ head/usr.sbin/fstyp/Makefile Thu Aug 2 18:55:55 2018 (r337160) @@ -31,7 +31,6 @@ CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libn CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/lib/libzpool/common CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/fs/zfs CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common -CFLAGS+= -I${SRCTOP}/sys/cddl/contrib/opensolaris/uts/common/sys CFLAGS+= -I${SRCTOP}/cddl/contrib/opensolaris/head .endif From owner-svn-src-all@freebsd.org Thu Aug 2 19:09:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B8D0104AF83; Thu, 2 Aug 2018 19:09:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2B0276C51; Thu, 2 Aug 2018 19:09:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B601415C6E; Thu, 2 Aug 2018 19:09:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72J9D4I076485; Thu, 2 Aug 2018 19:09:13 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72J9DfR076484; Thu, 2 Aug 2018 19:09:13 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808021909.w72J9DfR076484@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 19:09:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337161 - in vendor/illumos/dist: cmd/zfs lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor/illumos/dist: cmd/zfs lib/libzfs/common X-SVN-Commit-Revision: 337161 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 19:09:14 -0000 Author: mav Date: Thu Aug 2 19:09:13 2018 New Revision: 337161 URL: https://svnweb.freebsd.org/changeset/base/337161 Log: 9512 zfs remap poolname@snapname coredumps Only filesystems and volumes are valid "zfs remap" parameters: when passed a snapshot name zfs_remap_indirects() does not handle the EINVAL returned from libzfs_core, which results in failing an assertion and consequently crashing. illumos/illumos-gate@0b2e8253986c5c761129b58cfdac46d204903de1 Reviewed by: Matthew Ahrens Reviewed by: John Wren Kennedy Reviewed by: Sara Hartse Approved by: Matt Ahrens Author: loli10K Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Modified: vendor/illumos/dist/cmd/zfs/zfs_main.c ============================================================================== --- vendor/illumos/dist/cmd/zfs/zfs_main.c Thu Aug 2 18:55:55 2018 (r337160) +++ vendor/illumos/dist/cmd/zfs/zfs_main.c Thu Aug 2 19:09:13 2018 (r337161) @@ -6924,11 +6924,28 @@ zfs_do_diff(int argc, char **argv) return (err != 0); } +/* + * zfs remap + * + * Remap the indirect blocks in the given fileystem or volume. + */ static int zfs_do_remap(int argc, char **argv) { const char *fsname; int err = 0; + int c; + + /* check options */ + while ((c = getopt(argc, argv, "")) != -1) { + switch (c) { + case '?': + (void) fprintf(stderr, + gettext("invalid option '%c'\n"), optopt); + usage(B_FALSE); + } + } + if (argc != 2) { (void) fprintf(stderr, gettext("wrong number of arguments\n")); usage(B_FALSE); Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 18:55:55 2018 (r337160) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 19:09:13 2018 (r337161) @@ -3877,12 +3877,24 @@ zfs_remap_indirects(libzfs_handle_t *hdl, const char * char errbuf[1024]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, - "cannot remap filesystem '%s' "), fs); + "cannot remap dataset '%s'"), fs); err = lzc_remap(fs); if (err != 0) { - (void) zfs_standard_error(hdl, err, errbuf); + switch (err) { + case ENOTSUP: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "pool must be upgraded")); + (void) zfs_error(hdl, EZFS_BADVERSION, errbuf); + break; + case EINVAL: + (void) zfs_error(hdl, EZFS_BADTYPE, errbuf); + break; + default: + (void) zfs_standard_error(hdl, err, errbuf); + break; + } } return (err); From owner-svn-src-all@freebsd.org Thu Aug 2 19:13:47 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF38B104C35D; Thu, 2 Aug 2018 19:13:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8173377251; Thu, 2 Aug 2018 19:13:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4971A15E13; Thu, 2 Aug 2018 19:13:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72JDk7v081451; Thu, 2 Aug 2018 19:13:46 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72JDjpo081449; Thu, 2 Aug 2018 19:13:45 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808021913.w72JDjpo081449@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 19:13:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337163 - in head/cddl/contrib/opensolaris: cmd/zfs lib/libzfs/common X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/cddl/contrib/opensolaris: cmd/zfs lib/libzfs/common X-SVN-Commit-Revision: 337163 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 19:13:47 -0000 Author: mav Date: Thu Aug 2 19:13:45 2018 New Revision: 337163 URL: https://svnweb.freebsd.org/changeset/base/337163 Log: MFV r337161: 9512 zfs remap poolname@snapname coredumps Only filesystems and volumes are valid "zfs remap" parameters: when passed a snapshot name zfs_remap_indirects() does not handle the EINVAL returned from libzfs_core, which results in failing an assertion and consequently crashing. illumos/illumos-gate@0b2e8253986c5c761129b58cfdac46d204903de1 Reviewed by: Matthew Ahrens Reviewed by: John Wren Kennedy Reviewed by: Sara Hartse Approved by: Matt Ahrens Author: loli10K Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c Thu Aug 2 19:11:56 2018 (r337162) +++ head/cddl/contrib/opensolaris/cmd/zfs/zfs_main.c Thu Aug 2 19:13:45 2018 (r337163) @@ -7038,11 +7038,28 @@ zfs_do_diff(int argc, char **argv) return (err != 0); } +/* + * zfs remap + * + * Remap the indirect blocks in the given fileystem or volume. + */ static int zfs_do_remap(int argc, char **argv) { const char *fsname; int err = 0; + int c; + + /* check options */ + while ((c = getopt(argc, argv, "")) != -1) { + switch (c) { + case '?': + (void) fprintf(stderr, + gettext("invalid option '%c'\n"), optopt); + usage(B_FALSE); + } + } + if (argc != 2) { (void) fprintf(stderr, gettext("wrong number of arguments\n")); usage(B_FALSE); Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 19:11:56 2018 (r337162) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 19:13:45 2018 (r337163) @@ -3917,12 +3917,24 @@ zfs_remap_indirects(libzfs_handle_t *hdl, const char * char errbuf[1024]; (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, - "cannot remap filesystem '%s' "), fs); + "cannot remap dataset '%s'"), fs); err = lzc_remap(fs); if (err != 0) { - (void) zfs_standard_error(hdl, err, errbuf); + switch (err) { + case ENOTSUP: + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "pool must be upgraded")); + (void) zfs_error(hdl, EZFS_BADVERSION, errbuf); + break; + case EINVAL: + (void) zfs_error(hdl, EZFS_BADTYPE, errbuf); + break; + default: + (void) zfs_standard_error(hdl, err, errbuf); + break; + } } return (err); From owner-svn-src-all@freebsd.org Thu Aug 2 19:25:44 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E59D104CE00; Thu, 2 Aug 2018 19:25:44 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 20FE577AAA; Thu, 2 Aug 2018 19:25:44 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F232715FA6; Thu, 2 Aug 2018 19:25:43 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72JPhbp086583; Thu, 2 Aug 2018 19:25:43 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72JPhIu086582; Thu, 2 Aug 2018 19:25:43 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201808021925.w72JPhIu086582@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: Conrad Meyer Date: Thu, 2 Aug 2018 19:25:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337165 - head/sys/fs/fuse X-SVN-Group: head X-SVN-Commit-Author: cem X-SVN-Commit-Paths: head/sys/fs/fuse X-SVN-Commit-Revision: 337165 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 19:25:44 -0000 Author: cem Date: Thu Aug 2 19:25:43 2018 New Revision: 337165 URL: https://svnweb.freebsd.org/changeset/base/337165 Log: FUSE: Bump maximum IO size to enable more performant operation Various components restrict size of IO passed up to the userspace filesystem based on the mount's f_iosize value. The previous default of PAGE_SIZE is anemic, even for normal filesystems, but especially considering every FUSE operation involves a kernel <-> userspace IPC upcall. Bump to DFLTPHYS (currently 64kB) to match other FUSE implementations. Anecdotally, Jakub reports IO read performance increased from 600 MB/s -> 2700 MB/s with a basic RAM-backed FUSE filesystem. PR: 230260 Reported by: Peter (MooseFS) Tested by: Jakub Kruszona-Zawadzki MFC after: 3 days Modified: head/sys/fs/fuse/fuse_vfsops.c Modified: head/sys/fs/fuse/fuse_vfsops.c ============================================================================== --- head/sys/fs/fuse/fuse_vfsops.c Thu Aug 2 19:19:53 2018 (r337164) +++ head/sys/fs/fuse/fuse_vfsops.c Thu Aug 2 19:25:43 2018 (r337165) @@ -341,7 +341,7 @@ fuse_vfsop_mount(struct mount *mp) mp->mnt_kern_flag |= MNTK_USES_BCACHE; MNT_IUNLOCK(mp); /* We need this here as this slot is used by getnewvnode() */ - mp->mnt_stat.f_iosize = PAGE_SIZE; + mp->mnt_stat.f_iosize = DFLTPHYS; if (subtype) { strlcat(mp->mnt_stat.f_fstypename, ".", MFSNAMELEN); strlcat(mp->mnt_stat.f_fstypename, subtype, MFSNAMELEN); From owner-svn-src-all@freebsd.org Thu Aug 2 19:37:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65800104D7F0; Thu, 2 Aug 2018 19:37:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1BD3C786B9; Thu, 2 Aug 2018 19:37:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F12761615A; Thu, 2 Aug 2018 19:37:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72JbDel092498; Thu, 2 Aug 2018 19:37:13 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72JbDrZ092495; Thu, 2 Aug 2018 19:37:13 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808021937.w72JbDrZ092495@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 19:37:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337167 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337167 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 19:37:14 -0000 Author: mav Date: Thu Aug 2 19:37:13 2018 New Revision: 337167 URL: https://svnweb.freebsd.org/changeset/base/337167 Log: 9442 decrease indirect block size of spacemaps Updates to indirect blocks of spacemaps can contribute significantly to write inflation. Therefore we want to reduce the indirect block size of spacemaps from 128K to 16K. illumos/illumos-gate@221813c13b43ef48330b03725e00edee85108cf1 Reviewed by: Serapheim Dimitropoulos Reviewed by: George Wilson Reviewed by: Albert Lee Reviewed by: Igor Kozhukhov Approved by: Dan McDonald Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Thu Aug 2 19:32:32 2018 (r337166) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Thu Aug 2 19:37:13 2018 (r337167) @@ -32,7 +32,8 @@ #include uint64_t -dmu_object_alloc(objset_t *os, dmu_object_type_t ot, int blocksize, +dmu_object_alloc_ibs(objset_t *os, dmu_object_type_t ot, int blocksize, + int indirect_blockshift, dmu_object_type_t bonustype, int bonuslen, dmu_tx_t *tx) { uint64_t object; @@ -92,13 +93,22 @@ dmu_object_alloc(objset_t *os, dmu_object_type_t ot, i os->os_obj_next = object - 1; } - dnode_allocate(dn, ot, blocksize, 0, bonustype, bonuslen, tx); + dnode_allocate(dn, ot, blocksize, indirect_blockshift, + bonustype, bonuslen, tx); mutex_exit(&os->os_obj_lock); dmu_tx_add_new_object(tx, dn); dnode_rele(dn, FTAG); return (object); +} + +uint64_t +dmu_object_alloc(objset_t *os, dmu_object_type_t ot, int blocksize, + dmu_object_type_t bonustype, int bonuslen, dmu_tx_t *tx) +{ + return (dmu_object_alloc_ibs(os, ot, blocksize, 0, + bonustype, bonuslen, tx)); } int Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Thu Aug 2 19:32:32 2018 (r337166) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/space_map.c Thu Aug 2 19:37:13 2018 (r337167) @@ -52,6 +52,14 @@ */ boolean_t zfs_force_some_double_word_sm_entries = B_FALSE; +/* + * Override the default indirect block size of 128K, instead using 16K for + * spacemaps (2^14 bytes). This dramatically reduces write inflation since + * appending to a spacemap typically has to write one data block (4KB) and one + * or two indirect blocks (16K-32K, rather than 128K). + */ +int space_map_ibs = 14; + boolean_t sm_entry_is_debug(uint64_t e) { @@ -674,8 +682,8 @@ space_map_write_impl(space_map_t *sm, range_tree_t *rt * * [1] The feature is enabled. * [2] The offset or run is too big for a single-word entry, - * or the vdev_id is set (meaning not equal to - * SM_NO_VDEVID). + * or the vdev_id is set (meaning not equal to + * SM_NO_VDEVID). * * Note that for purposes of testing we've added the case that * we write two-word entries occasionally when the feature is @@ -834,7 +842,8 @@ space_map_truncate(space_map_t *sm, int blocksize, dmu */ if ((spa_feature_is_enabled(spa, SPA_FEATURE_SPACEMAP_HISTOGRAM) && doi.doi_bonus_size != sizeof (space_map_phys_t)) || - doi.doi_data_block_size != blocksize) { + doi.doi_data_block_size != blocksize || + doi.doi_metadata_block_size != 1 << space_map_ibs) { zfs_dbgmsg("txg %llu, spa %s, sm %p, reallocating " "object[%llu]: old bonus %u, old blocksz %u", dmu_tx_get_txg(tx), spa_name(spa), sm, sm->sm_object, @@ -890,8 +899,8 @@ space_map_alloc(objset_t *os, int blocksize, dmu_tx_t bonuslen = SPACE_MAP_SIZE_V0; } - object = dmu_object_alloc(os, DMU_OT_SPACE_MAP, blocksize, - DMU_OT_SPACE_MAP_HEADER, bonuslen, tx); + object = dmu_object_alloc_ibs(os, DMU_OT_SPACE_MAP, blocksize, + space_map_ibs, DMU_OT_SPACE_MAP_HEADER, bonuslen, tx); return (object); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 19:32:32 2018 (r337166) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 19:37:13 2018 (r337167) @@ -354,6 +354,9 @@ typedef struct dmu_buf { */ uint64_t dmu_object_alloc(objset_t *os, dmu_object_type_t ot, int blocksize, dmu_object_type_t bonus_type, int bonus_len, dmu_tx_t *tx); +uint64_t dmu_object_alloc_ibs(objset_t *os, dmu_object_type_t ot, int blocksize, + int indirect_blockshift, + dmu_object_type_t bonustype, int bonuslen, dmu_tx_t *tx); int dmu_object_claim(objset_t *os, uint64_t object, dmu_object_type_t ot, int blocksize, dmu_object_type_t bonus_type, int bonus_len, dmu_tx_t *tx); int dmu_object_reclaim(objset_t *os, uint64_t object, dmu_object_type_t ot, From owner-svn-src-all@freebsd.org Thu Aug 2 19:50:13 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01A9F104DFF8; Thu, 2 Aug 2018 19:50:13 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A362792B5; Thu, 2 Aug 2018 19:50:12 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 718351630C; Thu, 2 Aug 2018 19:50:12 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72JoCOx098071; Thu, 2 Aug 2018 19:50:12 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72JoCE9098070; Thu, 2 Aug 2018 19:50:12 GMT (envelope-from np@FreeBSD.org) Message-Id: <201808021950.w72JoCE9098070@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Thu, 2 Aug 2018 19:50:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337168 - head/sys/dev/cxgbe X-SVN-Group: head X-SVN-Commit-Author: np X-SVN-Commit-Paths: head/sys/dev/cxgbe X-SVN-Commit-Revision: 337168 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 19:50:13 -0000 Author: np Date: Thu Aug 2 19:50:12 2018 New Revision: 337168 URL: https://svnweb.freebsd.org/changeset/base/337168 Log: cxgbe(4): Use the tx credit limit for ethofld rather than TOE when initializing the softc for a per-flow rate limiter. The limit happens to be the same for both and the existing code worked by accident for common configurations. Reported by: gallatin@ Sponsored by: Chelsio Communications Modified: head/sys/dev/cxgbe/t4_sched.c Modified: head/sys/dev/cxgbe/t4_sched.c ============================================================================== --- head/sys/dev/cxgbe/t4_sched.c Thu Aug 2 19:37:13 2018 (r337167) +++ head/sys/dev/cxgbe/t4_sched.c Thu Aug 2 19:50:12 2018 (r337168) @@ -597,7 +597,7 @@ failed: cst->port_id = pi->port_id; cst->schedcl = schedcl; cst->max_rate = params->rate_limit.max_rate; - cst->tx_credits = sc->params.ofldq_wr_cred; + cst->tx_credits = sc->params.eo_wr_cred; cst->tx_total = cst->tx_credits; cst->plen = 0; cst->ctrl0 = htobe32(V_TXPKT_OPCODE(CPL_TX_PKT) | From owner-svn-src-all@freebsd.org Thu Aug 2 20:06:47 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC3A4104E6DE; Thu, 2 Aug 2018 20:06:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 55B1679BF3; Thu, 2 Aug 2018 20:06:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2C04D16649; Thu, 2 Aug 2018 20:06:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72K6kex007872; Thu, 2 Aug 2018 20:06:46 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72K6kWN007869; Thu, 2 Aug 2018 20:06:46 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022006.w72K6kWN007869@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:06:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337169 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337169 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:06:47 -0000 Author: mav Date: Thu Aug 2 20:06:46 2018 New Revision: 337169 URL: https://svnweb.freebsd.org/changeset/base/337169 Log: MFV r337167: 9442 decrease indirect block size of spacemaps Updates to indirect blocks of spacemaps can contribute significantly to write inflation. Therefore we want to reduce the indirect block size of spacemaps from 128K to 16K. illumos/illumos-gate@221813c13b43ef48330b03725e00edee85108cf1 Reviewed by: Serapheim Dimitropoulos Reviewed by: George Wilson Reviewed by: Albert Lee Reviewed by: Igor Kozhukhov Approved by: Dan McDonald Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c Thu Aug 2 19:50:12 2018 (r337168) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c Thu Aug 2 20:06:46 2018 (r337169) @@ -32,7 +32,8 @@ #include uint64_t -dmu_object_alloc(objset_t *os, dmu_object_type_t ot, int blocksize, +dmu_object_alloc_ibs(objset_t *os, dmu_object_type_t ot, int blocksize, + int indirect_blockshift, dmu_object_type_t bonustype, int bonuslen, dmu_tx_t *tx) { uint64_t object; @@ -92,13 +93,22 @@ dmu_object_alloc(objset_t *os, dmu_object_type_t ot, i os->os_obj_next = object - 1; } - dnode_allocate(dn, ot, blocksize, 0, bonustype, bonuslen, tx); + dnode_allocate(dn, ot, blocksize, indirect_blockshift, + bonustype, bonuslen, tx); mutex_exit(&os->os_obj_lock); dmu_tx_add_new_object(tx, dn); dnode_rele(dn, FTAG); return (object); +} + +uint64_t +dmu_object_alloc(objset_t *os, dmu_object_type_t ot, int blocksize, + dmu_object_type_t bonustype, int bonuslen, dmu_tx_t *tx) +{ + return (dmu_object_alloc_ibs(os, ot, blocksize, 0, + bonustype, bonuslen, tx)); } int Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c Thu Aug 2 19:50:12 2018 (r337168) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/space_map.c Thu Aug 2 20:06:46 2018 (r337169) @@ -54,6 +54,17 @@ SYSCTL_DECL(_vfs_zfs); */ boolean_t zfs_force_some_double_word_sm_entries = B_FALSE; +/* + * Override the default indirect block size of 128K, instead using 16K for + * spacemaps (2^14 bytes). This dramatically reduces write inflation since + * appending to a spacemap typically has to write one data block (4KB) and one + * or two indirect blocks (16K-32K, rather than 128K). + */ +int space_map_ibs = 14; + +SYSCTL_INT(_vfs_zfs, OID_AUTO, space_map_ibs, CTLFLAG_RWTUN, + &space_map_ibs, 0, "Space map indirect block shift"); + boolean_t sm_entry_is_debug(uint64_t e) { @@ -676,8 +687,8 @@ space_map_write_impl(space_map_t *sm, range_tree_t *rt * * [1] The feature is enabled. * [2] The offset or run is too big for a single-word entry, - * or the vdev_id is set (meaning not equal to - * SM_NO_VDEVID). + * or the vdev_id is set (meaning not equal to + * SM_NO_VDEVID). * * Note that for purposes of testing we've added the case that * we write two-word entries occasionally when the feature is @@ -836,7 +847,8 @@ space_map_truncate(space_map_t *sm, int blocksize, dmu */ if ((spa_feature_is_enabled(spa, SPA_FEATURE_SPACEMAP_HISTOGRAM) && doi.doi_bonus_size != sizeof (space_map_phys_t)) || - doi.doi_data_block_size != blocksize) { + doi.doi_data_block_size != blocksize || + doi.doi_metadata_block_size != 1 << space_map_ibs) { zfs_dbgmsg("txg %llu, spa %s, sm %p, reallocating " "object[%llu]: old bonus %u, old blocksz %u", dmu_tx_get_txg(tx), spa_name(spa), sm, sm->sm_object, @@ -892,8 +904,8 @@ space_map_alloc(objset_t *os, int blocksize, dmu_tx_t bonuslen = SPACE_MAP_SIZE_V0; } - object = dmu_object_alloc(os, DMU_OT_SPACE_MAP, blocksize, - DMU_OT_SPACE_MAP_HEADER, bonuslen, tx); + object = dmu_object_alloc_ibs(os, DMU_OT_SPACE_MAP, blocksize, + space_map_ibs, DMU_OT_SPACE_MAP_HEADER, bonuslen, tx); return (object); } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 19:50:12 2018 (r337168) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 20:06:46 2018 (r337169) @@ -356,6 +356,9 @@ typedef struct dmu_buf { */ uint64_t dmu_object_alloc(objset_t *os, dmu_object_type_t ot, int blocksize, dmu_object_type_t bonus_type, int bonus_len, dmu_tx_t *tx); +uint64_t dmu_object_alloc_ibs(objset_t *os, dmu_object_type_t ot, int blocksize, + int indirect_blockshift, + dmu_object_type_t bonustype, int bonuslen, dmu_tx_t *tx); int dmu_object_claim(objset_t *os, uint64_t object, dmu_object_type_t ot, int blocksize, dmu_object_type_t bonus_type, int bonus_len, dmu_tx_t *tx); int dmu_object_reclaim(objset_t *os, uint64_t object, dmu_object_type_t ot, From owner-svn-src-all@freebsd.org Thu Aug 2 20:11:00 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68CE5104E92E; Thu, 2 Aug 2018 20:11:00 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D86179DF0; Thu, 2 Aug 2018 20:11:00 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F34B016670; Thu, 2 Aug 2018 20:10:59 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72KAx93008100; Thu, 2 Aug 2018 20:10:59 GMT (envelope-from rmacklem@FreeBSD.org) Received: (from rmacklem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72KAx8h008099; Thu, 2 Aug 2018 20:10:59 GMT (envelope-from rmacklem@FreeBSD.org) Message-Id: <201808022010.w72KAx8h008099@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rmacklem set sender to rmacklem@FreeBSD.org using -f From: Rick Macklem Date: Thu, 2 Aug 2018 20:10:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337170 - head/sys/fs/nfsclient X-SVN-Group: head X-SVN-Commit-Author: rmacklem X-SVN-Commit-Paths: head/sys/fs/nfsclient X-SVN-Commit-Revision: 337170 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:11:00 -0000 Author: rmacklem Date: Thu Aug 2 20:10:59 2018 New Revision: 337170 URL: https://svnweb.freebsd.org/changeset/base/337170 Log: Silence newer gcc warnings. Newer versions of gcc generate "might not be initialized" warnings for several variables in nfsrpc_doiods(). I have checked and all of these variables are assigned values before they are used. In the one case of "tdrpc", it could have passed garbage as an argument to nfscl_dofflayoutio() when mirrorcnt is one. However nfscl_dofflayoutio() only uses the argument when mirrorcnt > 1, so it wasn't actually broken. This patch initializes "tdrpc" to avoid confusion and initializes the rest to make the compiler happy. Requested by: mmacy Modified: head/sys/fs/nfsclient/nfs_clrpcops.c Modified: head/sys/fs/nfsclient/nfs_clrpcops.c ============================================================================== --- head/sys/fs/nfsclient/nfs_clrpcops.c Thu Aug 2 20:06:46 2018 (r337169) +++ head/sys/fs/nfsclient/nfs_clrpcops.c Thu Aug 2 20:10:59 2018 (r337170) @@ -5619,10 +5619,10 @@ nfscl_doiods(vnode_t vp, struct uio *uiop, int *iomode int eof, error, firstmirror, i, iolaymode, mirrorcnt, recalled, timo; void *lckp; uint8_t *dev; - void *iovbase; - size_t iovlen; - off_t offs; - ssize_t resid; + void *iovbase = NULL; + size_t iovlen = 0; + off_t offs = 0; + ssize_t resid = 0; if (!NFSHASPNFS(nmp) || nfscl_enablecallb == 0 || nfs_numnfscbd == 0 || (np->n_flag & NNOLAYOUT) != 0) @@ -5696,7 +5696,7 @@ nfscl_doiods(vnode_t vp, struct uio *uiop, int *iomode * do all mirrors. */ m = NULL; - drpc = NULL; + tdrpc = drpc = NULL; firstmirror = 0; mirrorcnt = 1; if ((layp->nfsly_flags & NFSLY_FLEXFILE) != 0 && From owner-svn-src-all@freebsd.org Thu Aug 2 20:17:39 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C2BE104EA93; Thu, 2 Aug 2018 20:17:39 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E62CE7A271; Thu, 2 Aug 2018 20:17:38 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C4289167DF; Thu, 2 Aug 2018 20:17:38 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72KHch3013257; Thu, 2 Aug 2018 20:17:38 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72KHcoJ013254; Thu, 2 Aug 2018 20:17:38 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022017.w72KHcoJ013254@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:17:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337171 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337171 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:17:39 -0000 Author: mav Date: Thu Aug 2 20:17:37 2018 New Revision: 337171 URL: https://svnweb.freebsd.org/changeset/base/337171 Log: 9464 txg_kick() fails to see that we are quiescing, forcing transactions to their next stages without leaving them accumulate changes Ideally we would like txg_kick() to get triggered only when we are sure that we are not syncing AND not quiescing any txg. This way we can kick an open TXG to the quiescing state when we are sure that there is nothing going on and we would benefit from the different states running concurrently. illumos/illumos-gate@fa41d87de9ec9000964c605eb01d6dc19e4a1abe Reviewed by: Matt Ahrens Reviewed by: Brad Lewis Reviewed by: Andriy Gapon Approved by: Dan McDonald Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_tx.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg_impl.h vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_tx.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_tx.c Thu Aug 2 20:10:59 2018 (r337170) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_tx.c Thu Aug 2 20:17:37 2018 (r337171) @@ -1085,7 +1085,12 @@ dmu_tx_wait(dmu_tx_t *tx) mutex_exit(&dn->dn_mtx); tx->tx_needassign_txh = NULL; } else { - txg_wait_open(tx->tx_pool, tx->tx_lasttried_txg + 1); + /* + * If we have a lot of dirty data just wait until we sync + * out a TXG at which point we'll hopefully have synced + * a portion of the changes. + */ + txg_wait_synced(dp, spa_last_synced_txg(spa) + 1); } } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg_impl.h Thu Aug 2 20:10:59 2018 (r337170) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/txg_impl.h Thu Aug 2 20:17:37 2018 (r337171) @@ -25,7 +25,7 @@ */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2013, 2017 by Delphix. All rights reserved. */ #ifndef _SYS_TXG_IMPL_H @@ -92,6 +92,7 @@ typedef struct tx_state { kmutex_t tx_sync_lock; /* protects the rest of this struct */ uint64_t tx_open_txg; /* currently open txg id */ + uint64_t tx_quiescing_txg; /* currently quiescing txg id */ uint64_t tx_quiesced_txg; /* quiesced txg waiting for sync */ uint64_t tx_syncing_txg; /* currently syncing txg id */ uint64_t tx_synced_txg; /* last synced txg id */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c Thu Aug 2 20:10:59 2018 (r337170) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/txg.c Thu Aug 2 20:17:37 2018 (r337171) @@ -444,6 +444,30 @@ txg_dispatch_callbacks(dsl_pool_t *dp, uint64_t txg) } } +static boolean_t +txg_is_syncing(dsl_pool_t *dp) +{ + tx_state_t *tx = &dp->dp_tx; + ASSERT(MUTEX_HELD(&tx->tx_sync_lock)); + return (tx->tx_syncing_txg != 0); +} + +static boolean_t +txg_is_quiescing(dsl_pool_t *dp) +{ + tx_state_t *tx = &dp->dp_tx; + ASSERT(MUTEX_HELD(&tx->tx_sync_lock)); + return (tx->tx_quiescing_txg != 0); +} + +static boolean_t +txg_has_quiesced_to_sync(dsl_pool_t *dp) +{ + tx_state_t *tx = &dp->dp_tx; + ASSERT(MUTEX_HELD(&tx->tx_sync_lock)); + return (tx->tx_quiesced_txg != 0); +} + static void txg_sync_thread(void *arg) { @@ -470,7 +494,7 @@ txg_sync_thread(void *arg) while (!dsl_scan_active(dp->dp_scan) && !tx->tx_exiting && timer > 0 && tx->tx_synced_txg >= tx->tx_sync_txg_waiting && - tx->tx_quiesced_txg == 0 && + !txg_has_quiesced_to_sync(dp) && dp->dp_dirty_total < zfs_dirty_data_sync) { dprintf("waiting; tx_synced=%llu waiting=%llu dp=%p\n", tx->tx_synced_txg, tx->tx_sync_txg_waiting, dp); @@ -483,7 +507,7 @@ txg_sync_thread(void *arg) * Wait until the quiesce thread hands off a txg to us, * prompting it to do so if necessary. */ - while (!tx->tx_exiting && tx->tx_quiesced_txg == 0) { + while (!tx->tx_exiting && !txg_has_quiesced_to_sync(dp)) { if (tx->tx_quiesce_txg_waiting < tx->tx_open_txg+1) tx->tx_quiesce_txg_waiting = tx->tx_open_txg+1; cv_broadcast(&tx->tx_quiesce_more_cv); @@ -498,6 +522,7 @@ txg_sync_thread(void *arg) * us. This may cause the quiescing thread to now be * able to quiesce another txg, so we must signal it. */ + ASSERT(tx->tx_quiesced_txg != 0); txg = tx->tx_quiesced_txg; tx->tx_quiesced_txg = 0; tx->tx_syncing_txg = txg; @@ -546,7 +571,7 @@ txg_quiesce_thread(void *arg) */ while (!tx->tx_exiting && (tx->tx_open_txg >= tx->tx_quiesce_txg_waiting || - tx->tx_quiesced_txg != 0)) + txg_has_quiesced_to_sync(dp))) txg_thread_wait(tx, &cpr, &tx->tx_quiesce_more_cv, 0); if (tx->tx_exiting) @@ -556,6 +581,8 @@ txg_quiesce_thread(void *arg) dprintf("txg=%llu quiesce_txg=%llu sync_txg=%llu\n", txg, tx->tx_quiesce_txg_waiting, tx->tx_sync_txg_waiting); + tx->tx_quiescing_txg = txg; + mutex_exit(&tx->tx_sync_lock); txg_quiesce(dp, txg); mutex_enter(&tx->tx_sync_lock); @@ -564,6 +591,7 @@ txg_quiesce_thread(void *arg) * Hand this txg off to the sync thread. */ dprintf("quiesce done, handing off txg %llu\n", txg); + tx->tx_quiescing_txg = 0; tx->tx_quiesced_txg = txg; DTRACE_PROBE2(txg__quiesced, dsl_pool_t *, dp, uint64_t, txg); cv_broadcast(&tx->tx_sync_more_cv); @@ -661,7 +689,8 @@ txg_kick(dsl_pool_t *dp) ASSERT(!dsl_pool_config_held(dp)); mutex_enter(&tx->tx_sync_lock); - if (tx->tx_syncing_txg == 0 && + if (!txg_is_syncing(dp) && + !txg_is_quiescing(dp) && tx->tx_quiesce_txg_waiting <= tx->tx_open_txg && tx->tx_sync_txg_waiting <= tx->tx_synced_txg && tx->tx_quiesced_txg <= tx->tx_synced_txg) { From owner-svn-src-all@freebsd.org Thu Aug 2 20:18:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38EC3104EB5A; Thu, 2 Aug 2018 20:18:51 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC0157A3E2; Thu, 2 Aug 2018 20:18:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BA734167E0; Thu, 2 Aug 2018 20:18:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72KIots013360; Thu, 2 Aug 2018 20:18:50 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72KIoN6013357; Thu, 2 Aug 2018 20:18:50 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022018.w72KIoN6013357@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:18:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337172 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337172 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:18:51 -0000 Author: mav Date: Thu Aug 2 20:18:49 2018 New Revision: 337172 URL: https://svnweb.freebsd.org/changeset/base/337172 Log: MFV r337171: 9464 txg_kick() fails to see that we are quiescing, forcing transactions to their next stages without leaving them accumulate changes Ideally we would like txg_kick() to get triggered only when we are sure that we are not syncing AND not quiescing any txg. This way we can kick an open TXG to the quiescing state when we are sure that there is nothing going on and we would benefit from the different states running concurrently. illumos/illumos-gate@fa41d87de9ec9000964c605eb01d6dc19e4a1abe Reviewed by: Matt Ahrens Reviewed by: Brad Lewis Reviewed by: Andriy Gapon Approved by: Dan McDonald Author: Serapheim Dimitropoulos Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_tx.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/txg_impl.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_tx.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_tx.c Thu Aug 2 20:17:37 2018 (r337171) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_tx.c Thu Aug 2 20:18:49 2018 (r337172) @@ -1091,7 +1091,12 @@ dmu_tx_wait(dmu_tx_t *tx) mutex_exit(&dn->dn_mtx); tx->tx_needassign_txh = NULL; } else { - txg_wait_open(tx->tx_pool, tx->tx_lasttried_txg + 1); + /* + * If we have a lot of dirty data just wait until we sync + * out a TXG at which point we'll hopefully have synced + * a portion of the changes. + */ + txg_wait_synced(dp, spa_last_synced_txg(spa) + 1); } } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/txg_impl.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/txg_impl.h Thu Aug 2 20:17:37 2018 (r337171) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/txg_impl.h Thu Aug 2 20:18:49 2018 (r337172) @@ -25,7 +25,7 @@ */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2013, 2017 by Delphix. All rights reserved. */ #ifndef _SYS_TXG_IMPL_H @@ -92,6 +92,7 @@ typedef struct tx_state { kmutex_t tx_sync_lock; /* protects the rest of this struct */ uint64_t tx_open_txg; /* currently open txg id */ + uint64_t tx_quiescing_txg; /* currently quiescing txg id */ uint64_t tx_quiesced_txg; /* quiesced txg waiting for sync */ uint64_t tx_syncing_txg; /* currently syncing txg id */ uint64_t tx_synced_txg; /* last synced txg id */ Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c Thu Aug 2 20:17:37 2018 (r337171) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c Thu Aug 2 20:18:49 2018 (r337172) @@ -450,6 +450,30 @@ txg_dispatch_callbacks(dsl_pool_t *dp, uint64_t txg) } } +static boolean_t +txg_is_syncing(dsl_pool_t *dp) +{ + tx_state_t *tx = &dp->dp_tx; + ASSERT(MUTEX_HELD(&tx->tx_sync_lock)); + return (tx->tx_syncing_txg != 0); +} + +static boolean_t +txg_is_quiescing(dsl_pool_t *dp) +{ + tx_state_t *tx = &dp->dp_tx; + ASSERT(MUTEX_HELD(&tx->tx_sync_lock)); + return (tx->tx_quiescing_txg != 0); +} + +static boolean_t +txg_has_quiesced_to_sync(dsl_pool_t *dp) +{ + tx_state_t *tx = &dp->dp_tx; + ASSERT(MUTEX_HELD(&tx->tx_sync_lock)); + return (tx->tx_quiesced_txg != 0); +} + static void txg_sync_thread(void *arg) { @@ -476,7 +500,7 @@ txg_sync_thread(void *arg) while (!dsl_scan_active(dp->dp_scan) && !tx->tx_exiting && timer > 0 && tx->tx_synced_txg >= tx->tx_sync_txg_waiting && - tx->tx_quiesced_txg == 0 && + !txg_has_quiesced_to_sync(dp) && dp->dp_dirty_total < zfs_dirty_data_sync) { dprintf("waiting; tx_synced=%llu waiting=%llu dp=%p\n", tx->tx_synced_txg, tx->tx_sync_txg_waiting, dp); @@ -489,7 +513,7 @@ txg_sync_thread(void *arg) * Wait until the quiesce thread hands off a txg to us, * prompting it to do so if necessary. */ - while (!tx->tx_exiting && tx->tx_quiesced_txg == 0) { + while (!tx->tx_exiting && !txg_has_quiesced_to_sync(dp)) { if (tx->tx_quiesce_txg_waiting < tx->tx_open_txg+1) tx->tx_quiesce_txg_waiting = tx->tx_open_txg+1; cv_broadcast(&tx->tx_quiesce_more_cv); @@ -504,6 +528,7 @@ txg_sync_thread(void *arg) * us. This may cause the quiescing thread to now be * able to quiesce another txg, so we must signal it. */ + ASSERT(tx->tx_quiesced_txg != 0); txg = tx->tx_quiesced_txg; tx->tx_quiesced_txg = 0; tx->tx_syncing_txg = txg; @@ -552,7 +577,7 @@ txg_quiesce_thread(void *arg) */ while (!tx->tx_exiting && (tx->tx_open_txg >= tx->tx_quiesce_txg_waiting || - tx->tx_quiesced_txg != 0)) + txg_has_quiesced_to_sync(dp))) txg_thread_wait(tx, &cpr, &tx->tx_quiesce_more_cv, 0); if (tx->tx_exiting) @@ -562,6 +587,8 @@ txg_quiesce_thread(void *arg) dprintf("txg=%llu quiesce_txg=%llu sync_txg=%llu\n", txg, tx->tx_quiesce_txg_waiting, tx->tx_sync_txg_waiting); + tx->tx_quiescing_txg = txg; + mutex_exit(&tx->tx_sync_lock); txg_quiesce(dp, txg); mutex_enter(&tx->tx_sync_lock); @@ -570,6 +597,7 @@ txg_quiesce_thread(void *arg) * Hand this txg off to the sync thread. */ dprintf("quiesce done, handing off txg %llu\n", txg); + tx->tx_quiescing_txg = 0; tx->tx_quiesced_txg = txg; DTRACE_PROBE2(txg__quiesced, dsl_pool_t *, dp, uint64_t, txg); cv_broadcast(&tx->tx_sync_more_cv); @@ -667,7 +695,8 @@ txg_kick(dsl_pool_t *dp) ASSERT(!dsl_pool_config_held(dp)); mutex_enter(&tx->tx_sync_lock); - if (tx->tx_syncing_txg == 0 && + if (!txg_is_syncing(dp) && + !txg_is_quiescing(dp) && tx->tx_quiesce_txg_waiting <= tx->tx_open_txg && tx->tx_sync_txg_waiting <= tx->tx_synced_txg && tx->tx_quiesced_txg <= tx->tx_synced_txg) { From owner-svn-src-all@freebsd.org Thu Aug 2 20:29:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD4ED104F162; Thu, 2 Aug 2018 20:29:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 727387AC22; Thu, 2 Aug 2018 20:29:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 539C416992; Thu, 2 Aug 2018 20:29:59 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72KTxx1018624; Thu, 2 Aug 2018 20:29:59 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72KTxuW018623; Thu, 2 Aug 2018 20:29:59 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022029.w72KTxuW018623@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:29:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337175 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337175 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:30:00 -0000 Author: mav Date: Thu Aug 2 20:29:58 2018 New Revision: 337175 URL: https://svnweb.freebsd.org/changeset/base/337175 Log: 9487 Free objects when receiving full stream as clone All objects after the last written or freed object are not supposed to exist after receiving the stream. We should free them accordingly, as if a freeobjects record for them had been included in the stream. zfsonlinux/zfs@48fbb9ddbf2281911560dfbc2821aa8b74127315 illumos/illumos-gate@7864b8192b8d30471fa2240466d516292e5765b8 Reviewed by: Matthew Ahrens Approved by: Dan McDonald Author: Paul Dagnelie Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_send.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c Thu Aug 2 20:28:09 2018 (r337174) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_send.c Thu Aug 2 20:29:58 2018 (r337175) @@ -1757,6 +1757,7 @@ dmu_recv_begin(char *tofs, char *tosnap, dmu_replay_re drc->drc_force = force; drc->drc_resumable = resumable; drc->drc_cred = CRED(); + drc->drc_clone = (origin != NULL); if (drc->drc_drrb->drr_magic == BSWAP_64(DMU_BACKUP_MAGIC)) { drc->drc_byteswap = B_TRUE; @@ -1817,7 +1818,9 @@ struct receive_writer_arg { /* A map from guid to dataset to help handle dedup'd streams. */ avl_tree_t *guid_to_ds_map; boolean_t resumable; - uint64_t last_object, last_offset; + uint64_t last_object; + uint64_t last_offset; + uint64_t max_object; /* highest object ID referenced in stream */ uint64_t bytes_read; /* bytes read when current record created */ }; @@ -2089,6 +2092,9 @@ receive_object(struct receive_writer_arg *rwa, struct return (SET_ERROR(EINVAL)); object = err == 0 ? drro->drr_object : DMU_NEW_OBJECT; + if (drro->drr_object > rwa->max_object) + rwa->max_object = drro->drr_object; + /* * If we are losing blkptrs or changing the block size this must * be a new file instance. We must clear out the previous file @@ -2184,6 +2190,9 @@ receive_freeobjects(struct receive_writer_arg *rwa, err = dmu_free_long_object(rwa->os, obj); if (err != 0) return (err); + + if (obj > rwa->max_object) + rwa->max_object = obj; } if (next_err != ESRCH) return (next_err); @@ -2213,6 +2222,9 @@ receive_write(struct receive_writer_arg *rwa, struct d rwa->last_object = drrw->drr_object; rwa->last_offset = drrw->drr_offset; + if (rwa->last_object > rwa->max_object) + rwa->max_object = rwa->last_object; + if (dmu_object_info(rwa->os, drrw->drr_object, NULL) != 0) return (SET_ERROR(EINVAL)); @@ -2289,6 +2301,9 @@ receive_write_byref(struct receive_writer_arg *rwa, ref_os = rwa->os; } + if (drrwbr->drr_object > rwa->max_object) + rwa->max_object = drrwbr->drr_object; + err = dmu_buf_hold(ref_os, drrwbr->drr_refobject, drrwbr->drr_refoffset, FTAG, &dbp, DMU_READ_PREFETCH); if (err != 0) @@ -2331,6 +2346,9 @@ receive_write_embedded(struct receive_writer_arg *rwa, if (drrwe->drr_compression >= ZIO_COMPRESS_FUNCTIONS) return (EINVAL); + if (drrwe->drr_object > rwa->max_object) + rwa->max_object = drrwe->drr_object; + tx = dmu_tx_create(rwa->os); dmu_tx_hold_write(tx, drrwe->drr_object, @@ -2367,6 +2385,9 @@ receive_spill(struct receive_writer_arg *rwa, struct d if (dmu_object_info(rwa->os, drrs->drr_object, NULL) != 0) return (SET_ERROR(EINVAL)); + if (drrs->drr_object > rwa->max_object) + rwa->max_object = drrs->drr_object; + VERIFY0(dmu_bonus_hold(rwa->os, drrs->drr_object, FTAG, &db)); if ((err = dmu_spill_hold_by_bonus(db, FTAG, &db_spill)) != 0) { dmu_buf_rele(db, FTAG); @@ -2411,6 +2432,9 @@ receive_free(struct receive_writer_arg *rwa, struct dr if (dmu_object_info(rwa->os, drrf->drr_object, NULL) != 0) return (SET_ERROR(EINVAL)); + if (drrf->drr_object > rwa->max_object) + rwa->max_object = drrf->drr_object; + err = dmu_free_long_range(rwa->os, drrf->drr_object, drrf->drr_offset, drrf->drr_length); @@ -3028,6 +3052,41 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, vnode_t *vp, o cv_wait(&rwa.cv, &rwa.mutex); } mutex_exit(&rwa.mutex); + + /* + * If we are receiving a full stream as a clone, all object IDs which + * are greater than the maximum ID referenced in the stream are + * by definition unused and must be freed. Note that it's possible that + * we've resumed this send and the first record we received was the END + * record. In that case, max_object would be 0, but we shouldn't start + * freeing all objects from there; instead we should start from the + * resumeobj. + */ + if (drc->drc_clone && drc->drc_drrb->drr_fromguid == 0) { + uint64_t obj; + if (nvlist_lookup_uint64(begin_nvl, "resume_object", &obj) != 0) + obj = 0; + if (rwa.max_object > obj) + obj = rwa.max_object; + obj++; + int free_err = 0; + int next_err = 0; + + while (next_err == 0) { + free_err = dmu_free_long_object(rwa.os, obj); + if (free_err != 0 && free_err != ENOENT) + break; + + next_err = dmu_object_next(rwa.os, &obj, FALSE, 0); + } + + if (err == 0) { + if (free_err != 0 && free_err != ENOENT) + err = free_err; + else if (next_err != ESRCH) + err = next_err; + } + } cv_destroy(&rwa.cv); mutex_destroy(&rwa.mutex); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_send.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_send.h Thu Aug 2 20:28:09 2018 (r337174) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu_send.h Thu Aug 2 20:29:58 2018 (r337175) @@ -63,6 +63,7 @@ typedef struct dmu_recv_cookie { boolean_t drc_byteswap; boolean_t drc_force; boolean_t drc_resumable; + boolean_t drc_clone; struct avl_tree *drc_guid_to_ds_map; zio_cksum_t drc_cksum; uint64_t drc_newsnapobj; From owner-svn-src-all@freebsd.org Thu Aug 2 20:33:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDA5B104F91F; Thu, 2 Aug 2018 20:33:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82D4D7B305; Thu, 2 Aug 2018 20:33:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 63AB516B23; Thu, 2 Aug 2018 20:33:14 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72KXEvA023397; Thu, 2 Aug 2018 20:33:14 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72KXEiP023396; Thu, 2 Aug 2018 20:33:14 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022033.w72KXEiP023396@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:33:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337177 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337177 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:33:15 -0000 Author: mav Date: Thu Aug 2 20:33:13 2018 New Revision: 337177 URL: https://svnweb.freebsd.org/changeset/base/337177 Log: MFV r337175: 9487 Free objects when receiving full stream as clone All objects after the last written or freed object are not supposed to exist after receiving the stream. We should free them accordingly, as if a freeobjects record for them had been included in the stream. zfsonlinux/zfs@48fbb9ddbf2281911560dfbc2821aa8b74127315 illumos/illumos-gate@7864b8192b8d30471fa2240466d516292e5765b8 Reviewed by: Matthew Ahrens Approved by: Dan McDonald Author: Paul Dagnelie Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_send.h Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c Thu Aug 2 20:31:54 2018 (r337176) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c Thu Aug 2 20:33:13 2018 (r337177) @@ -1795,6 +1795,7 @@ dmu_recv_begin(char *tofs, char *tosnap, dmu_replay_re drc->drc_force = force; drc->drc_resumable = resumable; drc->drc_cred = CRED(); + drc->drc_clone = (origin != NULL); if (drc->drc_drrb->drr_magic == BSWAP_64(DMU_BACKUP_MAGIC)) { drc->drc_byteswap = B_TRUE; @@ -1855,7 +1856,9 @@ struct receive_writer_arg { /* A map from guid to dataset to help handle dedup'd streams. */ avl_tree_t *guid_to_ds_map; boolean_t resumable; - uint64_t last_object, last_offset; + uint64_t last_object; + uint64_t last_offset; + uint64_t max_object; /* highest object ID referenced in stream */ uint64_t bytes_read; /* bytes read when current record created */ }; @@ -2152,6 +2155,9 @@ receive_object(struct receive_writer_arg *rwa, struct return (SET_ERROR(EINVAL)); object = err == 0 ? drro->drr_object : DMU_NEW_OBJECT; + if (drro->drr_object > rwa->max_object) + rwa->max_object = drro->drr_object; + /* * If we are losing blkptrs or changing the block size this must * be a new file instance. We must clear out the previous file @@ -2247,6 +2253,9 @@ receive_freeobjects(struct receive_writer_arg *rwa, err = dmu_free_long_object(rwa->os, obj); if (err != 0) return (err); + + if (obj > rwa->max_object) + rwa->max_object = obj; } if (next_err != ESRCH) return (next_err); @@ -2276,6 +2285,9 @@ receive_write(struct receive_writer_arg *rwa, struct d rwa->last_object = drrw->drr_object; rwa->last_offset = drrw->drr_offset; + if (rwa->last_object > rwa->max_object) + rwa->max_object = rwa->last_object; + if (dmu_object_info(rwa->os, drrw->drr_object, NULL) != 0) return (SET_ERROR(EINVAL)); @@ -2352,6 +2364,9 @@ receive_write_byref(struct receive_writer_arg *rwa, ref_os = rwa->os; } + if (drrwbr->drr_object > rwa->max_object) + rwa->max_object = drrwbr->drr_object; + err = dmu_buf_hold(ref_os, drrwbr->drr_refobject, drrwbr->drr_refoffset, FTAG, &dbp, DMU_READ_PREFETCH); if (err != 0) @@ -2394,6 +2409,9 @@ receive_write_embedded(struct receive_writer_arg *rwa, if (drrwe->drr_compression >= ZIO_COMPRESS_FUNCTIONS) return (EINVAL); + if (drrwe->drr_object > rwa->max_object) + rwa->max_object = drrwe->drr_object; + tx = dmu_tx_create(rwa->os); dmu_tx_hold_write(tx, drrwe->drr_object, @@ -2430,6 +2448,9 @@ receive_spill(struct receive_writer_arg *rwa, struct d if (dmu_object_info(rwa->os, drrs->drr_object, NULL) != 0) return (SET_ERROR(EINVAL)); + if (drrs->drr_object > rwa->max_object) + rwa->max_object = drrs->drr_object; + VERIFY0(dmu_bonus_hold(rwa->os, drrs->drr_object, FTAG, &db)); if ((err = dmu_spill_hold_by_bonus(db, FTAG, &db_spill)) != 0) { dmu_buf_rele(db, FTAG); @@ -2474,6 +2495,9 @@ receive_free(struct receive_writer_arg *rwa, struct dr if (dmu_object_info(rwa->os, drrf->drr_object, NULL) != 0) return (SET_ERROR(EINVAL)); + if (drrf->drr_object > rwa->max_object) + rwa->max_object = drrf->drr_object; + err = dmu_free_long_range(rwa->os, drrf->drr_object, drrf->drr_offset, drrf->drr_length); @@ -3092,6 +3116,41 @@ dmu_recv_stream(dmu_recv_cookie_t *drc, struct file *f cv_wait(&rwa.cv, &rwa.mutex); } mutex_exit(&rwa.mutex); + + /* + * If we are receiving a full stream as a clone, all object IDs which + * are greater than the maximum ID referenced in the stream are + * by definition unused and must be freed. Note that it's possible that + * we've resumed this send and the first record we received was the END + * record. In that case, max_object would be 0, but we shouldn't start + * freeing all objects from there; instead we should start from the + * resumeobj. + */ + if (drc->drc_clone && drc->drc_drrb->drr_fromguid == 0) { + uint64_t obj; + if (nvlist_lookup_uint64(begin_nvl, "resume_object", &obj) != 0) + obj = 0; + if (rwa.max_object > obj) + obj = rwa.max_object; + obj++; + int free_err = 0; + int next_err = 0; + + while (next_err == 0) { + free_err = dmu_free_long_object(rwa.os, obj); + if (free_err != 0 && free_err != ENOENT) + break; + + next_err = dmu_object_next(rwa.os, &obj, FALSE, 0); + } + + if (err == 0) { + if (free_err != 0 && free_err != ENOENT) + err = free_err; + else if (next_err != ESRCH) + err = next_err; + } + } cv_destroy(&rwa.cv); mutex_destroy(&rwa.mutex); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_send.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_send.h Thu Aug 2 20:31:54 2018 (r337176) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_send.h Thu Aug 2 20:33:13 2018 (r337177) @@ -70,6 +70,7 @@ typedef struct dmu_recv_cookie { boolean_t drc_byteswap; boolean_t drc_force; boolean_t drc_resumable; + boolean_t drc_clone; struct avl_tree *drc_guid_to_ds_map; zio_cksum_t drc_cksum; uint64_t drc_newsnapobj; From owner-svn-src-all@freebsd.org Thu Aug 2 20:37:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83DF0104FAB6; Thu, 2 Aug 2018 20:37:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A2FF7B4E2; Thu, 2 Aug 2018 20:37:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1733516B25; Thu, 2 Aug 2018 20:37:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72KbrWN023637; Thu, 2 Aug 2018 20:37:53 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Kbro6023636; Thu, 2 Aug 2018 20:37:53 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022037.w72Kbro6023636@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:37:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337178 - vendor/illumos/dist/cmd/zdb X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor/illumos/dist/cmd/zdb X-SVN-Commit-Revision: 337178 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:37:54 -0000 Author: mav Date: Thu Aug 2 20:37:53 2018 New Revision: 337178 URL: https://svnweb.freebsd.org/changeset/base/337178 Log: 9523 Large alloc in zdb can cause trouble 16MB alloc in zdb_embedded_block() can cause cores in certain situations (clang, gcc55). OsX commit: https://github.com/openzfsonosx/zfs/commit/ced236a5da6e72ea7bf6d2919fe14e17cffe10f1 FreeBSD commit: https://svnweb.freebsd.org/base?view=revision&revision=326150 illumos/illumos-gate@03a4c2f4bfaca30115963b76445279b36468a614 Reviewed by: Igor Kozhukhov Reviewed by: Andriy Gapon Reviewed by: Matthew Ahrens Approved by: Dan McDonald Author: Jorgen Lundman Modified: vendor/illumos/dist/cmd/zdb/zdb.c Modified: vendor/illumos/dist/cmd/zdb/zdb.c ============================================================================== --- vendor/illumos/dist/cmd/zdb/zdb.c Thu Aug 2 20:33:13 2018 (r337177) +++ vendor/illumos/dist/cmd/zdb/zdb.c Thu Aug 2 20:37:53 2018 (r337178) @@ -4906,7 +4906,7 @@ zdb_embedded_block(char *thing) { blkptr_t bp; unsigned long long *words = (void *)&bp; - char buf[SPA_MAXBLOCKSIZE]; + char *buf; int err; bzero(&bp, sizeof (bp)); @@ -4917,16 +4917,22 @@ zdb_embedded_block(char *thing) words + 8, words + 9, words + 10, words + 11, words + 12, words + 13, words + 14, words + 15); if (err != 16) { - (void) printf("invalid input format\n"); + (void) fprintf(stderr, "invalid input format\n"); exit(1); } ASSERT3U(BPE_GET_LSIZE(&bp), <=, SPA_MAXBLOCKSIZE); + buf = malloc(SPA_MAXBLOCKSIZE); + if (buf == NULL) { + (void) fprintf(stderr, "out of memory\n"); + exit(1); + } err = decode_embedded_bp(&bp, buf, BPE_GET_LSIZE(&bp)); if (err != 0) { - (void) printf("decode failed: %u\n", err); + (void) fprintf(stderr, "decode failed: %u\n", err); exit(1); } zdb_dump_block_raw(buf, BPE_GET_LSIZE(&bp), 0); + free(buf); } static boolean_t From owner-svn-src-all@freebsd.org Thu Aug 2 20:44:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C013104FDAD; Thu, 2 Aug 2018 20:44:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C54FC7B94E; Thu, 2 Aug 2018 20:44:07 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A22E816CC7; Thu, 2 Aug 2018 20:44:07 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Ki7e6028599; Thu, 2 Aug 2018 20:44:07 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Ki7qv028598; Thu, 2 Aug 2018 20:44:07 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022044.w72Ki7qv028598@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:44:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337179 - head/cddl/contrib/opensolaris/cmd/zdb X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/cddl/contrib/opensolaris/cmd/zdb X-SVN-Commit-Revision: 337179 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:44:08 -0000 Author: mav Date: Thu Aug 2 20:44:07 2018 New Revision: 337179 URL: https://svnweb.freebsd.org/changeset/base/337179 Log: 9523 Large alloc in zdb can cause trouble 16MB alloc in zdb_embedded_block() can cause cores in certain situations (clang, gcc55). OsX commit: https://github.com/openzfsonosx/zfs/commit/ced236a5da6e72ea7bf6d2919fe14e17cffe10f1 FreeBSD commit: https://svnweb.freebsd.org/base?view=revision&revision=326150 illumos/illumos-gate@03a4c2f4bfaca30115963b76445279b36468a614 Reviewed by: Igor Kozhukhov Reviewed by: Andriy Gapon Reviewed by: Matthew Ahrens Approved by: Dan McDonald Author: Jorgen Lundman This is an update for r326150 (by avg), where this change comes from. Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/cddl/contrib/opensolaris/cmd/zdb/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zdb/zdb.c ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Thu Aug 2 20:37:53 2018 (r337178) +++ head/cddl/contrib/opensolaris/cmd/zdb/zdb.c Thu Aug 2 20:44:07 2018 (r337179) @@ -4950,19 +4950,18 @@ zdb_embedded_block(char *thing) words + 8, words + 9, words + 10, words + 11, words + 12, words + 13, words + 14, words + 15); if (err != 16) { - (void) printf("invalid input format\n"); + (void) fprintf(stderr, "invalid input format\n"); exit(1); } ASSERT3U(BPE_GET_LSIZE(&bp), <=, SPA_MAXBLOCKSIZE); buf = malloc(SPA_MAXBLOCKSIZE); if (buf == NULL) { - (void) fprintf(stderr, "%s: failed to allocate %llu bytes\n", - __func__, SPA_MAXBLOCKSIZE); + (void) fprintf(stderr, "out of memory\n"); exit(1); } err = decode_embedded_bp(&bp, buf, BPE_GET_LSIZE(&bp)); if (err != 0) { - (void) printf("decode failed: %u\n", err); + (void) fprintf(stderr, "decode failed: %u\n", err); free(buf); exit(1); } From owner-svn-src-all@freebsd.org Thu Aug 2 20:49:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6FF3105022C; Thu, 2 Aug 2018 20:49:09 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C4887BD9D; Thu, 2 Aug 2018 20:49:09 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4D4A816CF1; Thu, 2 Aug 2018 20:49:09 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Kn9AF028853; Thu, 2 Aug 2018 20:49:09 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Kn8ph028850; Thu, 2 Aug 2018 20:49:08 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022049.w72Kn8ph028850@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 20:49:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337180 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337180 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 20:49:10 -0000 Author: mav Date: Thu Aug 2 20:49:08 2018 New Revision: 337180 URL: https://svnweb.freebsd.org/changeset/base/337180 Log: 9539 Make zvol operations use _by_dnode routines Continues what was started in 7801 add more by-dnode routines by fully converting zvols to avoid unnecessary dnode_hold() calls. This saves a small amount of CPU time and slightly improves latencies of operations on zvols. illumos/illumos-gate@8dfe5547fbf0979fc1065a8b6fddc1e940a7cf4f Reviewed by: Matthew Ahrens Reviewed by: Brian Behlendorf Reviewed by: Rick McNeal Approved by: Dan McDonald Author: Richard Yao Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Thu Aug 2 20:44:07 2018 (r337179) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu.c Thu Aug 2 20:49:08 2018 (r337180) @@ -443,7 +443,7 @@ dmu_spill_hold_by_bonus(dmu_buf_t *bonus, void *tag, d * and can induce severe lock contention when writing to several files * whose dnodes are in the same block. */ -static int +int dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, uint64_t length, boolean_t read, void *tag, int *numbufsp, dmu_buf_t ***dbpp, uint32_t flags) { @@ -1302,7 +1302,7 @@ xuio_stat_wbuf_nocopy(void) } #ifdef _KERNEL -static int +int dmu_read_uio_dnode(dnode_t *dn, uio_t *uio, uint64_t size) { dmu_buf_t **dbp; @@ -1411,7 +1411,7 @@ dmu_read_uio(objset_t *os, uint64_t object, uio_t *uio return (err); } -static int +int dmu_write_uio_dnode(dnode_t *dn, uio_t *uio, uint64_t size, dmu_tx_t *tx) { dmu_buf_t **dbp; @@ -1600,22 +1600,17 @@ dmu_return_arcbuf(arc_buf_t *buf) * dmu_write(). */ void -dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, arc_buf_t *buf, +dmu_assign_arcbuf_dnode(dnode_t *dn, uint64_t offset, arc_buf_t *buf, dmu_tx_t *tx) { - dmu_buf_impl_t *dbuf = (dmu_buf_impl_t *)handle; - dnode_t *dn; dmu_buf_impl_t *db; uint32_t blksz = (uint32_t)arc_buf_lsize(buf); uint64_t blkid; - DB_DNODE_ENTER(dbuf); - dn = DB_DNODE(dbuf); rw_enter(&dn->dn_struct_rwlock, RW_READER); blkid = dbuf_whichblock(dn, 0, offset); VERIFY((db = dbuf_hold(dn, blkid, FTAG)) != NULL); rw_exit(&dn->dn_struct_rwlock); - DB_DNODE_EXIT(dbuf); /* * We can only assign if the offset is aligned, the arc buf is the @@ -1632,17 +1627,25 @@ dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, ASSERT3U(arc_get_compression(buf), ==, ZIO_COMPRESS_OFF); ASSERT(!(buf->b_flags & ARC_BUF_FLAG_COMPRESSED)); - DB_DNODE_ENTER(dbuf); - dn = DB_DNODE(dbuf); os = dn->dn_objset; object = dn->dn_object; - DB_DNODE_EXIT(dbuf); dbuf_rele(db, FTAG); dmu_write(os, object, offset, blksz, buf->b_data, tx); dmu_return_arcbuf(buf); XUIOSTAT_BUMP(xuiostat_wbuf_copied); } +} + +void +dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, arc_buf_t *buf, + dmu_tx_t *tx) +{ + dmu_buf_impl_t *dbuf = (dmu_buf_impl_t *)handle; + + DB_DNODE_ENTER(dbuf); + dmu_assign_arcbuf_dnode(DB_DNODE(dbuf), offset, buf, tx); + DB_DNODE_EXIT(dbuf); } typedef struct { Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 20:44:07 2018 (r337179) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 20:49:08 2018 (r337180) @@ -517,6 +517,9 @@ uint64_t dmu_buf_refcount(dmu_buf_t *db); int dmu_buf_hold_array_by_bonus(dmu_buf_t *db, uint64_t offset, uint64_t length, boolean_t read, void *tag, int *numbufsp, dmu_buf_t ***dbpp); +int dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, uint64_t length, + boolean_t read, void *tag, int *numbufsp, dmu_buf_t ***dbpp, + uint32_t flags); void dmu_buf_rele_array(dmu_buf_t **, int numbufs, void *tag); typedef void dmu_buf_evict_func_t(void *user_ptr); @@ -755,14 +758,19 @@ void dmu_prealloc(objset_t *os, uint64_t object, uint6 dmu_tx_t *tx); int dmu_read_uio(objset_t *os, uint64_t object, struct uio *uio, uint64_t size); int dmu_read_uio_dbuf(dmu_buf_t *zdb, struct uio *uio, uint64_t size); +int dmu_read_uio_dnode(dnode_t *dn, struct uio *uio, uint64_t size); int dmu_write_uio(objset_t *os, uint64_t object, struct uio *uio, uint64_t size, dmu_tx_t *tx); int dmu_write_uio_dbuf(dmu_buf_t *zdb, struct uio *uio, uint64_t size, dmu_tx_t *tx); +int dmu_write_uio_dnode(dnode_t *dn, struct uio *uio, uint64_t size, + dmu_tx_t *tx); int dmu_write_pages(objset_t *os, uint64_t object, uint64_t offset, uint64_t size, struct page *pp, dmu_tx_t *tx); struct arc_buf *dmu_request_arcbuf(dmu_buf_t *handle, int size); void dmu_return_arcbuf(struct arc_buf *buf); +void dmu_assign_arcbuf_dnode(dnode_t *handle, uint64_t offset, + struct arc_buf *buf, dmu_tx_t *tx); void dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, struct arc_buf *buf, dmu_tx_t *tx); int dmu_xuio_init(struct xuio *uio, int niov); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Thu Aug 2 20:44:07 2018 (r337179) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zvol.c Thu Aug 2 20:49:08 2018 (r337180) @@ -128,7 +128,7 @@ typedef struct zvol_state { zilog_t *zv_zilog; /* ZIL handle */ list_t zv_extents; /* List of extents for dump */ znode_t zv_znode; /* for range locking */ - dmu_buf_t *zv_dbuf; /* bonus handle */ + dnode_t *zv_dn; /* dnode hold */ } zvol_state_t; /* @@ -646,7 +646,7 @@ zvol_first_open(zvol_state_t *zv) return (error); } - error = dmu_bonus_hold(os, ZVOL_OBJ, zvol_tag, &zv->zv_dbuf); + error = dnode_hold(os, ZVOL_OBJ, zvol_tag, &zv->zv_dn); if (error) { dmu_objset_disown(os, zvol_tag); return (error); @@ -671,8 +671,8 @@ zvol_last_close(zvol_state_t *zv) zil_close(zv->zv_zilog); zv->zv_zilog = NULL; - dmu_buf_rele(zv->zv_dbuf, zvol_tag); - zv->zv_dbuf = NULL; + dnode_rele(zv->zv_dn, zvol_tag); + zv->zv_dn = NULL; /* * Evict cached data @@ -993,8 +993,6 @@ static int zvol_get_data(void *arg, lr_write_t *lr, char *buf, struct lwb *lwb, zio_t *zio) { zvol_state_t *zv = arg; - objset_t *os = zv->zv_objset; - uint64_t object = ZVOL_OBJ; uint64_t offset = lr->lr_offset; uint64_t size = lr->lr_length; /* length of user data */ dmu_buf_t *db; @@ -1018,7 +1016,7 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, st if (buf != NULL) { /* immediate write */ zgd->zgd_rl = zfs_range_lock(&zv->zv_znode, offset, size, RL_READER); - error = dmu_read(os, object, offset, size, buf, + error = dmu_read_by_dnode(zv->zv_dn, offset, size, buf, DMU_READ_NO_PREFETCH); } else { /* indirect write */ /* @@ -1031,7 +1029,7 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, st offset = P2ALIGN(offset, size); zgd->zgd_rl = zfs_range_lock(&zv->zv_znode, offset, size, RL_READER); - error = dmu_buf_hold(os, object, offset, zgd, &db, + error = dmu_buf_hold_by_dnode(zv->zv_dn, offset, zgd, &db, DMU_READ_NO_PREFETCH); if (error == 0) { blkptr_t *bp = &lr->lr_blkptr; @@ -1098,8 +1096,8 @@ zvol_log_write(zvol_state_t *zv, dmu_tx_t *tx, offset_ itx = zil_itx_create(TX_WRITE, sizeof (*lr) + (wr_state == WR_COPIED ? len : 0)); lr = (lr_write_t *)&itx->itx_lr; - if (wr_state == WR_COPIED && dmu_read(zv->zv_objset, - ZVOL_OBJ, off, len, lr + 1, DMU_READ_NO_PREFETCH) != 0) { + if (wr_state == WR_COPIED && dmu_read_by_dnode(zv->zv_dn, + off, len, lr + 1, DMU_READ_NO_PREFETCH) != 0) { zil_itx_destroy(itx); itx = zil_itx_create(TX_WRITE, sizeof (*lr)); lr = (lr_write_t *)&itx->itx_lr; @@ -1468,7 +1466,7 @@ zvol_write(dev_t dev, uio_t *uio, cred_t *cr) dmu_tx_abort(tx); break; } - error = dmu_write_uio_dbuf(zv->zv_dbuf, uio, bytes, tx); + error = dmu_write_uio_dnode(zv->zv_dn, uio, bytes, tx); if (error == 0) zvol_log_write(zv, tx, off, bytes, sync); dmu_tx_commit(tx); @@ -1549,7 +1547,7 @@ zvol_getefi(void *arg, int flag, uint64_t vs, uint8_t int zvol_get_volume_params(minor_t minor, uint64_t *blksize, uint64_t *max_xfer_len, void **minor_hdl, void **objset_hdl, void **zil_hdl, - void **rl_hdl, void **bonus_hdl) + void **rl_hdl, void **dnode_hdl) { zvol_state_t *zv; @@ -1560,7 +1558,7 @@ zvol_get_volume_params(minor_t minor, uint64_t *blksiz return (SET_ERROR(ENXIO)); ASSERT(blksize && max_xfer_len && minor_hdl && - objset_hdl && zil_hdl && rl_hdl && bonus_hdl); + objset_hdl && zil_hdl && rl_hdl && dnode_hdl); *blksize = zv->zv_volblocksize; *max_xfer_len = (uint64_t)zvol_maxphys; @@ -1568,7 +1566,7 @@ zvol_get_volume_params(minor_t minor, uint64_t *blksiz *objset_hdl = zv->zv_objset; *zil_hdl = zv->zv_zilog; *rl_hdl = &zv->zv_znode; - *bonus_hdl = zv->zv_dbuf; + *dnode_hdl = zv->zv_dn; return (0); } From owner-svn-src-all@freebsd.org Thu Aug 2 21:07:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0EFF31050880; Thu, 2 Aug 2018 21:07:06 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B9AC97C754; Thu, 2 Aug 2018 21:07:05 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9ACFE1702D; Thu, 2 Aug 2018 21:07:05 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72L75t6038984; Thu, 2 Aug 2018 21:07:05 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72L75LW038982; Thu, 2 Aug 2018 21:07:05 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022107.w72L75LW038982@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:07:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337181 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337181 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:07:06 -0000 Author: mav Date: Thu Aug 2 21:07:04 2018 New Revision: 337181 URL: https://svnweb.freebsd.org/changeset/base/337181 Log: 9539 Make zvol operations use _by_dnode routines Continues what was started in 7801 add more by-dnode routines by fully converting zvols to avoid unnecessary dnode_hold() calls. This saves a small amount of CPU time and slightly improves latencies of operations on zvols. illumos/illumos-gate@8dfe5547fbf0979fc1065a8b6fddc1e940a7cf4f Reviewed by: Matthew Ahrens Reviewed by: Brian Behlendorf Reviewed by: Rick McNeal Approved by: Dan McDonald Author: Richard Yao Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c Thu Aug 2 20:49:08 2018 (r337180) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c Thu Aug 2 21:07:04 2018 (r337181) @@ -449,7 +449,7 @@ dmu_spill_hold_by_bonus(dmu_buf_t *bonus, void *tag, d * and can induce severe lock contention when writing to several files * whose dnodes are in the same block. */ -static int +int dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, uint64_t length, boolean_t read, void *tag, int *numbufsp, dmu_buf_t ***dbpp, uint32_t flags) { @@ -1321,7 +1321,7 @@ xuio_stat_wbuf_nocopy(void) } #ifdef _KERNEL -static int +int dmu_read_uio_dnode(dnode_t *dn, uio_t *uio, uint64_t size) { dmu_buf_t **dbp; @@ -1437,7 +1437,7 @@ dmu_read_uio(objset_t *os, uint64_t object, uio_t *uio return (err); } -static int +int dmu_write_uio_dnode(dnode_t *dn, uio_t *uio, uint64_t size, dmu_tx_t *tx) { dmu_buf_t **dbp; @@ -1881,22 +1881,17 @@ dmu_return_arcbuf(arc_buf_t *buf) * dmu_write(). */ void -dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, arc_buf_t *buf, +dmu_assign_arcbuf_dnode(dnode_t *dn, uint64_t offset, arc_buf_t *buf, dmu_tx_t *tx) { - dmu_buf_impl_t *dbuf = (dmu_buf_impl_t *)handle; - dnode_t *dn; dmu_buf_impl_t *db; uint32_t blksz = (uint32_t)arc_buf_lsize(buf); uint64_t blkid; - DB_DNODE_ENTER(dbuf); - dn = DB_DNODE(dbuf); rw_enter(&dn->dn_struct_rwlock, RW_READER); blkid = dbuf_whichblock(dn, 0, offset); VERIFY((db = dbuf_hold(dn, blkid, FTAG)) != NULL); rw_exit(&dn->dn_struct_rwlock); - DB_DNODE_EXIT(dbuf); /* * We can only assign if the offset is aligned, the arc buf is the @@ -1924,17 +1919,25 @@ dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, ASSERT3U(arc_get_compression(buf), ==, ZIO_COMPRESS_OFF); ASSERT(!(buf->b_flags & ARC_BUF_FLAG_COMPRESSED)); - DB_DNODE_ENTER(dbuf); - dn = DB_DNODE(dbuf); os = dn->dn_objset; object = dn->dn_object; - DB_DNODE_EXIT(dbuf); dbuf_rele(db, FTAG); dmu_write(os, object, offset, blksz, buf->b_data, tx); dmu_return_arcbuf(buf); XUIOSTAT_BUMP(xuiostat_wbuf_copied); } +} + +void +dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, arc_buf_t *buf, + dmu_tx_t *tx) +{ + dmu_buf_impl_t *dbuf = (dmu_buf_impl_t *)handle; + + DB_DNODE_ENTER(dbuf); + dmu_assign_arcbuf_dnode(DB_DNODE(dbuf), offset, buf, tx); + DB_DNODE_EXIT(dbuf); } typedef struct { Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 20:49:08 2018 (r337180) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu.h Thu Aug 2 21:07:04 2018 (r337181) @@ -519,6 +519,9 @@ uint64_t dmu_buf_refcount(dmu_buf_t *db); int dmu_buf_hold_array_by_bonus(dmu_buf_t *db, uint64_t offset, uint64_t length, boolean_t read, void *tag, int *numbufsp, dmu_buf_t ***dbpp); +int dmu_buf_hold_array_by_dnode(dnode_t *dn, uint64_t offset, uint64_t length, + boolean_t read, void *tag, int *numbufsp, dmu_buf_t ***dbpp, + uint32_t flags); void dmu_buf_rele_array(dmu_buf_t **, int numbufs, void *tag); typedef void dmu_buf_evict_func_t(void *user_ptr); @@ -757,10 +760,13 @@ void dmu_prealloc(objset_t *os, uint64_t object, uint6 dmu_tx_t *tx); int dmu_read_uio(objset_t *os, uint64_t object, struct uio *uio, uint64_t size); int dmu_read_uio_dbuf(dmu_buf_t *zdb, struct uio *uio, uint64_t size); +int dmu_read_uio_dnode(dnode_t *dn, struct uio *uio, uint64_t size); int dmu_write_uio(objset_t *os, uint64_t object, struct uio *uio, uint64_t size, dmu_tx_t *tx); int dmu_write_uio_dbuf(dmu_buf_t *zdb, struct uio *uio, uint64_t size, dmu_tx_t *tx); +int dmu_write_uio_dnode(dnode_t *dn, struct uio *uio, uint64_t size, + dmu_tx_t *tx); #ifdef _KERNEL #ifdef illumos int dmu_write_pages(objset_t *os, uint64_t object, uint64_t offset, @@ -774,6 +780,8 @@ int dmu_read_pages(objset_t *os, uint64_t object, vm_p #endif struct arc_buf *dmu_request_arcbuf(dmu_buf_t *handle, int size); void dmu_return_arcbuf(struct arc_buf *buf); +void dmu_assign_arcbuf_dnode(dnode_t *handle, uint64_t offset, + struct arc_buf *buf, dmu_tx_t *tx); void dmu_assign_arcbuf(dmu_buf_t *handle, uint64_t offset, struct arc_buf *buf, dmu_tx_t *tx); int dmu_xuio_init(struct xuio *uio, int niov); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c Thu Aug 2 20:49:08 2018 (r337180) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c Thu Aug 2 21:07:04 2018 (r337181) @@ -174,7 +174,7 @@ typedef struct zvol_state { zilog_t *zv_zilog; /* ZIL handle */ list_t zv_extents; /* List of extents for dump */ znode_t zv_znode; /* for range locking */ - dmu_buf_t *zv_dbuf; /* bonus handle */ + dnode_t *zv_dn; /* dnode hold */ #ifndef illumos int zv_state; int zv_volmode; /* Provide GEOM or cdev */ @@ -868,7 +868,7 @@ zvol_first_open(zvol_state_t *zv) } zv->zv_volblocksize = doi.doi_data_block_size; - error = dmu_bonus_hold(os, ZVOL_OBJ, zvol_tag, &zv->zv_dbuf); + error = dnode_hold(os, ZVOL_OBJ, zvol_tag, &zv->zv_dn); if (error) { dmu_objset_disown(os, zvol_tag); return (error); @@ -893,8 +893,8 @@ zvol_last_close(zvol_state_t *zv) zil_close(zv->zv_zilog); zv->zv_zilog = NULL; - dmu_buf_rele(zv->zv_dbuf, zvol_tag); - zv->zv_dbuf = NULL; + dnode_rele(zv->zv_dn, zvol_tag); + zv->zv_dn = NULL; /* * Evict cached data @@ -1342,8 +1342,6 @@ static int zvol_get_data(void *arg, lr_write_t *lr, char *buf, struct lwb *lwb, zio_t *zio) { zvol_state_t *zv = arg; - objset_t *os = zv->zv_objset; - uint64_t object = ZVOL_OBJ; uint64_t offset = lr->lr_offset; uint64_t size = lr->lr_length; /* length of user data */ dmu_buf_t *db; @@ -1367,7 +1365,7 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, st if (buf != NULL) { /* immediate write */ zgd->zgd_rl = zfs_range_lock(&zv->zv_znode, offset, size, RL_READER); - error = dmu_read(os, object, offset, size, buf, + error = dmu_read_by_dnode(zv->zv_dn, offset, size, buf, DMU_READ_NO_PREFETCH); } else { /* indirect write */ /* @@ -1380,7 +1378,7 @@ zvol_get_data(void *arg, lr_write_t *lr, char *buf, st offset = P2ALIGN(offset, size); zgd->zgd_rl = zfs_range_lock(&zv->zv_znode, offset, size, RL_READER); - error = dmu_buf_hold(os, object, offset, zgd, &db, + error = dmu_buf_hold_by_dnode(zv->zv_dn, offset, zgd, &db, DMU_READ_NO_PREFETCH); if (error == 0) { blkptr_t *bp = &lr->lr_blkptr; @@ -1451,8 +1449,8 @@ zvol_log_write(zvol_state_t *zv, dmu_tx_t *tx, offset_ itx = zil_itx_create(TX_WRITE, sizeof (*lr) + (wr_state == WR_COPIED ? len : 0)); lr = (lr_write_t *)&itx->itx_lr; - if (wr_state == WR_COPIED && dmu_read(zv->zv_objset, - ZVOL_OBJ, off, len, lr + 1, DMU_READ_NO_PREFETCH) != 0) { + if (wr_state == WR_COPIED && dmu_read_by_dnode(zv->zv_dn, + off, len, lr + 1, DMU_READ_NO_PREFETCH) != 0) { zil_itx_destroy(itx); itx = zil_itx_create(TX_WRITE, sizeof (*lr)); lr = (lr_write_t *)&itx->itx_lr; @@ -1874,7 +1872,7 @@ zvol_read(struct cdev *dev, struct uio *uio, int iofla if (bytes > volsize - uio->uio_loffset) bytes = volsize - uio->uio_loffset; - error = dmu_read_uio_dbuf(zv->zv_dbuf, uio, bytes); + error = dmu_read_uio_dnode(zv->zv_dn, uio, bytes); if (error) { /* convert checksum errors into IO errors */ if (error == ECKSUM) @@ -1946,7 +1944,7 @@ zvol_write(struct cdev *dev, struct uio *uio, int iofl dmu_tx_abort(tx); break; } - error = dmu_write_uio_dbuf(zv->zv_dbuf, uio, bytes, tx); + error = dmu_write_uio_dnode(zv->zv_dn, uio, bytes, tx); if (error == 0) zvol_log_write(zv, tx, off, bytes, sync); dmu_tx_commit(tx); @@ -2028,7 +2026,7 @@ zvol_getefi(void *arg, int flag, uint64_t vs, uint8_t int zvol_get_volume_params(minor_t minor, uint64_t *blksize, uint64_t *max_xfer_len, void **minor_hdl, void **objset_hdl, void **zil_hdl, - void **rl_hdl, void **bonus_hdl) + void **rl_hdl, void **dnode_hdl) { zvol_state_t *zv; @@ -2039,7 +2037,7 @@ zvol_get_volume_params(minor_t minor, uint64_t *blksiz return (SET_ERROR(ENXIO)); ASSERT(blksize && max_xfer_len && minor_hdl && - objset_hdl && zil_hdl && rl_hdl && bonus_hdl); + objset_hdl && zil_hdl && rl_hdl && dnode_hdl); *blksize = zv->zv_volblocksize; *max_xfer_len = (uint64_t)zvol_maxphys; @@ -2047,7 +2045,7 @@ zvol_get_volume_params(minor_t minor, uint64_t *blksiz *objset_hdl = zv->zv_objset; *zil_hdl = zv->zv_zilog; *rl_hdl = &zv->zv_znode; - *bonus_hdl = zv->zv_dbuf; + *dnode_hdl = zv->zv_dn; return (0); } From owner-svn-src-all@freebsd.org Thu Aug 2 21:12:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E0411050C0F; Thu, 2 Aug 2018 21:12:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E6187CC20; Thu, 2 Aug 2018 21:12:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DE988171BD; Thu, 2 Aug 2018 21:12:53 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72LCrLT043807; Thu, 2 Aug 2018 21:12:53 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LCrDt043803; Thu, 2 Aug 2018 21:12:53 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022112.w72LCrDt043803@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:12:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337182 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 337182 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:12:54 -0000 Author: mav Date: Thu Aug 2 21:12:52 2018 New Revision: 337182 URL: https://svnweb.freebsd.org/changeset/base/337182 Log: 9330 stack overflow when creating a deeply nested dataset Datasets that are deeply nested (~100 levels) are impractical. We just put a limit of 50 levels to newly created datasets. Existing datasets should work without a problem. illumos/illumos-gate@5ac95da7d61660aa299c287a39277cb0372be959 Reviewed by: John Kennedy Reviewed by: Matt Ahrens Approved by: Garrett D'Amore Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/common/zfs/zfs_namecheck.c vendor-sys/illumos/dist/common/zfs/zfs_namecheck.h vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man1m/zfs.1m Modified: vendor-sys/illumos/dist/common/zfs/zfs_namecheck.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfs_namecheck.c Thu Aug 2 21:07:04 2018 (r337181) +++ vendor-sys/illumos/dist/common/zfs/zfs_namecheck.c Thu Aug 2 21:12:52 2018 (r337182) @@ -23,7 +23,7 @@ * Use is subject to license terms. */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2013, 2016 by Delphix. All rights reserved. */ /* @@ -34,8 +34,6 @@ * name is invalid. In the kernel, we only care whether it's valid or not. * Each routine therefore takes a 'namecheck_err_t' which describes exactly why * the name failed to validate. - * - * Each function returns 0 on success, -1 on error. */ #if defined(_KERNEL) @@ -50,6 +48,14 @@ #include "zfs_namecheck.h" #include "zfs_deleg.h" +/* + * Deeply nested datasets can overflow the stack, so we put a limit + * in the amount of nesting a path can have. zfs_max_dataset_nesting + * can be tuned temporarily to fix existing datasets that exceed our + * predefined limit. + */ +int zfs_max_dataset_nesting = 50; + static int valid_char(char c) { @@ -60,10 +66,35 @@ valid_char(char c) } /* + * Looks at a path and returns its level of nesting (depth). + */ +int +get_dataset_depth(const char *path) +{ + const char *loc = path; + int nesting = 0; + + /* + * Keep track of nesting until you hit the end of the + * path or found the snapshot/bookmark seperator. + */ + for (int i = 0; loc[i] != '\0' && + loc[i] != '@' && + loc[i] != '#'; i++) { + if (loc[i] == '/') + nesting++; + } + + return (nesting); +} + +/* * Snapshot names must be made up of alphanumeric characters plus the following * characters: * - * [-_.: ] + * [-_.: ] + * + * Returns 0 on success, -1 on error. */ int zfs_component_namecheck(const char *path, namecheck_err_t *why, char *what) @@ -99,6 +130,8 @@ zfs_component_namecheck(const char *path, namecheck_er * Permissions set name must start with the letter '@' followed by the * same character restrictions as snapshot names, except that the name * cannot exceed 64 characters. + * + * Returns 0 on success, -1 on error. */ int permset_namecheck(const char *path, namecheck_err_t *why, char *what) @@ -121,28 +154,40 @@ permset_namecheck(const char *path, namecheck_err_t *w } /* + * Dataset paths should not be deeper than zfs_max_dataset_nesting + * in terms of nesting. + * + * Returns 0 on success, -1 on error. + */ +int +dataset_nestcheck(const char *path) +{ + return ((get_dataset_depth(path) < zfs_max_dataset_nesting) ? 0 : -1); +} + +/* * Entity names must be of the following form: * - * [component/]*[component][(@|#)component]? + * [component/]*[component][(@|#)component]? * * Where each component is made up of alphanumeric characters plus the following * characters: * - * [-_.:%] + * [-_.:%] * * We allow '%' here as we use that character internally to create unique * names for temporary clones (for online recv). + * + * Returns 0 on success, -1 on error. */ int entity_namecheck(const char *path, namecheck_err_t *why, char *what) { - const char *start, *end; - int found_delim; + const char *end; /* * Make sure the name is not too long. */ - if (strlen(path) >= ZFS_MAX_DATASET_NAME_LEN) { if (why) *why = NAME_ERR_TOOLONG; @@ -162,8 +207,8 @@ entity_namecheck(const char *path, namecheck_err_t *wh return (-1); } - start = path; - found_delim = 0; + const char *start = path; + boolean_t found_delim = B_FALSE; for (;;) { /* Find the end of this component */ end = start; @@ -198,7 +243,7 @@ entity_namecheck(const char *path, namecheck_err_t *wh return (-1); } - found_delim = 1; + found_delim = B_TRUE; } /* Zero-length components are not allowed */ @@ -250,6 +295,8 @@ dataset_namecheck(const char *path, namecheck_err_t *w * mountpoint names must be of the following form: * * /[component][/]*[component][/] + * + * Returns 0 on success, -1 on error. */ int mountpoint_namecheck(const char *path, namecheck_err_t *why) @@ -294,6 +341,8 @@ mountpoint_namecheck(const char *path, namecheck_err_t * dataset names, with the additional restriction that the pool name must begin * with a letter. The pool names 'raidz' and 'mirror' are also reserved names * that cannot be used. + * + * Returns 0 on success, -1 on error. */ int pool_namecheck(const char *pool, namecheck_err_t *why, char *what) Modified: vendor-sys/illumos/dist/common/zfs/zfs_namecheck.h ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfs_namecheck.h Thu Aug 2 21:07:04 2018 (r337181) +++ vendor-sys/illumos/dist/common/zfs/zfs_namecheck.h Thu Aug 2 21:12:52 2018 (r337182) @@ -23,7 +23,7 @@ * Use is subject to license terms. */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2013, 2016 by Delphix. All rights reserved. */ #ifndef _ZFS_NAMECHECK_H @@ -48,9 +48,13 @@ typedef enum { #define ZFS_PERMSET_MAXLEN 64 +extern int zfs_max_dataset_nesting; + +int get_dataset_depth(const char *); int pool_namecheck(const char *, namecheck_err_t *, char *); int entity_namecheck(const char *, namecheck_err_t *, char *); int dataset_namecheck(const char *, namecheck_err_t *, char *); +int dataset_nestcheck(const char *); int mountpoint_namecheck(const char *, namecheck_err_t *); int zfs_component_namecheck(const char *, namecheck_err_t *, char *); int permset_namecheck(const char *, namecheck_err_t *, char *); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Thu Aug 2 21:07:04 2018 (r337181) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c Thu Aug 2 21:12:52 2018 (r337182) @@ -54,6 +54,7 @@ #include #include #include +#include "zfs_namecheck.h" /* * Needed to close a window in dnode_move() that allows the objset to be freed @@ -909,6 +910,9 @@ dmu_objset_create_check(void *arg, dmu_tx_t *tx) return (SET_ERROR(EINVAL)); if (strlen(doca->doca_name) >= ZFS_MAX_DATASET_NAME_LEN) + return (SET_ERROR(ENAMETOOLONG)); + + if (dataset_nestcheck(doca->doca_name) != 0) return (SET_ERROR(ENAMETOOLONG)); error = dsl_dir_hold(dp, doca->doca_name, FTAG, &pdd, &tail); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Thu Aug 2 21:07:04 2018 (r337181) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Thu Aug 2 21:12:52 2018 (r337182) @@ -1810,17 +1810,29 @@ typedef struct dsl_dir_rename_arg { cred_t *ddra_cred; } dsl_dir_rename_arg_t; +typedef struct dsl_valid_rename_arg { + int char_delta; + int nest_delta; +} dsl_valid_rename_arg_t; + /* ARGSUSED */ static int dsl_valid_rename(dsl_pool_t *dp, dsl_dataset_t *ds, void *arg) { - int *deltap = arg; + dsl_valid_rename_arg_t *dvra = arg; char namebuf[ZFS_MAX_DATASET_NAME_LEN]; dsl_dataset_name(ds, namebuf); - if (strlen(namebuf) + *deltap >= ZFS_MAX_DATASET_NAME_LEN) + ASSERT3U(strnlen(namebuf, ZFS_MAX_DATASET_NAME_LEN), + <, ZFS_MAX_DATASET_NAME_LEN); + int namelen = strlen(namebuf) + dvra->char_delta; + int depth = get_dataset_depth(namebuf) + dvra->nest_delta; + + if (namelen >= ZFS_MAX_DATASET_NAME_LEN) return (SET_ERROR(ENAMETOOLONG)); + if (dvra->nest_delta > 0 && depth >= zfs_max_dataset_nesting) + return (SET_ERROR(ENAMETOOLONG)); return (0); } @@ -1830,9 +1842,9 @@ dsl_dir_rename_check(void *arg, dmu_tx_t *tx) dsl_dir_rename_arg_t *ddra = arg; dsl_pool_t *dp = dmu_tx_pool(tx); dsl_dir_t *dd, *newparent; + dsl_valid_rename_arg_t dvra; const char *mynewname; int error; - int delta = strlen(ddra->ddra_newname) - strlen(ddra->ddra_oldname); /* target dir should exist */ error = dsl_dir_hold(dp, ddra->ddra_oldname, FTAG, &dd, NULL); @@ -1861,10 +1873,19 @@ dsl_dir_rename_check(void *arg, dmu_tx_t *tx) return (SET_ERROR(EEXIST)); } + ASSERT3U(strnlen(ddra->ddra_newname, ZFS_MAX_DATASET_NAME_LEN), + <, ZFS_MAX_DATASET_NAME_LEN); + ASSERT3U(strnlen(ddra->ddra_oldname, ZFS_MAX_DATASET_NAME_LEN), + <, ZFS_MAX_DATASET_NAME_LEN); + dvra.char_delta = strlen(ddra->ddra_newname) + - strlen(ddra->ddra_oldname); + dvra.nest_delta = get_dataset_depth(ddra->ddra_newname) + - get_dataset_depth(ddra->ddra_oldname); + /* if the name length is growing, validate child name lengths */ - if (delta > 0) { + if (dvra.char_delta > 0 || dvra.nest_delta > 0) { error = dmu_objset_find_dp(dp, dd->dd_object, dsl_valid_rename, - &delta, DS_FIND_CHILDREN | DS_FIND_SNAPSHOTS); + &dvra, DS_FIND_CHILDREN | DS_FIND_SNAPSHOTS); if (error != 0) { dsl_dir_rele(newparent, FTAG); dsl_dir_rele(dd, FTAG); From owner-svn-src-all@freebsd.org Thu Aug 2 21:12:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB9FD1050C12; Thu, 2 Aug 2018 21:12:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 912747CC21; Thu, 2 Aug 2018 21:12:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7232F171BE; Thu, 2 Aug 2018 21:12:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72LCsmG043814; Thu, 2 Aug 2018 21:12:54 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LCsT1043812; Thu, 2 Aug 2018 21:12:54 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022112.w72LCsT1043812@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:12:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337182 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 337182 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:12:55 -0000 Author: mav Date: Thu Aug 2 21:12:52 2018 New Revision: 337182 URL: https://svnweb.freebsd.org/changeset/base/337182 Log: 9330 stack overflow when creating a deeply nested dataset Datasets that are deeply nested (~100 levels) are impractical. We just put a limit of 50 levels to newly created datasets. Existing datasets should work without a problem. illumos/illumos-gate@5ac95da7d61660aa299c287a39277cb0372be959 Reviewed by: John Kennedy Reviewed by: Matt Ahrens Approved by: Garrett D'Amore Author: Serapheim Dimitropoulos Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man1m/zfs.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/common/zfs/zfs_namecheck.c vendor-sys/illumos/dist/common/zfs/zfs_namecheck.h vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_objset.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 21:07:04 2018 (r337181) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 21:12:52 2018 (r337182) @@ -3409,8 +3409,22 @@ zfs_create_ancestors(libzfs_handle_t *hdl, const char { int prefix; char *path_copy; + char errbuf[1024]; int rc = 0; + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, + "cannot create '%s'"), path); + + /* + * Check that we are not passing the nesting limit + * before we start creating any ancestors. + */ + if (dataset_nestcheck(path) != 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "maximum name nesting depth exceeded")); + return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); + } + if (check_parents(hdl, path, NULL, B_TRUE, &prefix) != 0) return (-1); @@ -3446,6 +3460,12 @@ zfs_create(libzfs_handle_t *hdl, const char *path, zfs if (!zfs_validate_name(hdl, path, type, B_TRUE)) return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); + if (dataset_nestcheck(path) != 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "maximum name nesting depth exceeded")); + return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); + } + /* validate parents exist */ if (check_parents(hdl, path, &zoned, B_FALSE, NULL) != 0) return (-1); @@ -4233,6 +4253,7 @@ zfs_rename(zfs_handle_t *zhp, const char *target, bool errbuf)); } } + if (!zfs_validate_name(hdl, target, zhp->zfs_type, B_TRUE)) return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); } else { Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Thu Aug 2 21:07:04 2018 (r337181) +++ vendor/illumos/dist/man/man1m/zfs.1m Thu Aug 2 21:12:52 2018 (r337182) @@ -289,7 +289,8 @@ pool/{filesystem,volume,snapshot} .Pp where the maximum length of a dataset name is .Dv MAXNAMELEN -.Pq 256 bytes . +.Pq 256 bytes +and the maximum amount of nesting allowed in a path is 50 levels deep. .Pp A dataset can be one of the following: .Bl -tag -width "file system" From owner-svn-src-all@freebsd.org Thu Aug 2 21:19:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC7F21050DF5; Thu, 2 Aug 2018 21:19:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6EE807D037; Thu, 2 Aug 2018 21:19:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 50BFA171C4; Thu, 2 Aug 2018 21:19:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72LJbD4044363; Thu, 2 Aug 2018 21:19:37 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LJaNY044357; Thu, 2 Aug 2018 21:19:36 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022119.w72LJaNY044357@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:19:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337183 - in head: cddl/contrib/opensolaris/cmd/zfs cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/zfs cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337183 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:19:38 -0000 Author: mav Date: Thu Aug 2 21:19:35 2018 New Revision: 337183 URL: https://svnweb.freebsd.org/changeset/base/337183 Log: MFV r337182: 9330 stack overflow when creating a deeply nested dataset Datasets that are deeply nested (~100 levels) are impractical. We just put a limit of 50 levels to newly created datasets. Existing datasets should work without a problem. illumos/illumos-gate@5ac95da7d61660aa299c287a39277cb0372be959 Reviewed by: John Kennedy Reviewed by: Matt Ahrens Approved by: Garrett D'Amore Author: Serapheim Dimitropoulos Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.c head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 Thu Aug 2 21:12:52 2018 (r337182) +++ head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 Thu Aug 2 21:19:35 2018 (r337183) @@ -320,7 +320,8 @@ namespace. For example: .Pp where the maximum length of a dataset name is .Dv MAXNAMELEN -(256 bytes). +(256 bytes) +and the maximum amount of nesting allowed in a path is 50 levels deep. .Pp A dataset can be one of the following: .Bl -hang -width 12n Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 21:12:52 2018 (r337182) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Thu Aug 2 21:19:35 2018 (r337183) @@ -3449,8 +3449,22 @@ zfs_create_ancestors(libzfs_handle_t *hdl, const char { int prefix; char *path_copy; + char errbuf[1024]; int rc = 0; + (void) snprintf(errbuf, sizeof (errbuf), dgettext(TEXT_DOMAIN, + "cannot create '%s'"), path); + + /* + * Check that we are not passing the nesting limit + * before we start creating any ancestors. + */ + if (dataset_nestcheck(path) != 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "maximum name nesting depth exceeded")); + return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); + } + if (check_parents(hdl, path, NULL, B_TRUE, &prefix) != 0) return (-1); @@ -3486,6 +3500,12 @@ zfs_create(libzfs_handle_t *hdl, const char *path, zfs if (!zfs_validate_name(hdl, path, type, B_TRUE)) return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); + if (dataset_nestcheck(path) != 0) { + zfs_error_aux(hdl, dgettext(TEXT_DOMAIN, + "maximum name nesting depth exceeded")); + return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); + } + /* validate parents exist */ if (check_parents(hdl, path, &zoned, B_FALSE, NULL) != 0) return (-1); @@ -4286,6 +4306,7 @@ zfs_rename(zfs_handle_t *zhp, const char *source, cons errbuf)); } } + if (!zfs_validate_name(hdl, target, zhp->zfs_type, B_TRUE)) return (zfs_error(hdl, EZFS_INVALIDNAME, errbuf)); } else { Modified: head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.c Thu Aug 2 21:12:52 2018 (r337182) +++ head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.c Thu Aug 2 21:19:35 2018 (r337183) @@ -23,7 +23,7 @@ * Use is subject to license terms. */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2013, 2016 by Delphix. All rights reserved. */ /* @@ -34,8 +34,6 @@ * name is invalid. In the kernel, we only care whether it's valid or not. * Each routine therefore takes a 'namecheck_err_t' which describes exactly why * the name failed to validate. - * - * Each function returns 0 on success, -1 on error. */ #if defined(_KERNEL) @@ -50,6 +48,14 @@ #include "zfs_namecheck.h" #include "zfs_deleg.h" +/* + * Deeply nested datasets can overflow the stack, so we put a limit + * in the amount of nesting a path can have. zfs_max_dataset_nesting + * can be tuned temporarily to fix existing datasets that exceed our + * predefined limit. + */ +int zfs_max_dataset_nesting = 50; + static int valid_char(char c) { @@ -60,10 +66,35 @@ valid_char(char c) } /* + * Looks at a path and returns its level of nesting (depth). + */ +int +get_dataset_depth(const char *path) +{ + const char *loc = path; + int nesting = 0; + + /* + * Keep track of nesting until you hit the end of the + * path or found the snapshot/bookmark seperator. + */ + for (int i = 0; loc[i] != '\0' && + loc[i] != '@' && + loc[i] != '#'; i++) { + if (loc[i] == '/') + nesting++; + } + + return (nesting); +} + +/* * Snapshot names must be made up of alphanumeric characters plus the following * characters: * - * [-_.: ] + * [-_.: ] + * + * Returns 0 on success, -1 on error. */ int zfs_component_namecheck(const char *path, namecheck_err_t *why, char *what) @@ -99,6 +130,8 @@ zfs_component_namecheck(const char *path, namecheck_er * Permissions set name must start with the letter '@' followed by the * same character restrictions as snapshot names, except that the name * cannot exceed 64 characters. + * + * Returns 0 on success, -1 on error. */ int permset_namecheck(const char *path, namecheck_err_t *why, char *what) @@ -121,28 +154,40 @@ permset_namecheck(const char *path, namecheck_err_t *w } /* + * Dataset paths should not be deeper than zfs_max_dataset_nesting + * in terms of nesting. + * + * Returns 0 on success, -1 on error. + */ +int +dataset_nestcheck(const char *path) +{ + return ((get_dataset_depth(path) < zfs_max_dataset_nesting) ? 0 : -1); +} + +/* * Entity names must be of the following form: * - * [component/]*[component][(@|#)component]? + * [component/]*[component][(@|#)component]? * * Where each component is made up of alphanumeric characters plus the following * characters: * - * [-_.:%] + * [-_.:%] * * We allow '%' here as we use that character internally to create unique * names for temporary clones (for online recv). + * + * Returns 0 on success, -1 on error. */ int entity_namecheck(const char *path, namecheck_err_t *why, char *what) { - const char *start, *end; - int found_delim; + const char *end; /* * Make sure the name is not too long. */ - if (strlen(path) >= ZFS_MAX_DATASET_NAME_LEN) { if (why) *why = NAME_ERR_TOOLONG; @@ -162,8 +207,8 @@ entity_namecheck(const char *path, namecheck_err_t *wh return (-1); } - start = path; - found_delim = 0; + const char *start = path; + boolean_t found_delim = B_FALSE; for (;;) { /* Find the end of this component */ end = start; @@ -198,7 +243,7 @@ entity_namecheck(const char *path, namecheck_err_t *wh return (-1); } - found_delim = 1; + found_delim = B_TRUE; } /* Zero-length components are not allowed */ @@ -250,6 +295,8 @@ dataset_namecheck(const char *path, namecheck_err_t *w * mountpoint names must be of the following form: * * /[component][/]*[component][/] + * + * Returns 0 on success, -1 on error. */ int mountpoint_namecheck(const char *path, namecheck_err_t *why) @@ -294,6 +341,8 @@ mountpoint_namecheck(const char *path, namecheck_err_t * dataset names, with the additional restriction that the pool name must begin * with a letter. The pool names 'raidz' and 'mirror' are also reserved names * that cannot be used. + * + * Returns 0 on success, -1 on error. */ int pool_namecheck(const char *pool, namecheck_err_t *why, char *what) Modified: head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.h Thu Aug 2 21:12:52 2018 (r337182) +++ head/sys/cddl/contrib/opensolaris/common/zfs/zfs_namecheck.h Thu Aug 2 21:19:35 2018 (r337183) @@ -23,7 +23,7 @@ * Use is subject to license terms. */ /* - * Copyright (c) 2013 by Delphix. All rights reserved. + * Copyright (c) 2013, 2016 by Delphix. All rights reserved. */ #ifndef _ZFS_NAMECHECK_H @@ -48,9 +48,13 @@ typedef enum { #define ZFS_PERMSET_MAXLEN 64 +extern int zfs_max_dataset_nesting; + +int get_dataset_depth(const char *); int pool_namecheck(const char *, namecheck_err_t *, char *); int entity_namecheck(const char *, namecheck_err_t *, char *); int dataset_namecheck(const char *, namecheck_err_t *, char *); +int dataset_nestcheck(const char *); int mountpoint_namecheck(const char *, namecheck_err_t *); int zfs_component_namecheck(const char *, namecheck_err_t *, char *); int permset_namecheck(const char *, namecheck_err_t *, char *); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c Thu Aug 2 21:12:52 2018 (r337182) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c Thu Aug 2 21:19:35 2018 (r337183) @@ -54,6 +54,7 @@ #include #include #include +#include "zfs_namecheck.h" /* * Needed to close a window in dnode_move() that allows the objset to be freed @@ -911,6 +912,9 @@ dmu_objset_create_check(void *arg, dmu_tx_t *tx) return (SET_ERROR(EINVAL)); if (strlen(doca->doca_name) >= ZFS_MAX_DATASET_NAME_LEN) + return (SET_ERROR(ENAMETOOLONG)); + + if (dataset_nestcheck(doca->doca_name) != 0) return (SET_ERROR(ENAMETOOLONG)); error = dsl_dir_hold(dp, doca->doca_name, FTAG, &pdd, &tail); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c Thu Aug 2 21:12:52 2018 (r337182) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c Thu Aug 2 21:19:35 2018 (r337183) @@ -1819,17 +1819,29 @@ typedef struct dsl_dir_rename_arg { cred_t *ddra_cred; } dsl_dir_rename_arg_t; +typedef struct dsl_valid_rename_arg { + int char_delta; + int nest_delta; +} dsl_valid_rename_arg_t; + /* ARGSUSED */ static int dsl_valid_rename(dsl_pool_t *dp, dsl_dataset_t *ds, void *arg) { - int *deltap = arg; + dsl_valid_rename_arg_t *dvra = arg; char namebuf[ZFS_MAX_DATASET_NAME_LEN]; dsl_dataset_name(ds, namebuf); - if (strlen(namebuf) + *deltap >= ZFS_MAX_DATASET_NAME_LEN) + ASSERT3U(strnlen(namebuf, ZFS_MAX_DATASET_NAME_LEN), + <, ZFS_MAX_DATASET_NAME_LEN); + int namelen = strlen(namebuf) + dvra->char_delta; + int depth = get_dataset_depth(namebuf) + dvra->nest_delta; + + if (namelen >= ZFS_MAX_DATASET_NAME_LEN) return (SET_ERROR(ENAMETOOLONG)); + if (dvra->nest_delta > 0 && depth >= zfs_max_dataset_nesting) + return (SET_ERROR(ENAMETOOLONG)); return (0); } @@ -1839,9 +1851,9 @@ dsl_dir_rename_check(void *arg, dmu_tx_t *tx) dsl_dir_rename_arg_t *ddra = arg; dsl_pool_t *dp = dmu_tx_pool(tx); dsl_dir_t *dd, *newparent; + dsl_valid_rename_arg_t dvra; const char *mynewname; int error; - int delta = strlen(ddra->ddra_newname) - strlen(ddra->ddra_oldname); /* target dir should exist */ error = dsl_dir_hold(dp, ddra->ddra_oldname, FTAG, &dd, NULL); @@ -1870,10 +1882,19 @@ dsl_dir_rename_check(void *arg, dmu_tx_t *tx) return (SET_ERROR(EEXIST)); } + ASSERT3U(strnlen(ddra->ddra_newname, ZFS_MAX_DATASET_NAME_LEN), + <, ZFS_MAX_DATASET_NAME_LEN); + ASSERT3U(strnlen(ddra->ddra_oldname, ZFS_MAX_DATASET_NAME_LEN), + <, ZFS_MAX_DATASET_NAME_LEN); + dvra.char_delta = strlen(ddra->ddra_newname) + - strlen(ddra->ddra_oldname); + dvra.nest_delta = get_dataset_depth(ddra->ddra_newname) + - get_dataset_depth(ddra->ddra_oldname); + /* if the name length is growing, validate child name lengths */ - if (delta > 0) { + if (dvra.char_delta > 0 || dvra.nest_delta > 0) { error = dmu_objset_find_dp(dp, dd->dd_object, dsl_valid_rename, - &delta, DS_FIND_CHILDREN | DS_FIND_SNAPSHOTS); + &dvra, DS_FIND_CHILDREN | DS_FIND_SNAPSHOTS); if (error != 0) { dsl_dir_rele(newparent, FTAG); dsl_dir_rele(dd, FTAG); From owner-svn-src-all@freebsd.org Thu Aug 2 21:24:05 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1A2110512B7; Thu, 2 Aug 2018 21:24:05 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 754DA7D5B9; Thu, 2 Aug 2018 21:24:05 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 567FC17355; Thu, 2 Aug 2018 21:24:05 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72LO5FX049470; Thu, 2 Aug 2018 21:24:05 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LO50A049469; Thu, 2 Aug 2018 21:24:05 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022124.w72LO50A049469@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:24:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337184 - vendor/illumos/dist/lib/libzfs/common X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor/illumos/dist/lib/libzfs/common X-SVN-Commit-Revision: 337184 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:24:06 -0000 Author: mav Date: Thu Aug 2 21:24:04 2018 New Revision: 337184 URL: https://svnweb.freebsd.org/changeset/base/337184 Log: 9457 libzfs_import.c:add_config() has a memory leak A memory leak occurs on lines 209 and 213 because the config is not freed in the error case. The interface to add_config() seems less than ideal - it would be better if it copied any data necessary from the config and the caller freed it. illumos/illumos-gate@ddfe901b12348d31c500fb57f9174e88860a4061 Reviewed by: Matt Ahrens Reviewed by: Serapheim Dimitropoulos Approved by: Robert Mustacchi Author: sara hartse Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_import.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Thu Aug 2 21:19:35 2018 (r337183) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_import.c Thu Aug 2 21:24:04 2018 (r337184) @@ -33,7 +33,7 @@ * ZFS label of each device. If we successfully read the label, then we * organize the configuration information in the following hierarchy: * - * pool guid -> toplevel vdev guid -> label txg + * pool guid -> toplevel vdev guid -> label txg * * Duplicate entries matching this same tuple will be discarded. Once we have * examined every device, we pick the best label txg config for each toplevel @@ -217,7 +217,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons ne->ne_next = pl->names; pl->names = ne; - nvlist_free(config); return (0); } @@ -237,7 +236,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons &top_guid) != 0 || nvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_TXG, &txg) != 0 || txg == 0) { - nvlist_free(config); return (0); } @@ -252,7 +250,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons if (pe == NULL) { if ((pe = zfs_alloc(hdl, sizeof (pool_entry_t))) == NULL) { - nvlist_free(config); return (-1); } pe->pe_guid = pool_guid; @@ -271,7 +268,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons if (ve == NULL) { if ((ve = zfs_alloc(hdl, sizeof (vdev_entry_t))) == NULL) { - nvlist_free(config); return (-1); } ve->ve_guid = top_guid; @@ -291,15 +287,12 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons if (ce == NULL) { if ((ce = zfs_alloc(hdl, sizeof (config_entry_t))) == NULL) { - nvlist_free(config); return (-1); } ce->ce_txg = txg; - ce->ce_config = config; + ce->ce_config = fnvlist_dup(config); ce->ce_next = ve->ve_configs; ve->ve_configs = ce; - } else { - nvlist_free(config); } /* @@ -1235,9 +1228,7 @@ zpool_find_import_impl(libzfs_handle_t *hdl, importarg &this_guid) == 0 && iarg->guid == this_guid; } - if (!matched) { - nvlist_free(config); - } else { + if (matched) { /* * use the non-raw path for the config */ @@ -1247,6 +1238,7 @@ zpool_find_import_impl(libzfs_handle_t *hdl, importarg config) != 0) config_failed = B_TRUE; } + nvlist_free(config); } free(slice->rn_name); free(slice); From owner-svn-src-all@freebsd.org Thu Aug 2 21:25:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65C9E1051346; Thu, 2 Aug 2018 21:25:33 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0CDDA7D71C; Thu, 2 Aug 2018 21:25:33 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DE37C17358; Thu, 2 Aug 2018 21:25:32 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72LPWCj049593; Thu, 2 Aug 2018 21:25:32 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LPWQ2049592; Thu, 2 Aug 2018 21:25:32 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022125.w72LPWQ2049592@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:25:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337185 - head/cddl/contrib/opensolaris/lib/libzfs/common X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/cddl/contrib/opensolaris/lib/libzfs/common X-SVN-Commit-Revision: 337185 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:25:33 -0000 Author: mav Date: Thu Aug 2 21:25:32 2018 New Revision: 337185 URL: https://svnweb.freebsd.org/changeset/base/337185 Log: MFV r337184: 9457 libzfs_import.c:add_config() has a memory leak A memory leak occurs on lines 209 and 213 because the config is not freed in the error case. The interface to add_config() seems less than ideal - it would be better if it copied any data necessary from the config and the caller freed it. illumos/illumos-gate@ddfe901b12348d31c500fb57f9174e88860a4061 Reviewed by: Matt Ahrens Reviewed by: Serapheim Dimitropoulos Approved by: Robert Mustacchi Author: sara hartse Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c Directory Properties: head/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c Thu Aug 2 21:24:04 2018 (r337184) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c Thu Aug 2 21:25:32 2018 (r337185) @@ -33,7 +33,7 @@ * ZFS label of each device. If we successfully read the label, then we * organize the configuration information in the following hierarchy: * - * pool guid -> toplevel vdev guid -> label txg + * pool guid -> toplevel vdev guid -> label txg * * Duplicate entries matching this same tuple will be discarded. Once we have * examined every device, we pick the best label txg config for each toplevel @@ -245,7 +245,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons ne->ne_next = pl->names; pl->names = ne; - nvlist_free(config); return (0); } @@ -265,7 +264,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons &top_guid) != 0 || nvlist_lookup_uint64(config, ZPOOL_CONFIG_POOL_TXG, &txg) != 0 || txg == 0) { - nvlist_free(config); return (0); } @@ -280,7 +278,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons if (pe == NULL) { if ((pe = zfs_alloc(hdl, sizeof (pool_entry_t))) == NULL) { - nvlist_free(config); return (-1); } pe->pe_guid = pool_guid; @@ -299,7 +296,6 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons if (ve == NULL) { if ((ve = zfs_alloc(hdl, sizeof (vdev_entry_t))) == NULL) { - nvlist_free(config); return (-1); } ve->ve_guid = top_guid; @@ -319,15 +315,12 @@ add_config(libzfs_handle_t *hdl, pool_list_t *pl, cons if (ce == NULL) { if ((ce = zfs_alloc(hdl, sizeof (config_entry_t))) == NULL) { - nvlist_free(config); return (-1); } ce->ce_txg = txg; - ce->ce_config = config; + ce->ce_config = fnvlist_dup(config); ce->ce_next = ve->ve_configs; ve->ve_configs = ce; - } else { - nvlist_free(config); } /* @@ -1396,9 +1389,7 @@ skipdir: &this_guid) == 0 && iarg->guid == this_guid; } - if (!matched) { - nvlist_free(config); - } else { + if (matched) { /* * use the non-raw path for the config */ @@ -1408,6 +1399,7 @@ skipdir: config) != 0) config_failed = B_TRUE; } + nvlist_free(config); } free(slice->rn_name); free(slice); From owner-svn-src-all@freebsd.org Thu Aug 2 21:33:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B14CC10515A6; Thu, 2 Aug 2018 21:33:48 +0000 (UTC) (envelope-from sjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 633D47DC17; Thu, 2 Aug 2018 21:33:48 +0000 (UTC) (envelope-from sjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 43FC617504; Thu, 2 Aug 2018 21:33:48 +0000 (UTC) (envelope-from sjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72LXmeX054615; Thu, 2 Aug 2018 21:33:48 GMT (envelope-from sjg@FreeBSD.org) Received: (from sjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LXjOq054601; Thu, 2 Aug 2018 21:33:45 GMT (envelope-from sjg@FreeBSD.org) Message-Id: <201808022133.w72LXjOq054601@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: sjg set sender to sjg@FreeBSD.org using -f From: "Simon J. Gerraty" Date: Thu, 2 Aug 2018 21:33:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337186 - head/share/mk X-SVN-Group: head X-SVN-Commit-Author: sjg X-SVN-Commit-Paths: head/share/mk X-SVN-Commit-Revision: 337186 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:33:49 -0000 Author: sjg Date: Thu Aug 2 21:33:45 2018 New Revision: 337186 URL: https://svnweb.freebsd.org/changeset/base/337186 Log: Update dirdeps.mk et al to latest dirdeps.mk and meta.autodep.mk will now look for Makefile.depend.options to handle optional dependencies, the work is all done by dirdeps-options.mk Also update to latest meta.stage.mk and gendirdeps.mk Reviewed by: bdrewery Differential Revision: https://reviews.freebsd.org/D15701 Added: head/share/mk/dirdeps-options.mk (contents, props changed) Modified: head/share/mk/Makefile head/share/mk/bsd.files.mk head/share/mk/bsd.sys.mk head/share/mk/dirdeps.mk head/share/mk/gendirdeps.mk head/share/mk/local.autodep.mk head/share/mk/local.dirdeps.mk head/share/mk/local.gendirdeps.mk head/share/mk/local.meta.sys.mk head/share/mk/meta.autodep.mk head/share/mk/meta.stage.mk head/share/mk/meta.sys.mk head/share/mk/meta2deps.py Modified: head/share/mk/Makefile ============================================================================== --- head/share/mk/Makefile Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/Makefile Thu Aug 2 21:33:45 2018 (r337186) @@ -57,6 +57,7 @@ FILES= \ bsd.sys.mk \ bsd.test.mk \ dirdeps.mk \ + dirdeps-options.mk \ gendirdeps.mk \ install-new.mk \ meta.autodep.mk \ Modified: head/share/mk/bsd.files.mk ============================================================================== --- head/share/mk/bsd.files.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/bsd.files.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -120,8 +120,10 @@ realinstall: installfiles .if ${MK_STAGING} != "no" .if !empty(STAGE_SETS) buildfiles: stage_files +STAGE_TARGETS+= stage_files .if !empty(STAGE_AS_SETS) buildfiles: stage_as +STAGE_TARGETS+= stage_as .endif .endif .endif Modified: head/share/mk/bsd.sys.mk ============================================================================== --- head/share/mk/bsd.sys.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/bsd.sys.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -340,7 +340,7 @@ STAGE_TARGETS+= $t STAGE_TARGETS+= stage_as .endif -.if !empty(_LIBS) || (${MK_STAGING_PROG} != "no" && !defined(INTERNALPROG)) +.if !empty(STAGE_TARGETS) || (${MK_STAGING_PROG} != "no" && !defined(INTERNALPROG)) .if !empty(LINKS) STAGE_TARGETS+= stage_links Added: head/share/mk/dirdeps-options.mk ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/share/mk/dirdeps-options.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -0,0 +1,73 @@ +# $FreeBSD$ +# $Id: dirdeps-options.mk,v 1.8 2018/05/29 22:31:21 sjg Exp $ +# +# @(#) Copyright (c) 2018, Simon J. Gerraty +# +# This file is provided in the hope that it will +# be of use. There is absolutely NO WARRANTY. +# Permission to copy, redistribute or otherwise +# use this file is hereby granted provided that +# the above copyright notice and this notice are +# left intact. +# +# Please send copies of changes and bug-fixes to: +# sjg@crufty.net +# + +## +# +# This makefile is used to deal with optional DIRDEPS. +# +# It is to be included by Makefile.depend.options in a +# directory which has DIRDEPS affected by optional features. +# Makefile.depend.options should set DIRDEPS_OPTIONS and +# may also set specific DIRDEPS.* for those options. +# +# If a Makefile.depend.options file exists, it will be included by +# dirdeps.mk and meta.autodep.mk +# +# We include local.dirdeps-options.mk which may also define DIRDEPS.* +# for options. +# +# Thus a directory, that is affected by an option FOO would have +# a Makefile.depend.options that sets +# DIRDEPS_OPTIONS= FOO +# It can also set either/both of +# DIRDEPS.FOO.yes +# DIRDEPS.FOO.no +# to whatever applies for that dir, or it can rely on globals +# set in local.dirdeps-options.mk +# Either way, we will .undef DIRDEPS.* when done. + +# This should have been set by Makefile.depend.options +# before including us +DIRDEPS_OPTIONS ?= + +# pickup any DIRDEPS.* we need +.-include + +.if ${.MAKE.LEVEL} == 0 +# :U below avoids potential errors when we := +.for o in ${DIRDEPS_OPTIONS:tu} +DIRDEPS += ${DIRDEPS.$o.${MK_$o:U}:U} +.endfor +DIRDEPS := ${DIRDEPS:O:u} +# avoid cross contamination +.for o in ${DIRDEPS_OPTIONS:tu} +.undef DIRDEPS.$o.yes DIRDEPS.$o.no +.endfor +.else +# whether options are enabled or not, +# we want to filter out the relevant DIRDEPS.* +# we should only be included by meta.autodep.mk +# if dependencies are to be updated +.for o in ${DIRDEPS_OPTIONS:tu} +.for d in ${DIRDEPS.$o.yes} ${DIRDEPS.$o.no} +.if exists(${SRCTOP}/$d) +GENDIRDEPS_FILTER += N$d* +.elif exists(${SRCTOP}/${d:R}) +GENDIRDEPS_FILTER += N${d:R}* +.endif +.endfor +.endfor +.endif Modified: head/share/mk/dirdeps.mk ============================================================================== --- head/share/mk/dirdeps.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/dirdeps.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -1,18 +1,18 @@ # $FreeBSD$ -# $Id: dirdeps.mk,v 1.86 2017/03/01 20:26:51 sjg Exp $ +# $Id: dirdeps.mk,v 1.96 2018/06/20 22:26:39 sjg Exp $ # Copyright (c) 2010-2013, Juniper Networks, Inc. # All rights reserved. -# +# # Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions -# are met: +# modification, are permitted provided that the following conditions +# are met: # 1. Redistributions of source code must retain the above copyright -# notice, this list of conditions and the following disclaimer. +# notice, this list of conditions and the following disclaimer. # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the -# documentation and/or other materials provided with the distribution. -# +# documentation and/or other materials provided with the distribution. +# # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR @@ -23,7 +23,7 @@ # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # Much of the complexity here is for supporting cross-building. # If a tree does not support that, simply using plain Makefile.depend @@ -57,7 +57,7 @@ # .MAKE.DEPENDFILE_PREFIX) to refer to these makefiles to # distinguish them from others. # -# Before each Makefile.depend file is read, we set +# Before each Makefile.depend file is read, we set # DEP_RELDIR to be the RELDIR (path relative to SRCTOP) for # its directory, and DEP_MACHINE etc according to the . # represented by the suffix of the corresponding target. @@ -90,7 +90,7 @@ # # For example: # -# # Always list MACHINE first, +# # Always list MACHINE first, # # other variables might be optional. # TARGET_SPEC_VARS = MACHINE TARGET_OS # .if ${TARGET_SPEC:Uno:M*,*} != "" @@ -102,7 +102,7 @@ # # and deal with MACHINE=${TARGET_SPEC} in the environment. # TARGET_SPEC = # # export but do not track -# .export-env TARGET_SPEC +# .export-env TARGET_SPEC # .export ${TARGET_SPEC_VARS} # .for v in ${TARGET_SPEC_VARS:O:u} # .if empty($v) @@ -138,6 +138,14 @@ # built for. # +.if !target(bootstrap) && (make(bootstrap) || \ + make(bootstrap-this) || \ + make(bootstrap-recurse) || \ + make(bootstrap-empty)) +# disable most of below +.MAKE.LEVEL = 1 +.endif + # touch this at your peril _DIRDEP_USE_LEVEL?= 0 .if ${.MAKE.LEVEL} == ${_DIRDEP_USE_LEVEL} @@ -321,7 +329,7 @@ _DEP_RELDIR := ${DEP_RELDIR} .endif # DIRDEPS_CACHE can be very handy for debugging. -# Also if repeatedly building the same target, +# Also if repeatedly building the same target, # we can avoid the overhead of re-computing the tree dependencies. MK_DIRDEPS_CACHE ?= no BUILD_DIRDEPS_CACHE ?= no @@ -434,7 +442,7 @@ _only_machines := ${_only_machines:O:u} # make sure we have a starting place? DIRDEPS ?= ${RELDIR} -.endif # target +.endif # target .if !defined(NO_DIRDEPS) && !defined(NO_DIRDEPS_BELOW) .if ${MK_DIRDEPS_CACHE} == "yes" @@ -444,7 +452,7 @@ build-dirdeps: M_oneperline = @x@\\${.newline} $$x@ -.if ${BUILD_DIRDEPS_CACHE} == "no" +.if ${BUILD_DIRDEPS_CACHE} == "no" .if !target(dirdeps-cached) # we do this via sub-make BUILD_DIRDEPS = no @@ -462,7 +470,7 @@ dirdeps-cached: ${DIRDEPS_CACHE} .MAKE BUILD_DIRDEPS_MAKEFILE ?= ${MAKEFILE} BUILD_DIRDEPS_TARGETS ?= ${.TARGETS} -# we need the .meta file to ensure we update if +# we need the .meta file to ensure we update if # any of the Makefile.depend* changed. # We do not want to compare the command line though. ${DIRDEPS_CACHE}: .META .NOMETA_CMP @@ -521,6 +529,7 @@ _this_dir := ${SRCTOP}/${DEP_RELDIR} # on rare occasions, there can be a need for extra help _dep_hack := ${_this_dir}/${.MAKE.DEPENDFILE_PREFIX}.inc .-include <${_dep_hack}> +.-include <${_dep_hack:R}.options> .if ${DEP_RELDIR} != ${_DEP_RELDIR} || ${DEP_TARGET_SPEC} != ${TARGET_SPEC} # this should be all @@ -529,10 +538,14 @@ _machines := ${DEP_MACHINE} # this is the machine list we actually use below _machines := ${_only_machines} -.if defined(HOSTPROG) || ${DEP_MACHINE} == "host" +.if defined(HOSTPROG) || ${DEP_MACHINE:Nhost*} == "" # we need to build this guy's dependencies for host as well. +.if ${DEP_MACHINE:Nhost*} == "" +_machines += ${DEP_MACHINE} +.else _machines += host .endif +.endif _machines := ${_machines:O:u} .endif @@ -570,7 +583,7 @@ _build_dirs += ${_machines:N${DEP_TARGET_SPEC}:@m@${_C .if ${_debug_reldir} .info ${DEP_RELDIR}.${DEP_TARGET_SPEC}: DIRDEPS='${DIRDEPS}' -.info ${DEP_RELDIR}.${DEP_TARGET_SPEC}: _machines='${_machines}' +.info ${DEP_RELDIR}.${DEP_TARGET_SPEC}: _machines='${_machines}' .endif .if !empty(DIRDEPS) @@ -578,7 +591,7 @@ _build_dirs += ${_machines:N${DEP_TARGET_SPEC}:@m@${_C DEP_DIRDEPS_FILTER = \ ${DIRDEPS_FILTER.${DEP_TARGET_SPEC}:U} \ ${TARGET_SPEC_VARS:@v@${DIRDEPS_FILTER.${DEP_$v}:U}@} \ - ${DIRDEPS_FILTER:U} + ${DIRDEPS_FILTER:U} .if empty(DEP_DIRDEPS_FILTER) # something harmless DEP_DIRDEPS_FILTER = U @@ -587,7 +600,7 @@ DEP_DIRDEPS_FILTER = U # this is what we start with __depdirs := ${DIRDEPS:${NSkipDir}:${DEP_DIRDEPS_FILTER:ts:}:C,//+,/,g:O:u:@d@${SRCTOP}/$d@} -# some entries may be qualified with . +# some entries may be qualified with . # the :M*/*/*.* just tries to limit the dirs we check to likely ones. # the ${d:E:M*/*} ensures we don't consider junos/usr.sbin/mgd __qual_depdirs := ${__depdirs:M*/*/*.*:@d@${exists($d):?:${"${d:E:M*/*}":?:${exists(${d:R}):?$d:}}}@} @@ -595,7 +608,8 @@ __unqual_depdirs := ${__depdirs:${__qual_depdirs:Uno:$ .if ${DEP_RELDIR} == ${_DEP_RELDIR} # if it was called out - we likely need it. -__hostdpadd := ${DPADD:U.:M${HOST_OBJTOP}/*:S,${HOST_OBJTOP}/,,:H:${NSkipDir}:${DIRDEPS_FILTER:ts:}:S,$,.host,:N.*:@d@${SRCTOP}/$d@} +__hostdpadd := ${DPADD:U.:M${HOST_OBJTOP}/*:S,${HOST_OBJTOP}/,,:H:${NSkipDir}:${DIRDEPS_FILTER:ts:}:S,$,.host,:N.*:@d@${SRCTOP}/$d@} \ + ${DPADD:U.:M${HOST_OBJTOP32:Uno}/*:S,${HOST_OBJTOP32:Uno}/,,:H:${NSkipDir}:${DIRDEPS_FILTER:ts:}:S,$,.host32,:N.*:@d@${SRCTOP}/$d@} __qual_depdirs += ${__hostdpadd} .endif @@ -629,6 +643,11 @@ _build_all_dirs := ${_build_all_dirs:O:u} x!= { echo; echo '\# ${DEP_RELDIR}.${DEP_TARGET_SPEC}'; \ echo 'dirdeps: ${_build_all_dirs:${M_oneperline}}'; echo; } >&3; echo x!= { ${_build_all_dirs:@x@${target($x):?:echo '$x: _DIRDEP_USE';}@} echo; } >&3; echo +.if !empty(DEP_EXPORT_VARS) +# Discouraged, but there are always exceptions. +# Handle it here rather than explain how. +x!= { echo; ${DEP_EXPORT_VARS:@v@echo '$v=${$v}';@} echo '.export ${DEP_EXPORT_VARS}'; echo; } >&3; echo +.endif .else # this makes it all happen dirdeps: ${_build_all_dirs} @@ -639,6 +658,11 @@ ${_build_all_dirs}: _DIRDEP_USE .info ${DEP_RELDIR}.${DEP_TARGET_SPEC}: needs: ${_build_dirs} .endif +.if !empty(DEP_EXPORT_VARS) +.export ${DEP_EXPORT_VARS} +DEP_EXPORT_VARS= +.endif + # this builds the dependency graph .for m in ${_machines} # it would be nice to do :N${.TARGET} @@ -689,7 +713,7 @@ DEP_${TARGET_SPEC_VARS:[$i]} := ${_dtspec:[$i]} .else DEP_MACHINE := ${_DEP_MACHINE} .endif -# Warning: there is an assumption here that MACHINE is always +# Warning: there is an assumption here that MACHINE is always # the first entry in TARGET_SPEC_VARS. # If TARGET_SPEC and MACHINE are insufficient, you have a problem. _m := ${.MAKE.DEPENDFILE_PREFERENCE:T:S;${TARGET_SPEC}$;${d:E};:S;${MACHINE};${d:E:C/,.*//};:@m@${exists(${d:R}/$m):?${d:R}/$m:}@:[1]} @@ -699,7 +723,7 @@ _qm := ${_m:C;(\.depend)$;\1.${d:E};:${M_dep_qual_fixe .if ${_debug_search} .info Looking for ${_qm} .endif -# set this "just in case" +# set this "just in case" # we can skip :tA since we computed the path above DEP_RELDIR := ${_m:H:S,${SRCTOP}/,,} # and reset this @@ -708,6 +732,8 @@ DIRDEPS = .info loading ${_m} for ${d:E} .endif .include <${_m}> +.else +.-include .endif .endif .endif @@ -722,6 +748,10 @@ DIRDEPS = # we are building something DEP_RELDIR := ${RELDIR} _DEP_RELDIR := ${RELDIR} +# Since we are/should be included by .MAKE.DEPENDFILE +# This is a final opportunity to add/hook global rules. +.-include + # pickup local dependencies .if ${MAKE_VERSION} < 20160220 .-include <.depend> @@ -758,7 +788,7 @@ bootstrap-this: .NOTMAIN @echo Bootstrapping ${RELDIR}/${_want:T} from ${_src:T}; \ echo You need to build ${RELDIR} to correctly populate it. .if ${_src:T} != ${.MAKE.DEPENDFILE_PREFIX:T} - (cd ${.CURDIR} && sed ${.MAKE.DEPENDFILE_BOOTSTRAP_SED} ${_src} > ${_want}) + (cd ${.CURDIR} && sed ${.MAKE.DEPENDFILE_BOOTSTRAP_SED} ${_src} > ${_want:T}) .else cp ${.CURDIR}/${_src:T} ${_want} .endif Modified: head/share/mk/gendirdeps.mk ============================================================================== --- head/share/mk/gendirdeps.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/gendirdeps.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -1,5 +1,5 @@ # $FreeBSD$ -# $Id: gendirdeps.mk,v 1.33 2016/10/11 22:37:28 sjg Exp $ +# $Id: gendirdeps.mk,v 1.39 2018/06/08 01:25:31 sjg Exp $ # Copyright (c) 2010-2013, Juniper Networks, Inc. # All rights reserved. @@ -161,17 +161,39 @@ META2DEPS_CMD += -S ${SB_BACKING_SB}/src M2D_OBJROOTS += ${SB_BACKING_SB}/${SB_OBJPREFIX} .endif +GENDIRDEPS_SEDCMDS += \ + -e 's,//*$$,,;s,\.${HOST_TARGET:Uhost}$$,.host,' \ + -e 's,\.${HOST_TARGET32:Uhost32}$$,.host32,' \ + -e 's,\.${MACHINE}$$,,' \ + -e 's:\.${TARGET_SPEC:U${MACHINE}}$$::' + # we are only interested in the dirs # specifically those we read something from. # we canonicalize them to keep things simple # if we are using a split-fs sandbox, it gets a little messier. _objtop := ${_OBJTOP:tA} + +# some people put *.meta in META_XTRAS to make sure we get here +_meta_files := ${META_FILES:N\*.meta:O:u} +# assume a big list +_meta_files_arg= @meta.list +.if empty(_meta_files) && ${META_FILES:M\*.meta} != "" +# XXX this should be considered a bad idea, +# since we cannot ignore stale .meta +x != cd ${_OBJDIR} && find . -name '*.meta' -print -o \( -type d ! -name . -prune \) | sed 's,^./,,' > meta.list; echo +.elif ${_meta_files:[#]} > 500 +.export _meta_files +x != echo; for m in $$_meta_files; do echo $$m; done > meta.list +.else +_meta_files_arg:= ${_meta_files} +.endif + dir_list != cd ${_OBJDIR} && \ ${META2DEPS_CMD} MACHINE=${MACHINE} \ SRCTOP=${SRCTOP} RELDIR=${RELDIR} CURDIR=${_CURDIR} \ ${META2DEPS_ARGS} \ - ${META_FILES:O:u} | ${META2DEPS_FILTER} ${_skip_gendirdeps} \ - sed 's,//*$$,,;s,\.${HOST_TARGET}$$,.host,' + ${_meta_files_arg} | ${META2DEPS_FILTER} ${_skip_gendirdeps} \ + sed ${GENDIRDEPS_SEDCMDS} .if ${dir_list:M*ERROR\:*} != "" .warning ${dir_list:tW:C,.*(ERROR),\1,} @@ -195,7 +217,7 @@ dpadd_dir_list += ${f:H:tA} .endfor .if !empty(ddep_list) ddeps != cat ${ddep_list:O:u} | ${META2DEPS_FILTER} ${_skip_gendirdeps} \ - sed 's,//*$$,,;s,\.${HOST_TARGET}$$,.host,;s,\.${MACHINE}$$,,' + sed ${GENDIRDEPS_SEDCMDS} .if ${DEBUG_GENDIRDEPS:Uno:@x@${RELDIR:M$x}@} != "" .info ${RELDIR}: raw_dir_list='${dir_list}' @@ -256,7 +278,9 @@ DIRDEPS += \ ${dirdep_list:M${RELDIR}/*:@d@${.MAKE.MAKEFILE_PREFERENCE:@m@${exists(${SRCTOP}/$d/$m):?$d:${exists(${SRCTOP}/${d:R}/$m):?$d:}}@}@} \ ${qualdir_list:M${RELDIR}/*:@d@${.MAKE.MAKEFILE_PREFERENCE:@m@${exists(${SRCTOP}/${d:R}/$m):?$d:}@}@} -DIRDEPS := ${DIRDEPS:${GENDIRDEPS_FILTER:UNno:ts:}:C,//+,/,g:O:u} +# what modifiers do we allow in GENDIRDEPS_FILTER +GENDIRDEPS_FILTER_MASK += @CMNS +DIRDEPS := ${DIRDEPS:${GENDIRDEPS_FILTER:UNno:M[${GENDIRDEPS_FILTER_MASK:O:u:ts}]*:ts:}:C,//+,/,g:O:u} .if ${DEBUG_GENDIRDEPS:Uno:@x@${RELDIR:M$x}@} != "" .info ${RELDIR}: M2D_OBJROOTS=${M2D_OBJROOTS} Modified: head/share/mk/local.autodep.mk ============================================================================== --- head/share/mk/local.autodep.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/local.autodep.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -2,7 +2,7 @@ .if ${.MAKE.DEPENDFILE:M*.${MACHINE}} == "" # by default only MACHINE0 does updates -UPDATE_DEPENDFILE_MACHINE?= ${MACHINE0} +UPDATE_DEPENDFILE_MACHINE?= ${MACHINE0:U${MACHINE}} .if ${MACHINE} != ${UPDATE_DEPENDFILE_MACHINE} UPDATE_DEPENDFILE= no .endif Modified: head/share/mk/local.dirdeps.mk ============================================================================== --- head/share/mk/local.dirdeps.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/local.dirdeps.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -6,9 +6,6 @@ .include "${SRCTOP}/share/mk/src.opts.mk" .endif -# DEP_MACHINE is set before we get here, this may not be. -DEP_RELDIR ?= ${RELDIR} - # making universe is special .if defined(UNIVERSE_GUARD) # these should be done by now @@ -39,6 +36,7 @@ DIRDEPS_FILTER.host = \ ${N_host_libs} \ Ninclude* \ Nlib/csu* \ + Nlib/libc \ Nlib/[mn]* \ Ngnu/lib/csu* \ Ngnu/lib/lib[a-r]* \ @@ -93,7 +91,7 @@ DIRDEPS += \ # Add in proper libgcc (gnu or LLVM) if not building libcc and libc is needed. # Add both gcc_s and gcc_eh as dependencies as the decision to build # -static or not is not known here. -.if ${DEP_RELDIR:M*libgcc*} == "" && ${DIRDEPS:Mlib/libc} +.if ${DEP_RELDIR:M*libgcc*} == "" && ${DIRDEPS:U:Mlib/libc} != "" .if ${MK_LLVM_LIBUNWIND} == "yes" DIRDEPS+= \ lib/libgcc_eh \ Modified: head/share/mk/local.gendirdeps.mk ============================================================================== --- head/share/mk/local.gendirdeps.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/local.gendirdeps.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -13,6 +13,8 @@ GENDIRDEPS_FILTER+= \ Ngnu/lib/libgcc \ Nlib/libgcc_eh \ Nlib/libgcc_s \ + Nstand/libsa/* \ + Nstand/libsa32/* \ Ntargets/pseudo/stage* \ Ntools/* Modified: head/share/mk/local.meta.sys.mk ============================================================================== --- head/share/mk/local.meta.sys.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/local.meta.sys.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -7,7 +7,7 @@ # we need this until there is an alternative MK_INSTALL_AS_USER= yes -.if !defined(HOST_TARGET) +.if !defined(HOST_TARGET) || !defined(HOST_MACHINE) # we need HOST_TARGET etc below. .include .export HOST_TARGET @@ -111,6 +111,7 @@ BUILD_AT_LEVEL0= no .error DIRDEPS_BUILD: Please run '${MAKE}' instead of '${MAKE} all'. .endif .endif +.endif # we want to end up with a singe stage tree for all machines .if ${MK_STAGING} == "yes" @@ -119,7 +120,6 @@ STAGE_ROOT?= ${OBJROOT}stage .export STAGE_ROOT .endif .endif -.endif .if ${MK_STAGING} == "yes" .if ${MACHINE} == "host" @@ -149,7 +149,7 @@ STAGE_INCSDIR= ${STAGE_OBJTOP}${INCSDIR:U/include} # the target is usually an absolute path STAGE_SYMLINKS_DIR= ${STAGE_OBJTOP} -LDFLAGS_LAST+= -Wl,-rpath-link,${STAGE_LIBDIR} +#LDFLAGS_LAST+= -Wl,-rpath-link,${STAGE_LIBDIR} .if ${MK_SYSROOT} == "yes" SYSROOT?= ${STAGE_OBJTOP} .else @@ -157,6 +157,8 @@ LDFLAGS_LAST+= -L${STAGE_LIBDIR} .endif .endif # MK_STAGING + +.-include "local.toolchain.mk" # this is sufficient for most of the tree. .MAKE.DEPENDFILE_DEFAULT = ${.MAKE.DEPENDFILE_PREFIX} Modified: head/share/mk/meta.autodep.mk ============================================================================== --- head/share/mk/meta.autodep.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/meta.autodep.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -1,5 +1,5 @@ # $FreeBSD$ -# $Id: meta.autodep.mk,v 1.40 2016/02/22 22:44:58 sjg Exp $ +# $Id: meta.autodep.mk,v 1.50 2018/06/08 01:25:31 sjg Exp $ # # @(#) Copyright (c) 2010, Simon J. Gerraty @@ -7,10 +7,10 @@ # This file is provided in the hope that it will # be of use. There is absolutely NO WARRANTY. # Permission to copy, redistribute or otherwise -# use this file is hereby granted provided that +# use this file is hereby granted provided that # the above copyright notice and this notice are -# left intact. -# +# left intact. +# # Please send copies of changes and bug-fixes to: # sjg@crufty.net # @@ -21,9 +21,12 @@ __${_this}__: .NOTMAIN .-include +PICO?= .pico +NOSSPPICO?= .nossppico + .if defined(SRCS) # it would be nice to be able to query .SUFFIXES -OBJ_EXTENSIONS+= .o .po .lo .pico .nossppico +OBJ_EXTENSIONS+= .o .po .lo ${PICO} ${NOSSPPICO} # explicit dependencies help short-circuit .SUFFIX searches SRCS_DEP_FILTER+= N*.[hly] @@ -40,7 +43,7 @@ ${s:T:R}$e: $s # you are supposed to know what you are doing! UPDATE_DEPENDFILE = yes .elif !empty(.TARGETS) && !make(all) -# do not update the *depend* files +# do not update the *depend* files # unless we are building the entire directory or the default target. # NO means don't update .depend - or Makefile.depend* # no means update .depend but not Makefile.depend* @@ -120,7 +123,7 @@ META_FILE_FILTER ?= N.meta META_FILE_FILTER += Ndirdeps.cache* .if !empty(DPADD) -# if we have any non-libs in DPADD, +# if we have any non-libs in DPADD, # they probably need to be paid attention to .if !empty(DPLIBS) FORCE_DPADD = ${DPADD:${DPLIBS:${M_ListToSkip}}:${DPADD_LAST:${M_ListToSkip}}} @@ -139,8 +142,8 @@ FORCE_DPADD += ${_nonlibs:@x@${DPADD:M*/$x}@} # if we don't have OBJS, then .depend isn't useful .if !target(.depend) && (!empty(OBJS) || ${.ALLTARGETS:M*.o} != "") # some makefiles and/or targets contain -# circular dependencies if you dig too deep -# (as meta mode is apt to do) +# circular dependencies if you dig too deep +# (as meta mode is apt to do) # so we provide a means of suppressing them. # the input to the loop below is target: dependency # with just one dependency per line. @@ -156,13 +159,13 @@ SUPPRESS_DEPEND += \ # we use ${.MAKE.META.CREATED} to trigger an update but # we process using ${.MAKE.META.FILES} # the double $$ defers initial evaluation -# if necessary, we fake .po dependencies, just so the result +# if necessary, we fake .po dependencies, just so the result # in Makefile.depend* is stable # The current objdir may be referred to in various ways OBJDIR_REFS += ${.OBJDIR} ${.OBJDIR:tA} ${_OBJDIR} ${RELOBJTOP}/${RELDIR} _depend = .depend # it would be nice to be able to get .SUFFIXES as ${.SUFFIXES} -# we actually only care about the .SUFFIXES of files that might be +# we actually only care about the .SUFFIXES of files that might be # generated by tools like yacc. DEPEND_SUFFIXES += .c .h .cpp .hpp .cxx .hxx .cc .hh .depend: .NOMETA $${.MAKE.META.CREATED} ${_this} @@ -179,7 +182,7 @@ DEPEND_SUFFIXES += .c .h .cpp .hpp .cxx .hxx .cc .hh @case "${.MAKE.META.FILES:T:M*.po.*}" in \ *.po.*) mv $@.${.MAKE.PID} $@;; \ *) { cat $@.${.MAKE.PID}; \ - sed 's,\.nossppico:,.o:,;s,\.pico:,.o:,;s,\.o:,.po:,' $@.${.MAKE.PID}; } | sort -u > $@; \ + sed 's,\${NOSSPPICO}:,.o:,;s,\${PICO}:,.o:,;s,\.o:,.po:,' $@.${.MAKE.PID}; } | sort -u > $@; \ rm -f $@.${.MAKE.PID};; \ esac .else @@ -244,7 +247,7 @@ META_FILES = *.meta .elif ${OPTIMIZE_OBJECT_META_FILES:Uno:tl} == "no" META_FILES = ${.MAKE.META.FILES:T:N.depend*:O:u} .else -# if we have 1000's of .o.meta, .pico.meta etc we need only look at one set +# if we have 1000's of .o.meta, ${PICO}.meta etc we need only look at one set # it is left as an exercise for the reader to work out what this does META_FILES = ${.MAKE.META.FILES:T:N.depend*:N*o.meta:O:u} \ ${.MAKE.META.FILES:T:M*.${.MAKE.META.FILES:M*o.meta:R:E:O:u:[1]}.meta:O:u} @@ -254,20 +257,27 @@ META_FILES = ${.MAKE.META.FILES:T:N.depend*:N*o.meta:O .info ${_DEPENDFILE:S,${SRCTOP}/,,}: ${_depend} ${.PARSEDIR}/gendirdeps.mk ${META2DEPS} xtras=${META_XTRAS} .endif -.if ${.MAKE.LEVEL} > 0 && !empty(GENDIRDEPS_FILTER) +.if ${.MAKE.LEVEL} > 0 +.if ${UPDATE_DEPENDFILE} == "yes" +.-include <${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}.options> +.endif +.if !empty(GENDIRDEPS_FILTER) .export GENDIRDEPS_FILTER .endif +# export to avoid blowing command line limit +META_FILES := ${META_XTRAS:U:O:u} ${META_FILES:U:T:O:u:${META_FILE_FILTER:ts:}} +.export META_FILES +.endif # we might have .../ in MAKESYSPATH _makesyspath:= ${_PARSEDIR} ${_DEPENDFILE}: ${_depend} ${.PARSEDIR}/gendirdeps.mk ${META2DEPS} $${.MAKE.META.CREATED} @echo Checking $@: ${.OODATE:T:[1..8]} - @(cd . && \ + @(cd . && ${GENDIRDEPS_ENV} \ SKIP_GENDIRDEPS='${SKIP_GENDIRDEPS:O:u}' \ DPADD='${FORCE_DPADD:O:u}' ${_gendirdeps_mutex} \ MAKESYSPATH=${_makesyspath} \ - ${.MAKE} -f gendirdeps.mk RELDIR=${RELDIR} _DEPENDFILE=${_DEPENDFILE} \ - META_FILES='${META_XTRAS:O:u} ${META_FILES:T:O:u:${META_FILE_FILTER:ts:}}') + ${.MAKE} -f gendirdeps.mk RELDIR=${RELDIR} _DEPENDFILE=${_DEPENDFILE}) @test -s $@ && touch $@; : .endif Modified: head/share/mk/meta.stage.mk ============================================================================== --- head/share/mk/meta.stage.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/meta.stage.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -1,5 +1,5 @@ # $FreeBSD$ -# $Id: meta.stage.mk,v 1.48 2017/03/01 22:48:07 sjg Exp $ +# $Id: meta.stage.mk,v 1.55 2017/10/27 01:17:09 sjg Exp $ # # @(#) Copyright (c) 2011-2017, Simon J. Gerraty # @@ -14,14 +14,16 @@ # sjg@crufty.net # +.ifndef NO_STAGING + .if !target(__${.PARSEFILE}__) # the guard target is defined later .if ${.MAKE.DEPENDFILE_PREFERENCE:U${.MAKE.DEPENDFILE}:M*.${MACHINE}} != "" # this is generally safer anyway -_dirdep = ${RELDIR}.${MACHINE} +_dirdep ?= ${RELDIR}.${MACHINE} .else -_dirdep = ${RELDIR} +_dirdep ?= ${RELDIR} .endif CLEANFILES+= .dirdep @@ -135,11 +137,31 @@ _STAGE_AS_BASENAME_USE: .USE .dirdep ${.TARGET: @${STAGE_AS_SCRIPT}; StageAs ${.TARGET:H:${STAGE_DIR_FILTER}} ${.TARGET:T} ${STAGE_AS_${.TARGET:T}:U${.TARGET:T}} +.endif # first time + + +.if !empty(STAGE_INCSDIR) +.if !empty(STAGE_INCS) +stage_incs: ${STAGE_INCS} +.endif +.if target(stage_incs) || !empty(.ALLTARGETS:Mstage_includes) +STAGE_TARGETS += stage_incs +STAGE_INCS ?= ${.ALLSRC:N.dirdep:Nstage_*} stage_includes: stage_incs stage_incs: .dirdep @${STAGE_FILE_SCRIPT}; StageFiles ${STAGE_INCSDIR:${STAGE_DIR_FILTER}} ${STAGE_INCS} @touch $@ +.endif +.endif + +.if !empty(STAGE_LIBDIR) +.if !empty(STAGE_LIBS) +stage_libs: ${STAGE_LIBS} +.endif +.if target(stage_libs) +STAGE_TARGETS += stage_libs +STAGE_LIBS ?= ${.ALLSRC:N.dirdep:Nstage_*} stage_libs: .dirdep @${STAGE_FILE_SCRIPT}; StageFiles ${STAGE_LIBDIR:${STAGE_DIR_FILTER}} ${STAGE_LIBS} .if !defined(NO_SHLIB_LINKS) @@ -151,18 +173,7 @@ stage_libs: .dirdep .endif .endif @touch $@ - -.endif # first time - - -.if !empty(STAGE_INCSDIR) -STAGE_TARGETS += stage_incs -STAGE_INCS ?= ${.ALLSRC:N.dirdep:Nstage_*} .endif - -.if !empty(STAGE_LIBDIR) -STAGE_TARGETS += stage_libs -STAGE_LIBS ?= ${.ALLSRC:N.dirdep:Nstage_*} .endif .if !empty(STAGE_DIR) @@ -173,8 +184,6 @@ STAGE_SYMLINKS_DIR._default = ${STAGE_SYMLINKS_DIR:U${ STAGE_FILES._default = ${STAGE_FILES} STAGE_LINKS._default = ${STAGE_LINKS} STAGE_SYMLINKS._default = ${STAGE_SYMLINKS} -STAGE_FILES ?= ${.ALLSRC:N.dirdep:Nstage_*} -STAGE_SYMLINKS ?= ${.ALLSRC:T:N.dirdep:Nstage_*} .endif .if !empty(STAGE_SETS) @@ -182,24 +191,31 @@ CLEANFILES += ${STAGE_SETS:@s@stage*$s@} # some makefiles need to populate multiple directories .for s in ${STAGE_SETS:O:u} -STAGE_FILES.$s ?= ${.ALLSRC:N.dirdep:Nstage_*} -STAGE_SYMLINKS.$s ?= ${.ALLSRC:N.dirdep:Nstage_*} -STAGE_LINKS_DIR.$s ?= ${STAGE_OBJTOP} -STAGE_SYMLINKS_DIR.$s ?= ${STAGE_OBJTOP} - +.if !empty(STAGE_FILES.$s) +stage_files.$s: ${STAGE_FILES.$s} +.endif +.if target(stage_files.$s) || target(stage_files${s:S,^,.,:N._default}) STAGE_TARGETS += stage_files +STAGE_FILES.$s ?= ${.ALLSRC:N.dirdep:Nstage_*} .if !target(.stage_files.$s) .stage_files.$s: .if $s != "_default" stage_files: stage_files.$s stage_files.$s: .dirdep .else +STAGE_FILES ?= ${.ALLSRC:N.dirdep:Nstage_*} stage_files: .dirdep .endif @${STAGE_FILE_SCRIPT}; StageFiles ${FLAGS.$@} ${STAGE_FILES_DIR.$s:U${STAGE_DIR.$s}:${STAGE_DIR_FILTER}} ${STAGE_FILES.$s} @touch $@ .endif +.endif +.if !empty(STAGE_LINKS.$s) +stage_links.$s: +.endif +.if target(stage_links.$s) || target(stage_links${s:S,^,.,:N._default}) +STAGE_LINKS_DIR.$s ?= ${STAGE_OBJTOP} STAGE_TARGETS += stage_links .if !target(.stage_links.$s) .stage_links.$s: @@ -212,7 +228,13 @@ stage_links: .dirdep @${STAGE_LINKS_SCRIPT}; StageLinks ${STAGE_LINKS_DIR.$s:U${STAGE_DIR.$s}:${STAGE_DIR_FILTER}} ${STAGE_LINKS.$s} @touch $@ .endif +.endif +.if !empty(STAGE_SYMLINKS.$s) +stage_symlinks.$s: +.endif +.if target(stage_symlinks.$s) || target(stage_symlinks${s:S,^,.,:N._default}) +STAGE_SYMLINKS_DIR.$s ?= ${STAGE_OBJTOP} STAGE_TARGETS += stage_symlinks .if !target(.stage_symlinks.$s) .stage_symlinks.$s: @@ -225,6 +247,7 @@ stage_symlinks: .dirdep @${STAGE_LINKS_SCRIPT}; StageLinks -s ${STAGE_SYMLINKS_DIR.$s:U${STAGE_DIR.$s}:${STAGE_DIR_FILTER}} ${STAGE_SYMLINKS.$s} @touch $@ .endif +.endif .endfor .endif @@ -232,8 +255,6 @@ stage_symlinks: .dirdep .if !empty(STAGE_AS_SETS) CLEANFILES += ${STAGE_AS_SETS:@s@stage*$s@} -STAGE_TARGETS += stage_as stage_as_and_symlink - # sometimes things need to be renamed as they are staged # each ${file} will be staged as ${STAGE_AS_${file:T}} # one could achieve the same with SYMLINKS @@ -241,9 +262,12 @@ STAGE_TARGETS += stage_as stage_as_and_symlink # it is the same as using stage_as and stage_symlinks but ensures # both operations happen together .for s in ${STAGE_AS_SETS:O:u} +.if !empty(STAGE_AS.$s) +stage_as.$s: ${STAGE_AS.$s} +.endif +.if target(stage_as.$s) +STAGE_TARGETS += stage_as STAGE_AS.$s ?= ${.ALLSRC:N.dirdep:Nstage_*} -STAGE_AS_AND_SYMLINK.$s ?= ${.ALLSRC:N.dirdep:Nstage_*} - .if !target(.stage_as.$s) .stage_as.$s: stage_as: stage_as.$s @@ -251,7 +275,14 @@ stage_as.$s: .dirdep @${STAGE_AS_SCRIPT}; StageAs ${FLAGS.$@} ${STAGE_FILES_DIR.$s:U${STAGE_DIR.$s}:${STAGE_DIR_FILTER}} ${STAGE_AS.$s:@f@$f ${STAGE_AS_${f:tA}:U${STAGE_AS_${f:T}:U${f:T}}}@} @touch $@ .endif +.endif +.if !empty(STAGE_AS_AND_SYMLINK.$s) +stage_as_and_symlink.$s: ${STAGE_AS_AND_SYMLINK.$s} +.endif +.if target(stage_as_and_symlink.$s) +STAGE_TARGETS += stage_as_and_symlink +STAGE_AS_AND_SYMLINK.$s ?= ${.ALLSRC:N.dirdep:Nstage_*} .if !target(.stage_as_and_symlink.$s) .stage_as_and_symlink.$s: stage_as_and_symlink: stage_as_and_symlink.$s @@ -260,6 +291,7 @@ stage_as_and_symlink.$s: .dirdep @${STAGE_LINKS_SCRIPT}; StageLinks -s ${STAGE_FILES_DIR.$s:U${STAGE_DIR.$s}:${STAGE_DIR_FILTER}} ${STAGE_AS_AND_SYMLINK.$s:@f@${STAGE_AS_${f:tA}:U${STAGE_AS_${f:T}:U${f:T}}} $f@} @touch $@ .endif +.endif .endfor .endif @@ -303,6 +335,9 @@ beforeinstall: .dirdep .NOPATH: ${STAGE_FILES} .if !empty(STAGE_TARGETS) +# for backwards compat make sure they exist +${STAGE_TARGETS}: + .NOPATH: ${CLEANFILES} MK_STALE_STAGED?= no @@ -322,6 +357,7 @@ stale_staged: staging .NOMETA echo "Removing stale staged files..."; \ sed 's,.*,& &.dirdep,' ${.TARGET}.stale | xargs rm -f; } +.endif .endif .endif .endif Modified: head/share/mk/meta.sys.mk ============================================================================== --- head/share/mk/meta.sys.mk Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/meta.sys.mk Thu Aug 2 21:33:45 2018 (r337186) @@ -1,5 +1,5 @@ # $FreeBSD$ -# $Id: meta.sys.mk,v 1.19 2014/08/02 23:16:02 sjg Exp $ +# $Id: meta.sys.mk,v 1.32 2017/06/11 03:24:04 sjg Exp $ # # @(#) Copyright (c) 2010, Simon J. Gerraty @@ -21,7 +21,7 @@ .if ${MAKE_VERSION:U0} > 20100901 .if !target(.ERROR) -.-include "local.meta.sys.mk" +.-include # absolute path to what we are reading. _PARSEDIR = ${.PARSEDIR:tA} @@ -56,17 +56,6 @@ META_MODE += silent=yes .endif .endif -# make defaults .MAKE.DEPENDFILE to .depend -# that won't work for us. -.if ${.MAKE.DEPENDFILE} == ".depend" -.undef .MAKE.DEPENDFILE -.endif - -# if you don't cross build for multiple MACHINEs concurrently, then -# .MAKE.DEPENDFILE = Makefile.depend -# probably makes sense - you can set that in local.sys.mk -.MAKE.DEPENDFILE ?= Makefile.depend.${MACHINE} - # we use the pseudo machine "host" for the build host. # this should be taken care of before we get here .if ${OBJTOP:Ua} == ${HOST_OBJTOP:Ub} @@ -118,10 +107,21 @@ _metaError: .NOMETA .NOTMAIN .if ${.MAKE.MODE:Uno:Mmeta*} != "" MKDEP_MK = meta.autodep.mk +.if ${.MAKE.MAKEFILES:M*sys.dependfile.mk} == "" +# this does all the smarts of setting .MAKE.DEPENDFILE +.-include +# check if we got anything sane +.if ${.MAKE.DEPENDFILE} == ".depend" +.undef .MAKE.DEPENDFILE +.endif +.MAKE.DEPENDFILE ?= Makefile.depend +.endif + # we can afford to use cookies to prevent some targets # re-running needlessly META_COOKIE_TOUCH?= touch ${COOKIE.${.TARGET}:U${.OBJDIR}/${.TARGET:T}} META_NOPHONY= +META_NOECHO= : # some targets involve old pre-built targets # ignore mtime of shell @@ -131,11 +131,16 @@ META_NOPHONY= ${SHELL} \ ${SYS_MK_DIR} -# if we think we are updating dependencies, -# then filemon had better be present -.if ${UPDATE_DEPENDFILE:Uyes:tl} != "no" && !exists(/dev/filemon) +.if ${UPDATE_DEPENDFILE:Uyes:tl} != "no" +.if ${.MAKEFLAGS:Uno:M-k} != "" +# make this more obvious +.warning Setting UPDATE_DEPENDFILE=NO due to -k +UPDATE_DEPENDFILE= NO +.export UPDATE_DEPENDFILE +.elif !exists(/dev/filemon) .error ${.newline}ERROR: The filemon module (/dev/filemon) is not loaded. .endif +.endif .if ${.MAKE.LEVEL} == 0 # make sure dirdeps target exists and do it first @@ -160,5 +165,6 @@ BUILD_AT_LEVEL0 ?= no META_COOKIE_TOUCH= # some targets need to be .PHONY in non-meta mode META_NOPHONY= .PHONY +META_NOECHO= echo .endif .endif Modified: head/share/mk/meta2deps.py ============================================================================== --- head/share/mk/meta2deps.py Thu Aug 2 21:25:32 2018 (r337185) +++ head/share/mk/meta2deps.py Thu Aug 2 21:33:45 2018 (r337186) @@ -38,7 +38,7 @@ We only pay attention to a subset of the information i """ RCSid: $FreeBSD$ - $Id: meta2deps.py,v 1.24 2017/02/08 22:17:10 sjg Exp $ + $Id: meta2deps.py,v 1.27 2017/05/24 00:04:04 sjg Exp $ Copyright (c) 2011-2013, Juniper Networks, Inc. All rights reserved. @@ -91,6 +91,12 @@ def resolve(path, cwd, last_dir=None, debug=0, debug_o for d in [last_dir, cwd]: if not d: continue + if path == '..': + dw = d.split('/') + p = '/'.join(dw[:-1]) + if not p: + p = '/' + return p p = '/'.join([d,path]) if debug > 2: print("looking for:", p, end=' ', file=debug_out) @@ -116,8 +122,11 @@ def cleanpath(path): if not d or d == '.': continue if d == '..': - p.pop() - continue + try: + p.pop() + continue + except: + break p.append(d) return r + '/'.join(p) From owner-svn-src-all@freebsd.org Thu Aug 2 21:52:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27EB1105207D; Thu, 2 Aug 2018 21:52:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC31D7E82A; Thu, 2 Aug 2018 21:52:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AAD3317844; Thu, 2 Aug 2018 21:52:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Lqlx4064747; Thu, 2 Aug 2018 21:52:47 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Lqlvh064746; Thu, 2 Aug 2018 21:52:47 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022152.w72Lqlvh064746@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:52:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337187 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337187 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:52:48 -0000 Author: mav Date: Thu Aug 2 21:52:47 2018 New Revision: 337187 URL: https://svnweb.freebsd.org/changeset/base/337187 Log: 9479 fix wrong format specifier for vdev_id illumos/illumos-gate@c7a7b2fa084facd68c3d1e860dfabe20c713877b Reviewed by: Matt Ahrens Approved by: Robert Mustacchi Author: Andriy Gapon Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Thu Aug 2 21:33:45 2018 (r337186) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Thu Aug 2 21:52:47 2018 (r337187) @@ -168,7 +168,7 @@ vdev_dbgmsg_print_tree(vdev_t *vd, int indent) } zfs_dbgmsg("%*svdev %u: %s%s, guid: %llu, path: %s, %s", indent, - "", vd->vdev_id, vd->vdev_ops->vdev_op_type, + "", (int)vd->vdev_id, vd->vdev_ops->vdev_op_type, vd->vdev_islog ? " (log)" : "", (u_longlong_t)vd->vdev_guid, vd->vdev_path ? vd->vdev_path : "N/A", state); From owner-svn-src-all@freebsd.org Thu Aug 2 21:53:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C9A11052125; Thu, 2 Aug 2018 21:53:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 234277E989; Thu, 2 Aug 2018 21:53:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DF19C17847; Thu, 2 Aug 2018 21:53:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Lrl9n064846; Thu, 2 Aug 2018 21:53:47 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LrlJS064845; Thu, 2 Aug 2018 21:53:47 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022153.w72LrlJS064845@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:53:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337188 - head/sys/cddl/contrib/opensolaris X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris X-SVN-Commit-Revision: 337188 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:53:48 -0000 Author: mav Date: Thu Aug 2 21:53:47 2018 New Revision: 337188 URL: https://svnweb.freebsd.org/changeset/base/337188 Log: Record r337187 as merged, since it originated from us. Modified: Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) From owner-svn-src-all@freebsd.org Thu Aug 2 21:54:35 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA4DB10521B1; Thu, 2 Aug 2018 21:54:35 +0000 (UTC) (envelope-from oshogbo@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E1447F01A; Thu, 2 Aug 2018 21:54:35 +0000 (UTC) (envelope-from oshogbo@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3F3C717860; Thu, 2 Aug 2018 21:54:35 +0000 (UTC) (envelope-from oshogbo@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72LsZoR064921; Thu, 2 Aug 2018 21:54:35 GMT (envelope-from oshogbo@FreeBSD.org) Received: (from oshogbo@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LsZSO064920; Thu, 2 Aug 2018 21:54:35 GMT (envelope-from oshogbo@FreeBSD.org) Message-Id: <201808022154.w72LsZSO064920@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: oshogbo set sender to oshogbo@FreeBSD.org using -f From: Mariusz Zaborski Date: Thu, 2 Aug 2018 21:54:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337189 - head/usr.sbin/bhyve X-SVN-Group: head X-SVN-Commit-Author: oshogbo X-SVN-Commit-Paths: head/usr.sbin/bhyve X-SVN-Commit-Revision: 337189 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:54:35 -0000 Author: oshogbo Date: Thu Aug 2 21:54:34 2018 New Revision: 337189 URL: https://svnweb.freebsd.org/changeset/base/337189 Log: bhyve: set title before entering capability mode PR: 230082 Submitted by: Yuichiro NAITO Modified: head/usr.sbin/bhyve/bhyverun.c Modified: head/usr.sbin/bhyve/bhyverun.c ============================================================================== --- head/usr.sbin/bhyve/bhyverun.c Thu Aug 2 21:53:47 2018 (r337188) +++ head/usr.sbin/bhyve/bhyverun.c Thu Aug 2 21:54:34 2018 (r337189) @@ -1112,6 +1112,11 @@ main(int argc, char *argv[]) if (lpc_bootrom()) fwctl_init(); + /* + * Change the proc title to include the VM name. + */ + setproctitle("%s", vmname); + #ifndef WITHOUT_CAPSICUM caph_cache_catpages(); @@ -1122,11 +1127,6 @@ main(int argc, char *argv[]) errx(EX_OSERR, "cap_enter() failed"); #endif - /* - * Change the proc title to include the VM name. - */ - setproctitle("%s", vmname); - /* * Add CPU 0 */ From owner-svn-src-all@freebsd.org Thu Aug 2 21:58:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77F3F1052349; Thu, 2 Aug 2018 21:58:01 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E0037F310; Thu, 2 Aug 2018 21:58:01 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0AD4E17872; Thu, 2 Aug 2018 21:58:01 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Lw0Og065328; Thu, 2 Aug 2018 21:58:00 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Lw0G0065323; Thu, 2 Aug 2018 21:58:00 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022158.w72Lw0G0065323@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:58:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337190 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337190 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:58:01 -0000 Author: mav Date: Thu Aug 2 21:57:59 2018 New Revision: 337190 URL: https://svnweb.freebsd.org/changeset/base/337190 Log: 9486 reduce memory used by device removal on fragmented pools In the most fragmented real-world cases, this reduces memory used by the mapping from ~1GB to ~50MB of RAM per 1TB of storage removed. Less fragmented cases will typically also see around 50-100MB of RAM per 1TB of storage. illumos/illumos-gate@cfd63e1b1bcf7ba4bf72f55ddbd87ce008d2986d Reviewed by: George Wilson Reviewed by: Serapheim Dimitropoulos Reviewed by: Brian Behlendorf Reviewed by: Tim Chase Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/range_tree.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c Thu Aug 2 21:54:34 2018 (r337189) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/range_tree.c Thu Aug 2 21:57:59 2018 (r337190) @@ -298,7 +298,6 @@ range_tree_remove(void *arg, uint64_t start, uint64_t static range_seg_t * range_tree_find_impl(range_tree_t *rt, uint64_t start, uint64_t size) { - avl_index_t where; range_seg_t rsearch; uint64_t end = start + size; @@ -306,7 +305,7 @@ range_tree_find_impl(range_tree_t *rt, uint64_t start, rsearch.rs_start = start; rsearch.rs_end = end; - return (avl_find(&rt->rt_root, &rsearch, &where)); + return (avl_find(&rt->rt_root, &rsearch, NULL)); } static range_seg_t * @@ -406,4 +405,24 @@ range_tree_is_empty(range_tree_t *rt) { ASSERT(rt != NULL); return (range_tree_space(rt) == 0); +} + +uint64_t +range_tree_min(range_tree_t *rt) +{ + range_seg_t *rs = avl_first(&rt->rt_root); + return (rs != NULL ? rs->rs_start : 0); +} + +uint64_t +range_tree_max(range_tree_t *rt) +{ + range_seg_t *rs = avl_last(&rt->rt_root); + return (rs != NULL ? rs->rs_end : 0); +} + +uint64_t +range_tree_span(range_tree_t *rt) +{ + return (range_tree_max(rt) - range_tree_min(rt)); } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/range_tree.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/range_tree.h Thu Aug 2 21:54:34 2018 (r337189) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/range_tree.h Thu Aug 2 21:57:59 2018 (r337190) @@ -86,6 +86,9 @@ boolean_t range_tree_is_empty(range_tree_t *rt); void range_tree_verify(range_tree_t *rt, uint64_t start, uint64_t size); void range_tree_swap(range_tree_t **rtsrc, range_tree_t **rtdst); void range_tree_stat_verify(range_tree_t *rt); +uint64_t range_tree_min(range_tree_t *rt); +uint64_t range_tree_max(range_tree_t *rt); +uint64_t range_tree_span(range_tree_t *rt); void range_tree_add(void *arg, uint64_t start, uint64_t size); void range_tree_remove(void *arg, uint64_t start, uint64_t size); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h Thu Aug 2 21:54:34 2018 (r337189) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/vdev_removal.h Thu Aug 2 21:57:59 2018 (r337190) @@ -86,6 +86,9 @@ extern void spa_vdev_remove_suspend(spa_t *); extern int spa_vdev_remove_cancel(spa_t *); extern void spa_vdev_removal_destroy(spa_vdev_removal_t *svr); +extern int vdev_removal_max_span; +extern int zfs_remove_max_segment; + #ifdef __cplusplus } #endif Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Thu Aug 2 21:54:34 2018 (r337189) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_label.c Thu Aug 2 21:57:59 2018 (r337190) @@ -33,15 +33,15 @@ * 1. Uniquely identify this device as part of a ZFS pool and confirm its * identity within the pool. * - * 2. Verify that all the devices given in a configuration are present + * 2. Verify that all the devices given in a configuration are present * within the pool. * - * 3. Determine the uberblock for the pool. + * 3. Determine the uberblock for the pool. * - * 4. In case of an import operation, determine the configuration of the + * 4. In case of an import operation, determine the configuration of the * toplevel vdev of which it is a part. * - * 5. If an import operation cannot find all the devices in the pool, + * 5. If an import operation cannot find all the devices in the pool, * provide enough information to the administrator to determine which * devices are missing. * @@ -77,9 +77,9 @@ * In order to identify which labels are valid, the labels are written in the * following manner: * - * 1. For each vdev, update 'L1' to the new label - * 2. Update the uberblock - * 3. For each vdev, update 'L2' to the new label + * 1. For each vdev, update 'L1' to the new label + * 2. Update the uberblock + * 3. For each vdev, update 'L2' to the new label * * Given arbitrary failure, we can determine the correct label to use based on * the transaction group. If we fail after updating L1 but before updating the @@ -117,19 +117,19 @@ * * The nvlist describing the pool and vdev contains the following elements: * - * version ZFS on-disk version - * name Pool name - * state Pool state - * txg Transaction group in which this label was written - * pool_guid Unique identifier for this pool - * vdev_tree An nvlist describing vdev tree. + * version ZFS on-disk version + * name Pool name + * state Pool state + * txg Transaction group in which this label was written + * pool_guid Unique identifier for this pool + * vdev_tree An nvlist describing vdev tree. * features_for_read * An nvlist of the features necessary for reading the MOS. * * Each leaf device label also contains the following: * - * top_guid Unique ID for top-level vdev in which this is contained - * guid Unique ID for the leaf vdev + * top_guid Unique ID for top-level vdev in which this is contained + * guid Unique ID for the leaf vdev * * The 'vs' configuration follows the format described in 'spa_config.c'. */ @@ -390,22 +390,33 @@ vdev_config_generate(spa_t *spa, vdev_t *vd, boolean_t * histograms. */ uint64_t seg_count = 0; + uint64_t to_alloc = vd->vdev_stat.vs_alloc; /* * There are the same number of allocated segments * as free segments, so we will have at least one - * entry per free segment. + * entry per free segment. However, small free + * segments (smaller than vdev_removal_max_span) + * will be combined with adjacent allocated segments + * as a single mapping. */ for (int i = 0; i < RANGE_TREE_HISTOGRAM_SIZE; i++) { - seg_count += vd->vdev_mg->mg_histogram[i]; + if (1ULL << (i + 1) < vdev_removal_max_span) { + to_alloc += + vd->vdev_mg->mg_histogram[i] << + i + 1; + } else { + seg_count += + vd->vdev_mg->mg_histogram[i]; + } } /* - * The maximum length of a mapping is SPA_MAXBLOCKSIZE, - * so we need at least one entry per SPA_MAXBLOCKSIZE - * of allocated data. + * The maximum length of a mapping is + * zfs_remove_max_segment, so we need at least one entry + * per zfs_remove_max_segment of allocated data. */ - seg_count += vd->vdev_stat.vs_alloc / SPA_MAXBLOCKSIZE; + seg_count += to_alloc / zfs_remove_max_segment; fnvlist_add_uint64(nv, ZPOOL_CONFIG_INDIRECT_SIZE, seg_count * Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Thu Aug 2 21:54:34 2018 (r337189) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev_removal.c Thu Aug 2 21:57:59 2018 (r337190) @@ -106,6 +106,24 @@ int zfs_remove_max_copy_bytes = 64 * 1024 * 1024; int zfs_remove_max_segment = 1024 * 1024; /* + * Allow a remap segment to span free chunks of at most this size. The main + * impact of a larger span is that we will read and write larger, more + * contiguous chunks, with more "unnecessary" data -- trading off bandwidth + * for iops. The value here was chosen to align with + * zfs_vdev_read_gap_limit, which is a similar concept when doing regular + * reads (but there's no reason it has to be the same). + * + * Additionally, a higher span will have the following relatively minor + * effects: + * - the mapping will be smaller, since one entry can cover more allocated + * segments + * - more of the fragmentation in the removing device will be preserved + * - we'll do larger allocations, which may fail and fall back on smaller + * allocations + */ +int vdev_removal_max_span = 32 * 1024; + +/* * This is used by the test suite so that it can ensure that certain * actions happen while in the middle of a removal. */ @@ -726,13 +744,52 @@ vdev_mapping_sync(void *arg, dmu_tx_t *tx) spa_sync_removing_state(spa, tx); } +typedef struct vdev_copy_segment_arg { + spa_t *vcsa_spa; + dva_t *vcsa_dest_dva; + uint64_t vcsa_txg; + range_tree_t *vcsa_obsolete_segs; +} vdev_copy_segment_arg_t; + +static void +unalloc_seg(void *arg, uint64_t start, uint64_t size) +{ + vdev_copy_segment_arg_t *vcsa = arg; + spa_t *spa = vcsa->vcsa_spa; + blkptr_t bp = { 0 }; + + BP_SET_BIRTH(&bp, TXG_INITIAL, TXG_INITIAL); + BP_SET_LSIZE(&bp, size); + BP_SET_PSIZE(&bp, size); + BP_SET_COMPRESS(&bp, ZIO_COMPRESS_OFF); + BP_SET_CHECKSUM(&bp, ZIO_CHECKSUM_OFF); + BP_SET_TYPE(&bp, DMU_OT_NONE); + BP_SET_LEVEL(&bp, 0); + BP_SET_DEDUP(&bp, 0); + BP_SET_BYTEORDER(&bp, ZFS_HOST_BYTEORDER); + + DVA_SET_VDEV(&bp.blk_dva[0], DVA_GET_VDEV(vcsa->vcsa_dest_dva)); + DVA_SET_OFFSET(&bp.blk_dva[0], + DVA_GET_OFFSET(vcsa->vcsa_dest_dva) + start); + DVA_SET_ASIZE(&bp.blk_dva[0], size); + + zio_free(spa, vcsa->vcsa_txg, &bp); +} + /* * All reads and writes associated with a call to spa_vdev_copy_segment() * are done. */ static void -spa_vdev_copy_nullzio_done(zio_t *zio) +spa_vdev_copy_segment_done(zio_t *zio) { + vdev_copy_segment_arg_t *vcsa = zio->io_private; + + range_tree_vacate(vcsa->vcsa_obsolete_segs, + unalloc_seg, vcsa); + range_tree_destroy(vcsa->vcsa_obsolete_segs); + kmem_free(vcsa, sizeof (*vcsa)); + spa_config_exit(zio->io_spa, SCL_STATE, zio->io_spa); } @@ -849,7 +906,8 @@ spa_vdev_copy_one_child(vdev_copy_arg_t *vca, zio_t *n * read from the old location and write to the new location. */ static int -spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint64_t size, uint64_t txg, +spa_vdev_copy_segment(vdev_t *vd, range_tree_t *segs, + uint64_t maxalloc, uint64_t txg, vdev_copy_arg_t *vca, zio_alloc_list_t *zal) { metaslab_group_t *mg = vd->vdev_mg; @@ -857,9 +915,40 @@ spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint spa_vdev_removal_t *svr = spa->spa_vdev_removal; vdev_indirect_mapping_entry_t *entry; dva_t dst = { 0 }; + uint64_t start = range_tree_min(segs); - ASSERT3U(size, <=, SPA_MAXBLOCKSIZE); + ASSERT3U(maxalloc, <=, SPA_MAXBLOCKSIZE); + uint64_t size = range_tree_span(segs); + if (range_tree_span(segs) > maxalloc) { + /* + * We can't allocate all the segments. Prefer to end + * the allocation at the end of a segment, thus avoiding + * additional split blocks. + */ + range_seg_t search; + avl_index_t where; + search.rs_start = start + maxalloc; + search.rs_end = search.rs_start; + range_seg_t *rs = avl_find(&segs->rt_root, &search, &where); + if (rs == NULL) { + rs = avl_nearest(&segs->rt_root, where, AVL_BEFORE); + } else { + rs = AVL_PREV(&segs->rt_root, rs); + } + if (rs != NULL) { + size = rs->rs_end - start; + } else { + /* + * There are no segments that end before maxalloc. + * I.e. the first segment is larger than maxalloc, + * so we must split it. + */ + size = maxalloc; + } + } + ASSERT3U(size, <=, maxalloc); + /* * We use allocator 0 for this I/O because we don't expect device remap * to be the steady state of the system, so parallelizing is not as @@ -873,6 +962,31 @@ spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint return (error); /* + * Determine the ranges that are not actually needed. Offsets are + * relative to the start of the range to be copied (i.e. relative to the + * local variable "start"). + */ + range_tree_t *obsolete_segs = range_tree_create(NULL, NULL); + + range_seg_t *rs = avl_first(&segs->rt_root); + ASSERT3U(rs->rs_start, ==, start); + uint64_t prev_seg_end = rs->rs_end; + while ((rs = AVL_NEXT(&segs->rt_root, rs)) != NULL) { + if (rs->rs_start >= start + size) { + break; + } else { + range_tree_add(obsolete_segs, + prev_seg_end - start, + rs->rs_start - prev_seg_end); + } + prev_seg_end = rs->rs_end; + } + /* We don't end in the middle of an obsolete range */ + ASSERT3U(start + size, <=, prev_seg_end); + + range_tree_clear(segs, start, size); + + /* * We can't have any padding of the allocated size, otherwise we will * misunderstand what's allocated, and the size of the mapping. * The caller ensures this will be true by passing in a size that is @@ -883,13 +997,22 @@ spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint entry = kmem_zalloc(sizeof (vdev_indirect_mapping_entry_t), KM_SLEEP); DVA_MAPPING_SET_SRC_OFFSET(&entry->vime_mapping, start); entry->vime_mapping.vimep_dst = dst; + if (spa_feature_is_enabled(spa, SPA_FEATURE_OBSOLETE_COUNTS)) { + entry->vime_obsolete_count = range_tree_space(obsolete_segs); + } + vdev_copy_segment_arg_t *vcsa = kmem_zalloc(sizeof (*vcsa), KM_SLEEP); + vcsa->vcsa_dest_dva = &entry->vime_mapping.vimep_dst; + vcsa->vcsa_obsolete_segs = obsolete_segs; + vcsa->vcsa_spa = spa; + vcsa->vcsa_txg = txg; + /* * See comment before spa_vdev_copy_one_child(). */ spa_config_enter(spa, SCL_STATE, spa, RW_READER); zio_t *nzio = zio_null(spa->spa_txg_zio[txg & TXG_MASK], spa, NULL, - spa_vdev_copy_nullzio_done, NULL, 0); + spa_vdev_copy_segment_done, vcsa, 0); vdev_t *dest_vd = vdev_lookup_top(spa, DVA_GET_VDEV(&dst)); if (dest_vd->vdev_ops == &vdev_mirror_ops) { for (int i = 0; i < dest_vd->vdev_children; i++) { @@ -1092,39 +1215,78 @@ spa_vdev_copy_impl(vdev_t *vd, spa_vdev_removal_t *svr mutex_enter(&svr->svr_lock); - range_seg_t *rs = avl_first(&svr->svr_allocd_segs->rt_root); - if (rs == NULL) { + /* + * Determine how big of a chunk to copy. We can allocate up + * to max_alloc bytes, and we can span up to vdev_removal_max_span + * bytes of unallocated space at a time. "segs" will track the + * allocated segments that we are copying. We may also be copying + * free segments (of up to vdev_removal_max_span bytes). + */ + range_tree_t *segs = range_tree_create(NULL, NULL); + for (;;) { + range_seg_t *rs = avl_first(&svr->svr_allocd_segs->rt_root); + if (rs == NULL) + break; + + uint64_t seg_length; + + if (range_tree_is_empty(segs)) { + /* need to truncate the first seg based on max_alloc */ + seg_length = + MIN(rs->rs_end - rs->rs_start, *max_alloc); + } else { + if (rs->rs_start - range_tree_max(segs) > + vdev_removal_max_span) { + /* + * Including this segment would cause us to + * copy a larger unneeded chunk than is allowed. + */ + break; + } else if (rs->rs_end - range_tree_min(segs) > + *max_alloc) { + /* + * This additional segment would extend past + * max_alloc. Rather than splitting this + * segment, leave it for the next mapping. + */ + break; + } else { + seg_length = rs->rs_end - rs->rs_start; + } + } + + range_tree_add(segs, rs->rs_start, seg_length); + range_tree_remove(svr->svr_allocd_segs, + rs->rs_start, seg_length); + } + + if (range_tree_is_empty(segs)) { mutex_exit(&svr->svr_lock); + range_tree_destroy(segs); return; } - uint64_t offset = rs->rs_start; - uint64_t length = MIN(rs->rs_end - rs->rs_start, *max_alloc); - range_tree_remove(svr->svr_allocd_segs, offset, length); - if (svr->svr_max_offset_to_sync[txg & TXG_MASK] == 0) { dsl_sync_task_nowait(dmu_tx_pool(tx), vdev_mapping_sync, svr, 0, ZFS_SPACE_CHECK_NONE, tx); } - svr->svr_max_offset_to_sync[txg & TXG_MASK] = offset + length; + svr->svr_max_offset_to_sync[txg & TXG_MASK] = range_tree_max(segs); /* * Note: this is the amount of *allocated* space * that we are taking care of each txg. */ - svr->svr_bytes_done[txg & TXG_MASK] += length; + svr->svr_bytes_done[txg & TXG_MASK] += range_tree_space(segs); mutex_exit(&svr->svr_lock); zio_alloc_list_t zal; metaslab_trace_init(&zal); - uint64_t thismax = *max_alloc; - while (length > 0) { - uint64_t mylen = MIN(length, thismax); - + uint64_t thismax = SPA_MAXBLOCKSIZE; + while (!range_tree_is_empty(segs)) { int error = spa_vdev_copy_segment(vd, - offset, mylen, txg, vca, &zal); + segs, thismax, txg, vca, &zal); if (error == ENOSPC) { /* @@ -1138,18 +1300,17 @@ spa_vdev_copy_impl(vdev_t *vd, spa_vdev_removal_t *svr */ ASSERT3U(spa->spa_max_ashift, >=, SPA_MINBLOCKSHIFT); ASSERT3U(spa->spa_max_ashift, ==, spa->spa_min_ashift); - thismax = P2ROUNDUP(mylen / 2, + uint64_t attempted = + MIN(range_tree_span(segs), thismax); + thismax = P2ROUNDUP(attempted / 2, 1 << spa->spa_max_ashift); - ASSERT3U(thismax, <, mylen); /* * The minimum-size allocation can not fail. */ - ASSERT3U(mylen, >, 1 << spa->spa_max_ashift); - *max_alloc = mylen - (1 << spa->spa_max_ashift); + ASSERT3U(attempted, >, 1 << spa->spa_max_ashift); + *max_alloc = attempted - (1 << spa->spa_max_ashift); } else { ASSERT0(error); - length -= mylen; - offset += mylen; /* * We've performed an allocation, so reset the @@ -1160,6 +1321,7 @@ spa_vdev_copy_impl(vdev_t *vd, spa_vdev_removal_t *svr } } metaslab_trace_fini(&zal); + range_tree_destroy(segs); } /* From owner-svn-src-all@freebsd.org Thu Aug 2 21:59:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1E1610523E9; Thu, 2 Aug 2018 21:59:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F5BF7F469; Thu, 2 Aug 2018 21:59:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3D1A817873; Thu, 2 Aug 2018 21:59:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Lxmig065455; Thu, 2 Aug 2018 21:59:48 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72LxlaQ065450; Thu, 2 Aug 2018 21:59:47 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022159.w72LxlaQ065450@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 21:59:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337191 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337191 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 21:59:49 -0000 Author: mav Date: Thu Aug 2 21:59:46 2018 New Revision: 337191 URL: https://svnweb.freebsd.org/changeset/base/337191 Log: MFV r337190: 9486 reduce memory used by device removal on fragmented pools In the most fragmented real-world cases, this reduces memory used by the mapping from ~1GB to ~50MB of RAM per 1TB of storage removed. Less fragmented cases will typically also see around 50-100MB of RAM per 1TB of storage. illumos/illumos-gate@cfd63e1b1bcf7ba4bf72f55ddbd87ce008d2986d Reviewed by: George Wilson Reviewed by: Serapheim Dimitropoulos Reviewed by: Brian Behlendorf Reviewed by: Tim Chase Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/range_tree.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c Thu Aug 2 21:57:59 2018 (r337190) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/range_tree.c Thu Aug 2 21:59:46 2018 (r337191) @@ -491,7 +491,6 @@ range_tree_resize_segment(range_tree_t *rt, range_seg_ static range_seg_t * range_tree_find_impl(range_tree_t *rt, uint64_t start, uint64_t size) { - avl_index_t where; range_seg_t rsearch; uint64_t end = start + size; @@ -499,7 +498,7 @@ range_tree_find_impl(range_tree_t *rt, uint64_t start, rsearch.rs_start = start; rsearch.rs_end = end; - return (avl_find(&rt->rt_root, &rsearch, &where)); + return (avl_find(&rt->rt_root, &rsearch, NULL)); } range_seg_t * @@ -650,4 +649,24 @@ range_tree_is_empty(range_tree_t *rt) { ASSERT(rt != NULL); return (range_tree_space(rt) == 0); +} + +uint64_t +range_tree_min(range_tree_t *rt) +{ + range_seg_t *rs = avl_first(&rt->rt_root); + return (rs != NULL ? rs->rs_start : 0); +} + +uint64_t +range_tree_max(range_tree_t *rt) +{ + range_seg_t *rs = avl_last(&rt->rt_root); + return (rs != NULL ? rs->rs_end : 0); +} + +uint64_t +range_tree_span(range_tree_t *rt) +{ + return (range_tree_max(rt) - range_tree_min(rt)); } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/range_tree.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/range_tree.h Thu Aug 2 21:57:59 2018 (r337190) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/range_tree.h Thu Aug 2 21:59:46 2018 (r337191) @@ -95,6 +95,9 @@ boolean_t range_tree_is_empty(range_tree_t *rt); void range_tree_verify(range_tree_t *rt, uint64_t start, uint64_t size); void range_tree_swap(range_tree_t **rtsrc, range_tree_t **rtdst); void range_tree_stat_verify(range_tree_t *rt); +uint64_t range_tree_min(range_tree_t *rt); +uint64_t range_tree_max(range_tree_t *rt); +uint64_t range_tree_span(range_tree_t *rt); void range_tree_add(void *arg, uint64_t start, uint64_t size); void range_tree_remove(void *arg, uint64_t start, uint64_t size); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h Thu Aug 2 21:57:59 2018 (r337190) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_removal.h Thu Aug 2 21:59:46 2018 (r337191) @@ -86,6 +86,9 @@ extern void spa_vdev_remove_suspend(spa_t *); extern int spa_vdev_remove_cancel(spa_t *); extern void spa_vdev_removal_destroy(spa_vdev_removal_t *svr); +extern int vdev_removal_max_span; +extern int zfs_remove_max_segment; + #ifdef __cplusplus } #endif Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Thu Aug 2 21:57:59 2018 (r337190) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c Thu Aug 2 21:59:46 2018 (r337191) @@ -33,15 +33,15 @@ * 1. Uniquely identify this device as part of a ZFS pool and confirm its * identity within the pool. * - * 2. Verify that all the devices given in a configuration are present + * 2. Verify that all the devices given in a configuration are present * within the pool. * - * 3. Determine the uberblock for the pool. + * 3. Determine the uberblock for the pool. * - * 4. In case of an import operation, determine the configuration of the + * 4. In case of an import operation, determine the configuration of the * toplevel vdev of which it is a part. * - * 5. If an import operation cannot find all the devices in the pool, + * 5. If an import operation cannot find all the devices in the pool, * provide enough information to the administrator to determine which * devices are missing. * @@ -77,9 +77,9 @@ * In order to identify which labels are valid, the labels are written in the * following manner: * - * 1. For each vdev, update 'L1' to the new label - * 2. Update the uberblock - * 3. For each vdev, update 'L2' to the new label + * 1. For each vdev, update 'L1' to the new label + * 2. Update the uberblock + * 3. For each vdev, update 'L2' to the new label * * Given arbitrary failure, we can determine the correct label to use based on * the transaction group. If we fail after updating L1 but before updating the @@ -117,19 +117,19 @@ * * The nvlist describing the pool and vdev contains the following elements: * - * version ZFS on-disk version - * name Pool name - * state Pool state - * txg Transaction group in which this label was written - * pool_guid Unique identifier for this pool - * vdev_tree An nvlist describing vdev tree. + * version ZFS on-disk version + * name Pool name + * state Pool state + * txg Transaction group in which this label was written + * pool_guid Unique identifier for this pool + * vdev_tree An nvlist describing vdev tree. * features_for_read * An nvlist of the features necessary for reading the MOS. * * Each leaf device label also contains the following: * - * top_guid Unique ID for top-level vdev in which this is contained - * guid Unique ID for the leaf vdev + * top_guid Unique ID for top-level vdev in which this is contained + * guid Unique ID for the leaf vdev * * The 'vs' configuration follows the format described in 'spa_config.c'. */ @@ -396,22 +396,33 @@ vdev_config_generate(spa_t *spa, vdev_t *vd, boolean_t * histograms. */ uint64_t seg_count = 0; + uint64_t to_alloc = vd->vdev_stat.vs_alloc; /* * There are the same number of allocated segments * as free segments, so we will have at least one - * entry per free segment. + * entry per free segment. However, small free + * segments (smaller than vdev_removal_max_span) + * will be combined with adjacent allocated segments + * as a single mapping. */ for (int i = 0; i < RANGE_TREE_HISTOGRAM_SIZE; i++) { - seg_count += vd->vdev_mg->mg_histogram[i]; + if (1ULL << (i + 1) < vdev_removal_max_span) { + to_alloc += + vd->vdev_mg->mg_histogram[i] << + i + 1; + } else { + seg_count += + vd->vdev_mg->mg_histogram[i]; + } } /* - * The maximum length of a mapping is SPA_MAXBLOCKSIZE, - * so we need at least one entry per SPA_MAXBLOCKSIZE - * of allocated data. + * The maximum length of a mapping is + * zfs_remove_max_segment, so we need at least one entry + * per zfs_remove_max_segment of allocated data. */ - seg_count += vd->vdev_stat.vs_alloc / SPA_MAXBLOCKSIZE; + seg_count += to_alloc / zfs_remove_max_segment; fnvlist_add_uint64(nv, ZPOOL_CONFIG_INDIRECT_SIZE, seg_count * Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c Thu Aug 2 21:57:59 2018 (r337190) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_removal.c Thu Aug 2 21:59:46 2018 (r337191) @@ -106,6 +106,24 @@ int zfs_remove_max_copy_bytes = 64 * 1024 * 1024; int zfs_remove_max_segment = 1024 * 1024; /* + * Allow a remap segment to span free chunks of at most this size. The main + * impact of a larger span is that we will read and write larger, more + * contiguous chunks, with more "unnecessary" data -- trading off bandwidth + * for iops. The value here was chosen to align with + * zfs_vdev_read_gap_limit, which is a similar concept when doing regular + * reads (but there's no reason it has to be the same). + * + * Additionally, a higher span will have the following relatively minor + * effects: + * - the mapping will be smaller, since one entry can cover more allocated + * segments + * - more of the fragmentation in the removing device will be preserved + * - we'll do larger allocations, which may fail and fall back on smaller + * allocations + */ +int vdev_removal_max_span = 32 * 1024; + +/* * This is used by the test suite so that it can ensure that certain * actions happen while in the middle of a removal. */ @@ -726,13 +744,52 @@ vdev_mapping_sync(void *arg, dmu_tx_t *tx) spa_sync_removing_state(spa, tx); } +typedef struct vdev_copy_segment_arg { + spa_t *vcsa_spa; + dva_t *vcsa_dest_dva; + uint64_t vcsa_txg; + range_tree_t *vcsa_obsolete_segs; +} vdev_copy_segment_arg_t; + +static void +unalloc_seg(void *arg, uint64_t start, uint64_t size) +{ + vdev_copy_segment_arg_t *vcsa = arg; + spa_t *spa = vcsa->vcsa_spa; + blkptr_t bp = { 0 }; + + BP_SET_BIRTH(&bp, TXG_INITIAL, TXG_INITIAL); + BP_SET_LSIZE(&bp, size); + BP_SET_PSIZE(&bp, size); + BP_SET_COMPRESS(&bp, ZIO_COMPRESS_OFF); + BP_SET_CHECKSUM(&bp, ZIO_CHECKSUM_OFF); + BP_SET_TYPE(&bp, DMU_OT_NONE); + BP_SET_LEVEL(&bp, 0); + BP_SET_DEDUP(&bp, 0); + BP_SET_BYTEORDER(&bp, ZFS_HOST_BYTEORDER); + + DVA_SET_VDEV(&bp.blk_dva[0], DVA_GET_VDEV(vcsa->vcsa_dest_dva)); + DVA_SET_OFFSET(&bp.blk_dva[0], + DVA_GET_OFFSET(vcsa->vcsa_dest_dva) + start); + DVA_SET_ASIZE(&bp.blk_dva[0], size); + + zio_free(spa, vcsa->vcsa_txg, &bp); +} + /* * All reads and writes associated with a call to spa_vdev_copy_segment() * are done. */ static void -spa_vdev_copy_nullzio_done(zio_t *zio) +spa_vdev_copy_segment_done(zio_t *zio) { + vdev_copy_segment_arg_t *vcsa = zio->io_private; + + range_tree_vacate(vcsa->vcsa_obsolete_segs, + unalloc_seg, vcsa); + range_tree_destroy(vcsa->vcsa_obsolete_segs); + kmem_free(vcsa, sizeof (*vcsa)); + spa_config_exit(zio->io_spa, SCL_STATE, zio->io_spa); } @@ -849,7 +906,8 @@ spa_vdev_copy_one_child(vdev_copy_arg_t *vca, zio_t *n * read from the old location and write to the new location. */ static int -spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint64_t size, uint64_t txg, +spa_vdev_copy_segment(vdev_t *vd, range_tree_t *segs, + uint64_t maxalloc, uint64_t txg, vdev_copy_arg_t *vca, zio_alloc_list_t *zal) { metaslab_group_t *mg = vd->vdev_mg; @@ -857,9 +915,40 @@ spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint spa_vdev_removal_t *svr = spa->spa_vdev_removal; vdev_indirect_mapping_entry_t *entry; dva_t dst = { 0 }; + uint64_t start = range_tree_min(segs); - ASSERT3U(size, <=, SPA_MAXBLOCKSIZE); + ASSERT3U(maxalloc, <=, SPA_MAXBLOCKSIZE); + uint64_t size = range_tree_span(segs); + if (range_tree_span(segs) > maxalloc) { + /* + * We can't allocate all the segments. Prefer to end + * the allocation at the end of a segment, thus avoiding + * additional split blocks. + */ + range_seg_t search; + avl_index_t where; + search.rs_start = start + maxalloc; + search.rs_end = search.rs_start; + range_seg_t *rs = avl_find(&segs->rt_root, &search, &where); + if (rs == NULL) { + rs = avl_nearest(&segs->rt_root, where, AVL_BEFORE); + } else { + rs = AVL_PREV(&segs->rt_root, rs); + } + if (rs != NULL) { + size = rs->rs_end - start; + } else { + /* + * There are no segments that end before maxalloc. + * I.e. the first segment is larger than maxalloc, + * so we must split it. + */ + size = maxalloc; + } + } + ASSERT3U(size, <=, maxalloc); + /* * We use allocator 0 for this I/O because we don't expect device remap * to be the steady state of the system, so parallelizing is not as @@ -873,6 +962,31 @@ spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint return (error); /* + * Determine the ranges that are not actually needed. Offsets are + * relative to the start of the range to be copied (i.e. relative to the + * local variable "start"). + */ + range_tree_t *obsolete_segs = range_tree_create(NULL, NULL); + + range_seg_t *rs = avl_first(&segs->rt_root); + ASSERT3U(rs->rs_start, ==, start); + uint64_t prev_seg_end = rs->rs_end; + while ((rs = AVL_NEXT(&segs->rt_root, rs)) != NULL) { + if (rs->rs_start >= start + size) { + break; + } else { + range_tree_add(obsolete_segs, + prev_seg_end - start, + rs->rs_start - prev_seg_end); + } + prev_seg_end = rs->rs_end; + } + /* We don't end in the middle of an obsolete range */ + ASSERT3U(start + size, <=, prev_seg_end); + + range_tree_clear(segs, start, size); + + /* * We can't have any padding of the allocated size, otherwise we will * misunderstand what's allocated, and the size of the mapping. * The caller ensures this will be true by passing in a size that is @@ -883,13 +997,22 @@ spa_vdev_copy_segment(vdev_t *vd, uint64_t start, uint entry = kmem_zalloc(sizeof (vdev_indirect_mapping_entry_t), KM_SLEEP); DVA_MAPPING_SET_SRC_OFFSET(&entry->vime_mapping, start); entry->vime_mapping.vimep_dst = dst; + if (spa_feature_is_enabled(spa, SPA_FEATURE_OBSOLETE_COUNTS)) { + entry->vime_obsolete_count = range_tree_space(obsolete_segs); + } + vdev_copy_segment_arg_t *vcsa = kmem_zalloc(sizeof (*vcsa), KM_SLEEP); + vcsa->vcsa_dest_dva = &entry->vime_mapping.vimep_dst; + vcsa->vcsa_obsolete_segs = obsolete_segs; + vcsa->vcsa_spa = spa; + vcsa->vcsa_txg = txg; + /* * See comment before spa_vdev_copy_one_child(). */ spa_config_enter(spa, SCL_STATE, spa, RW_READER); zio_t *nzio = zio_null(spa->spa_txg_zio[txg & TXG_MASK], spa, NULL, - spa_vdev_copy_nullzio_done, NULL, 0); + spa_vdev_copy_segment_done, vcsa, 0); vdev_t *dest_vd = vdev_lookup_top(spa, DVA_GET_VDEV(&dst)); if (dest_vd->vdev_ops == &vdev_mirror_ops) { for (int i = 0; i < dest_vd->vdev_children; i++) { @@ -1092,39 +1215,78 @@ spa_vdev_copy_impl(vdev_t *vd, spa_vdev_removal_t *svr mutex_enter(&svr->svr_lock); - range_seg_t *rs = avl_first(&svr->svr_allocd_segs->rt_root); - if (rs == NULL) { + /* + * Determine how big of a chunk to copy. We can allocate up + * to max_alloc bytes, and we can span up to vdev_removal_max_span + * bytes of unallocated space at a time. "segs" will track the + * allocated segments that we are copying. We may also be copying + * free segments (of up to vdev_removal_max_span bytes). + */ + range_tree_t *segs = range_tree_create(NULL, NULL); + for (;;) { + range_seg_t *rs = avl_first(&svr->svr_allocd_segs->rt_root); + if (rs == NULL) + break; + + uint64_t seg_length; + + if (range_tree_is_empty(segs)) { + /* need to truncate the first seg based on max_alloc */ + seg_length = + MIN(rs->rs_end - rs->rs_start, *max_alloc); + } else { + if (rs->rs_start - range_tree_max(segs) > + vdev_removal_max_span) { + /* + * Including this segment would cause us to + * copy a larger unneeded chunk than is allowed. + */ + break; + } else if (rs->rs_end - range_tree_min(segs) > + *max_alloc) { + /* + * This additional segment would extend past + * max_alloc. Rather than splitting this + * segment, leave it for the next mapping. + */ + break; + } else { + seg_length = rs->rs_end - rs->rs_start; + } + } + + range_tree_add(segs, rs->rs_start, seg_length); + range_tree_remove(svr->svr_allocd_segs, + rs->rs_start, seg_length); + } + + if (range_tree_is_empty(segs)) { mutex_exit(&svr->svr_lock); + range_tree_destroy(segs); return; } - uint64_t offset = rs->rs_start; - uint64_t length = MIN(rs->rs_end - rs->rs_start, *max_alloc); - range_tree_remove(svr->svr_allocd_segs, offset, length); - if (svr->svr_max_offset_to_sync[txg & TXG_MASK] == 0) { dsl_sync_task_nowait(dmu_tx_pool(tx), vdev_mapping_sync, svr, 0, ZFS_SPACE_CHECK_NONE, tx); } - svr->svr_max_offset_to_sync[txg & TXG_MASK] = offset + length; + svr->svr_max_offset_to_sync[txg & TXG_MASK] = range_tree_max(segs); /* * Note: this is the amount of *allocated* space * that we are taking care of each txg. */ - svr->svr_bytes_done[txg & TXG_MASK] += length; + svr->svr_bytes_done[txg & TXG_MASK] += range_tree_space(segs); mutex_exit(&svr->svr_lock); zio_alloc_list_t zal; metaslab_trace_init(&zal); - uint64_t thismax = *max_alloc; - while (length > 0) { - uint64_t mylen = MIN(length, thismax); - + uint64_t thismax = SPA_MAXBLOCKSIZE; + while (!range_tree_is_empty(segs)) { int error = spa_vdev_copy_segment(vd, - offset, mylen, txg, vca, &zal); + segs, thismax, txg, vca, &zal); if (error == ENOSPC) { /* @@ -1138,18 +1300,17 @@ spa_vdev_copy_impl(vdev_t *vd, spa_vdev_removal_t *svr */ ASSERT3U(spa->spa_max_ashift, >=, SPA_MINBLOCKSHIFT); ASSERT3U(spa->spa_max_ashift, ==, spa->spa_min_ashift); - thismax = P2ROUNDUP(mylen / 2, + uint64_t attempted = + MIN(range_tree_span(segs), thismax); + thismax = P2ROUNDUP(attempted / 2, 1 << spa->spa_max_ashift); - ASSERT3U(thismax, <, mylen); /* * The minimum-size allocation can not fail. */ - ASSERT3U(mylen, >, 1 << spa->spa_max_ashift); - *max_alloc = mylen - (1 << spa->spa_max_ashift); + ASSERT3U(attempted, >, 1 << spa->spa_max_ashift); + *max_alloc = attempted - (1 << spa->spa_max_ashift); } else { ASSERT0(error); - length -= mylen; - offset += mylen; /* * We've performed an allocation, so reset the @@ -1160,6 +1321,7 @@ spa_vdev_copy_impl(vdev_t *vd, spa_vdev_removal_t *svr } } metaslab_trace_fini(&zal); + range_tree_destroy(segs); } /* From owner-svn-src-all@freebsd.org Thu Aug 2 22:52:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA7BE105337C; Thu, 2 Aug 2018 22:52:06 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 80E5D80C9C; Thu, 2 Aug 2018 22:52:06 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 491A518291; Thu, 2 Aug 2018 22:52:06 +0000 (UTC) (envelope-from np@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Mq6lM095700; Thu, 2 Aug 2018 22:52:06 GMT (envelope-from np@FreeBSD.org) Received: (from np@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Mq582095697; Thu, 2 Aug 2018 22:52:05 GMT (envelope-from np@FreeBSD.org) Message-Id: <201808022252.w72Mq582095697@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: np set sender to np@FreeBSD.org using -f From: Navdeep Parhar Date: Thu, 2 Aug 2018 22:52:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337192 - in head/sys/dev/cxgbe: . common X-SVN-Group: head X-SVN-Commit-Author: np X-SVN-Commit-Paths: in head/sys/dev/cxgbe: . common X-SVN-Commit-Revision: 337192 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 22:52:07 -0000 Author: np Date: Thu Aug 2 22:52:05 2018 New Revision: 337192 URL: https://svnweb.freebsd.org/changeset/base/337192 Log: cxgbe(4): Improvements in TID management. - Ignore any type of TID where the start/end values are not in the correct order. There are situations where the firmware isn't able to reserve room for the number requested in the config file but doesn't report a failure during configuration and instead sets end <= start. - Track start/end in tid_tab and remove some redundant copies from adapter->params. - Move all the start/end and other read-only parameters to a quiet part of tid_tab, away from the tid locks. MFC after: 1 week Sponsored by: Chelsio Communications Modified: head/sys/dev/cxgbe/common/common.h head/sys/dev/cxgbe/offload.h head/sys/dev/cxgbe/t4_main.c Modified: head/sys/dev/cxgbe/common/common.h ============================================================================== --- head/sys/dev/cxgbe/common/common.h Thu Aug 2 21:59:46 2018 (r337191) +++ head/sys/dev/cxgbe/common/common.h Thu Aug 2 22:52:05 2018 (r337192) @@ -355,11 +355,6 @@ struct adapter_params { unsigned short a_wnd[NCCTRL_WIN]; unsigned short b_wnd[NCCTRL_WIN]; - u_int ftid_min; - u_int ftid_max; - u_int etid_min; - u_int etid_max; - unsigned int cim_la_size; uint8_t nports; /* # of ethernet ports */ @@ -442,14 +437,15 @@ struct link_config { static inline int is_ftid(const struct adapter *sc, u_int tid) { - return (tid >= sc->params.ftid_min && tid <= sc->params.ftid_max); + return (sc->tids.nftids > 0 && tid >= sc->tids.ftid_base && + tid <= sc->tids.ftid_end); } static inline int is_etid(const struct adapter *sc, u_int tid) { - return (sc->params.etid_min > 0 && tid >= sc->params.etid_min && - tid <= sc->params.etid_max); + return (sc->tids.netids > 0 && tid >= sc->tids.etid_base && + tid <= sc->tids.etid_end); } static inline int is_offload(const struct adapter *adap) Modified: head/sys/dev/cxgbe/offload.h ============================================================================== --- head/sys/dev/cxgbe/offload.h Thu Aug 2 21:59:46 2018 (r337191) +++ head/sys/dev/cxgbe/offload.h Thu Aug 2 22:52:05 2018 (r337192) @@ -120,18 +120,27 @@ union etid_entry { }; /* - * Holds the size, base address, free list start, etc of the TID, server TID, - * and active-open TID tables. The tables themselves are allocated dynamically. + * Holds the size, base address, start, end, etc. of various types of TIDs. The + * tables themselves are allocated dynamically. */ struct tid_info { - void **tid_tab; + u_int nstids; + u_int stid_base; + + u_int natids; + + u_int nftids; + u_int ftid_base; + u_int ftid_end; + u_int ntids; - u_int tids_in_use; + u_int netids; + u_int etid_base; + u_int etid_end; + struct mtx stid_lock __aligned(CACHE_LINE_SIZE); struct listen_ctx **stid_tab; - u_int nstids; - u_int stid_base; u_int stids_in_use; u_int nstids_free_head; /* # of available stids at the beginning */ struct stid_head stids; @@ -139,26 +148,28 @@ struct tid_info { struct mtx atid_lock __aligned(CACHE_LINE_SIZE); union aopen_entry *atid_tab; union aopen_entry *afree; - u_int natids; u_int atids_in_use; struct mtx ftid_lock __aligned(CACHE_LINE_SIZE); struct cv ftid_cv; struct filter_entry *ftid_tab; - u_int nftids; - u_int ftid_base; u_int ftids_in_use; + /* + * hashfilter and TOE are mutually exclusive and both use ntids and + * tids_in_use. The lock and cv are used only by hashfilter. + */ struct mtx hftid_lock __aligned(CACHE_LINE_SIZE); struct cv hftid_cv; - void **hftid_tab; - /* ntids, tids_in_use */ + union { + void **hftid_tab; + void **tid_tab; + }; + u_int tids_in_use; struct mtx etid_lock __aligned(CACHE_LINE_SIZE); union etid_entry *etid_tab; union etid_entry *efree; - u_int netids; - u_int etid_base; u_int etids_in_use; }; Modified: head/sys/dev/cxgbe/t4_main.c ============================================================================== --- head/sys/dev/cxgbe/t4_main.c Thu Aug 2 21:59:46 2018 (r337191) +++ head/sys/dev/cxgbe/t4_main.c Thu Aug 2 22:52:05 2018 (r337192) @@ -3841,10 +3841,11 @@ get_params__post_init(struct adapter *sc) sc->sge.iq_start = val[0]; sc->sge.eq_start = val[1]; - sc->tids.ftid_base = val[2]; - sc->tids.nftids = val[3] - val[2] + 1; - sc->params.ftid_min = val[2]; - sc->params.ftid_max = val[3]; + if (val[3] > val[2]) { + sc->tids.ftid_base = val[2]; + sc->tids.ftid_end = val[3]; + sc->tids.nftids = val[3] - val[2] + 1; + } sc->vres.l2t.start = val[4]; sc->vres.l2t.size = val[5] - val[4] + 1; KASSERT(sc->vres.l2t.size <= L2T_SIZE, @@ -3928,12 +3929,13 @@ get_params__post_init(struct adapter *sc) "failed to query NIC parameters: %d.\n", rc); return (rc); } - sc->tids.etid_base = val[0]; - sc->params.etid_min = val[0]; - sc->params.etid_max = val[1]; - sc->tids.netids = val[1] - val[0] + 1; - sc->params.eo_wr_cred = val[2]; - sc->params.ethoffload = 1; + if (val[1] > val[0]) { + sc->tids.etid_base = val[0]; + sc->tids.etid_end = val[1]; + sc->tids.netids = val[1] - val[0] + 1; + sc->params.eo_wr_cred = val[2]; + sc->params.ethoffload = 1; + } } if (sc->toecaps) { /* query offload-related parameters */ @@ -3951,8 +3953,10 @@ get_params__post_init(struct adapter *sc) } sc->tids.ntids = val[0]; sc->tids.natids = min(sc->tids.ntids / 2, MAX_ATIDS); - sc->tids.stid_base = val[1]; - sc->tids.nstids = val[2] - val[1] + 1; + if (val[2] > val[1]) { + sc->tids.stid_base = val[1]; + sc->tids.nstids = val[2] - val[1] + 1; + } sc->vres.ddp.start = val[3]; sc->vres.ddp.size = val[4] - val[3] + 1; sc->params.ofldq_wr_cred = val[5]; From owner-svn-src-all@freebsd.org Thu Aug 2 23:13:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5282E1053835; Thu, 2 Aug 2018 23:13:41 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EC854814E2; Thu, 2 Aug 2018 23:13:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CCA3118604; Thu, 2 Aug 2018 23:13:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NDeC5006035; Thu, 2 Aug 2018 23:13:40 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NDeui006034; Thu, 2 Aug 2018 23:13:40 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022313.w72NDeui006034@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:13:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337193 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337193 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:13:41 -0000 Author: mav Date: Thu Aug 2 23:13:40 2018 New Revision: 337193 URL: https://svnweb.freebsd.org/changeset/base/337193 Log: 9424 ztest failure: "unprotected error in call to Lua API (Invalid value type 'function' for key 'error')" illumos/illumos-gate@fe3ba4d1227d8746116ece7240682b13595c3142 Reviewed by: Sebastien Roy Reviewed by: Paul Dagnelie Reviewed by: Don Brady Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Thu Aug 2 22:52:05 2018 (r337192) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zcp.c Thu Aug 2 23:13:40 2018 (r337193) @@ -426,7 +426,7 @@ zcp_lua_to_nvlist_impl(lua_State *state, int index, nv /* * Convert a lua value to an nvpair, adding it to an nvlist with the given key. */ -void +static void zcp_lua_to_nvlist(lua_State *state, int index, nvlist_t *nvl, const char *key) { /* @@ -438,7 +438,7 @@ zcp_lua_to_nvlist(lua_State *state, int index, nvlist_ (void) lua_error(state); } -int +static int zcp_lua_to_nvlist_helper(lua_State *state) { nvlist_t *nv = (nvlist_t *)lua_touserdata(state, 2); @@ -447,11 +447,12 @@ zcp_lua_to_nvlist_helper(lua_State *state) return (0); } -void +static void zcp_convert_return_values(lua_State *state, nvlist_t *nvl, const char *key, zcp_eval_arg_t *evalargs) { int err; + VERIFY3U(1, ==, lua_gettop(state)); lua_pushcfunction(state, zcp_lua_to_nvlist_helper); lua_pushlightuserdata(state, (char *)key); lua_pushlightuserdata(state, nvl); @@ -897,6 +898,7 @@ zcp_eval_impl(dmu_tx_t *tx, boolean_t sync, zcp_eval_a ZCP_RET_RETURN, evalargs); } else if (return_count > 1) { evalargs->ea_result = SET_ERROR(ECHRNG); + lua_settop(state, 0); (void) lua_pushfstring(state, "Multiple return " "values not supported"); zcp_convert_return_values(state, evalargs->ea_outnvl, @@ -958,6 +960,7 @@ static void zcp_pool_error(zcp_eval_arg_t *evalargs, const char *poolname) { evalargs->ea_result = SET_ERROR(ECHRNG); + lua_settop(evalargs->ea_state, 0); (void) lua_pushfstring(evalargs->ea_state, "Could not open pool: %s", poolname); zcp_convert_return_values(evalargs->ea_state, evalargs->ea_outnvl, From owner-svn-src-all@freebsd.org Thu Aug 2 23:15:11 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0AA331053933; Thu, 2 Aug 2018 23:15:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACB4681686; Thu, 2 Aug 2018 23:15:10 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8A88D18607; Thu, 2 Aug 2018 23:15:10 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NFAeC006163; Thu, 2 Aug 2018 23:15:10 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NFAEE006162; Thu, 2 Aug 2018 23:15:10 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022315.w72NFAEE006162@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:15:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337194 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337194 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:15:11 -0000 Author: mav Date: Thu Aug 2 23:15:10 2018 New Revision: 337194 URL: https://svnweb.freebsd.org/changeset/base/337194 Log: MFV r337193: 9424 ztest failure: "unprotected error in call to Lua API (Invalid value type 'f unction' for key 'error')" illumos/illumos-gate@fe3ba4d1227d8746116ece7240682b13595c3142 Reviewed by: Sebastien Roy Reviewed by: Paul Dagnelie Reviewed by: Don Brady Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zcp.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zcp.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zcp.c Thu Aug 2 23:13:40 2018 (r337193) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zcp.c Thu Aug 2 23:15:10 2018 (r337194) @@ -433,7 +433,7 @@ zcp_lua_to_nvlist_impl(lua_State *state, int index, nv /* * Convert a lua value to an nvpair, adding it to an nvlist with the given key. */ -void +static void zcp_lua_to_nvlist(lua_State *state, int index, nvlist_t *nvl, const char *key) { /* @@ -445,7 +445,7 @@ zcp_lua_to_nvlist(lua_State *state, int index, nvlist_ (void) lua_error(state); } -int +static int zcp_lua_to_nvlist_helper(lua_State *state) { nvlist_t *nv = (nvlist_t *)lua_touserdata(state, 2); @@ -454,11 +454,12 @@ zcp_lua_to_nvlist_helper(lua_State *state) return (0); } -void +static void zcp_convert_return_values(lua_State *state, nvlist_t *nvl, const char *key, zcp_eval_arg_t *evalargs) { int err; + VERIFY3U(1, ==, lua_gettop(state)); lua_pushcfunction(state, zcp_lua_to_nvlist_helper); lua_pushlightuserdata(state, (char *)key); lua_pushlightuserdata(state, nvl); @@ -904,6 +905,7 @@ zcp_eval_impl(dmu_tx_t *tx, boolean_t sync, zcp_eval_a ZCP_RET_RETURN, evalargs); } else if (return_count > 1) { evalargs->ea_result = SET_ERROR(ECHRNG); + lua_settop(state, 0); (void) lua_pushfstring(state, "Multiple return " "values not supported"); zcp_convert_return_values(state, evalargs->ea_outnvl, @@ -965,6 +967,7 @@ static void zcp_pool_error(zcp_eval_arg_t *evalargs, const char *poolname) { evalargs->ea_result = SET_ERROR(ECHRNG); + lua_settop(evalargs->ea_state, 0); (void) lua_pushfstring(evalargs->ea_state, "Could not open pool: %s", poolname); zcp_convert_return_values(evalargs->ea_state, evalargs->ea_outnvl, From owner-svn-src-all@freebsd.org Thu Aug 2 23:17:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F7D11053A41; Thu, 2 Aug 2018 23:17:12 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8656F8187F; Thu, 2 Aug 2018 23:17:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 642F618608; Thu, 2 Aug 2018 23:17:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NHB11006288; Thu, 2 Aug 2018 23:17:11 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NHBv3006287; Thu, 2 Aug 2018 23:17:11 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022317.w72NHBv3006287@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:17:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337195 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337195 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:17:12 -0000 Author: mav Date: Thu Aug 2 23:17:10 2018 New Revision: 337195 URL: https://svnweb.freebsd.org/changeset/base/337195 Log: 9454 ::zfs_blkstats should count embedded blocks illumos/illumos-gate@dec267e7ea9828898b1c64462daa6636c4ef5e29 Reviewed by: Dan Kimmel Reviewed by: George Wilson Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Thu Aug 2 23:15:10 2018 (r337194) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_scan.c Thu Aug 2 23:17:10 2018 (r337195) @@ -1951,14 +1951,14 @@ dsl_scan_scrub_cb(dsl_pool_t *dp, int zio_flags = ZIO_FLAG_SCAN_THREAD | ZIO_FLAG_RAW | ZIO_FLAG_CANFAIL; int scan_delay = 0; + count_block(dp->dp_blkstats, bp); + if (phys_birth <= scn->scn_phys.scn_min_txg || phys_birth >= scn->scn_phys.scn_max_txg) return (0); - count_block(dp->dp_blkstats, bp); - - if (BP_IS_EMBEDDED(bp)) - return (0); + /* Embedded BP's have phys_birth==0, so we reject them above. */ + ASSERT(!BP_IS_EMBEDDED(bp)); ASSERT(DSL_SCAN_IS_SCRUB_RESILVER(scn)); if (scn->scn_phys.scn_func == POOL_SCAN_SCRUB) { From owner-svn-src-all@freebsd.org Thu Aug 2 23:23:11 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABB631053D80; Thu, 2 Aug 2018 23:23:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5797E81D1E; Thu, 2 Aug 2018 23:23:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3590318796; Thu, 2 Aug 2018 23:23:11 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NNArG011226; Thu, 2 Aug 2018 23:23:10 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NNAcF011225; Thu, 2 Aug 2018 23:23:10 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022323.w72NNAcF011225@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:23:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337196 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337196 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:23:11 -0000 Author: mav Date: Thu Aug 2 23:23:10 2018 New Revision: 337196 URL: https://svnweb.freebsd.org/changeset/base/337196 Log: MFV r337195: 9454 ::zfs_blkstats should count embedded blocks illumos/illumos-gate@dec267e7ea9828898b1c64462daa6636c4ef5e29 Reviewed by: Dan Kimmel Reviewed by: George Wilson Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Thu Aug 2 23:17:10 2018 (r337195) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Thu Aug 2 23:23:10 2018 (r337196) @@ -3552,14 +3552,14 @@ dsl_scan_scrub_cb(dsl_pool_t *dp, int zio_flags = ZIO_FLAG_SCAN_THREAD | ZIO_FLAG_RAW | ZIO_FLAG_CANFAIL; int d; + count_block(dp->dp_blkstats, bp); + if (phys_birth <= scn->scn_phys.scn_min_txg || phys_birth >= scn->scn_phys.scn_max_txg) return (0); - if (BP_IS_EMBEDDED(bp)) { - count_block(scn, dp->dp_blkstats, bp); - return (0); - } + /* Embedded BP's have phys_birth==0, so we reject them above. */ + ASSERT(!BP_IS_EMBEDDED(bp)); ASSERT(DSL_SCAN_IS_SCRUB_RESILVER(scn)); if (scn->scn_phys.scn_func == POOL_SCAN_SCRUB) { From owner-svn-src-all@freebsd.org Thu Aug 2 23:24:42 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E379E1053E68; Thu, 2 Aug 2018 23:24:41 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 894F381EB9; Thu, 2 Aug 2018 23:24:41 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 679961879A; Thu, 2 Aug 2018 23:24:41 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NOfUD011324; Thu, 2 Aug 2018 23:24:41 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NOfPG011323; Thu, 2 Aug 2018 23:24:41 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022324.w72NOfPG011323@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:24:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337197 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337197 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:24:42 -0000 Author: mav Date: Thu Aug 2 23:24:40 2018 New Revision: 337197 URL: https://svnweb.freebsd.org/changeset/base/337197 Log: 9456 ztest failure in zil_commit_waiter_timeout illumos/illumos-gate@b6031810da58df96413bf76e068638fcab1f228a Reviewed by: Matt Ahrens Reviewed by: Serapheim Dimitropoulos Approved by: Matt Ahrens Author: Prakash Surya Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Thu Aug 2 23:23:10 2018 (r337196) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zil.c Thu Aug 2 23:24:40 2018 (r337197) @@ -2290,7 +2290,7 @@ zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_w */ lwb_t *nlwb = zil_lwb_write_issue(zilog, lwb); - ASSERT3S(lwb->lwb_state, !=, LWB_STATE_OPENED); + IMPLY(nlwb != NULL, lwb->lwb_state != LWB_STATE_OPENED); /* * Since the lwb's zio hadn't been issued by the time this thread From owner-svn-src-all@freebsd.org Thu Aug 2 23:25:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 090D61053EDA; Thu, 2 Aug 2018 23:25:50 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A7E0E81FF9; Thu, 2 Aug 2018 23:25:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 87B881879B; Thu, 2 Aug 2018 23:25:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NPn2P011436; Thu, 2 Aug 2018 23:25:49 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NPnr6011435; Thu, 2 Aug 2018 23:25:49 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022325.w72NPnr6011435@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:25:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337198 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337198 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:25:50 -0000 Author: mav Date: Thu Aug 2 23:25:49 2018 New Revision: 337198 URL: https://svnweb.freebsd.org/changeset/base/337198 Log: MFV r337197: 9456 ztest failure in zil_commit_waiter_timeout illumos/illumos-gate@b6031810da58df96413bf76e068638fcab1f228a Reviewed by: Matt Ahrens Reviewed by: Serapheim Dimitropoulos Approved by: Matt Ahrens Author: Prakash Surya Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c Thu Aug 2 23:24:40 2018 (r337197) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c Thu Aug 2 23:25:49 2018 (r337198) @@ -2299,7 +2299,7 @@ zil_commit_waiter_timeout(zilog_t *zilog, zil_commit_w */ lwb_t *nlwb = zil_lwb_write_issue(zilog, lwb); - ASSERT3S(lwb->lwb_state, !=, LWB_STATE_OPENED); + IMPLY(nlwb != NULL, lwb->lwb_state != LWB_STATE_OPENED); /* * Since the lwb's zio hadn't been issued by the time this thread From owner-svn-src-all@freebsd.org Thu Aug 2 23:28:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA9621054013; Thu, 2 Aug 2018 23:28:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 761118218F; Thu, 2 Aug 2018 23:28:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 52EAF1879F; Thu, 2 Aug 2018 23:28:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NSmpr011596; Thu, 2 Aug 2018 23:28:48 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NSmHq011595; Thu, 2 Aug 2018 23:28:48 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022328.w72NSmHq011595@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:28:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337199 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m vendor/illumos/dist/man/man5 X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m vendor/illumos/dist/man/man5 X-SVN-Commit-Revision: 337199 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:28:49 -0000 Author: mav Date: Thu Aug 2 23:28:47 2018 New Revision: 337199 URL: https://svnweb.freebsd.org/changeset/base/337199 Log: 8906 uts: illumos rootfs should support salted cksum illumos/illumos-gate@0dd498c0020c4a0dd34d350b07f97399a475d102 Reviewed by: C Fraire Reviewed by: Robert Mustacchi Reviewed by: Yuri Pankov Approved by: Dan McDonald Author: Toomas Soome Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Changes in other areas also in this revision: Modified: vendor/illumos/dist/man/man1m/zfs.1m vendor/illumos/dist/man/man5/zpool-features.5 Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Thu Aug 2 23:25:49 2018 (r337198) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Thu Aug 2 23:28:47 2018 (r337199) @@ -4069,16 +4069,7 @@ zfs_check_settable(const char *dsname, nvpair_t *pair, if ((err = spa_open(dsname, &spa, FTAG)) != 0) return (err); - /* - * Salted checksums are not supported on root pools. - */ - if (spa_bootfs(spa) != 0 && - intval < ZIO_CHECKSUM_FUNCTIONS && - (zio_checksum_table[intval].ci_flags & - ZCHECKSUM_FLAG_SALTED)) { - spa_close(spa, FTAG); - return (SET_ERROR(ERANGE)); - } + if (!spa_feature_is_enabled(spa, feature)) { spa_close(spa, FTAG); return (SET_ERROR(ENOTSUP)); From owner-svn-src-all@freebsd.org Thu Aug 2 23:28:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5050F1054017; Thu, 2 Aug 2018 23:28:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB1F382190; Thu, 2 Aug 2018 23:28:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C8483187A0; Thu, 2 Aug 2018 23:28:48 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NSmGr011603; Thu, 2 Aug 2018 23:28:48 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NSmK5011601; Thu, 2 Aug 2018 23:28:48 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022328.w72NSmK5011601@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:28:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337199 - vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m vendor/illumos/dist/man/man5 X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs vendor/illumos/dist/man/man1m vendor/illumos/dist/man/man5 X-SVN-Commit-Revision: 337199 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:28:49 -0000 Author: mav Date: Thu Aug 2 23:28:47 2018 New Revision: 337199 URL: https://svnweb.freebsd.org/changeset/base/337199 Log: 8906 uts: illumos rootfs should support salted cksum illumos/illumos-gate@0dd498c0020c4a0dd34d350b07f97399a475d102 Reviewed by: C Fraire Reviewed by: Robert Mustacchi Reviewed by: Yuri Pankov Approved by: Dan McDonald Author: Toomas Soome Modified: vendor/illumos/dist/man/man1m/zfs.1m vendor/illumos/dist/man/man5/zpool-features.5 Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_ioctl.c Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Thu Aug 2 23:25:49 2018 (r337198) +++ vendor/illumos/dist/man/man1m/zfs.1m Thu Aug 2 23:28:47 2018 (r337199) @@ -29,7 +29,7 @@ .\" Copyright 2017 Nexenta Systems, Inc. .\" Copyright 2018 Joyent, Inc. .\" -.Dd December 6, 2017 +.Dd Feb 10, 2018 .Dt ZFS 1M .Os .Sh NAME @@ -1002,10 +1002,6 @@ Please see for more information on these algorithms. .Pp Changing this property affects only newly-written data. -.Pp -Salted checksum algorithms -.Pq Cm edonr , skein -are currently not supported for any filesystem on the boot pools. .It Xo .Sy compression Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy gzip Ns | Ns .Sy gzip- Ns Em N Ns | Ns Sy lz4 Ns | Ns Sy lzjb Ns | Ns Sy zle Modified: vendor/illumos/dist/man/man5/zpool-features.5 ============================================================================== --- vendor/illumos/dist/man/man5/zpool-features.5 Thu Aug 2 23:25:49 2018 (r337198) +++ vendor/illumos/dist/man/man5/zpool-features.5 Thu Aug 2 23:28:47 2018 (r337199) @@ -15,7 +15,7 @@ .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your .\" own identifying information: .\" Portions Copyright [yyyy] [name of copyright owner] -.TH ZPOOL-FEATURES 5 "Aug 27, 2013" +.TH ZPOOL-FEATURES 5 "Jun 8, 2018" .SH NAME zpool\-features \- ZFS pool feature descriptions .SH DESCRIPTION @@ -561,8 +561,7 @@ can turn on the \fBsha512\fR checksum on any dataset u and will return to being \fBenabled\fR once all filesystems that have ever had their checksum set to \fBsha512\fR are destroyed. -Booting off of pools utilizing SHA-512/256 is supported (provided that -the updated GRUB stage2 module is installed). +Booting off of pools utilizing SHA-512/256 is supported. .RE @@ -596,9 +595,7 @@ can turn on the \fBskein\fR checksum on any dataset us and will return to being \fBenabled\fR once all filesystems that have ever had their checksum set to \fBskein\fR are destroyed. -Booting off of pools using \fBskein\fR is \fBNOT\fR supported --- any attempt to enable \fBskein\fR on a root pool will fail with an -error. +Booting off of pools using \fBskein\fR is supported. .RE @@ -638,9 +635,7 @@ can turn on the \fBedonr\fR checksum on any dataset us and will return to being \fBenabled\fR once all filesystems that have ever had their checksum set to \fBedonr\fR are destroyed. -Booting off of pools using \fBedonr\fR is \fBNOT\fR supported --- any attempt to enable \fBedonr\fR on a root pool will fail with an -error. +Booting off of pools using \fBedonr\fR is supported. .SH "SEE ALSO" \fBzpool\fR(1M) From owner-svn-src-all@freebsd.org Thu Aug 2 23:37:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D915A105437C; Thu, 2 Aug 2018 23:37:27 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8389682759; Thu, 2 Aug 2018 23:37:27 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 628E418936; Thu, 2 Aug 2018 23:37:27 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NbRfs016776; Thu, 2 Aug 2018 23:37:27 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NbQXB016770; Thu, 2 Aug 2018 23:37:26 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022337.w72NbQXB016770@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:37:26 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337200 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337200 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:37:28 -0000 Author: mav Date: Thu Aug 2 23:37:26 2018 New Revision: 337200 URL: https://svnweb.freebsd.org/changeset/base/337200 Log: 9438 Holes can lose birth time info if a block has a mix of birth times Ultimately, the problem here is that when you truncate and write a file in the same transaction group, the dbuf for the indirect block will be zeroed out to deal with the truncation, and then written for the write. During this process, we will lose hole birth time information for any holes in the range. In the case where a dnode is being freed, we need to determine whether the block should be converted to a higher-level hole in the zio pipeline, and if so do it when the dnode is being synced out. illumos/illumos-gate@738e2a3ce3b2579222d6855e7fe75b5bcfcddf8d Reviewed by: Matt Ahrens Reviewed by: George Wilson Approved by: Robert Mustacchi Author: Paul Dagnelie Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Thu Aug 2 23:28:47 2018 (r337199) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dmu_object.c Thu Aug 2 23:37:26 2018 (r337200) @@ -167,6 +167,10 @@ dmu_object_free(objset_t *os, uint64_t object, dmu_tx_ return (err); ASSERT(dn->dn_type != DMU_OT_NONE); + /* + * If we don't create this free range, we'll leak indirect blocks when + * we get to freeing the dnode in syncing context. + */ dnode_free_range(dn, 0, DMU_OBJECT_END, tx); dnode_free(dn, tx); dnode_rele(dn, FTAG); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:28:47 2018 (r337199) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:37:26 2018 (r337200) @@ -1507,6 +1507,72 @@ dnode_dirty_l1(dnode_t *dn, uint64_t l1blkid, dmu_tx_t } } +/* + * Dirty all the in-core level-1 dbufs in the range specified by start_blkid + * and end_blkid. + */ +static void +dnode_dirty_l1range(dnode_t *dn, uint64_t start_blkid, uint64_t end_blkid, + dmu_tx_t *tx) +{ + dmu_buf_impl_t db_search; + dmu_buf_impl_t *db; + avl_index_t where; + + mutex_enter(&dn->dn_dbufs_mtx); + + db_search.db_level = 1; + db_search.db_blkid = start_blkid + 1; + db_search.db_state = DB_SEARCH; + for (;;) { + + db = avl_find(&dn->dn_dbufs, &db_search, &where); + if (db == NULL) + db = avl_nearest(&dn->dn_dbufs, where, AVL_AFTER); + + if (db == NULL || db->db_level != 1 || + db->db_blkid >= end_blkid) { + break; + } + + /* + * Setup the next blkid we want to search for. + */ + db_search.db_blkid = db->db_blkid + 1; + ASSERT3U(db->db_blkid, >=, start_blkid); + + /* + * If the dbuf transitions to DB_EVICTING while we're trying + * to dirty it, then we will be unable to discover it in + * the dbuf hash table. This will result in a call to + * dbuf_create() which needs to acquire the dn_dbufs_mtx + * lock. To avoid a deadlock, we drop the lock before + * dirtying the level-1 dbuf. + */ + mutex_exit(&dn->dn_dbufs_mtx); + dnode_dirty_l1(dn, db->db_blkid, tx); + mutex_enter(&dn->dn_dbufs_mtx); + } + +#ifdef ZFS_DEBUG + /* + * Walk all the in-core level-1 dbufs and verify they have been dirtied. + */ + db_search.db_level = 1; + db_search.db_blkid = start_blkid + 1; + db_search.db_state = DB_SEARCH; + db = avl_find(&dn->dn_dbufs, &db_search, &where); + if (db == NULL) + db = avl_nearest(&dn->dn_dbufs, where, AVL_AFTER); + for (; db != NULL; db = AVL_NEXT(&dn->dn_dbufs, db)) { + if (db->db_level != 1 || db->db_blkid >= end_blkid) + break; + ASSERT(db->db_dirtycnt > 0); + } +#endif + mutex_exit(&dn->dn_dbufs_mtx); +} + void dnode_free_range(dnode_t *dn, uint64_t off, uint64_t len, dmu_tx_t *tx) { @@ -1657,6 +1723,8 @@ dnode_free_range(dnode_t *dn, uint64_t off, uint64_t l last = (blkid + nblks - 1) >> epbs; if (last != first) dnode_dirty_l1(dn, last, tx); + + dnode_dirty_l1range(dn, first, last, tx); int shift = dn->dn_datablkshift + dn->dn_indblkshift - SPA_BLKPTRSHIFT; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:28:47 2018 (r337199) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:37:26 2018 (r337200) @@ -229,9 +229,24 @@ free_verify(dmu_buf_impl_t *db, uint64_t start, uint64 } #endif +/* + * We don't usually free the indirect blocks here. If in one txg we have a + * free_range and a write to the same indirect block, it's important that we + * preserve the hole's birth times. Therefore, we don't free any any indirect + * blocks in free_children(). If an indirect block happens to turn into all + * holes, it will be freed by dbuf_write_children_ready, which happens at a + * point in the syncing process where we know for certain the contents of the + * indirect block. + * + * However, if we're freeing a dnode, its space accounting must go to zero + * before we actually try to free the dnode, or we will trip an assertion. In + * addition, we know the case described above cannot occur, because the dnode is + * being freed. Therefore, we free the indirect blocks immediately in that + * case. + */ static void free_children(dmu_buf_impl_t *db, uint64_t blkid, uint64_t nblks, - dmu_tx_t *tx) + boolean_t free_indirects, dmu_tx_t *tx) { dnode_t *dn; blkptr_t *bp; @@ -283,32 +298,16 @@ free_children(dmu_buf_impl_t *db, uint64_t blkid, uint rw_exit(&dn->dn_struct_rwlock); ASSERT3P(bp, ==, subdb->db_blkptr); - free_children(subdb, blkid, nblks, tx); + free_children(subdb, blkid, nblks, free_indirects, tx); dbuf_rele(subdb, FTAG); } } - /* If this whole block is free, free ourself too. */ - for (i = 0, bp = db->db.db_data; i < 1 << epbs; i++, bp++) { - if (!BP_IS_HOLE(bp)) - break; - } - if (i == 1 << epbs) { - /* - * We only found holes. Grab the rwlock to prevent - * anybody from reading the blocks we're about to - * zero out. - */ - rw_enter(&dn->dn_struct_rwlock, RW_WRITER); + if (free_indirects) { + for (i = 0, bp = db->db.db_data; i < 1 << epbs; i++, bp++) + ASSERT(BP_IS_HOLE(bp)); bzero(db->db.db_data, db->db.db_size); - rw_exit(&dn->dn_struct_rwlock); free_blocks(dn, db->db_blkptr, 1, tx); - } else { - /* - * Partial block free; must be marked dirty so that it - * will be written out. - */ - ASSERT(db->db_dirtycnt > 0); } DB_DNODE_EXIT(db); @@ -321,7 +320,7 @@ free_children(dmu_buf_impl_t *db, uint64_t blkid, uint */ static void dnode_sync_free_range_impl(dnode_t *dn, uint64_t blkid, uint64_t nblks, - dmu_tx_t *tx) + boolean_t free_indirects, dmu_tx_t *tx) { blkptr_t *bp = dn->dn_phys->dn_blkptr; int dnlevel = dn->dn_phys->dn_nlevels; @@ -361,7 +360,7 @@ dnode_sync_free_range_impl(dnode_t *dn, uint64_t blkid TRUE, FALSE, FTAG, &db)); rw_exit(&dn->dn_struct_rwlock); - free_children(db, blkid, nblks, tx); + free_children(db, blkid, nblks, free_indirects, tx); dbuf_rele(db, FTAG); } } @@ -380,6 +379,7 @@ dnode_sync_free_range_impl(dnode_t *dn, uint64_t blkid typedef struct dnode_sync_free_range_arg { dnode_t *dsfra_dnode; dmu_tx_t *dsfra_tx; + boolean_t dsfra_free_indirects; } dnode_sync_free_range_arg_t; static void @@ -389,7 +389,8 @@ dnode_sync_free_range(void *arg, uint64_t blkid, uint6 dnode_t *dn = dsfra->dsfra_dnode; mutex_exit(&dn->dn_mtx); - dnode_sync_free_range_impl(dn, blkid, nblks, dsfra->dsfra_tx); + dnode_sync_free_range_impl(dn, blkid, nblks, + dsfra->dsfra_free_indirects, dsfra->dsfra_tx); mutex_enter(&dn->dn_mtx); } @@ -670,6 +671,11 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) dnode_sync_free_range_arg_t dsfra; dsfra.dsfra_dnode = dn; dsfra.dsfra_tx = tx; + dsfra.dsfra_free_indirects = freeing_dnode; + if (freeing_dnode) { + ASSERT(range_tree_contains(dn->dn_free_ranges[txgoff], + 0, dn->dn_maxblkid + 1)); + } mutex_enter(&dn->dn_mtx); range_tree_vacate(dn->dn_free_ranges[txgoff], dnode_sync_free_range, &dsfra); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Thu Aug 2 23:28:47 2018 (r337199) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/zfs_znode.c Thu Aug 2 23:37:26 2018 (r337200) @@ -1603,7 +1603,8 @@ zfs_trunc(znode_t *zp, uint64_t end) return (0); } - error = dmu_free_long_range(zfsvfs->z_os, zp->z_id, end, -1); + error = dmu_free_long_range(zfsvfs->z_os, zp->z_id, end, + DMU_OBJECT_END); if (error) { zfs_range_unlock(rl); return (error); From owner-svn-src-all@freebsd.org Thu Aug 2 23:40:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A19E1054433; Thu, 2 Aug 2018 23:40:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DD95828D8; Thu, 2 Aug 2018 23:40:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 197C21893E; Thu, 2 Aug 2018 23:40:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NeSF9016972; Thu, 2 Aug 2018 23:40:28 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NeS1s016971; Thu, 2 Aug 2018 23:40:28 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022340.w72NeS1s016971@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:40:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337201 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337201 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:40:29 -0000 Author: mav Date: Thu Aug 2 23:40:28 2018 New Revision: 337201 URL: https://svnweb.freebsd.org/changeset/base/337201 Log: Fix build after r337196 mismerge. Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Thu Aug 2 23:37:26 2018 (r337200) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c Thu Aug 2 23:40:28 2018 (r337201) @@ -3552,7 +3552,7 @@ dsl_scan_scrub_cb(dsl_pool_t *dp, int zio_flags = ZIO_FLAG_SCAN_THREAD | ZIO_FLAG_RAW | ZIO_FLAG_CANFAIL; int d; - count_block(dp->dp_blkstats, bp); + count_block(scn, dp->dp_blkstats, bp); if (phys_birth <= scn->scn_phys.scn_min_txg || phys_birth >= scn->scn_phys.scn_max_txg) From owner-svn-src-all@freebsd.org Thu Aug 2 23:43:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DB141054975; Thu, 2 Aug 2018 23:43:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B6A7836C1; Thu, 2 Aug 2018 23:43:02 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C23B918AE8; Thu, 2 Aug 2018 23:43:02 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Nh2w3023340; Thu, 2 Aug 2018 23:43:02 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Nh1wD023318; Thu, 2 Aug 2018 23:43:01 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022343.w72Nh1wD023318@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:43:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337202 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337202 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:43:04 -0000 Author: mav Date: Thu Aug 2 23:43:01 2018 New Revision: 337202 URL: https://svnweb.freebsd.org/changeset/base/337202 Log: MFV r337200: 9438 Holes can lose birth time info if a block has a mix of birth times Ultimately, the problem here is that when you truncate and write a file in the same transaction group, the dbuf for the indirect block will be zeroed out to deal with the truncation, and then written for the write. During this process, we will lose hole birth time information for any holes in the range. In the case where a dnode is being freed, we need to determine whether the block should be converted to a higher-level hole in the zio pipeline, and if so do it when the dnode is being synced out. illumos/illumos-gate@738e2a3ce3b2579222d6855e7fe75b5bcfcddf8d Reviewed by: Matt Ahrens Reviewed by: George Wilson Approved by: Robert Mustacchi Author: Paul Dagnelie Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c Thu Aug 2 23:40:28 2018 (r337201) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_object.c Thu Aug 2 23:43:01 2018 (r337202) @@ -167,6 +167,10 @@ dmu_object_free(objset_t *os, uint64_t object, dmu_tx_ return (err); ASSERT(dn->dn_type != DMU_OT_NONE); + /* + * If we don't create this free range, we'll leak indirect blocks when + * we get to freeing the dnode in syncing context. + */ dnode_free_range(dn, 0, DMU_OBJECT_END, tx); dnode_free(dn, tx); dnode_rele(dn, FTAG); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Thu Aug 2 23:40:28 2018 (r337201) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Thu Aug 2 23:43:01 2018 (r337202) @@ -1518,6 +1518,72 @@ dnode_dirty_l1(dnode_t *dn, uint64_t l1blkid, dmu_tx_t } } +/* + * Dirty all the in-core level-1 dbufs in the range specified by start_blkid + * and end_blkid. + */ +static void +dnode_dirty_l1range(dnode_t *dn, uint64_t start_blkid, uint64_t end_blkid, + dmu_tx_t *tx) +{ + dmu_buf_impl_t db_search; + dmu_buf_impl_t *db; + avl_index_t where; + + mutex_enter(&dn->dn_dbufs_mtx); + + db_search.db_level = 1; + db_search.db_blkid = start_blkid + 1; + db_search.db_state = DB_SEARCH; + for (;;) { + + db = avl_find(&dn->dn_dbufs, &db_search, &where); + if (db == NULL) + db = avl_nearest(&dn->dn_dbufs, where, AVL_AFTER); + + if (db == NULL || db->db_level != 1 || + db->db_blkid >= end_blkid) { + break; + } + + /* + * Setup the next blkid we want to search for. + */ + db_search.db_blkid = db->db_blkid + 1; + ASSERT3U(db->db_blkid, >=, start_blkid); + + /* + * If the dbuf transitions to DB_EVICTING while we're trying + * to dirty it, then we will be unable to discover it in + * the dbuf hash table. This will result in a call to + * dbuf_create() which needs to acquire the dn_dbufs_mtx + * lock. To avoid a deadlock, we drop the lock before + * dirtying the level-1 dbuf. + */ + mutex_exit(&dn->dn_dbufs_mtx); + dnode_dirty_l1(dn, db->db_blkid, tx); + mutex_enter(&dn->dn_dbufs_mtx); + } + +#ifdef ZFS_DEBUG + /* + * Walk all the in-core level-1 dbufs and verify they have been dirtied. + */ + db_search.db_level = 1; + db_search.db_blkid = start_blkid + 1; + db_search.db_state = DB_SEARCH; + db = avl_find(&dn->dn_dbufs, &db_search, &where); + if (db == NULL) + db = avl_nearest(&dn->dn_dbufs, where, AVL_AFTER); + for (; db != NULL; db = AVL_NEXT(&dn->dn_dbufs, db)) { + if (db->db_level != 1 || db->db_blkid >= end_blkid) + break; + ASSERT(db->db_dirtycnt > 0); + } +#endif + mutex_exit(&dn->dn_dbufs_mtx); +} + void dnode_free_range(dnode_t *dn, uint64_t off, uint64_t len, dmu_tx_t *tx) { @@ -1668,6 +1734,8 @@ dnode_free_range(dnode_t *dn, uint64_t off, uint64_t l last = (blkid + nblks - 1) >> epbs; if (last != first) dnode_dirty_l1(dn, last, tx); + + dnode_dirty_l1range(dn, first, last, tx); int shift = dn->dn_datablkshift + dn->dn_indblkshift - SPA_BLKPTRSHIFT; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:40:28 2018 (r337201) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:43:01 2018 (r337202) @@ -229,9 +229,24 @@ free_verify(dmu_buf_impl_t *db, uint64_t start, uint64 } #endif +/* + * We don't usually free the indirect blocks here. If in one txg we have a + * free_range and a write to the same indirect block, it's important that we + * preserve the hole's birth times. Therefore, we don't free any any indirect + * blocks in free_children(). If an indirect block happens to turn into all + * holes, it will be freed by dbuf_write_children_ready, which happens at a + * point in the syncing process where we know for certain the contents of the + * indirect block. + * + * However, if we're freeing a dnode, its space accounting must go to zero + * before we actually try to free the dnode, or we will trip an assertion. In + * addition, we know the case described above cannot occur, because the dnode is + * being freed. Therefore, we free the indirect blocks immediately in that + * case. + */ static void free_children(dmu_buf_impl_t *db, uint64_t blkid, uint64_t nblks, - dmu_tx_t *tx) + boolean_t free_indirects, dmu_tx_t *tx) { dnode_t *dn; blkptr_t *bp; @@ -283,32 +298,16 @@ free_children(dmu_buf_impl_t *db, uint64_t blkid, uint rw_exit(&dn->dn_struct_rwlock); ASSERT3P(bp, ==, subdb->db_blkptr); - free_children(subdb, blkid, nblks, tx); + free_children(subdb, blkid, nblks, free_indirects, tx); dbuf_rele(subdb, FTAG); } } - /* If this whole block is free, free ourself too. */ - for (i = 0, bp = db->db.db_data; i < 1 << epbs; i++, bp++) { - if (!BP_IS_HOLE(bp)) - break; - } - if (i == 1 << epbs) { - /* - * We only found holes. Grab the rwlock to prevent - * anybody from reading the blocks we're about to - * zero out. - */ - rw_enter(&dn->dn_struct_rwlock, RW_WRITER); + if (free_indirects) { + for (i = 0, bp = db->db.db_data; i < 1 << epbs; i++, bp++) + ASSERT(BP_IS_HOLE(bp)); bzero(db->db.db_data, db->db.db_size); - rw_exit(&dn->dn_struct_rwlock); free_blocks(dn, db->db_blkptr, 1, tx); - } else { - /* - * Partial block free; must be marked dirty so that it - * will be written out. - */ - ASSERT(db->db_dirtycnt > 0); } DB_DNODE_EXIT(db); @@ -321,7 +320,7 @@ free_children(dmu_buf_impl_t *db, uint64_t blkid, uint */ static void dnode_sync_free_range_impl(dnode_t *dn, uint64_t blkid, uint64_t nblks, - dmu_tx_t *tx) + boolean_t free_indirects, dmu_tx_t *tx) { blkptr_t *bp = dn->dn_phys->dn_blkptr; int dnlevel = dn->dn_phys->dn_nlevels; @@ -361,7 +360,7 @@ dnode_sync_free_range_impl(dnode_t *dn, uint64_t blkid TRUE, FALSE, FTAG, &db)); rw_exit(&dn->dn_struct_rwlock); - free_children(db, blkid, nblks, tx); + free_children(db, blkid, nblks, free_indirects, tx); dbuf_rele(db, FTAG); } } @@ -380,6 +379,7 @@ dnode_sync_free_range_impl(dnode_t *dn, uint64_t blkid typedef struct dnode_sync_free_range_arg { dnode_t *dsfra_dnode; dmu_tx_t *dsfra_tx; + boolean_t dsfra_free_indirects; } dnode_sync_free_range_arg_t; static void @@ -389,7 +389,8 @@ dnode_sync_free_range(void *arg, uint64_t blkid, uint6 dnode_t *dn = dsfra->dsfra_dnode; mutex_exit(&dn->dn_mtx); - dnode_sync_free_range_impl(dn, blkid, nblks, dsfra->dsfra_tx); + dnode_sync_free_range_impl(dn, blkid, nblks, + dsfra->dsfra_free_indirects, dsfra->dsfra_tx); mutex_enter(&dn->dn_mtx); } @@ -670,6 +671,11 @@ dnode_sync(dnode_t *dn, dmu_tx_t *tx) dnode_sync_free_range_arg_t dsfra; dsfra.dsfra_dnode = dn; dsfra.dsfra_tx = tx; + dsfra.dsfra_free_indirects = freeing_dnode; + if (freeing_dnode) { + ASSERT(range_tree_contains(dn->dn_free_ranges[txgoff], + 0, dn->dn_maxblkid + 1)); + } mutex_enter(&dn->dn_mtx); range_tree_vacate(dn->dn_free_ranges[txgoff], dnode_sync_free_range, &dsfra); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c Thu Aug 2 23:40:28 2018 (r337201) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c Thu Aug 2 23:43:01 2018 (r337202) @@ -1691,7 +1691,8 @@ zfs_trunc(znode_t *zp, uint64_t end) return (0); } - error = dmu_free_long_range(zfsvfs->z_os, zp->z_id, end, -1); + error = dmu_free_long_range(zfsvfs->z_os, zp->z_id, end, + DMU_OBJECT_END); if (error) { zfs_range_unlock(rl); return (error); From owner-svn-src-all@freebsd.org Thu Aug 2 23:45:15 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 975B91054C95; Thu, 2 Aug 2018 23:45:15 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D8D883FE4; Thu, 2 Aug 2018 23:45:15 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3012518B05; Thu, 2 Aug 2018 23:45:15 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NjFJL024208; Thu, 2 Aug 2018 23:45:15 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NjE8Q024206; Thu, 2 Aug 2018 23:45:14 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201808022345.w72NjE8Q024206@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: Conrad Meyer Date: Thu, 2 Aug 2018 23:45:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337203 - head/usr.bin/wc X-SVN-Group: head X-SVN-Commit-Author: cem X-SVN-Commit-Paths: head/usr.bin/wc X-SVN-Commit-Revision: 337203 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:45:15 -0000 Author: cem Date: Thu Aug 2 23:45:14 2018 New Revision: 337203 URL: https://svnweb.freebsd.org/changeset/base/337203 Log: wc(1): Fix 'wc -L' I inadvertently broke 'wc -L' in r326736. We must skip the fast path if -L was specified, in addition to the existing check for the -l option. Document long-standing -L behavior (count varies depending on whether wc(1) is run with the -m option or not) in wc.1. That behavior dates back to the introduction of the -L option, but was not documented. PR: 230300 Reported by: Sponsored by: Dell EMC Isilon Modified: head/usr.bin/wc/wc.1 head/usr.bin/wc/wc.c Modified: head/usr.bin/wc/wc.1 ============================================================================== --- head/usr.bin/wc/wc.1 Thu Aug 2 23:43:01 2018 (r337202) +++ head/usr.bin/wc/wc.1 Thu Aug 2 23:45:14 2018 (r337203) @@ -31,7 +31,7 @@ .\" @(#)wc.1 8.2 (Berkeley) 4/19/94 .\" $FreeBSD$ .\" -.Dd December 1, 2015 +.Dd August 2, 2018 .Dt WC 1 .Os .Sh NAME @@ -76,8 +76,11 @@ See .Xr xo_parse_args 3 for details on command line arguments. .It Fl L -The number of characters in the longest input line -is written to the standard output. +Write the length of the line containing the most bytes (default) or characters +(when +.Fl m +is provided) +to standard output. When more than one .Ar file argument is specified, the longest input line of Modified: head/usr.bin/wc/wc.c ============================================================================== --- head/usr.bin/wc/wc.c Thu Aug 2 23:43:01 2018 (r337202) +++ head/usr.bin/wc/wc.c Thu Aug 2 23:45:14 2018 (r337203) @@ -216,7 +216,7 @@ cnt(const char *file) * If all we need is the number of characters and it's a regular file, * just stat it. */ - if (doline == 0) { + if (doline == 0 && dolongline == 0) { if (fstat(fd, &sb)) { xo_warn("%s: fstat", file); (void)close(fd); @@ -246,7 +246,7 @@ cnt(const char *file) if (siginfo) show_cnt(file, linect, wordct, charct, llct); charct += len; - if (doline) { + if (doline || dolongline) { for (p = buf; len--; ++p) if (*p == '\n') { if (tmpll > llct) @@ -262,10 +262,8 @@ cnt(const char *file) tlinect += linect; if (dochar) tcharct += charct; - if (dolongline) { - if (llct > tlongline) - tlongline = llct; - } + if (dolongline && llct > tlongline) + tlongline = llct; show_cnt(file, linect, wordct, charct, llct); (void)close(fd); return (0); @@ -331,10 +329,8 @@ word: gotsp = 1; twordct += wordct; if (dochar || domulti) tcharct += charct; - if (dolongline) { - if (llct > tlongline) - tlongline = llct; - } + if (dolongline && llct > tlongline) + tlongline = llct; show_cnt(file, linect, wordct, charct, llct); (void)close(fd); return (0); From owner-svn-src-all@freebsd.org Thu Aug 2 23:45:26 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D80911054CE1; Thu, 2 Aug 2018 23:45:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F4CA840FB; Thu, 2 Aug 2018 23:45:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5987F18B08; Thu, 2 Aug 2018 23:45:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NjPa6024275; Thu, 2 Aug 2018 23:45:25 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NjP41024273; Thu, 2 Aug 2018 23:45:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022345.w72NjP41024273@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:45:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337204 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337204 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:45:26 -0000 Author: mav Date: Thu Aug 2 23:45:24 2018 New Revision: 337204 URL: https://svnweb.freebsd.org/changeset/base/337204 Log: 9439 ZFS double-free due to failure to dirty indirect block illumos/illumos-gate@99a19144e82244f3426f055cc73af8a937c0135c Reviewed by: George Wilson Reviewed by: Paul Dagnelie Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:45:14 2018 (r337203) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:45:24 2018 (r337204) @@ -1605,13 +1605,11 @@ dnode_free_range(dnode_t *dn, uint64_t off, uint64_t l if (off == 0 && len >= blksz) { /* * Freeing the whole block; fast-track this request. - * Note that we won't dirty any indirect blocks, - * which is fine because we will be freeing the entire - * file and thus all indirect blocks will be freed - * by free_children(). */ blkid = 0; nblks = 1; + if (dn->dn_nlevels > 1) + dnode_dirty_l1(dn, 0, tx); goto done; } else if (off >= blksz) { /* Freeing past end-of-data */ Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:45:14 2018 (r337203) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:45:24 2018 (r337204) @@ -263,6 +263,24 @@ free_children(dmu_buf_impl_t *db, uint64_t blkid, uint if (db->db_state != DB_CACHED) (void) dbuf_read(db, NULL, DB_RF_MUST_SUCCEED); + /* + * If we modify this indirect block, and we are not freeing the + * dnode (!free_indirects), then this indirect block needs to get + * written to disk by dbuf_write(). If it is dirty, we know it will + * be written (otherwise, we would have incorrect on-disk state + * because the space would be freed but still referenced by the BP + * in this indirect block). Therefore we VERIFY that it is + * dirty. + * + * Our VERIFY covers some cases that do not actually have to be + * dirty, but the open-context code happens to dirty. E.g. if the + * blocks we are freeing are all holes, because in that case, we + * are only freeing part of this indirect block, so it is an + * ancestor of the first or last block to be freed. The first and + * last L1 indirect blocks are always dirtied by dnode_free_range(). + */ + VERIFY(BP_GET_FILL(db->db_blkptr) == 0 || db->db_dirtycnt > 0); + dbuf_release_bp(db); bp = db->db.db_data; From owner-svn-src-all@freebsd.org Thu Aug 2 23:46:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D790C1054E49; Thu, 2 Aug 2018 23:46:31 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CB148434F; Thu, 2 Aug 2018 23:46:31 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5514218B0A; Thu, 2 Aug 2018 23:46:31 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NkVZQ024378; Thu, 2 Aug 2018 23:46:31 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NkV5U024376; Thu, 2 Aug 2018 23:46:31 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022346.w72NkV5U024376@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:46:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337205 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337205 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:46:32 -0000 Author: mav Date: Thu Aug 2 23:46:30 2018 New Revision: 337205 URL: https://svnweb.freebsd.org/changeset/base/337205 Log: MFV r337204: 9439 ZFS double-free due to failure to dirty indirect block illumos/illumos-gate@99a19144e82244f3426f055cc73af8a937c0135c Reviewed by: George Wilson Reviewed by: Paul Dagnelie Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Thu Aug 2 23:45:24 2018 (r337204) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Thu Aug 2 23:46:30 2018 (r337205) @@ -1616,13 +1616,11 @@ dnode_free_range(dnode_t *dn, uint64_t off, uint64_t l if (off == 0 && len >= blksz) { /* * Freeing the whole block; fast-track this request. - * Note that we won't dirty any indirect blocks, - * which is fine because we will be freeing the entire - * file and thus all indirect blocks will be freed - * by free_children(). */ blkid = 0; nblks = 1; + if (dn->dn_nlevels > 1) + dnode_dirty_l1(dn, 0, tx); goto done; } else if (off >= blksz) { /* Freeing past end-of-data */ Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:45:24 2018 (r337204) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:46:30 2018 (r337205) @@ -263,6 +263,24 @@ free_children(dmu_buf_impl_t *db, uint64_t blkid, uint if (db->db_state != DB_CACHED) (void) dbuf_read(db, NULL, DB_RF_MUST_SUCCEED); + /* + * If we modify this indirect block, and we are not freeing the + * dnode (!free_indirects), then this indirect block needs to get + * written to disk by dbuf_write(). If it is dirty, we know it will + * be written (otherwise, we would have incorrect on-disk state + * because the space would be freed but still referenced by the BP + * in this indirect block). Therefore we VERIFY that it is + * dirty. + * + * Our VERIFY covers some cases that do not actually have to be + * dirty, but the open-context code happens to dirty. E.g. if the + * blocks we are freeing are all holes, because in that case, we + * are only freeing part of this indirect block, so it is an + * ancestor of the first or last block to be freed. The first and + * last L1 indirect blocks are always dirtied by dnode_free_range(). + */ + VERIFY(BP_GET_FILL(db->db_blkptr) == 0 || db->db_dirtycnt > 0); + dbuf_release_bp(db); bp = db->db.db_data; From owner-svn-src-all@freebsd.org Thu Aug 2 23:48:20 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16C8D1055000; Thu, 2 Aug 2018 23:48:20 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BFE80845F4; Thu, 2 Aug 2018 23:48:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9D39418B0B; Thu, 2 Aug 2018 23:48:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NmJ6b024487; Thu, 2 Aug 2018 23:48:19 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NmJDQ024486; Thu, 2 Aug 2018 23:48:19 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022348.w72NmJDQ024486@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:48:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337206 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337206 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:48:20 -0000 Author: mav Date: Thu Aug 2 23:48:19 2018 New Revision: 337206 URL: https://svnweb.freebsd.org/changeset/base/337206 Log: 9338 moved dnode has incorrect dn_next_type illumos/illumos-gate@c7fbe46df966ea665df63b6e6071808987e839d1 Reviewed by: Prashanth Sreenivasa Reviewed by: Serapheim Dimitropoulos Reviewed by: Dan Kimmel Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:46:30 2018 (r337205) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:48:19 2018 (r337206) @@ -733,6 +733,8 @@ dnode_move_impl(dnode_t *odn, dnode_t *ndn) ndn->dn_datablkszsec = odn->dn_datablkszsec; ndn->dn_datablksz = odn->dn_datablksz; ndn->dn_maxblkid = odn->dn_maxblkid; + bcopy(&odn->dn_next_type[0], &ndn->dn_next_type[0], + sizeof (odn->dn_next_type)); bcopy(&odn->dn_next_nblkptr[0], &ndn->dn_next_nblkptr[0], sizeof (odn->dn_next_nblkptr)); bcopy(&odn->dn_next_nlevels[0], &ndn->dn_next_nlevels[0], From owner-svn-src-all@freebsd.org Thu Aug 2 23:50:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88DF71055126; Thu, 2 Aug 2018 23:50:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3EAD9847E6; Thu, 2 Aug 2018 23:50:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1BBED18B10; Thu, 2 Aug 2018 23:50:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72No3S0024644; Thu, 2 Aug 2018 23:50:03 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72No3Vf024643; Thu, 2 Aug 2018 23:50:03 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022350.w72No3Vf024643@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:50:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337207 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337207 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:50:04 -0000 Author: mav Date: Thu Aug 2 23:50:03 2018 New Revision: 337207 URL: https://svnweb.freebsd.org/changeset/base/337207 Log: MFV r337206: 9338 moved dnode has incorrect dn_next_type illumos/illumos-gate@c7fbe46df966ea665df63b6e6071808987e839d1 Reviewed by: Prashanth Sreenivasa Reviewed by: Serapheim Dimitropoulos Reviewed by: Dan Kimmel Approved by: Robert Mustacchi Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Thu Aug 2 23:48:19 2018 (r337206) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Thu Aug 2 23:50:03 2018 (r337207) @@ -742,6 +742,8 @@ dnode_move_impl(dnode_t *odn, dnode_t *ndn) ndn->dn_datablkszsec = odn->dn_datablkszsec; ndn->dn_datablksz = odn->dn_datablksz; ndn->dn_maxblkid = odn->dn_maxblkid; + bcopy(&odn->dn_next_type[0], &ndn->dn_next_type[0], + sizeof (odn->dn_next_type)); bcopy(&odn->dn_next_nblkptr[0], &ndn->dn_next_nblkptr[0], sizeof (odn->dn_next_nblkptr)); bcopy(&odn->dn_next_nlevels[0], &ndn->dn_next_nlevels[0], From owner-svn-src-all@freebsd.org Thu Aug 2 23:52:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A56F3105547C; Thu, 2 Aug 2018 23:52:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B0A984D5E; Thu, 2 Aug 2018 23:52:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2219218CB0; Thu, 2 Aug 2018 23:52:30 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NqTLJ029166; Thu, 2 Aug 2018 23:52:29 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NqTvj029147; Thu, 2 Aug 2018 23:52:29 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022352.w72NqTvj029147@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:52:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337208 - vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/uts/common/fs/zfs X-SVN-Commit-Revision: 337208 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:52:31 -0000 Author: mav Date: Thu Aug 2 23:52:29 2018 New Revision: 337208 URL: https://svnweb.freebsd.org/changeset/base/337208 Log: 9591 ms_shift can be incorrectly changed in MOS config for indirect vdevs that have been historically expanded illumos/illumos-gate@11f6a9680e013a7c9c57dc0b64d3e91e2eee1a6b Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: John Kennedy Reviewed by: Prashanth Sreenivasa Reviewed by: Tim Chase Approved by: Richard Lowe Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c Thu Aug 2 23:50:03 2018 (r337207) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_config.c Thu Aug 2 23:52:29 2018 (r337208) @@ -22,7 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2011, 2015 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright 2017 Joyent, Inc. */ @@ -510,6 +510,18 @@ spa_config_update(spa_t *spa, int what) */ for (c = 0; c < rvd->vdev_children; c++) { vdev_t *tvd = rvd->vdev_child[c]; + + /* + * Explicitly skip vdevs that are indirect or + * log vdevs that are being removed. The reason + * is that both of those can have vdev_ms_array + * set to 0 and we wouldn't want to change their + * metaslab size nor call vdev_expand() on them. + */ + if (!vdev_is_concrete(tvd) || + (tvd->vdev_islog && tvd->vdev_removing)) + continue; + if (tvd->vdev_ms_array == 0) vdev_metaslab_set_size(tvd); vdev_expand(tvd, txg); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Thu Aug 2 23:50:03 2018 (r337207) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/vdev.c Thu Aug 2 23:52:29 2018 (r337208) @@ -4086,11 +4086,11 @@ vdev_expand(vdev_t *vd, uint64_t txg) { ASSERT(vd->vdev_top == vd); ASSERT(spa_config_held(vd->vdev_spa, SCL_ALL, RW_WRITER) == SCL_ALL); + ASSERT(vdev_is_concrete(vd)); vdev_set_deflate_ratio(vd); - if ((vd->vdev_asize >> vd->vdev_ms_shift) > vd->vdev_ms_count && - vdev_is_concrete(vd)) { + if ((vd->vdev_asize >> vd->vdev_ms_shift) > vd->vdev_ms_count) { VERIFY(vdev_metaslab_init(vd, txg) == 0); vdev_config_dirty(vd); } From owner-svn-src-all@freebsd.org Thu Aug 2 23:56:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE6C11055727; Thu, 2 Aug 2018 23:56:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7250985115; Thu, 2 Aug 2018 23:56:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 53B9518CB4; Thu, 2 Aug 2018 23:56:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72Nu8dD029496; Thu, 2 Aug 2018 23:56:08 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72Nu81g029494; Thu, 2 Aug 2018 23:56:08 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022356.w72Nu81g029494@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:56:07 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337209 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-SVN-Commit-Revision: 337209 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:56:09 -0000 Author: mav Date: Thu Aug 2 23:56:07 2018 New Revision: 337209 URL: https://svnweb.freebsd.org/changeset/base/337209 Log: MFV r337208: 9591 ms_shift can be incorrectly changed in MOS config for indirect vdevs that have been historically expanded illumos/illumos-gate@11f6a9680e013a7c9c57dc0b64d3e91e2eee1a6b Reviewed by: Matthew Ahrens Reviewed by: George Wilson Reviewed by: John Kennedy Reviewed by: Prashanth Sreenivasa Reviewed by: Tim Chase Approved by: Richard Lowe Author: Serapheim Dimitropoulos Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c Thu Aug 2 23:52:29 2018 (r337208) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_config.c Thu Aug 2 23:56:07 2018 (r337209) @@ -22,7 +22,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. * Copyright 2011 Nexenta Systems, Inc. All rights reserved. - * Copyright (c) 2011, 2015 by Delphix. All rights reserved. + * Copyright (c) 2011, 2018 by Delphix. All rights reserved. * Copyright 2017 Joyent, Inc. */ @@ -559,6 +559,18 @@ spa_config_update(spa_t *spa, int what) */ for (c = 0; c < rvd->vdev_children; c++) { vdev_t *tvd = rvd->vdev_child[c]; + + /* + * Explicitly skip vdevs that are indirect or + * log vdevs that are being removed. The reason + * is that both of those can have vdev_ms_array + * set to 0 and we wouldn't want to change their + * metaslab size nor call vdev_expand() on them. + */ + if (!vdev_is_concrete(tvd) || + (tvd->vdev_islog && tvd->vdev_removing)) + continue; + if (tvd->vdev_ms_array == 0) { vdev_ashift_optimize(tvd); vdev_metaslab_set_size(tvd); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Thu Aug 2 23:52:29 2018 (r337208) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Thu Aug 2 23:56:07 2018 (r337209) @@ -4284,11 +4284,11 @@ vdev_expand(vdev_t *vd, uint64_t txg) { ASSERT(vd->vdev_top == vd); ASSERT(spa_config_held(vd->vdev_spa, SCL_ALL, RW_WRITER) == SCL_ALL); + ASSERT(vdev_is_concrete(vd)); vdev_set_deflate_ratio(vd); - if ((vd->vdev_asize >> vd->vdev_ms_shift) > vd->vdev_ms_count && - vdev_is_concrete(vd)) { + if ((vd->vdev_asize >> vd->vdev_ms_shift) > vd->vdev_ms_count) { VERIFY(vdev_metaslab_init(vd, txg) == 0); vdev_config_dirty(vd); } From owner-svn-src-all@freebsd.org Thu Aug 2 23:59:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16B9D1055A55; Thu, 2 Aug 2018 23:59:54 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE6BB85365; Thu, 2 Aug 2018 23:59:53 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9F50B18CBD; Thu, 2 Aug 2018 23:59:53 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w72NxrvX029688; Thu, 2 Aug 2018 23:59:53 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w72NxqhY029681; Thu, 2 Aug 2018 23:59:52 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808022359.w72NxqhY029681@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 2 Aug 2018 23:59:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337210 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337210 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Aug 2018 23:59:54 -0000 Author: mav Date: Thu Aug 2 23:59:52 2018 New Revision: 337210 URL: https://svnweb.freebsd.org/changeset/base/337210 Log: 9577 remove zfs_dbuf_evict_key tsd The zfs_dbuf_evict_key TSD (thread-specific data) is not necessary - we can instead pass a flag down in a few places to prevent recursive dbuf eviction. Making this change has 3 benefits: 1. The code semantics are easier to understand. 2. On Linux, performance is improved, because creating/removing TSD values (by setting to NULL vs non-NULL) is expensive, and we do it very often. 3. According to Nexenta, the current semantics can cause a deadlock when concurrently calling dmu_objset_evict_dbufs() (which is rare today, but they are working on a "parallel unmount" change that triggers this more easily) illumos/illumos-gate@c2919acbea007fa95c709b60d073db9a24526e01 Reviewed by: George Wilson Reviewed by: Serapheim Dimitropoulos Reviewed by: Brian Behlendorf Reviewed by: Andy Stormont Approved by: Richard Lowe Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Thu Aug 2 23:56:07 2018 (r337209) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dbuf.c Thu Aug 2 23:59:52 2018 (r337210) @@ -51,8 +51,6 @@ #include #include -uint_t zfs_dbuf_evict_key; - static boolean_t dbuf_undirty(dmu_buf_impl_t *db, dmu_tx_t *tx); static void dbuf_write(dbuf_dirty_record_t *dr, arc_buf_t *data, dmu_tx_t *tx); @@ -524,14 +522,6 @@ dbuf_evict_one(void) ASSERT(!MUTEX_HELD(&dbuf_evict_lock)); - /* - * Set the thread's tsd to indicate that it's processing evictions. - * Once a thread stops evicting from the dbuf cache it will - * reset its tsd to NULL. - */ - ASSERT3P(tsd_get(zfs_dbuf_evict_key), ==, NULL); - (void) tsd_set(zfs_dbuf_evict_key, (void *)B_TRUE); - dmu_buf_impl_t *db = multilist_sublist_tail(mls); while (db != NULL && mutex_tryenter(&db->db_mtx) == 0) { db = multilist_sublist_prev(mls, db); @@ -551,7 +541,6 @@ dbuf_evict_one(void) } else { multilist_sublist_unlock(mls); } - (void) tsd_set(zfs_dbuf_evict_key, NULL); } /* @@ -605,30 +594,7 @@ dbuf_evict_thread(void *unused) static void dbuf_evict_notify(void) { - /* - * We use thread specific data to track when a thread has - * started processing evictions. This allows us to avoid deeply - * nested stacks that would have a call flow similar to this: - * - * dbuf_rele()-->dbuf_rele_and_unlock()-->dbuf_evict_notify() - * ^ | - * | | - * +-----dbuf_destroy()<--dbuf_evict_one()<--------+ - * - * The dbuf_eviction_thread will always have its tsd set until - * that thread exits. All other threads will only set their tsd - * if they are participating in the eviction process. This only - * happens if the eviction thread is unable to process evictions - * fast enough. To keep the dbuf cache size in check, other threads - * can evict from the dbuf cache directly. Those threads will set - * their tsd values so that we ensure that they only evict one dbuf - * from the dbuf cache. - */ - if (tsd_get(zfs_dbuf_evict_key) != NULL) - return; - - /* * We check if we should evict without holding the dbuf_evict_lock, * because it's OK to occasionally make the wrong decision here, * and grabbing the lock results in massive lock contention. @@ -704,7 +670,6 @@ retry: refcount_create(&dbuf_caches[dcs].size); } - tsd_create(&zfs_dbuf_evict_key, NULL); dbuf_evict_thread_exit = B_FALSE; mutex_init(&dbuf_evict_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&dbuf_evict_cv, NULL, CV_DEFAULT, NULL); @@ -731,7 +696,6 @@ dbuf_fini(void) cv_wait(&dbuf_evict_cv, &dbuf_evict_lock); } mutex_exit(&dbuf_evict_lock); - tsd_destroy(&zfs_dbuf_evict_key); mutex_destroy(&dbuf_evict_lock); cv_destroy(&dbuf_evict_cv); @@ -1011,7 +975,7 @@ dbuf_read_done(zio_t *zio, arc_buf_t *buf, void *vdb) db->db_state = DB_CACHED; } cv_broadcast(&db->db_changed); - dbuf_rele_and_unlock(db, NULL); + dbuf_rele_and_unlock(db, NULL, B_FALSE); } static void @@ -2171,7 +2135,8 @@ dbuf_destroy(dmu_buf_impl_t *db) * value in dnode_move(), since DB_DNODE_EXIT doesn't actually * release any lock. */ - dnode_rele(dn, db); + mutex_enter(&dn->dn_mtx); + dnode_rele_and_unlock(dn, db, B_TRUE); db->db_dnode_handle = NULL; dbuf_hash_remove(db); @@ -2198,8 +2163,10 @@ dbuf_destroy(dmu_buf_impl_t *db) * If this dbuf is referenced from an indirect dbuf, * decrement the ref count on the indirect dbuf. */ - if (parent && parent != dndb) - dbuf_rele(parent, db); + if (parent && parent != dndb) { + mutex_enter(&parent->db_mtx); + dbuf_rele_and_unlock(parent, db, B_TRUE); + } } /* @@ -2823,7 +2790,7 @@ void dbuf_rele(dmu_buf_impl_t *db, void *tag) { mutex_enter(&db->db_mtx); - dbuf_rele_and_unlock(db, tag); + dbuf_rele_and_unlock(db, tag, B_FALSE); } void @@ -2834,10 +2801,19 @@ dmu_buf_rele(dmu_buf_t *db, void *tag) /* * dbuf_rele() for an already-locked dbuf. This is necessary to allow - * db_dirtycnt and db_holds to be updated atomically. + * db_dirtycnt and db_holds to be updated atomically. The 'evicting' + * argument should be set if we are already in the dbuf-evicting code + * path, in which case we don't want to recursively evict. This allows us to + * avoid deeply nested stacks that would have a call flow similar to this: + * + * dbuf_rele()-->dbuf_rele_and_unlock()-->dbuf_evict_notify() + * ^ | + * | | + * +-----dbuf_destroy()<--dbuf_evict_one()<--------+ + * */ void -dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag) +dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag, boolean_t evicting) { int64_t holds; @@ -2940,7 +2916,8 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag) db->db.db_size, db); mutex_exit(&db->db_mtx); - if (db->db_caching_status == DB_DBUF_CACHE) { + if (db->db_caching_status == DB_DBUF_CACHE && + !evicting) { dbuf_evict_notify(); } } @@ -3203,7 +3180,7 @@ dbuf_sync_leaf(dbuf_dirty_record_t *dr, dmu_tx_t *tx) kmem_free(dr, sizeof (dbuf_dirty_record_t)); ASSERT(db->db_dirtycnt > 0); db->db_dirtycnt -= 1; - dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg); + dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg, B_FALSE); return; } @@ -3553,7 +3530,7 @@ dbuf_write_done(zio_t *zio, arc_buf_t *buf, void *vdb) ASSERT(db->db_dirtycnt > 0); db->db_dirtycnt -= 1; db->db_data_pending = NULL; - dbuf_rele_and_unlock(db, (void *)(uintptr_t)tx->tx_txg); + dbuf_rele_and_unlock(db, (void *)(uintptr_t)tx->tx_txg, B_FALSE); } static void Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:56:07 2018 (r337209) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode.c Thu Aug 2 23:59:52 2018 (r337210) @@ -1229,11 +1229,11 @@ void dnode_rele(dnode_t *dn, void *tag) { mutex_enter(&dn->dn_mtx); - dnode_rele_and_unlock(dn, tag); + dnode_rele_and_unlock(dn, tag, B_FALSE); } void -dnode_rele_and_unlock(dnode_t *dn, void *tag) +dnode_rele_and_unlock(dnode_t *dn, void *tag, boolean_t evicting) { uint64_t refs; /* Get while the hold prevents the dnode from moving. */ @@ -1264,7 +1264,8 @@ dnode_rele_and_unlock(dnode_t *dn, void *tag) * that the handle has zero references, but that will be * asserted anyway when the handle gets destroyed. */ - dbuf_rele(db, dnh); + mutex_enter(&db->db_mtx); + dbuf_rele_and_unlock(db, dnh, evicting); } } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:56:07 2018 (r337209) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:59:52 2018 (r337210) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. */ @@ -439,6 +439,19 @@ dnode_evict_dbufs(dnode_t *dn) avl_insert_here(&dn->dn_dbufs, &db_marker, db, AVL_BEFORE); + /* + * We need to use the "marker" dbuf rather than + * simply getting the next dbuf, because + * dbuf_destroy() may actually remove multiple dbufs. + * It can call itself recursively on the parent dbuf, + * which may also be removed from dn_dbufs. The code + * flow would look like: + * + * dbuf_destroy(): + * dnode_rele_and_unlock(parent_dbuf, evicting=TRUE): + * if (!cacheable || pending_evict) + * dbuf_destroy() + */ dbuf_destroy(db); db_next = AVL_NEXT(&dn->dn_dbufs, &db_marker); @@ -497,7 +510,7 @@ dnode_undirty_dbufs(list_t *list) list_destroy(&dr->dt.di.dr_children); } kmem_free(dr, sizeof (dbuf_dirty_record_t)); - dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg); + dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg, B_FALSE); } } Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Thu Aug 2 23:56:07 2018 (r337209) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dbuf.h Thu Aug 2 23:59:52 2018 (r337210) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2015 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. */ @@ -303,7 +303,7 @@ boolean_t dbuf_try_add_ref(dmu_buf_t *db, objset_t *os uint64_t dbuf_refcount(dmu_buf_impl_t *db); void dbuf_rele(dmu_buf_impl_t *db, void *tag); -void dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag); +void dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag, boolean_t evicting); dmu_buf_impl_t *dbuf_find(struct objset *os, uint64_t object, uint8_t level, uint64_t blkid); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h Thu Aug 2 23:56:07 2018 (r337209) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/dnode.h Thu Aug 2 23:59:52 2018 (r337210) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. */ @@ -291,7 +291,7 @@ int dnode_hold_impl(struct objset *dd, uint64_t object void *ref, dnode_t **dnp); boolean_t dnode_add_ref(dnode_t *dn, void *ref); void dnode_rele(dnode_t *dn, void *ref); -void dnode_rele_and_unlock(dnode_t *dn, void *tag); +void dnode_rele_and_unlock(dnode_t *dn, void *tag, boolean_t evicting); void dnode_setdirty(dnode_t *dn, dmu_tx_t *tx); void dnode_sync(dnode_t *dn, dmu_tx_t *tx); void dnode_allocate(dnode_t *dn, dmu_object_type_t ot, int blocksize, int ibs, From owner-svn-src-all@freebsd.org Fri Aug 3 00:01:49 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9FE41055E4E; Fri, 3 Aug 2018 00:01:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 700CC8590F; Fri, 3 Aug 2018 00:01:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5159A18D21; Fri, 3 Aug 2018 00:01:49 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7301n8T031481; Fri, 3 Aug 2018 00:01:49 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7301mtr031476; Fri, 3 Aug 2018 00:01:48 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030001.w7301mtr031476@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 00:01:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337211 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337211 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 00:01:50 -0000 Author: mav Date: Fri Aug 3 00:01:48 2018 New Revision: 337211 URL: https://svnweb.freebsd.org/changeset/base/337211 Log: MFV r337210: 9577 remove zfs_dbuf_evict_key tsd The zfs_dbuf_evict_key TSD (thread-specific data) is not necessary - we can instead pass a flag down in a few places to prevent recursive dbuf eviction. Making this change has 3 benefits: 1. The code semantics are easier to understand. 2. On Linux, performance is improved, because creating/removing TSD values (by setting to NULL vs non-NULL) is expensive, and we do it very often. 3. According to Nexenta, the current semantics can cause a deadlock when concurrently calling dmu_objset_evict_dbufs() (which is rare today, but they are working on a "parallel unmount" change that triggers this more easily) illumos/illumos-gate@c2919acbea007fa95c709b60d073db9a24526e01 Reviewed by: George Wilson Reviewed by: Serapheim Dimitropoulos Reviewed by: Brian Behlendorf Reviewed by: Andy Stormont Approved by: Richard Lowe Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dnode.h Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c Thu Aug 2 23:59:52 2018 (r337210) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c Fri Aug 3 00:01:48 2018 (r337211) @@ -51,8 +51,6 @@ #include #include -uint_t zfs_dbuf_evict_key; - static boolean_t dbuf_undirty(dmu_buf_impl_t *db, dmu_tx_t *tx); static void dbuf_write(dbuf_dirty_record_t *dr, arc_buf_t *data, dmu_tx_t *tx); @@ -534,14 +532,6 @@ dbuf_evict_one(void) ASSERT(!MUTEX_HELD(&dbuf_evict_lock)); - /* - * Set the thread's tsd to indicate that it's processing evictions. - * Once a thread stops evicting from the dbuf cache it will - * reset its tsd to NULL. - */ - ASSERT3P(tsd_get(zfs_dbuf_evict_key), ==, NULL); - (void) tsd_set(zfs_dbuf_evict_key, (void *)B_TRUE); - dmu_buf_impl_t *db = multilist_sublist_tail(mls); while (db != NULL && mutex_tryenter(&db->db_mtx) == 0) { db = multilist_sublist_prev(mls, db); @@ -561,7 +551,6 @@ dbuf_evict_one(void) } else { multilist_sublist_unlock(mls); } - (void) tsd_set(zfs_dbuf_evict_key, NULL); } /* @@ -615,30 +604,7 @@ dbuf_evict_thread(void *unused __unused) static void dbuf_evict_notify(void) { - /* - * We use thread specific data to track when a thread has - * started processing evictions. This allows us to avoid deeply - * nested stacks that would have a call flow similar to this: - * - * dbuf_rele()-->dbuf_rele_and_unlock()-->dbuf_evict_notify() - * ^ | - * | | - * +-----dbuf_destroy()<--dbuf_evict_one()<--------+ - * - * The dbuf_eviction_thread will always have its tsd set until - * that thread exits. All other threads will only set their tsd - * if they are participating in the eviction process. This only - * happens if the eviction thread is unable to process evictions - * fast enough. To keep the dbuf cache size in check, other threads - * can evict from the dbuf cache directly. Those threads will set - * their tsd values so that we ensure that they only evict one dbuf - * from the dbuf cache. - */ - if (tsd_get(zfs_dbuf_evict_key) != NULL) - return; - - /* * We check if we should evict without holding the dbuf_evict_lock, * because it's OK to occasionally make the wrong decision here, * and grabbing the lock results in massive lock contention. @@ -714,7 +680,6 @@ retry: refcount_create(&dbuf_caches[dcs].size); } - tsd_create(&zfs_dbuf_evict_key, NULL); dbuf_evict_thread_exit = B_FALSE; mutex_init(&dbuf_evict_lock, NULL, MUTEX_DEFAULT, NULL); cv_init(&dbuf_evict_cv, NULL, CV_DEFAULT, NULL); @@ -741,7 +706,6 @@ dbuf_fini(void) cv_wait(&dbuf_evict_cv, &dbuf_evict_lock); } mutex_exit(&dbuf_evict_lock); - tsd_destroy(&zfs_dbuf_evict_key); mutex_destroy(&dbuf_evict_lock); cv_destroy(&dbuf_evict_cv); @@ -1026,7 +990,7 @@ dbuf_read_done(zio_t *zio, const zbookmark_phys_t *zb, db->db_state = DB_CACHED; } cv_broadcast(&db->db_changed); - dbuf_rele_and_unlock(db, NULL); + dbuf_rele_and_unlock(db, NULL, B_FALSE); } static void @@ -2186,7 +2150,8 @@ dbuf_destroy(dmu_buf_impl_t *db) * value in dnode_move(), since DB_DNODE_EXIT doesn't actually * release any lock. */ - dnode_rele(dn, db); + mutex_enter(&dn->dn_mtx); + dnode_rele_and_unlock(dn, db, B_TRUE); db->db_dnode_handle = NULL; dbuf_hash_remove(db); @@ -2213,8 +2178,10 @@ dbuf_destroy(dmu_buf_impl_t *db) * If this dbuf is referenced from an indirect dbuf, * decrement the ref count on the indirect dbuf. */ - if (parent && parent != dndb) - dbuf_rele(parent, db); + if (parent && parent != dndb) { + mutex_enter(&parent->db_mtx); + dbuf_rele_and_unlock(parent, db, B_TRUE); + } } /* @@ -2846,7 +2813,7 @@ void dbuf_rele(dmu_buf_impl_t *db, void *tag) { mutex_enter(&db->db_mtx); - dbuf_rele_and_unlock(db, tag); + dbuf_rele_and_unlock(db, tag, B_FALSE); } void @@ -2857,10 +2824,19 @@ dmu_buf_rele(dmu_buf_t *db, void *tag) /* * dbuf_rele() for an already-locked dbuf. This is necessary to allow - * db_dirtycnt and db_holds to be updated atomically. + * db_dirtycnt and db_holds to be updated atomically. The 'evicting' + * argument should be set if we are already in the dbuf-evicting code + * path, in which case we don't want to recursively evict. This allows us to + * avoid deeply nested stacks that would have a call flow similar to this: + * + * dbuf_rele()-->dbuf_rele_and_unlock()-->dbuf_evict_notify() + * ^ | + * | | + * +-----dbuf_destroy()<--dbuf_evict_one()<--------+ + * */ void -dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag) +dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag, boolean_t evicting) { int64_t holds; @@ -2963,7 +2939,8 @@ dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag) db->db.db_size, db); mutex_exit(&db->db_mtx); - if (db->db_caching_status == DB_DBUF_CACHE) { + if (db->db_caching_status == DB_DBUF_CACHE && + !evicting) { dbuf_evict_notify(); } } @@ -3230,7 +3207,7 @@ dbuf_sync_leaf(dbuf_dirty_record_t *dr, dmu_tx_t *tx) kmem_free(dr, sizeof (dbuf_dirty_record_t)); ASSERT(db->db_dirtycnt > 0); db->db_dirtycnt -= 1; - dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg); + dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg, B_FALSE); return; } @@ -3580,7 +3557,7 @@ dbuf_write_done(zio_t *zio, arc_buf_t *buf, void *vdb) ASSERT(db->db_dirtycnt > 0); db->db_dirtycnt -= 1; db->db_data_pending = NULL; - dbuf_rele_and_unlock(db, (void *)(uintptr_t)tx->tx_txg); + dbuf_rele_and_unlock(db, (void *)(uintptr_t)tx->tx_txg, B_FALSE); } static void Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Thu Aug 2 23:59:52 2018 (r337210) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode.c Fri Aug 3 00:01:48 2018 (r337211) @@ -1240,11 +1240,11 @@ void dnode_rele(dnode_t *dn, void *tag) { mutex_enter(&dn->dn_mtx); - dnode_rele_and_unlock(dn, tag); + dnode_rele_and_unlock(dn, tag, B_FALSE); } void -dnode_rele_and_unlock(dnode_t *dn, void *tag) +dnode_rele_and_unlock(dnode_t *dn, void *tag, boolean_t evicting) { uint64_t refs; /* Get while the hold prevents the dnode from moving. */ @@ -1275,7 +1275,8 @@ dnode_rele_and_unlock(dnode_t *dn, void *tag) * that the handle has zero references, but that will be * asserted anyway when the handle gets destroyed. */ - dbuf_rele(db, dnh); + mutex_enter(&db->db_mtx); + dbuf_rele_and_unlock(db, dnh, evicting); } } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c Thu Aug 2 23:59:52 2018 (r337210) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dnode_sync.c Fri Aug 3 00:01:48 2018 (r337211) @@ -21,7 +21,7 @@ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. */ @@ -439,6 +439,19 @@ dnode_evict_dbufs(dnode_t *dn) avl_insert_here(&dn->dn_dbufs, &db_marker, db, AVL_BEFORE); + /* + * We need to use the "marker" dbuf rather than + * simply getting the next dbuf, because + * dbuf_destroy() may actually remove multiple dbufs. + * It can call itself recursively on the parent dbuf, + * which may also be removed from dn_dbufs. The code + * flow would look like: + * + * dbuf_destroy(): + * dnode_rele_and_unlock(parent_dbuf, evicting=TRUE): + * if (!cacheable || pending_evict) + * dbuf_destroy() + */ dbuf_destroy(db); db_next = AVL_NEXT(&dn->dn_dbufs, &db_marker); @@ -497,7 +510,7 @@ dnode_undirty_dbufs(list_t *list) list_destroy(&dr->dt.di.dr_children); } kmem_free(dr, sizeof (dbuf_dirty_record_t)); - dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg); + dbuf_rele_and_unlock(db, (void *)(uintptr_t)txg, B_FALSE); } } Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h Thu Aug 2 23:59:52 2018 (r337210) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dbuf.h Fri Aug 3 00:01:48 2018 (r337211) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2015 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2013 by Saso Kiselkov. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. */ @@ -303,7 +303,7 @@ boolean_t dbuf_try_add_ref(dmu_buf_t *db, objset_t *os uint64_t dbuf_refcount(dmu_buf_impl_t *db); void dbuf_rele(dmu_buf_impl_t *db, void *tag); -void dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag); +void dbuf_rele_and_unlock(dmu_buf_impl_t *db, void *tag, boolean_t evicting); dmu_buf_impl_t *dbuf_find(struct objset *os, uint64_t object, uint8_t level, uint64_t blkid); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dnode.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dnode.h Thu Aug 2 23:59:52 2018 (r337210) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dnode.h Fri Aug 3 00:01:48 2018 (r337211) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012, 2017 by Delphix. All rights reserved. + * Copyright (c) 2012, 2018 by Delphix. All rights reserved. * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved. */ @@ -291,7 +291,7 @@ int dnode_hold_impl(struct objset *dd, uint64_t object void *ref, dnode_t **dnp); boolean_t dnode_add_ref(dnode_t *dn, void *ref); void dnode_rele(dnode_t *dn, void *ref); -void dnode_rele_and_unlock(dnode_t *dn, void *tag); +void dnode_rele_and_unlock(dnode_t *dn, void *tag, boolean_t evicting); void dnode_setdirty(dnode_t *dn, dmu_tx_t *tx); void dnode_sync(dnode_t *dn, dmu_tx_t *tx); void dnode_allocate(dnode_t *dn, dmu_object_type_t ot, int blocksize, int ibs, From owner-svn-src-all@freebsd.org Fri Aug 3 00:11:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90354105603E; Fri, 3 Aug 2018 00:11:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4503985CD1; Fri, 3 Aug 2018 00:11:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2618218E82; Fri, 3 Aug 2018 00:11:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w730B7A6036771; Fri, 3 Aug 2018 00:11:07 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w730B6vs036762; Fri, 3 Aug 2018 00:11:06 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030011.w730B6vs036762@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 00:11:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337212 - in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in vendor-sys/illumos/dist/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337212 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 00:11:08 -0000 Author: mav Date: Fri Aug 3 00:11:06 2018 New Revision: 337212 URL: https://svnweb.freebsd.org/changeset/base/337212 Log: 9465 ARC check for 'anon_size > arc_c/2' can stall the system illumos/illumos-gate@abe1fd01ce5a83718c5a840daeab4abdaec1c104 Reviewed by: Sebastien Roy Reviewed by: Matt Ahrens Reviewed by: Prashanth Sreenivasa Approved by: Robert Mustacchi Author: Don Brady Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Fri Aug 3 00:01:48 2018 (r337211) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/arc.c Fri Aug 3 00:11:06 2018 (r337212) @@ -379,6 +379,13 @@ int zfs_arc_shrink_shift = 0; int zfs_arc_p_min_shift = 0; int zfs_arc_average_blocksize = 8 * 1024; /* 8KB */ +/* + * ARC dirty data constraints for arc_tempreserve_space() throttle + */ +uint_t zfs_arc_dirty_limit_percent = 50; /* total dirty data limit */ +uint_t zfs_arc_anon_limit_percent = 25; /* anon block dirty limit */ +uint_t zfs_arc_pool_dirty_percent = 20; /* each pool's anon allowance */ + boolean_t zfs_compressed_arc_enabled = B_TRUE; /* @@ -5839,12 +5846,10 @@ arc_write(zio_t *pio, spa_t *spa, uint64_t txg, blkptr } static int -arc_memory_throttle(uint64_t reserve, uint64_t txg) +arc_memory_throttle(spa_t *spa, uint64_t reserve, uint64_t txg) { #ifdef _KERNEL uint64_t available_memory = ptob(freemem); - static uint64_t page_load = 0; - static uint64_t last_txg = 0; #if defined(__i386) available_memory = @@ -5854,9 +5859,9 @@ arc_memory_throttle(uint64_t reserve, uint64_t txg) if (freemem > physmem * arc_lotsfree_percent / 100) return (0); - if (txg > last_txg) { - last_txg = txg; - page_load = 0; + if (txg > spa->spa_lowmem_last_txg) { + spa->spa_lowmem_last_txg = txg; + spa->spa_lowmem_page_load = 0; } /* * If we are in pageout, we know that memory is already tight, @@ -5864,18 +5869,19 @@ arc_memory_throttle(uint64_t reserve, uint64_t txg) * continue to let page writes occur as quickly as possible. */ if (curproc == proc_pageout) { - if (page_load > MAX(ptob(minfree), available_memory) / 4) + if (spa->spa_lowmem_page_load > + MAX(ptob(minfree), available_memory) / 4) return (SET_ERROR(ERESTART)); /* Note: reserve is inflated, so we deflate */ - page_load += reserve / 8; + atomic_add_64(&spa->spa_lowmem_page_load, reserve / 8); return (0); - } else if (page_load > 0 && arc_reclaim_needed()) { + } else if (spa->spa_lowmem_page_load > 0 && arc_reclaim_needed()) { /* memory is low, delay before restarting */ ARCSTAT_INCR(arcstat_memory_throttle_count, 1); return (SET_ERROR(EAGAIN)); } - page_load = 0; -#endif + spa->spa_lowmem_page_load = 0; +#endif /* _KERNEL */ return (0); } @@ -5887,7 +5893,7 @@ arc_tempreserve_clear(uint64_t reserve) } int -arc_tempreserve_space(uint64_t reserve, uint64_t txg) +arc_tempreserve_space(spa_t *spa, uint64_t reserve, uint64_t txg) { int error; uint64_t anon_size; @@ -5914,7 +5920,7 @@ arc_tempreserve_space(uint64_t reserve, uint64_t txg) * in order to compress/encrypt/etc the data. We therefore need to * make sure that there is sufficient available memory for this. */ - error = arc_memory_throttle(reserve, txg); + error = arc_memory_throttle(spa, reserve, txg); if (error != 0) return (error); @@ -5922,12 +5928,24 @@ arc_tempreserve_space(uint64_t reserve, uint64_t txg) * Throttle writes when the amount of dirty data in the cache * gets too large. We try to keep the cache less than half full * of dirty blocks so that our sync times don't grow too large. + * + * In the case of one pool being built on another pool, we want + * to make sure we don't end up throttling the lower (backing) + * pool when the upper pool is the majority contributor to dirty + * data. To insure we make forward progress during throttling, we + * also check the current pool's net dirty data and only throttle + * if it exceeds zfs_arc_pool_dirty_percent of the anonymous dirty + * data in the cache. + * * Note: if two requests come in concurrently, we might let them * both succeed, when one of them should fail. Not a huge deal. */ + uint64_t total_dirty = reserve + arc_tempreserve + anon_size; + uint64_t spa_dirty_anon = spa_dirty_data(spa); - if (reserve + arc_tempreserve + anon_size > arc_c / 2 && - anon_size > arc_c / 4) { + if (total_dirty > arc_c * zfs_arc_dirty_limit_percent / 100 && + anon_size > arc_c * zfs_arc_anon_limit_percent / 100 && + spa_dirty_anon > anon_size * zfs_arc_pool_dirty_percent / 100) { uint64_t meta_esize = refcount_count(&arc_anon->arcs_esize[ARC_BUFC_METADATA]); uint64_t data_esize = Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Fri Aug 3 00:01:48 2018 (r337211) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/dsl_dir.c Fri Aug 3 00:11:06 2018 (r337212) @@ -1379,7 +1379,7 @@ dsl_dir_tempreserve_space(dsl_dir_t *dd, uint64_t lsiz offsetof(struct tempreserve, tr_node)); ASSERT3S(asize, >, 0); - err = arc_tempreserve_space(lsize, tx->tx_txg); + err = arc_tempreserve_space(dd->dd_pool->dp_spa, lsize, tx->tx_txg); if (err == 0) { struct tempreserve *tr; Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Fri Aug 3 00:01:48 2018 (r337211) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/spa_misc.c Fri Aug 3 00:11:06 2018 (r337212) @@ -1910,6 +1910,12 @@ bp_get_dsize(spa_t *spa, const blkptr_t *bp) return (dsize); } +uint64_t +spa_dirty_data(spa_t *spa) +{ + return (spa->spa_dsl_pool->dp_dirty_total); +} + /* * ========================================================================== * Initialization and Termination Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Fri Aug 3 00:01:48 2018 (r337211) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/arc.h Fri Aug 3 00:11:06 2018 (r337212) @@ -190,7 +190,7 @@ void arc_freed(spa_t *spa, const blkptr_t *bp); void arc_flush(spa_t *spa, boolean_t retry); void arc_tempreserve_clear(uint64_t reserve); -int arc_tempreserve_space(uint64_t reserve, uint64_t txg); +int arc_tempreserve_space(spa_t *spa, uint64_t reserve, uint64_t txg); uint64_t arc_max_bytes(void); void arc_init(void); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Fri Aug 3 00:01:48 2018 (r337211) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa.h Fri Aug 3 00:11:06 2018 (r337212) @@ -813,6 +813,7 @@ extern uint64_t spa_bootfs(spa_t *spa); extern uint64_t spa_delegation(spa_t *spa); extern objset_t *spa_meta_objset(spa_t *spa); extern uint64_t spa_deadman_synctime(spa_t *spa); +extern uint64_t spa_dirty_data(spa_t *spa); /* Miscellaneous support routines */ extern void spa_load_failed(spa_t *spa, const char *fmt, ...); Modified: vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Fri Aug 3 00:01:48 2018 (r337211) +++ vendor-sys/illumos/dist/uts/common/fs/zfs/sys/spa_impl.h Fri Aug 3 00:11:06 2018 (r337212) @@ -373,6 +373,10 @@ struct spa { int spa_queued; } spa_queue_stats[ZIO_PRIORITY_NUM_QUEUEABLE]; + /* arc_memory_throttle() parameters during low memory condition */ + uint64_t spa_lowmem_page_load; /* memory load during txg */ + uint64_t spa_lowmem_last_txg; /* txg window start */ + hrtime_t spa_ccw_fail_time; /* Conf cache write fail time */ /* From owner-svn-src-all@freebsd.org Fri Aug 3 00:14:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28C1E10562CA; Fri, 3 Aug 2018 00:14:38 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D2EBF86353; Fri, 3 Aug 2018 00:14:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B3F8A18FEF; Fri, 3 Aug 2018 00:14:37 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w730Ebfs040099; Fri, 3 Aug 2018 00:14:37 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w730Ea6L040093; Fri, 3 Aug 2018 00:14:36 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030014.w730Ea6L040093@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 00:14:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337213 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337213 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 00:14:38 -0000 Author: mav Date: Fri Aug 3 00:14:36 2018 New Revision: 337213 URL: https://svnweb.freebsd.org/changeset/base/337213 Log: MFV r337212: 9465 ARC check for 'anon_size > arc_c/2' can stall the system illumos/illumos-gate@abe1fd01ce5a83718c5a840daeab4abdaec1c104 Reviewed by: Sebastien Roy Reviewed by: Matt Ahrens Reviewed by: Prashanth Sreenivasa Approved by: Robert Mustacchi Author: Don Brady Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Fri Aug 3 00:11:06 2018 (r337212) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Fri Aug 3 00:14:36 2018 (r337213) @@ -377,6 +377,13 @@ u_int zfs_arc_free_target = 0; /* Absolute min for arc min / max is 16MB. */ static uint64_t arc_abs_min = 16 << 20; +/* + * ARC dirty data constraints for arc_tempreserve_space() throttle + */ +uint_t zfs_arc_dirty_limit_percent = 50; /* total dirty data limit */ +uint_t zfs_arc_anon_limit_percent = 25; /* anon block dirty limit */ +uint_t zfs_arc_pool_dirty_percent = 20; /* each pool's anon allowance */ + boolean_t zfs_compressed_arc_enabled = B_TRUE; static int sysctl_vfs_zfs_arc_free_target(SYSCTL_HANDLER_ARGS); @@ -6308,12 +6315,10 @@ arc_write(zio_t *pio, spa_t *spa, uint64_t txg, blkptr } static int -arc_memory_throttle(uint64_t reserve, uint64_t txg) +arc_memory_throttle(spa_t *spa, uint64_t reserve, uint64_t txg) { #ifdef _KERNEL uint64_t available_memory = ptob(freemem); - static uint64_t page_load = 0; - static uint64_t last_txg = 0; #if defined(__i386) || !defined(UMA_MD_SMALL_ALLOC) available_memory = MIN(available_memory, uma_avail()); @@ -6322,9 +6327,9 @@ arc_memory_throttle(uint64_t reserve, uint64_t txg) if (freemem > (uint64_t)physmem * arc_lotsfree_percent / 100) return (0); - if (txg > last_txg) { - last_txg = txg; - page_load = 0; + if (txg > spa->spa_lowmem_last_txg) { + spa->spa_lowmem_last_txg = txg; + spa->spa_lowmem_page_load = 0; } /* * If we are in pageout, we know that memory is already tight, @@ -6332,18 +6337,19 @@ arc_memory_throttle(uint64_t reserve, uint64_t txg) * continue to let page writes occur as quickly as possible. */ if (curproc == pageproc) { - if (page_load > MAX(ptob(minfree), available_memory) / 4) + if (spa->spa_lowmem_page_load > + MAX(ptob(minfree), available_memory) / 4) return (SET_ERROR(ERESTART)); /* Note: reserve is inflated, so we deflate */ - page_load += reserve / 8; + atomic_add_64(&spa->spa_lowmem_page_load, reserve / 8); return (0); - } else if (page_load > 0 && arc_reclaim_needed()) { + } else if (spa->spa_lowmem_page_load > 0 && arc_reclaim_needed()) { /* memory is low, delay before restarting */ ARCSTAT_INCR(arcstat_memory_throttle_count, 1); return (SET_ERROR(EAGAIN)); } - page_load = 0; -#endif + spa->spa_lowmem_page_load = 0; +#endif /* _KERNEL */ return (0); } @@ -6355,7 +6361,7 @@ arc_tempreserve_clear(uint64_t reserve) } int -arc_tempreserve_space(uint64_t reserve, uint64_t txg) +arc_tempreserve_space(spa_t *spa, uint64_t reserve, uint64_t txg) { int error; uint64_t anon_size; @@ -6384,7 +6390,7 @@ arc_tempreserve_space(uint64_t reserve, uint64_t txg) * in order to compress/encrypt/etc the data. We therefore need to * make sure that there is sufficient available memory for this. */ - error = arc_memory_throttle(reserve, txg); + error = arc_memory_throttle(spa, reserve, txg); if (error != 0) return (error); @@ -6392,12 +6398,24 @@ arc_tempreserve_space(uint64_t reserve, uint64_t txg) * Throttle writes when the amount of dirty data in the cache * gets too large. We try to keep the cache less than half full * of dirty blocks so that our sync times don't grow too large. + * + * In the case of one pool being built on another pool, we want + * to make sure we don't end up throttling the lower (backing) + * pool when the upper pool is the majority contributor to dirty + * data. To insure we make forward progress during throttling, we + * also check the current pool's net dirty data and only throttle + * if it exceeds zfs_arc_pool_dirty_percent of the anonymous dirty + * data in the cache. + * * Note: if two requests come in concurrently, we might let them * both succeed, when one of them should fail. Not a huge deal. */ + uint64_t total_dirty = reserve + arc_tempreserve + anon_size; + uint64_t spa_dirty_anon = spa_dirty_data(spa); - if (reserve + arc_tempreserve + anon_size > arc_c / 2 && - anon_size > arc_c / 4) { + if (total_dirty > arc_c * zfs_arc_dirty_limit_percent / 100 && + anon_size > arc_c * zfs_arc_anon_limit_percent / 100 && + spa_dirty_anon > anon_size * zfs_arc_pool_dirty_percent / 100) { uint64_t meta_esize = refcount_count(&arc_anon->arcs_esize[ARC_BUFC_METADATA]); uint64_t data_esize = Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c Fri Aug 3 00:11:06 2018 (r337212) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c Fri Aug 3 00:14:36 2018 (r337213) @@ -1388,7 +1388,7 @@ dsl_dir_tempreserve_space(dsl_dir_t *dd, uint64_t lsiz offsetof(struct tempreserve, tr_node)); ASSERT3S(asize, >, 0); - err = arc_tempreserve_space(lsize, tx->tx_txg); + err = arc_tempreserve_space(dd->dd_pool->dp_spa, lsize, tx->tx_txg); if (err == 0) { struct tempreserve *tr; Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Fri Aug 3 00:11:06 2018 (r337212) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_misc.c Fri Aug 3 00:14:36 2018 (r337213) @@ -2033,6 +2033,12 @@ bp_get_dsize(spa_t *spa, const blkptr_t *bp) return (dsize); } +uint64_t +spa_dirty_data(spa_t *spa) +{ + return (spa->spa_dsl_pool->dp_dirty_total); +} + /* * ========================================================================== * Initialization and Termination Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h Fri Aug 3 00:11:06 2018 (r337212) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h Fri Aug 3 00:14:36 2018 (r337213) @@ -194,7 +194,7 @@ void arc_freed(spa_t *spa, const blkptr_t *bp); void arc_flush(spa_t *spa, boolean_t retry); void arc_tempreserve_clear(uint64_t reserve); -int arc_tempreserve_space(uint64_t reserve, uint64_t txg); +int arc_tempreserve_space(spa_t *spa, uint64_t reserve, uint64_t txg); uint64_t arc_max_bytes(void); void arc_init(void); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h Fri Aug 3 00:11:06 2018 (r337212) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h Fri Aug 3 00:14:36 2018 (r337213) @@ -833,6 +833,7 @@ extern uint64_t spa_bootfs(spa_t *spa); extern uint64_t spa_delegation(spa_t *spa); extern objset_t *spa_meta_objset(spa_t *spa); extern uint64_t spa_deadman_synctime(spa_t *spa); +extern uint64_t spa_dirty_data(spa_t *spa); /* Miscellaneous support routines */ extern void spa_load_failed(spa_t *spa, const char *fmt, ...); Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h Fri Aug 3 00:11:06 2018 (r337212) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h Fri Aug 3 00:14:36 2018 (r337213) @@ -388,6 +388,10 @@ struct spa { int spa_queued; } spa_queue_stats[ZIO_PRIORITY_NUM_QUEUEABLE]; #endif + /* arc_memory_throttle() parameters during low memory condition */ + uint64_t spa_lowmem_page_load; /* memory load during txg */ + uint64_t spa_lowmem_last_txg; /* txg window start */ + hrtime_t spa_ccw_fail_time; /* Conf cache write fail time */ /* From owner-svn-src-all@freebsd.org Fri Aug 3 00:20:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5E111056469; Fri, 3 Aug 2018 00:20:00 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9848B867C6; Fri, 3 Aug 2018 00:20:00 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 756FF18FF6; Fri, 3 Aug 2018 00:20:00 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w730K0eG040492; Fri, 3 Aug 2018 00:20:00 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w730K08w040491; Fri, 3 Aug 2018 00:20:00 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030020.w730K08w040491@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 00:20:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337214 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 337214 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 00:20:01 -0000 Author: mav Date: Fri Aug 3 00:19:59 2018 New Revision: 337214 URL: https://svnweb.freebsd.org/changeset/base/337214 Log: 9621 Make createtxg and guid properties public illumos/illumos-gate@e8d4a73c868afb740396041be80ed2b141065e76 Reviewed by: Andy Stormont Reviewed by: Paul Dagnelie Reviewed by: Matt Ahrens Reviewed by: Yuri Pankov Approved by: Robert Mustacchi Author: Josh Paetzel Modified: vendor-sys/illumos/dist/common/zfs/zfs_prop.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man1m/zfs.1m Modified: vendor-sys/illumos/dist/common/zfs/zfs_prop.c ============================================================================== --- vendor-sys/illumos/dist/common/zfs/zfs_prop.c Fri Aug 3 00:14:36 2018 (r337213) +++ vendor-sys/illumos/dist/common/zfs/zfs_prop.c Fri Aug 3 00:19:59 2018 (r337214) @@ -411,6 +411,10 @@ zfs_prop_init(void) zprop_register_number(ZFS_PROP_SNAPSHOT_COUNT, "snapshot_count", UINT64_MAX, PROP_DEFAULT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", "SSCOUNT"); + zprop_register_number(ZFS_PROP_GUID, "guid", 0, PROP_READONLY, + ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", "GUID"); + zprop_register_number(ZFS_PROP_CREATETXG, "createtxg", 0, PROP_READONLY, + ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", "CREATETXG"); /* inherit number properties */ zprop_register_number(ZFS_PROP_RECORDSIZE, "recordsize", @@ -418,8 +422,6 @@ zfs_prop_init(void) ZFS_TYPE_FILESYSTEM, "512 to 1M, power of 2", "RECSIZE"); /* hidden properties */ - zprop_register_hidden(ZFS_PROP_CREATETXG, "createtxg", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "CREATETXG"); zprop_register_hidden(ZFS_PROP_REMAPTXG, "remaptxg", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "REMAPTXG"); zprop_register_hidden(ZFS_PROP_NUMCLONES, "numclones", PROP_TYPE_NUMBER, @@ -431,8 +433,6 @@ zfs_prop_init(void) zprop_register_hidden(ZFS_PROP_STMF_SHAREINFO, "stmf_sbd_lu", PROP_TYPE_STRING, PROP_INHERIT, ZFS_TYPE_VOLUME, "STMF_SBD_LU"); - zprop_register_hidden(ZFS_PROP_GUID, "guid", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "GUID"); zprop_register_hidden(ZFS_PROP_USERACCOUNTING, "useraccounting", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "USERACCOUNTING"); Modified: vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Fri Aug 3 00:14:36 2018 (r337213) +++ vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Fri Aug 3 00:19:59 2018 (r337214) @@ -115,7 +115,7 @@ typedef enum { ZFS_PROP_SNAPDIR, ZFS_PROP_ACLMODE, ZFS_PROP_ACLINHERIT, - ZFS_PROP_CREATETXG, /* not exposed to the user */ + ZFS_PROP_CREATETXG, ZFS_PROP_NAME, /* not exposed to the user */ ZFS_PROP_CANMOUNT, ZFS_PROP_ISCSIOPTIONS, /* not exposed to the user */ From owner-svn-src-all@freebsd.org Fri Aug 3 00:20:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D6EF1056471; Fri, 3 Aug 2018 00:20:01 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30EC4867C7; Fri, 3 Aug 2018 00:20:01 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EEDF618FF7; Fri, 3 Aug 2018 00:20:00 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w730K0Dj040500; Fri, 3 Aug 2018 00:20:00 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w730K0Zs040498; Fri, 3 Aug 2018 00:20:00 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030020.w730K0Zs040498@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 00:20:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337214 - vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/zfs vendor-sys/illumos/dist/uts/common/sys/fs vendor/illumos/dist/lib/libzfs/common vendor/illumos/dist/man/man1m X-SVN-Commit-Revision: 337214 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 00:20:01 -0000 Author: mav Date: Fri Aug 3 00:19:59 2018 New Revision: 337214 URL: https://svnweb.freebsd.org/changeset/base/337214 Log: 9621 Make createtxg and guid properties public illumos/illumos-gate@e8d4a73c868afb740396041be80ed2b141065e76 Reviewed by: Andy Stormont Reviewed by: Paul Dagnelie Reviewed by: Matt Ahrens Reviewed by: Yuri Pankov Approved by: Robert Mustacchi Author: Josh Paetzel Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c vendor/illumos/dist/man/man1m/zfs.1m Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/common/zfs/zfs_prop.c vendor-sys/illumos/dist/uts/common/sys/fs/zfs.h Modified: vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Fri Aug 3 00:14:36 2018 (r337213) +++ vendor/illumos/dist/lib/libzfs/common/libzfs_dataset.c Fri Aug 3 00:19:59 2018 (r337214) @@ -2753,6 +2753,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char break; case ZFS_PROP_GUID: + case ZFS_PROP_CREATETXG: /* * GUIDs are stored as numbers, but they are identifiers. * We don't want them to be pretty printed, because pretty Modified: vendor/illumos/dist/man/man1m/zfs.1m ============================================================================== --- vendor/illumos/dist/man/man1m/zfs.1m Fri Aug 3 00:14:36 2018 (r337213) +++ vendor/illumos/dist/man/man1m/zfs.1m Fri Aug 3 00:19:59 2018 (r337214) @@ -550,6 +550,13 @@ Compression can be turned on by running: .Nm zfs Cm set Sy compression Ns = Ns Sy on Ar dataset . The default value is .Sy off . +.It Sy createtxg +The transaction group (txg) in which the dataset was created. +Bookmarks have the same +.Sy createtxg +as the snapshot they are initially tied to. +This property is suitable for ordering a list of snapshots, +e.g. for incremental send and receive. .It Sy creation The time this dataset was created. .It Sy clones @@ -581,6 +588,14 @@ the dataset tree. This value is only available when a .Sy filesystem_limit has been set somewhere in the tree under which the dataset resides. +.It Sy guid +The 64 bit GUID of this dataset or bookmark which does not change over its +entire lifetime. +When a snapshot is sent to another pool, the received snapshot has the same +GUID. +Thus, the +.Sy guid +is suitable to identify a snapshot across pools. .It Sy logicalreferenced The amount of space that is .Qq logically From owner-svn-src-all@freebsd.org Fri Aug 3 00:24:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93D4B105678D; Fri, 3 Aug 2018 00:24:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4504286F40; Fri, 3 Aug 2018 00:24:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0CB5019198; Fri, 3 Aug 2018 00:24:29 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w730OSOg045152; Fri, 3 Aug 2018 00:24:28 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w730OSa2045149; Fri, 3 Aug 2018 00:24:28 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030024.w730OSa2045149@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 00:24:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337215 - in head: cddl/contrib/opensolaris/cmd/zfs cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/sys/fs X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/cmd/zfs cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contrib/opensolaris/uts/common/sys/fs X-SVN-Commit-Revision: 337215 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 00:24:29 -0000 Author: mav Date: Fri Aug 3 00:24:27 2018 New Revision: 337215 URL: https://svnweb.freebsd.org/changeset/base/337215 Log: MFV 337214: 9621 Make createtxg and guid properties public illumos/illumos-gate@e8d4a73c868afb740396041be80ed2b141065e76 Reviewed by: Andy Stormont Reviewed by: Paul Dagnelie Reviewed by: Matt Ahrens Reviewed by: Yuri Pankov Approved by: Robert Mustacchi Author: Josh Paetzel Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c head/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c head/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/cddl/contrib/opensolaris/lib/libzfs/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 ============================================================================== --- head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 Fri Aug 3 00:19:59 2018 (r337214) +++ head/cddl/contrib/opensolaris/cmd/zfs/zfs.8 Fri Aug 3 00:24:27 2018 (r337215) @@ -548,6 +548,13 @@ property. Compression can be turned on by running: .Qq Nm Cm set compression=on Ar dataset The default value is .Cm off . +.It Sy createtxg +The transaction group (txg) in which the dataset was created. +Bookmarks have the same +.Sy createtxg +as the snapshot they are initially tied to. +This property is suitable for ordering a list of snapshots, +e.g. for incremental send and receive. .It Sy creation The time this dataset was created. .It Sy clones @@ -575,6 +582,14 @@ This value is only available when a .Sy filesystem_limit has been set somewhere in the tree under which the dataset resides. +.It Sy guid +The 64 bit GUID of this dataset or bookmark which does not change over its +entire lifetime. +When a snapshot is sent to another pool, the received snapshot has the same +GUID. +Thus, the +.Sy guid +is suitable to identify a snapshot across pools. .It Sy logicalreferenced The amount of space that is .Qq logically Modified: head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Fri Aug 3 00:19:59 2018 (r337214) +++ head/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c Fri Aug 3 00:24:27 2018 (r337215) @@ -2784,6 +2784,7 @@ zfs_prop_get(zfs_handle_t *zhp, zfs_prop_t prop, char break; case ZFS_PROP_GUID: + case ZFS_PROP_CREATETXG: /* * GUIDs are stored as numbers, but they are identifiers. * We don't want them to be pretty printed, because pretty Modified: head/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c Fri Aug 3 00:19:59 2018 (r337214) +++ head/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c Fri Aug 3 00:24:27 2018 (r337215) @@ -427,6 +427,10 @@ zfs_prop_init(void) zprop_register_number(ZFS_PROP_SNAPSHOT_COUNT, "snapshot_count", UINT64_MAX, PROP_DEFAULT, ZFS_TYPE_FILESYSTEM | ZFS_TYPE_VOLUME, "", "SSCOUNT"); + zprop_register_number(ZFS_PROP_GUID, "guid", 0, PROP_READONLY, + ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", "GUID"); + zprop_register_number(ZFS_PROP_CREATETXG, "createtxg", 0, PROP_READONLY, + ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "", "CREATETXG"); /* inherit number properties */ zprop_register_number(ZFS_PROP_RECORDSIZE, "recordsize", @@ -434,8 +438,6 @@ zfs_prop_init(void) ZFS_TYPE_FILESYSTEM, "512 to 1M, power of 2", "RECSIZE"); /* hidden properties */ - zprop_register_hidden(ZFS_PROP_CREATETXG, "createtxg", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "CREATETXG"); zprop_register_hidden(ZFS_PROP_REMAPTXG, "remaptxg", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "REMAPTXG"); zprop_register_hidden(ZFS_PROP_NUMCLONES, "numclones", PROP_TYPE_NUMBER, @@ -447,8 +449,6 @@ zfs_prop_init(void) zprop_register_hidden(ZFS_PROP_STMF_SHAREINFO, "stmf_sbd_lu", PROP_TYPE_STRING, PROP_INHERIT, ZFS_TYPE_VOLUME, "STMF_SBD_LU"); - zprop_register_hidden(ZFS_PROP_GUID, "guid", PROP_TYPE_NUMBER, - PROP_READONLY, ZFS_TYPE_DATASET | ZFS_TYPE_BOOKMARK, "GUID"); zprop_register_hidden(ZFS_PROP_USERACCOUNTING, "useraccounting", PROP_TYPE_NUMBER, PROP_READONLY, ZFS_TYPE_DATASET, "USERACCOUNTING"); Modified: head/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h Fri Aug 3 00:19:59 2018 (r337214) +++ head/sys/cddl/contrib/opensolaris/uts/common/sys/fs/zfs.h Fri Aug 3 00:24:27 2018 (r337215) @@ -118,7 +118,7 @@ typedef enum { ZFS_PROP_SNAPDIR, ZFS_PROP_ACLMODE, ZFS_PROP_ACLINHERIT, - ZFS_PROP_CREATETXG, /* not exposed to the user */ + ZFS_PROP_CREATETXG, ZFS_PROP_NAME, /* not exposed to the user */ ZFS_PROP_CANMOUNT, ZFS_PROP_ISCSIOPTIONS, /* not exposed to the user */ From owner-svn-src-all@freebsd.org Fri Aug 3 00:47:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C90921056F1C; Fri, 3 Aug 2018 00:47:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A166882BF; Fri, 3 Aug 2018 00:47:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 57DE9194C0; Fri, 3 Aug 2018 00:47:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w730lPcR055636; Fri, 3 Aug 2018 00:47:25 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w730lPL3055635; Fri, 3 Aug 2018 00:47:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030047.w730lPL3055635@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 00:47:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337216 - vendor-sys/illumos/dist/common/nvpair X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/nvpair X-SVN-Commit-Revision: 337216 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 00:47:26 -0000 Author: mav Date: Fri Aug 3 00:47:24 2018 New Revision: 337216 URL: https://svnweb.freebsd.org/changeset/base/337216 Log: 7263 deeply nested nvlist can overflow stack illumos/illumos-gate@9ca527c3d3dfa7c8f304b34a9e03b5eddace838f Reviewed by: Adam Leventhal Reviewed by: George Wilson Reviewed by: Robert Mustacchi Approved by: Dan McDonald Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c ============================================================================== --- vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 00:24:27 2018 (r337215) +++ vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 00:47:24 2018 (r337216) @@ -21,6 +21,7 @@ /* * Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2015, 2016 by Delphix. All rights reserved. */ #include @@ -138,6 +139,11 @@ static int nvlist_add_common(nvlist_t *nvl, const char #define NVPAIR2I_NVP(nvp) \ ((i_nvp_t *)((size_t)(nvp) - offsetof(i_nvp_t, nvi_nvp))) +#ifdef _KERNEL +int nvpair_max_recursion = 20; +#else +int nvpair_max_recursion = 100; +#endif int nv_alloc_init(nv_alloc_t *nva, const nv_alloc_ops_t *nvo, /* args */ ...) @@ -2014,6 +2020,7 @@ typedef struct { const nvs_ops_t *nvs_ops; void *nvs_private; nvpriv_t *nvs_priv; + int nvs_recursion; } nvstream_t; /* @@ -2165,9 +2172,16 @@ static int nvs_embedded(nvstream_t *nvs, nvlist_t *embedded) { switch (nvs->nvs_op) { - case NVS_OP_ENCODE: - return (nvs_operation(nvs, embedded, NULL)); + case NVS_OP_ENCODE: { + int err; + if (nvs->nvs_recursion >= nvpair_max_recursion) + return (EINVAL); + nvs->nvs_recursion++; + err = nvs_operation(nvs, embedded, NULL); + nvs->nvs_recursion--; + return (err); + } case NVS_OP_DECODE: { nvpriv_t *priv; int err; @@ -2180,8 +2194,12 @@ nvs_embedded(nvstream_t *nvs, nvlist_t *embedded) nvlist_init(embedded, embedded->nvl_nvflag, priv); + if (nvs->nvs_recursion >= nvpair_max_recursion) + return (EINVAL); + nvs->nvs_recursion++; if ((err = nvs_operation(nvs, embedded, NULL)) != 0) nvlist_free(embedded); + nvs->nvs_recursion--; return (err); } default: @@ -2269,6 +2287,7 @@ nvlist_common(nvlist_t *nvl, char *buf, size_t *buflen return (EINVAL); nvs.nvs_op = nvs_op; + nvs.nvs_recursion = 0; /* * For NVS_OP_ENCODE and NVS_OP_DECODE make sure an nvlist and From owner-svn-src-all@freebsd.org Fri Aug 3 01:09:13 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C5391057461; Fri, 3 Aug 2018 01:09:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31E2288A63; Fri, 3 Aug 2018 01:09:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0ED10197F3; Fri, 3 Aug 2018 01:09:13 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7319CtD065864; Fri, 3 Aug 2018 01:09:12 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7319CHW065863; Fri, 3 Aug 2018 01:09:12 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030109.w7319CHW065863@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:09:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337217 - head/sys/cddl/contrib/opensolaris/common/nvpair X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/common/nvpair X-SVN-Commit-Revision: 337217 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:09:13 -0000 Author: mav Date: Fri Aug 3 01:09:12 2018 New Revision: 337217 URL: https://svnweb.freebsd.org/changeset/base/337217 Log: MFV r337216: 7263 deeply nested nvlist can overflow stack illumos/illumos-gate@9ca527c3d3dfa7c8f304b34a9e03b5eddace838f Reviewed by: Adam Leventhal Reviewed by: George Wilson Reviewed by: Robert Mustacchi Approved by: Dan McDonald Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 00:47:24 2018 (r337216) +++ head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 01:09:12 2018 (r337217) @@ -21,6 +21,7 @@ /* * Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2015, 2016 by Delphix. All rights reserved. */ #include @@ -142,6 +143,11 @@ static int nvlist_add_common(nvlist_t *nvl, const char #define NVPAIR2I_NVP(nvp) \ ((i_nvp_t *)((size_t)(nvp) - offsetof(i_nvp_t, nvi_nvp))) +#ifdef _KERNEL +int nvpair_max_recursion = 20; +#else +int nvpair_max_recursion = 100; +#endif int nv_alloc_init(nv_alloc_t *nva, const nv_alloc_ops_t *nvo, /* args */ ...) @@ -2018,6 +2024,7 @@ typedef struct { const nvs_ops_t *nvs_ops; void *nvs_private; nvpriv_t *nvs_priv; + int nvs_recursion; } nvstream_t; /* @@ -2169,9 +2176,16 @@ static int nvs_embedded(nvstream_t *nvs, nvlist_t *embedded) { switch (nvs->nvs_op) { - case NVS_OP_ENCODE: - return (nvs_operation(nvs, embedded, NULL)); + case NVS_OP_ENCODE: { + int err; + if (nvs->nvs_recursion >= nvpair_max_recursion) + return (EINVAL); + nvs->nvs_recursion++; + err = nvs_operation(nvs, embedded, NULL); + nvs->nvs_recursion--; + return (err); + } case NVS_OP_DECODE: { nvpriv_t *priv; int err; @@ -2184,8 +2198,12 @@ nvs_embedded(nvstream_t *nvs, nvlist_t *embedded) nvlist_init(embedded, embedded->nvl_nvflag, priv); + if (nvs->nvs_recursion >= nvpair_max_recursion) + return (EINVAL); + nvs->nvs_recursion++; if ((err = nvs_operation(nvs, embedded, NULL)) != 0) nvlist_free(embedded); + nvs->nvs_recursion--; return (err); } default: @@ -2273,6 +2291,7 @@ nvlist_common(nvlist_t *nvl, char *buf, size_t *buflen return (EINVAL); nvs.nvs_op = nvs_op; + nvs.nvs_recursion = 0; /* * For NVS_OP_ENCODE and NVS_OP_DECODE make sure an nvlist and From owner-svn-src-all@freebsd.org Fri Aug 3 01:17:15 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F28410578FD; Fri, 3 Aug 2018 01:17:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 420A888F9F; Fri, 3 Aug 2018 01:17:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1E34719988; Fri, 3 Aug 2018 01:17:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731HEFp070868; Fri, 3 Aug 2018 01:17:14 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731HE1Q070867; Fri, 3 Aug 2018 01:17:14 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030117.w731HE1Q070867@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:17:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337218 - vendor-sys/illumos/dist/common/nvpair X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/nvpair X-SVN-Commit-Revision: 337218 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:17:15 -0000 Author: mav Date: Fri Aug 3 01:17:14 2018 New Revision: 337218 URL: https://svnweb.freebsd.org/changeset/base/337218 Log: 7261 nvlist code should enforce name length limit illumos/illumos-gate@48dd5e630c9b1773b7b10d08a3b90b6c9062d713 Reviewed by: Sebastien Roy Reviewed by: George Wilson Reviewed by: Robert Mustacchi Approved by: Dan McDonald Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c ============================================================================== --- vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 01:09:12 2018 (r337217) +++ vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 01:17:14 2018 (r337218) @@ -910,6 +910,8 @@ nvlist_add_common(nvlist_t *nvl, const char *name, /* calculate sizes of the nvpair elements and the nvpair itself */ name_sz = strlen(name) + 1; + if (name_sz >= 1ULL << (sizeof (nvp->nvp_name_sz) * NBBY - 1)) + return (EINVAL); nvp_sz = NVP_SIZE_CALC(name_sz, value_sz); From owner-svn-src-all@freebsd.org Fri Aug 3 01:26:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A2421057C36; Fri, 3 Aug 2018 01:26:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5030889478; Fri, 3 Aug 2018 01:26:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2CD3819B36; Fri, 3 Aug 2018 01:26:08 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731Q82X075931; Fri, 3 Aug 2018 01:26:08 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731Q8Eg075930; Fri, 3 Aug 2018 01:26:08 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030126.w731Q8Eg075930@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:26:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337219 - head/sys/cddl/contrib/opensolaris/common/nvpair X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/common/nvpair X-SVN-Commit-Revision: 337219 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:26:08 -0000 Author: mav Date: Fri Aug 3 01:26:07 2018 New Revision: 337219 URL: https://svnweb.freebsd.org/changeset/base/337219 Log: MFV r337218: 7261 nvlist code should enforce name length limit illumos/illumos-gate@48dd5e630c9b1773b7b10d08a3b90b6c9062d713 Reviewed by: Sebastien Roy Reviewed by: George Wilson Reviewed by: Robert Mustacchi Approved by: Dan McDonald Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 01:17:14 2018 (r337218) +++ head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 01:26:07 2018 (r337219) @@ -914,6 +914,8 @@ nvlist_add_common(nvlist_t *nvl, const char *name, /* calculate sizes of the nvpair elements and the nvpair itself */ name_sz = strlen(name) + 1; + if (name_sz >= 1ULL << (sizeof (nvp->nvp_name_sz) * 8 - 1)) + return (EINVAL); nvp_sz = NVP_SIZE_CALC(name_sz, value_sz); From owner-svn-src-all@freebsd.org Fri Aug 3 01:28:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A040D1057CE9; Fri, 3 Aug 2018 01:28:12 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56843895DF; Fri, 3 Aug 2018 01:28:12 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3368719B37; Fri, 3 Aug 2018 01:28:12 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731SBK3076054; Fri, 3 Aug 2018 01:28:11 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731SBjN076053; Fri, 3 Aug 2018 01:28:11 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030128.w731SBjN076053@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:28:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337220 - vendor-sys/illumos/dist/common/nvpair X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/nvpair X-SVN-Commit-Revision: 337220 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:28:12 -0000 Author: mav Date: Fri Aug 3 01:28:11 2018 New Revision: 337220 URL: https://svnweb.freebsd.org/changeset/base/337220 Log: 8375 Kernel memory leak in nvpair code illumos/illumos-gate@843c2111b160463f014d325560ad4b051711928e Reviewed by: Pavel Zakharov Reviewed by: George Wilson Reviewed by: Prashanth Sreenivasa Reviewed by: Robert Mustacchi Approved by: Dan McDonald Author: Matthew Ahrens Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c ============================================================================== --- vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 01:26:07 2018 (r337219) +++ vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 01:28:11 2018 (r337220) @@ -21,7 +21,7 @@ /* * Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2015, 2016 by Delphix. All rights reserved. + * Copyright (c) 2015, 2017 by Delphix. All rights reserved. */ #include @@ -2196,8 +2196,10 @@ nvs_embedded(nvstream_t *nvs, nvlist_t *embedded) nvlist_init(embedded, embedded->nvl_nvflag, priv); - if (nvs->nvs_recursion >= nvpair_max_recursion) + if (nvs->nvs_recursion >= nvpair_max_recursion) { + nvlist_free(embedded); return (EINVAL); + } nvs->nvs_recursion++; if ((err = nvs_operation(nvs, embedded, NULL)) != 0) nvlist_free(embedded); From owner-svn-src-all@freebsd.org Fri Aug 3 01:30:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A5A1F1057DA5; Fri, 3 Aug 2018 01:30:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5915789730; Fri, 3 Aug 2018 01:30:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3A45319B3C; Fri, 3 Aug 2018 01:30:04 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731U401076214; Fri, 3 Aug 2018 01:30:04 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731U46d076213; Fri, 3 Aug 2018 01:30:04 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030130.w731U46d076213@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:30:04 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337221 - head/sys/cddl/contrib/opensolaris/common/nvpair X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: head/sys/cddl/contrib/opensolaris/common/nvpair X-SVN-Commit-Revision: 337221 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:30:04 -0000 Author: mav Date: Fri Aug 3 01:30:03 2018 New Revision: 337221 URL: https://svnweb.freebsd.org/changeset/base/337221 Log: MFV r337220: 8375 Kernel memory leak in nvpair code illumos/illumos-gate@843c2111b160463f014d325560ad4b051711928e Reviewed by: Pavel Zakharov Reviewed by: George Wilson Reviewed by: Prashanth Sreenivasa Reviewed by: Robert Mustacchi Approved by: Dan McDonald Author: Matthew Ahrens Modified: head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Directory Properties: head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 01:28:11 2018 (r337220) +++ head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 01:30:03 2018 (r337221) @@ -21,7 +21,7 @@ /* * Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2015, 2016 by Delphix. All rights reserved. + * Copyright (c) 2015, 2017 by Delphix. All rights reserved. */ #include @@ -2200,8 +2200,10 @@ nvs_embedded(nvstream_t *nvs, nvlist_t *embedded) nvlist_init(embedded, embedded->nvl_nvflag, priv); - if (nvs->nvs_recursion >= nvpair_max_recursion) + if (nvs->nvs_recursion >= nvpair_max_recursion) { + nvlist_free(embedded); return (EINVAL); + } nvs->nvs_recursion++; if ((err = nvs_operation(nvs, embedded, NULL)) != 0) nvlist_free(embedded); From owner-svn-src-all@freebsd.org Fri Aug 3 01:37:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01C541057FFD; Fri, 3 Aug 2018 01:37:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9906989B5E; Fri, 3 Aug 2018 01:37:01 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7489419CC8; Fri, 3 Aug 2018 01:37:01 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731b1CW081230; Fri, 3 Aug 2018 01:37:01 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731b0bB081225; Fri, 3 Aug 2018 01:37:00 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808030137.w731b0bB081225@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 01:37:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337222 - in head: sys/kern sys/sys tests/sys/kern X-SVN-Group: head X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in head: sys/kern sys/sys tests/sys/kern X-SVN-Commit-Revision: 337222 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:37:02 -0000 Author: asomers Date: Fri Aug 3 01:37:00 2018 New Revision: 337222 URL: https://svnweb.freebsd.org/changeset/base/337222 Log: Fix LOCAL_PEERCRED with socketpair(2) Enable the LOCAL_PEERCRED socket option for unix domain stream sockets created with socketpair(2). Previously, it only worked with unix domain stream sockets created with socket(2)/listen(2)/connect(2)/accept(2). PR: 176419 Reported by: Nicholas Wilson MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D16350 Added: head/tests/sys/kern/unix_socketpair_test.c (contents, props changed) Modified: head/sys/kern/uipc_syscalls.c head/sys/kern/uipc_usrreq.c head/sys/sys/unpcb.h head/tests/sys/kern/Makefile Modified: head/sys/kern/uipc_syscalls.c ============================================================================== --- head/sys/kern/uipc_syscalls.c Fri Aug 3 01:30:03 2018 (r337221) +++ head/sys/kern/uipc_syscalls.c Fri Aug 3 01:37:00 2018 (r337222) @@ -58,6 +58,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #ifdef KTRACE #include #endif @@ -612,6 +614,15 @@ kern_socketpair(struct thread *td, int domain, int typ error = soconnect2(so2, so1); if (error != 0) goto free4; + } else if (so1->so_proto->pr_flags & PR_CONNREQUIRED) { + struct unpcb *unp, *unp2; + unp = sotounpcb(so1); + unp2 = sotounpcb(so2); + /* + * No need to lock the unps, because the sockets are brand-new. + * No other threads can be using them yet + */ + unp_copy_peercred(td, unp, unp2, unp); } finit(fp1, FREAD | FWRITE | fflag, DTYPE_SOCKET, fp1->f_data, &socketops); Modified: head/sys/kern/uipc_usrreq.c ============================================================================== --- head/sys/kern/uipc_usrreq.c Fri Aug 3 01:30:03 2018 (r337221) +++ head/sys/kern/uipc_usrreq.c Fri Aug 3 01:37:00 2018 (r337222) @@ -1609,24 +1609,8 @@ unp_connectat(int fd, struct socket *so, struct sockad sa = NULL; } - /* - * The connector's (client's) credentials are copied from its - * process structure at the time of connect() (which is now). - */ - cru2x(td->td_ucred, &unp3->unp_peercred); - unp3->unp_flags |= UNP_HAVEPC; + unp_copy_peercred(td, unp3, unp, unp2); - /* - * The receiver's (server's) credentials are copied from the - * unp_peercred member of socket on which the former called - * listen(); uipc_listen() cached that process's credentials - * at that time so we can use them now. - */ - memcpy(&unp->unp_peercred, &unp2->unp_peercred, - sizeof(unp->unp_peercred)); - unp->unp_flags |= UNP_HAVEPC; - if (unp2->unp_flags & UNP_WANTCRED) - unp3->unp_flags |= UNP_WANTCRED; UNP_PCB_UNLOCK(unp2); unp2 = unp3; unp_pcb_owned_lock2(unp2, unp, freed); @@ -1663,6 +1647,27 @@ bad: unp->unp_flags &= ~UNP_CONNECTING; UNP_PCB_UNLOCK(unp); return (error); +} + +/* + * Set socket peer credentials at connection time. + * + * The client's PCB credentials are copied from its process structure. The + * server's PCB credentials are copied from the socket on which it called + * listen(2). uipc_listen cached that process's credentials at the time. + */ +void +unp_copy_peercred(struct thread *td, struct unpcb *client_unp, + struct unpcb *server_unp, struct unpcb *listen_unp) +{ + cru2x(td->td_ucred, &client_unp->unp_peercred); + client_unp->unp_flags |= UNP_HAVEPC; + + memcpy(&server_unp->unp_peercred, &listen_unp->unp_peercred, + sizeof(server_unp->unp_peercred)); + server_unp->unp_flags |= UNP_HAVEPC; + if (listen_unp->unp_flags & UNP_WANTCRED) + client_unp->unp_flags |= UNP_WANTCRED; } static int Modified: head/sys/sys/unpcb.h ============================================================================== --- head/sys/sys/unpcb.h Fri Aug 3 01:30:03 2018 (r337221) +++ head/sys/sys/unpcb.h Fri Aug 3 01:37:00 2018 (r337222) @@ -166,4 +166,13 @@ struct xunpgen { } __aligned(8);; #endif /* _SYS_SOCKETVAR_H_ */ +#if defined(_KERNEL) +struct thread; + +/* In uipc_userreq.c */ +void +unp_copy_peercred(struct thread *td, struct unpcb *client_unp, + struct unpcb *server_unp, struct unpcb *listen_unp); +#endif + #endif /* _SYS_UNPCB_H_ */ Modified: head/tests/sys/kern/Makefile ============================================================================== --- head/tests/sys/kern/Makefile Fri Aug 3 01:30:03 2018 (r337221) +++ head/tests/sys/kern/Makefile Fri Aug 3 01:37:00 2018 (r337222) @@ -12,9 +12,10 @@ TEST_METADATA.ptrace_test+= timeout="15" ATF_TESTS_C+= reaper PLAIN_TESTS_C+= subr_unit_test ATF_TESTS_C+= sys_getrandom -ATF_TESTS_C+= unix_seqpacket_test ATF_TESTS_C+= unix_passfd_test +ATF_TESTS_C+= unix_seqpacket_test TEST_METADATA.unix_seqpacket_test+= timeout="15" +ATF_TESTS_C+= unix_socketpair_test ATF_TESTS_C+= waitpid_nohang ATF_TESTS_C+= pdeathsig Added: head/tests/sys/kern/unix_socketpair_test.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/tests/sys/kern/unix_socketpair_test.c Fri Aug 3 01:37:00 2018 (r337222) @@ -0,0 +1,76 @@ +/*- + * Copyright (c) 2018 Alan Somers + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +__FBSDID("$FreeBSD$"); + +#include +#include +#include +#include +#include +#include + +#include + +#include + +/* getpeereid(3) should work with stream sockets created via socketpair(2) */ +ATF_TC_WITHOUT_HEAD(getpeereid); +ATF_TC_BODY(getpeereid, tc) +{ + int sv[2]; + int s; + uid_t real_euid, euid; + gid_t real_egid, egid; + + real_euid = geteuid(); + real_egid = getegid(); + + s = socketpair(PF_LOCAL, SOCK_STREAM, 0, sv); + ATF_CHECK_EQ(0, s); + ATF_CHECK(sv[0] >= 0); + ATF_CHECK(sv[1] >= 0); + ATF_CHECK(sv[0] != sv[1]); + + ATF_REQUIRE_EQ(0, getpeereid(sv[0], &euid, &egid)); + ATF_CHECK_EQ(real_euid, euid); + ATF_CHECK_EQ(real_egid, egid); + + ATF_REQUIRE_EQ(0, getpeereid(sv[1], &euid, &egid)); + ATF_CHECK_EQ(real_euid, euid); + ATF_CHECK_EQ(real_egid, egid); + + close(sv[0]); + close(sv[1]); +} + + +ATF_TP_ADD_TCS(tp) +{ + ATF_TP_ADD_TC(tp, getpeereid); + + return atf_no_error(); +} From owner-svn-src-all@freebsd.org Fri Aug 3 01:37:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C66ED105801E; Fri, 3 Aug 2018 01:37:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A72989C6D; Fri, 3 Aug 2018 01:37:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 42D7219CCE; Fri, 3 Aug 2018 01:37:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731bF23081298; Fri, 3 Aug 2018 01:37:15 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731bEir081295; Fri, 3 Aug 2018 01:37:14 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030137.w731bEir081295@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:37:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337223 - vendor-sys/illumos/dist/common/nvpair vendor-sys/illumos/dist/uts/common/sys vendor/illumos/dist/lib/libnvpair X-SVN-Group: vendor-sys X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/nvpair vendor-sys/illumos/dist/uts/common/sys vendor/illumos/dist/lib/libnvpair X-SVN-Commit-Revision: 337223 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:37:16 -0000 Author: mav Date: Fri Aug 3 01:37:14 2018 New Revision: 337223 URL: https://svnweb.freebsd.org/changeset/base/337223 Log: 9580 Add a hash-table on top of nvlist to speed-up operations illumos/illumos-gate@2ec7644aab2a726a64681fa66c6db8731b160de1 Reviewed by: Matt Ahrens Reviewed by: Sebastien Roy Approved by: Robert Mustacchi Author: Serapheim Dimitropoulos Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c vendor-sys/illumos/dist/uts/common/sys/nvpair.h vendor-sys/illumos/dist/uts/common/sys/nvpair_impl.h Changes in other areas also in this revision: Modified: vendor/illumos/dist/lib/libnvpair/nvpair_json.c Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c ============================================================================== --- vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 01:37:00 2018 (r337222) +++ vendor-sys/illumos/dist/common/nvpair/nvpair.c Fri Aug 3 01:37:14 2018 (r337223) @@ -145,6 +145,8 @@ int nvpair_max_recursion = 20; int nvpair_max_recursion = 100; #endif +uint64_t nvlist_hashtable_init_size = (1 << 4); + int nv_alloc_init(nv_alloc_t *nva, const nv_alloc_ops_t *nvo, /* args */ ...) { @@ -252,7 +254,292 @@ nv_priv_alloc_embedded(nvpriv_t *priv) return (emb_priv); } +static int +nvt_tab_alloc(nvpriv_t *priv, uint64_t buckets) +{ + ASSERT3P(priv->nvp_hashtable, ==, NULL); + ASSERT0(priv->nvp_nbuckets); + ASSERT0(priv->nvp_nentries); + + i_nvp_t **tab = nv_mem_zalloc(priv, buckets * sizeof (i_nvp_t *)); + if (tab == NULL) + return (ENOMEM); + + priv->nvp_hashtable = tab; + priv->nvp_nbuckets = buckets; + return (0); +} + static void +nvt_tab_free(nvpriv_t *priv) +{ + i_nvp_t **tab = priv->nvp_hashtable; + if (tab == NULL) { + ASSERT0(priv->nvp_nbuckets); + ASSERT0(priv->nvp_nentries); + return; + } + + nv_mem_free(priv, tab, priv->nvp_nbuckets * sizeof (i_nvp_t *)); + + priv->nvp_hashtable = NULL; + priv->nvp_nbuckets = 0; + priv->nvp_nentries = 0; +} + +static uint32_t +nvt_hash(const char *p) +{ + uint32_t g, hval = 0; + + while (*p) { + hval = (hval << 4) + *p++; + if ((g = (hval & 0xf0000000)) != 0) + hval ^= g >> 24; + hval &= ~g; + } + return (hval); +} + +static boolean_t +nvt_nvpair_match(nvpair_t *nvp1, nvpair_t *nvp2, uint32_t nvflag) +{ + boolean_t match = B_FALSE; + if (nvflag & NV_UNIQUE_NAME_TYPE) { + if (strcmp(NVP_NAME(nvp1), NVP_NAME(nvp2)) == 0 && + NVP_TYPE(nvp1) == NVP_TYPE(nvp2)) + match = B_TRUE; + } else { + ASSERT(nvflag == 0 || nvflag & NV_UNIQUE_NAME); + if (strcmp(NVP_NAME(nvp1), NVP_NAME(nvp2)) == 0) + match = B_TRUE; + } + return (match); +} + +static nvpair_t * +nvt_lookup_name_type(nvlist_t *nvl, const char *name, data_type_t type) +{ + nvpriv_t *priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv; + ASSERT(priv != NULL); + + i_nvp_t **tab = priv->nvp_hashtable; + + if (tab == NULL) { + ASSERT3P(priv->nvp_list, ==, NULL); + ASSERT0(priv->nvp_nbuckets); + ASSERT0(priv->nvp_nentries); + return (NULL); + } else { + ASSERT(priv->nvp_nbuckets != 0); + } + + uint64_t hash = nvt_hash(name); + uint64_t index = hash & (priv->nvp_nbuckets - 1); + + ASSERT3U(index, <, priv->nvp_nbuckets); + i_nvp_t *entry = tab[index]; + + for (i_nvp_t *e = entry; e != NULL; e = e->nvi_hashtable_next) { + if (strcmp(NVP_NAME(&e->nvi_nvp), name) == 0 && + (type == DATA_TYPE_DONTCARE || + NVP_TYPE(&e->nvi_nvp) == type)) + return (&e->nvi_nvp); + } + return (NULL); +} + +static nvpair_t * +nvt_lookup_name(nvlist_t *nvl, const char *name) +{ + return (nvt_lookup_name_type(nvl, name, DATA_TYPE_DONTCARE)); +} + +static int +nvt_resize(nvpriv_t *priv, uint32_t new_size) +{ + i_nvp_t **tab = priv->nvp_hashtable; + + /* + * Migrate all the entries from the current table + * to a newly-allocated table with the new size by + * re-adjusting the pointers of their entries. + */ + uint32_t size = priv->nvp_nbuckets; + uint32_t new_mask = new_size - 1; + ASSERT(ISP2(new_size)); + + i_nvp_t **new_tab = nv_mem_zalloc(priv, new_size * sizeof (i_nvp_t *)); + if (new_tab == NULL) + return (ENOMEM); + + uint32_t nentries = 0; + for (uint32_t i = 0; i < size; i++) { + i_nvp_t *next, *e = tab[i]; + + while (e != NULL) { + next = e->nvi_hashtable_next; + + uint32_t hash = nvt_hash(NVP_NAME(&e->nvi_nvp)); + uint32_t index = hash & new_mask; + + e->nvi_hashtable_next = new_tab[index]; + new_tab[index] = e; + nentries++; + + e = next; + } + tab[i] = NULL; + } + ASSERT3U(nentries, ==, priv->nvp_nentries); + + nvt_tab_free(priv); + + priv->nvp_hashtable = new_tab; + priv->nvp_nbuckets = new_size; + priv->nvp_nentries = nentries; + + return (0); +} + +static boolean_t +nvt_needs_togrow(nvpriv_t *priv) +{ + /* + * Grow only when we have more elements than buckets + * and the # of buckets doesn't overflow. + */ + return (priv->nvp_nentries > priv->nvp_nbuckets && + (UINT32_MAX >> 1) >= priv->nvp_nbuckets); +} + +/* + * Allocate a new table that's twice the size of the old one, + * and migrate all the entries from the old one to the new + * one by re-adjusting their pointers. + */ +static int +nvt_grow(nvpriv_t *priv) +{ + uint32_t current_size = priv->nvp_nbuckets; + /* ensure we won't overflow */ + ASSERT3U(UINT32_MAX >> 1, >=, current_size); + return (nvt_resize(priv, current_size << 1)); +} + +static boolean_t +nvt_needs_toshrink(nvpriv_t *priv) +{ + /* + * Shrink only when the # of elements is less than or + * equal to 1/4 the # of buckets. Never shrink less than + * nvlist_hashtable_init_size. + */ + ASSERT3U(priv->nvp_nbuckets, >=, nvlist_hashtable_init_size); + if (priv->nvp_nbuckets == nvlist_hashtable_init_size) + return (B_FALSE); + return (priv->nvp_nentries <= (priv->nvp_nbuckets >> 2)); +} + +/* + * Allocate a new table that's half the size of the old one, + * and migrate all the entries from the old one to the new + * one by re-adjusting their pointers. + */ +static int +nvt_shrink(nvpriv_t *priv) +{ + uint32_t current_size = priv->nvp_nbuckets; + /* ensure we won't overflow */ + ASSERT3U(current_size, >=, nvlist_hashtable_init_size); + return (nvt_resize(priv, current_size >> 1)); +} + +static int +nvt_remove_nvpair(nvlist_t *nvl, nvpair_t *nvp) +{ + nvpriv_t *priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv; + + if (nvt_needs_toshrink(priv)) { + int err = nvt_shrink(priv); + if (err != 0) + return (err); + } + i_nvp_t **tab = priv->nvp_hashtable; + + char *name = NVP_NAME(nvp); + uint64_t hash = nvt_hash(name); + uint64_t index = hash & (priv->nvp_nbuckets - 1); + + ASSERT3U(index, <, priv->nvp_nbuckets); + i_nvp_t *bucket = tab[index]; + + for (i_nvp_t *prev = NULL, *e = bucket; + e != NULL; prev = e, e = e->nvi_hashtable_next) { + if (nvt_nvpair_match(&e->nvi_nvp, nvp, nvl->nvl_flag)) { + if (prev != NULL) { + prev->nvi_hashtable_next = + e->nvi_hashtable_next; + } else { + ASSERT3P(e, ==, bucket); + tab[index] = e->nvi_hashtable_next; + } + e->nvi_hashtable_next = NULL; + priv->nvp_nentries--; + break; + } + } + + return (0); +} + +static int +nvt_add_nvpair(nvlist_t *nvl, nvpair_t *nvp) +{ + nvpriv_t *priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv; + + /* initialize nvpair table now if it doesn't exist. */ + if (priv->nvp_hashtable == NULL) { + int err = nvt_tab_alloc(priv, nvlist_hashtable_init_size); + if (err != 0) + return (err); + } + + /* + * if we don't allow duplicate entries, make sure to + * unlink any existing entries from the table. + */ + if (nvl->nvl_nvflag != 0) { + int err = nvt_remove_nvpair(nvl, nvp); + if (err != 0) + return (err); + } + + if (nvt_needs_togrow(priv)) { + int err = nvt_grow(priv); + if (err != 0) + return (err); + } + i_nvp_t **tab = priv->nvp_hashtable; + + char *name = NVP_NAME(nvp); + uint64_t hash = nvt_hash(name); + uint64_t index = hash & (priv->nvp_nbuckets - 1); + + ASSERT3U(index, <, priv->nvp_nbuckets); + i_nvp_t *bucket = tab[index]; + + /* insert link at the beginning of the bucket */ + i_nvp_t *new_entry = NVPAIR2I_NVP(nvp); + ASSERT3P(new_entry->nvi_hashtable_next, ==, NULL); + new_entry->nvi_hashtable_next = bucket; + tab[index] = new_entry; + + priv->nvp_nentries++; + return (0); +} + +static void nvlist_init(nvlist_t *nvl, uint32_t nvflag, nvpriv_t *priv) { nvl->nvl_version = NV_VERSION; @@ -584,6 +871,7 @@ nvlist_free(nvlist_t *nvl) else nvl->nvl_priv = 0; + nvt_tab_free(priv); nv_mem_free(priv, priv, sizeof (nvpriv_t)); } @@ -644,26 +932,14 @@ nvlist_xdup(nvlist_t *nvl, nvlist_t **nvlp, nv_alloc_t int nvlist_remove_all(nvlist_t *nvl, const char *name) { - nvpriv_t *priv; - i_nvp_t *curr; int error = ENOENT; - if (nvl == NULL || name == NULL || - (priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv) == NULL) + if (nvl == NULL || name == NULL || nvl->nvl_priv == 0) return (EINVAL); - curr = priv->nvp_list; - while (curr != NULL) { - nvpair_t *nvp = &curr->nvi_nvp; - - curr = curr->nvi_next; - if (strcmp(name, NVP_NAME(nvp)) != 0) - continue; - - nvp_buf_unlink(nvl, nvp); - nvpair_free(nvp); - nvp_buf_free(nvl, nvp); - + nvpair_t *nvp; + while ((nvp = nvt_lookup_name(nvl, name)) != NULL) { + VERIFY0(nvlist_remove_nvpair(nvl, nvp)); error = 0; } @@ -676,28 +952,14 @@ nvlist_remove_all(nvlist_t *nvl, const char *name) int nvlist_remove(nvlist_t *nvl, const char *name, data_type_t type) { - nvpriv_t *priv; - i_nvp_t *curr; - - if (nvl == NULL || name == NULL || - (priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv) == NULL) + if (nvl == NULL || name == NULL || nvl->nvl_priv == 0) return (EINVAL); - curr = priv->nvp_list; - while (curr != NULL) { - nvpair_t *nvp = &curr->nvi_nvp; + nvpair_t *nvp = nvt_lookup_name_type(nvl, name, type); + if (nvp == NULL) + return (ENOENT); - if (strcmp(name, NVP_NAME(nvp)) == 0 && NVP_TYPE(nvp) == type) { - nvp_buf_unlink(nvl, nvp); - nvpair_free(nvp); - nvp_buf_free(nvl, nvp); - - return (0); - } - curr = curr->nvi_next; - } - - return (ENOENT); + return (nvlist_remove_nvpair(nvl, nvp)); } int @@ -706,6 +968,10 @@ nvlist_remove_nvpair(nvlist_t *nvl, nvpair_t *nvp) if (nvl == NULL || nvp == NULL) return (EINVAL); + int err = nvt_remove_nvpair(nvl, nvp); + if (err != 0) + return (err); + nvp_buf_unlink(nvl, nvp); nvpair_free(nvp); nvp_buf_free(nvl, nvp); @@ -983,6 +1249,12 @@ nvlist_add_common(nvlist_t *nvl, const char *name, else if (nvl->nvl_nvflag & NV_UNIQUE_NAME_TYPE) (void) nvlist_remove(nvl, name, type); + err = nvt_add_nvpair(nvl, nvp); + if (err != 0) { + nvpair_free(nvp); + nvp_buf_free(nvl, nvp); + return (err); + } nvp_buf_link(nvl, nvp); return (0); @@ -1332,25 +1604,17 @@ static int nvlist_lookup_common(nvlist_t *nvl, const char *name, data_type_t type, uint_t *nelem, void *data) { - nvpriv_t *priv; - nvpair_t *nvp; - i_nvp_t *curr; - - if (name == NULL || nvl == NULL || - (priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv) == NULL) + if (name == NULL || nvl == NULL || nvl->nvl_priv == 0) return (EINVAL); if (!(nvl->nvl_nvflag & (NV_UNIQUE_NAME | NV_UNIQUE_NAME_TYPE))) return (ENOTSUP); - for (curr = priv->nvp_list; curr != NULL; curr = curr->nvi_next) { - nvp = &curr->nvi_nvp; + nvpair_t *nvp = nvt_lookup_name_type(nvl, name, type); + if (nvp == NULL) + return (ENOENT); - if (strcmp(name, NVP_NAME(nvp)) == 0 && NVP_TYPE(nvp) == type) - return (nvpair_value_common(nvp, type, nelem, data)); - } - - return (ENOENT); + return (nvpair_value_common(nvp, type, nelem, data)); } int @@ -2108,6 +2372,12 @@ nvs_decode_pairs(nvstream_t *nvs, nvlist_t *nvl) return (EFAULT); } + err = nvt_add_nvpair(nvl, nvp); + if (err != 0) { + nvpair_free(nvp); + nvp_buf_free(nvl, nvp); + return (err); + } nvp_buf_link(nvl, nvp); } return (err); Modified: vendor-sys/illumos/dist/uts/common/sys/nvpair.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/nvpair.h Fri Aug 3 01:37:00 2018 (r337222) +++ vendor-sys/illumos/dist/uts/common/sys/nvpair.h Fri Aug 3 01:37:14 2018 (r337223) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. */ #ifndef _SYS_NVPAIR_H @@ -40,6 +40,7 @@ extern "C" { #endif typedef enum { + DATA_TYPE_DONTCARE = -1, DATA_TYPE_UNKNOWN = 0, DATA_TYPE_BOOLEAN, DATA_TYPE_BYTE, Modified: vendor-sys/illumos/dist/uts/common/sys/nvpair_impl.h ============================================================================== --- vendor-sys/illumos/dist/uts/common/sys/nvpair_impl.h Fri Aug 3 01:37:00 2018 (r337222) +++ vendor-sys/illumos/dist/uts/common/sys/nvpair_impl.h Fri Aug 3 01:37:14 2018 (r337223) @@ -24,11 +24,13 @@ * Use is subject to license terms. */ +/* + * Copyright (c) 2017 by Delphix. All rights reserved. + */ + #ifndef _NVPAIR_IMPL_H #define _NVPAIR_IMPL_H -#pragma ident "%Z%%M% %I% %E% SMI" - #ifdef __cplusplus extern "C" { #endif @@ -47,16 +49,27 @@ typedef struct i_nvp i_nvp_t; struct i_nvp { union { - uint64_t _nvi_align; /* ensure alignment */ + /* ensure alignment */ + uint64_t _nvi_align; + struct { - i_nvp_t *_nvi_next; /* pointer to next nvpair */ - i_nvp_t *_nvi_prev; /* pointer to prev nvpair */ + /* pointer to next nvpair */ + i_nvp_t *_nvi_next; + + /* pointer to prev nvpair */ + i_nvp_t *_nvi_prev; + + /* next pair in table bucket */ + i_nvp_t *_nvi_hashtable_next; } _nvi; } _nvi_un; - nvpair_t nvi_nvp; /* nvpair */ + + /* nvpair */ + nvpair_t nvi_nvp; }; #define nvi_next _nvi_un._nvi._nvi_next #define nvi_prev _nvi_un._nvi._nvi_prev +#define nvi_hashtable_next _nvi_un._nvi._nvi_hashtable_next typedef struct { i_nvp_t *nvp_list; /* linked list of nvpairs */ @@ -64,6 +77,10 @@ typedef struct { i_nvp_t *nvp_curr; /* current walker nvpair */ nv_alloc_t *nvp_nva; /* pluggable allocator */ uint32_t nvp_stat; /* internal state */ + + i_nvp_t **nvp_hashtable; /* table of entries used for lookup */ + uint32_t nvp_nbuckets; /* # of buckets in hash table */ + uint32_t nvp_nentries; /* # of entries in hash table */ } nvpriv_t; #ifdef __cplusplus From owner-svn-src-all@freebsd.org Fri Aug 3 01:37:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09CD3105801F; Fri, 3 Aug 2018 01:37:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B20F889C6E; Fri, 3 Aug 2018 01:37:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 94AAF19CCF; Fri, 3 Aug 2018 01:37:15 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731bFXp081305; Fri, 3 Aug 2018 01:37:15 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731bFvx081304; Fri, 3 Aug 2018 01:37:15 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030137.w731bFvx081304@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:37:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337223 - vendor-sys/illumos/dist/common/nvpair vendor-sys/illumos/dist/uts/common/sys vendor/illumos/dist/lib/libnvpair X-SVN-Group: vendor X-SVN-Commit-Author: mav X-SVN-Commit-Paths: vendor-sys/illumos/dist/common/nvpair vendor-sys/illumos/dist/uts/common/sys vendor/illumos/dist/lib/libnvpair X-SVN-Commit-Revision: 337223 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:37:16 -0000 Author: mav Date: Fri Aug 3 01:37:14 2018 New Revision: 337223 URL: https://svnweb.freebsd.org/changeset/base/337223 Log: 9580 Add a hash-table on top of nvlist to speed-up operations illumos/illumos-gate@2ec7644aab2a726a64681fa66c6db8731b160de1 Reviewed by: Matt Ahrens Reviewed by: Sebastien Roy Approved by: Robert Mustacchi Author: Serapheim Dimitropoulos Modified: vendor/illumos/dist/lib/libnvpair/nvpair_json.c Changes in other areas also in this revision: Modified: vendor-sys/illumos/dist/common/nvpair/nvpair.c vendor-sys/illumos/dist/uts/common/sys/nvpair.h vendor-sys/illumos/dist/uts/common/sys/nvpair_impl.h Modified: vendor/illumos/dist/lib/libnvpair/nvpair_json.c ============================================================================== --- vendor/illumos/dist/lib/libnvpair/nvpair_json.c Fri Aug 3 01:37:00 2018 (r337222) +++ vendor/illumos/dist/lib/libnvpair/nvpair_json.c Fri Aug 3 01:37:14 2018 (r337223) @@ -10,6 +10,7 @@ */ /* * Copyright (c) 2014, Joyent, Inc. + * Copyright (c) 2017 by Delphix. All rights reserved. */ #include @@ -394,8 +395,10 @@ nvlist_print_json(FILE *fp, nvlist_t *nvl) } case DATA_TYPE_UNKNOWN: + case DATA_TYPE_DONTCARE: return (-1); } + } FPRINTF(fp, "}"); From owner-svn-src-all@freebsd.org Fri Aug 3 01:52:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3B651058B7E; Fri, 3 Aug 2018 01:52:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97F0D8AC15; Fri, 3 Aug 2018 01:52:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 78CEE19FF1; Fri, 3 Aug 2018 01:52:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w731qQDE091271; Fri, 3 Aug 2018 01:52:26 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w731qPpL091267; Fri, 3 Aug 2018 01:52:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030152.w731qPpL091267@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 01:52:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337227 - in head: cddl/contrib/opensolaris/lib/libnvpair sys/cddl/contrib/opensolaris/common/nvpair sys/cddl/contrib/opensolaris/uts/common/sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head: cddl/contrib/opensolaris/lib/libnvpair sys/cddl/contrib/opensolaris/common/nvpair sys/cddl/contrib/opensolaris/uts/common/sys X-SVN-Commit-Revision: 337227 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 01:52:27 -0000 Author: mav Date: Fri Aug 3 01:52:25 2018 New Revision: 337227 URL: https://svnweb.freebsd.org/changeset/base/337227 Log: MFV r337223: 9580 Add a hash-table on top of nvlist to speed-up operations illumos/illumos-gate@2ec7644aab2a726a64681fa66c6db8731b160de1 Reviewed by: Matt Ahrens Reviewed by: Sebastien Roy Approved by: Robert Mustacchi Author: Serapheim Dimitropoulos Modified: head/cddl/contrib/opensolaris/lib/libnvpair/nvpair_json.c head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair.h head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair_impl.h Directory Properties: head/cddl/contrib/opensolaris/ (props changed) head/sys/cddl/contrib/opensolaris/ (props changed) Modified: head/cddl/contrib/opensolaris/lib/libnvpair/nvpair_json.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libnvpair/nvpair_json.c Fri Aug 3 01:51:44 2018 (r337226) +++ head/cddl/contrib/opensolaris/lib/libnvpair/nvpair_json.c Fri Aug 3 01:52:25 2018 (r337227) @@ -10,6 +10,7 @@ */ /* * Copyright (c) 2014, Joyent, Inc. + * Copyright (c) 2017 by Delphix. All rights reserved. */ #include @@ -394,8 +395,10 @@ nvlist_print_json(FILE *fp, nvlist_t *nvl) } case DATA_TYPE_UNKNOWN: + case DATA_TYPE_DONTCARE: return (-1); } + } FPRINTF(fp, "}"); Modified: head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 01:51:44 2018 (r337226) +++ head/sys/cddl/contrib/opensolaris/common/nvpair/opensolaris_nvpair.c Fri Aug 3 01:52:25 2018 (r337227) @@ -149,6 +149,8 @@ int nvpair_max_recursion = 20; int nvpair_max_recursion = 100; #endif +uint64_t nvlist_hashtable_init_size = (1 << 4); + int nv_alloc_init(nv_alloc_t *nva, const nv_alloc_ops_t *nvo, /* args */ ...) { @@ -256,7 +258,292 @@ nv_priv_alloc_embedded(nvpriv_t *priv) return (emb_priv); } +static int +nvt_tab_alloc(nvpriv_t *priv, uint64_t buckets) +{ + ASSERT3P(priv->nvp_hashtable, ==, NULL); + ASSERT0(priv->nvp_nbuckets); + ASSERT0(priv->nvp_nentries); + + i_nvp_t **tab = nv_mem_zalloc(priv, buckets * sizeof (i_nvp_t *)); + if (tab == NULL) + return (ENOMEM); + + priv->nvp_hashtable = tab; + priv->nvp_nbuckets = buckets; + return (0); +} + static void +nvt_tab_free(nvpriv_t *priv) +{ + i_nvp_t **tab = priv->nvp_hashtable; + if (tab == NULL) { + ASSERT0(priv->nvp_nbuckets); + ASSERT0(priv->nvp_nentries); + return; + } + + nv_mem_free(priv, tab, priv->nvp_nbuckets * sizeof (i_nvp_t *)); + + priv->nvp_hashtable = NULL; + priv->nvp_nbuckets = 0; + priv->nvp_nentries = 0; +} + +static uint32_t +nvt_hash(const char *p) +{ + uint32_t g, hval = 0; + + while (*p) { + hval = (hval << 4) + *p++; + if ((g = (hval & 0xf0000000)) != 0) + hval ^= g >> 24; + hval &= ~g; + } + return (hval); +} + +static boolean_t +nvt_nvpair_match(nvpair_t *nvp1, nvpair_t *nvp2, uint32_t nvflag) +{ + boolean_t match = B_FALSE; + if (nvflag & NV_UNIQUE_NAME_TYPE) { + if (strcmp(NVP_NAME(nvp1), NVP_NAME(nvp2)) == 0 && + NVP_TYPE(nvp1) == NVP_TYPE(nvp2)) + match = B_TRUE; + } else { + ASSERT(nvflag == 0 || nvflag & NV_UNIQUE_NAME); + if (strcmp(NVP_NAME(nvp1), NVP_NAME(nvp2)) == 0) + match = B_TRUE; + } + return (match); +} + +static nvpair_t * +nvt_lookup_name_type(nvlist_t *nvl, const char *name, data_type_t type) +{ + nvpriv_t *priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv; + ASSERT(priv != NULL); + + i_nvp_t **tab = priv->nvp_hashtable; + + if (tab == NULL) { + ASSERT3P(priv->nvp_list, ==, NULL); + ASSERT0(priv->nvp_nbuckets); + ASSERT0(priv->nvp_nentries); + return (NULL); + } else { + ASSERT(priv->nvp_nbuckets != 0); + } + + uint64_t hash = nvt_hash(name); + uint64_t index = hash & (priv->nvp_nbuckets - 1); + + ASSERT3U(index, <, priv->nvp_nbuckets); + i_nvp_t *entry = tab[index]; + + for (i_nvp_t *e = entry; e != NULL; e = e->nvi_hashtable_next) { + if (strcmp(NVP_NAME(&e->nvi_nvp), name) == 0 && + (type == DATA_TYPE_DONTCARE || + NVP_TYPE(&e->nvi_nvp) == type)) + return (&e->nvi_nvp); + } + return (NULL); +} + +static nvpair_t * +nvt_lookup_name(nvlist_t *nvl, const char *name) +{ + return (nvt_lookup_name_type(nvl, name, DATA_TYPE_DONTCARE)); +} + +static int +nvt_resize(nvpriv_t *priv, uint32_t new_size) +{ + i_nvp_t **tab = priv->nvp_hashtable; + + /* + * Migrate all the entries from the current table + * to a newly-allocated table with the new size by + * re-adjusting the pointers of their entries. + */ + uint32_t size = priv->nvp_nbuckets; + uint32_t new_mask = new_size - 1; + ASSERT(((new_size) & ((new_size) - 1)) == 0); + + i_nvp_t **new_tab = nv_mem_zalloc(priv, new_size * sizeof (i_nvp_t *)); + if (new_tab == NULL) + return (ENOMEM); + + uint32_t nentries = 0; + for (uint32_t i = 0; i < size; i++) { + i_nvp_t *next, *e = tab[i]; + + while (e != NULL) { + next = e->nvi_hashtable_next; + + uint32_t hash = nvt_hash(NVP_NAME(&e->nvi_nvp)); + uint32_t index = hash & new_mask; + + e->nvi_hashtable_next = new_tab[index]; + new_tab[index] = e; + nentries++; + + e = next; + } + tab[i] = NULL; + } + ASSERT3U(nentries, ==, priv->nvp_nentries); + + nvt_tab_free(priv); + + priv->nvp_hashtable = new_tab; + priv->nvp_nbuckets = new_size; + priv->nvp_nentries = nentries; + + return (0); +} + +static boolean_t +nvt_needs_togrow(nvpriv_t *priv) +{ + /* + * Grow only when we have more elements than buckets + * and the # of buckets doesn't overflow. + */ + return (priv->nvp_nentries > priv->nvp_nbuckets && + (UINT32_MAX >> 1) >= priv->nvp_nbuckets); +} + +/* + * Allocate a new table that's twice the size of the old one, + * and migrate all the entries from the old one to the new + * one by re-adjusting their pointers. + */ +static int +nvt_grow(nvpriv_t *priv) +{ + uint32_t current_size = priv->nvp_nbuckets; + /* ensure we won't overflow */ + ASSERT3U(UINT32_MAX >> 1, >=, current_size); + return (nvt_resize(priv, current_size << 1)); +} + +static boolean_t +nvt_needs_toshrink(nvpriv_t *priv) +{ + /* + * Shrink only when the # of elements is less than or + * equal to 1/4 the # of buckets. Never shrink less than + * nvlist_hashtable_init_size. + */ + ASSERT3U(priv->nvp_nbuckets, >=, nvlist_hashtable_init_size); + if (priv->nvp_nbuckets == nvlist_hashtable_init_size) + return (B_FALSE); + return (priv->nvp_nentries <= (priv->nvp_nbuckets >> 2)); +} + +/* + * Allocate a new table that's half the size of the old one, + * and migrate all the entries from the old one to the new + * one by re-adjusting their pointers. + */ +static int +nvt_shrink(nvpriv_t *priv) +{ + uint32_t current_size = priv->nvp_nbuckets; + /* ensure we won't overflow */ + ASSERT3U(current_size, >=, nvlist_hashtable_init_size); + return (nvt_resize(priv, current_size >> 1)); +} + +static int +nvt_remove_nvpair(nvlist_t *nvl, nvpair_t *nvp) +{ + nvpriv_t *priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv; + + if (nvt_needs_toshrink(priv)) { + int err = nvt_shrink(priv); + if (err != 0) + return (err); + } + i_nvp_t **tab = priv->nvp_hashtable; + + char *name = NVP_NAME(nvp); + uint64_t hash = nvt_hash(name); + uint64_t index = hash & (priv->nvp_nbuckets - 1); + + ASSERT3U(index, <, priv->nvp_nbuckets); + i_nvp_t *bucket = tab[index]; + + for (i_nvp_t *prev = NULL, *e = bucket; + e != NULL; prev = e, e = e->nvi_hashtable_next) { + if (nvt_nvpair_match(&e->nvi_nvp, nvp, nvl->nvl_flag)) { + if (prev != NULL) { + prev->nvi_hashtable_next = + e->nvi_hashtable_next; + } else { + ASSERT3P(e, ==, bucket); + tab[index] = e->nvi_hashtable_next; + } + e->nvi_hashtable_next = NULL; + priv->nvp_nentries--; + break; + } + } + + return (0); +} + +static int +nvt_add_nvpair(nvlist_t *nvl, nvpair_t *nvp) +{ + nvpriv_t *priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv; + + /* initialize nvpair table now if it doesn't exist. */ + if (priv->nvp_hashtable == NULL) { + int err = nvt_tab_alloc(priv, nvlist_hashtable_init_size); + if (err != 0) + return (err); + } + + /* + * if we don't allow duplicate entries, make sure to + * unlink any existing entries from the table. + */ + if (nvl->nvl_nvflag != 0) { + int err = nvt_remove_nvpair(nvl, nvp); + if (err != 0) + return (err); + } + + if (nvt_needs_togrow(priv)) { + int err = nvt_grow(priv); + if (err != 0) + return (err); + } + i_nvp_t **tab = priv->nvp_hashtable; + + char *name = NVP_NAME(nvp); + uint64_t hash = nvt_hash(name); + uint64_t index = hash & (priv->nvp_nbuckets - 1); + + ASSERT3U(index, <, priv->nvp_nbuckets); + i_nvp_t *bucket = tab[index]; + + /* insert link at the beginning of the bucket */ + i_nvp_t *new_entry = NVPAIR2I_NVP(nvp); + ASSERT3P(new_entry->nvi_hashtable_next, ==, NULL); + new_entry->nvi_hashtable_next = bucket; + tab[index] = new_entry; + + priv->nvp_nentries++; + return (0); +} + +static void nvlist_init(nvlist_t *nvl, uint32_t nvflag, nvpriv_t *priv) { nvl->nvl_version = NV_VERSION; @@ -588,6 +875,7 @@ nvlist_free(nvlist_t *nvl) else nvl->nvl_priv = 0; + nvt_tab_free(priv); nv_mem_free(priv, priv, sizeof (nvpriv_t)); } @@ -648,26 +936,14 @@ nvlist_xdup(nvlist_t *nvl, nvlist_t **nvlp, nv_alloc_t int nvlist_remove_all(nvlist_t *nvl, const char *name) { - nvpriv_t *priv; - i_nvp_t *curr; int error = ENOENT; - if (nvl == NULL || name == NULL || - (priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv) == NULL) + if (nvl == NULL || name == NULL || nvl->nvl_priv == 0) return (EINVAL); - curr = priv->nvp_list; - while (curr != NULL) { - nvpair_t *nvp = &curr->nvi_nvp; - - curr = curr->nvi_next; - if (strcmp(name, NVP_NAME(nvp)) != 0) - continue; - - nvp_buf_unlink(nvl, nvp); - nvpair_free(nvp); - nvp_buf_free(nvl, nvp); - + nvpair_t *nvp; + while ((nvp = nvt_lookup_name(nvl, name)) != NULL) { + VERIFY0(nvlist_remove_nvpair(nvl, nvp)); error = 0; } @@ -680,28 +956,14 @@ nvlist_remove_all(nvlist_t *nvl, const char *name) int nvlist_remove(nvlist_t *nvl, const char *name, data_type_t type) { - nvpriv_t *priv; - i_nvp_t *curr; - - if (nvl == NULL || name == NULL || - (priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv) == NULL) + if (nvl == NULL || name == NULL || nvl->nvl_priv == 0) return (EINVAL); - curr = priv->nvp_list; - while (curr != NULL) { - nvpair_t *nvp = &curr->nvi_nvp; + nvpair_t *nvp = nvt_lookup_name_type(nvl, name, type); + if (nvp == NULL) + return (ENOENT); - if (strcmp(name, NVP_NAME(nvp)) == 0 && NVP_TYPE(nvp) == type) { - nvp_buf_unlink(nvl, nvp); - nvpair_free(nvp); - nvp_buf_free(nvl, nvp); - - return (0); - } - curr = curr->nvi_next; - } - - return (ENOENT); + return (nvlist_remove_nvpair(nvl, nvp)); } int @@ -710,6 +972,10 @@ nvlist_remove_nvpair(nvlist_t *nvl, nvpair_t *nvp) if (nvl == NULL || nvp == NULL) return (EINVAL); + int err = nvt_remove_nvpair(nvl, nvp); + if (err != 0) + return (err); + nvp_buf_unlink(nvl, nvp); nvpair_free(nvp); nvp_buf_free(nvl, nvp); @@ -987,6 +1253,12 @@ nvlist_add_common(nvlist_t *nvl, const char *name, else if (nvl->nvl_nvflag & NV_UNIQUE_NAME_TYPE) (void) nvlist_remove(nvl, name, type); + err = nvt_add_nvpair(nvl, nvp); + if (err != 0) { + nvpair_free(nvp); + nvp_buf_free(nvl, nvp); + return (err); + } nvp_buf_link(nvl, nvp); return (0); @@ -1336,25 +1608,17 @@ static int nvlist_lookup_common(nvlist_t *nvl, const char *name, data_type_t type, uint_t *nelem, void *data) { - nvpriv_t *priv; - nvpair_t *nvp; - i_nvp_t *curr; - - if (name == NULL || nvl == NULL || - (priv = (nvpriv_t *)(uintptr_t)nvl->nvl_priv) == NULL) + if (name == NULL || nvl == NULL || nvl->nvl_priv == 0) return (EINVAL); if (!(nvl->nvl_nvflag & (NV_UNIQUE_NAME | NV_UNIQUE_NAME_TYPE))) return (ENOTSUP); - for (curr = priv->nvp_list; curr != NULL; curr = curr->nvi_next) { - nvp = &curr->nvi_nvp; + nvpair_t *nvp = nvt_lookup_name_type(nvl, name, type); + if (nvp == NULL) + return (ENOENT); - if (strcmp(name, NVP_NAME(nvp)) == 0 && NVP_TYPE(nvp) == type) - return (nvpair_value_common(nvp, type, nelem, data)); - } - - return (ENOENT); + return (nvpair_value_common(nvp, type, nelem, data)); } int @@ -2112,6 +2376,12 @@ nvs_decode_pairs(nvstream_t *nvs, nvlist_t *nvl) return (EFAULT); } + err = nvt_add_nvpair(nvl, nvp); + if (err != 0) { + nvpair_free(nvp); + nvp_buf_free(nvl, nvp); + return (err); + } nvp_buf_link(nvl, nvp); } return (err); Modified: head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair.h Fri Aug 3 01:51:44 2018 (r337226) +++ head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair.h Fri Aug 3 01:52:25 2018 (r337227) @@ -20,7 +20,7 @@ */ /* * Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. - * Copyright (c) 2012 by Delphix. All rights reserved. + * Copyright (c) 2012, 2017 by Delphix. All rights reserved. */ #ifndef _SYS_NVPAIR_H @@ -39,6 +39,7 @@ extern "C" { #endif typedef enum { + DATA_TYPE_DONTCARE = -1, DATA_TYPE_UNKNOWN = 0, DATA_TYPE_BOOLEAN, DATA_TYPE_BYTE, Modified: head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair_impl.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair_impl.h Fri Aug 3 01:51:44 2018 (r337226) +++ head/sys/cddl/contrib/opensolaris/uts/common/sys/nvpair_impl.h Fri Aug 3 01:52:25 2018 (r337227) @@ -24,11 +24,13 @@ * Use is subject to license terms. */ +/* + * Copyright (c) 2017 by Delphix. All rights reserved. + */ + #ifndef _NVPAIR_IMPL_H #define _NVPAIR_IMPL_H -#pragma ident "%Z%%M% %I% %E% SMI" - #ifdef __cplusplus extern "C" { #endif @@ -47,16 +49,27 @@ typedef struct i_nvp i_nvp_t; struct i_nvp { union { - uint64_t _nvi_align; /* ensure alignment */ + /* ensure alignment */ + uint64_t _nvi_align; + struct { - i_nvp_t *_nvi_next; /* pointer to next nvpair */ - i_nvp_t *_nvi_prev; /* pointer to prev nvpair */ + /* pointer to next nvpair */ + i_nvp_t *_nvi_next; + + /* pointer to prev nvpair */ + i_nvp_t *_nvi_prev; + + /* next pair in table bucket */ + i_nvp_t *_nvi_hashtable_next; } _nvi; } _nvi_un; - nvpair_t nvi_nvp; /* nvpair */ + + /* nvpair */ + nvpair_t nvi_nvp; }; #define nvi_next _nvi_un._nvi._nvi_next #define nvi_prev _nvi_un._nvi._nvi_prev +#define nvi_hashtable_next _nvi_un._nvi._nvi_hashtable_next typedef struct { i_nvp_t *nvp_list; /* linked list of nvpairs */ @@ -64,6 +77,10 @@ typedef struct { i_nvp_t *nvp_curr; /* current walker nvpair */ nv_alloc_t *nvp_nva; /* pluggable allocator */ uint32_t nvp_stat; /* internal state */ + + i_nvp_t **nvp_hashtable; /* table of entries used for lookup */ + uint32_t nvp_nbuckets; /* # of buckets in hash table */ + uint32_t nvp_nentries; /* # of entries in hash table */ } nvpriv_t; #ifdef __cplusplus From owner-svn-src-all@freebsd.org Fri Aug 3 02:16:47 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0759A1059088; Fri, 3 Aug 2018 02:16:47 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E7858B4FA; Fri, 3 Aug 2018 02:16:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 69D211A36B; Fri, 3 Aug 2018 02:16:46 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w732GkCq001960; Fri, 3 Aug 2018 02:16:46 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w732Gk8A001958; Fri, 3 Aug 2018 02:16:46 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808030216.w732Gk8A001958@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 3 Aug 2018 02:16:46 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337229 - in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-SVN-Commit-Revision: 337229 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 02:16:47 -0000 Author: mav Date: Fri Aug 3 02:16:45 2018 New Revision: 337229 URL: https://svnweb.freebsd.org/changeset/base/337229 Log: Reduce taskq and context-switch cost of zio pipe When doing a read from disk, ZFS creates 3 ZIO's: a zio_null(), the logical zio_read(), and then a physical zio. Currently, each of these results in a separate taskq_dispatch(zio_execute). On high-read-iops workloads, this causes a significant performance impact. By processing all 3 ZIO's in a single taskq entry, we reduce the overhead on taskq locking and context switching. We accomplish this by allowing zio_done() to return a "next zio to execute" to zio_execute(). This results in a ~12% performance increase for random reads, from 96,000 iops to 108,000 iops (with recordsize=8k, on SSD's). Reviewed by: Pavel Zakharov Reviewed-by: Brian Behlendorf Reviewed by: George Wilson Signed-off-by: Matthew Ahrens External-issue: DLPX-59292 Closes #7736 zfsonlinux/zfs@62840030a7dceaee013ddbcc1eebcfc7922edf7c Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h Fri Aug 3 02:04:57 2018 (r337228) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h Fri Aug 3 02:16:45 2018 (r337229) @@ -217,7 +217,7 @@ enum zio_child { #define ZIO_CHILD_DDT_BIT ZIO_CHILD_BIT(ZIO_CHILD_DDT) #define ZIO_CHILD_LOGICAL_BIT ZIO_CHILD_BIT(ZIO_CHILD_LOGICAL) #define ZIO_CHILD_ALL_BITS \ - (ZIO_CHILD_VDEV_BIT | ZIO_CHILD_GANG_BIT | \ + (ZIO_CHILD_VDEV_BIT | ZIO_CHILD_GANG_BIT | \ ZIO_CHILD_DDT_BIT | ZIO_CHILD_LOGICAL_BIT) enum zio_wait_type { @@ -356,7 +356,7 @@ typedef struct zio_transform { struct zio_transform *zt_next; } zio_transform_t; -typedef int zio_pipe_stage_t(zio_t *zio); +typedef zio_t *zio_pipe_stage_t(zio_t *zio); /* * The io_reexecute flags are distinct from io_flags because the child must Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Fri Aug 3 02:04:57 2018 (r337228) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c Fri Aug 3 02:16:45 2018 (r337229) @@ -100,9 +100,6 @@ kmem_cache_t *zio_data_buf_cache[SPA_MAXBLOCKSIZE >> S extern vmem_t *zio_alloc_arena; #endif -#define ZIO_PIPELINE_CONTINUE 0x100 -#define ZIO_PIPELINE_STOP 0x101 - #define BP_SPANB(indblkshift, level) \ (((uint64_t)1) << ((level) * ((indblkshift) - SPA_BLKPTRSHIFT))) #define COMPARE_META_LEVEL 0x80000000ul @@ -539,7 +536,8 @@ zio_wait_for_children(zio_t *zio, uint8_t childbits, e } static void -zio_notify_parent(zio_t *pio, zio_t *zio, enum zio_wait_type wait) +zio_notify_parent(zio_t *pio, zio_t *zio, enum zio_wait_type wait, + zio_t **next_to_executep) { uint64_t *countp = &pio->io_children[zio->io_child_type][wait]; int *errorp = &pio->io_child_error[zio->io_child_type]; @@ -558,13 +556,33 @@ zio_notify_parent(zio_t *pio, zio_t *zio, enum zio_wai ZIO_TASKQ_INTERRUPT; pio->io_stall = NULL; mutex_exit(&pio->io_lock); + /* - * Dispatch the parent zio in its own taskq so that - * the child can continue to make progress. This also - * prevents overflowing the stack when we have deeply nested - * parent-child relationships. + * If we can tell the caller to execute this parent next, do + * so. Otherwise dispatch the parent zio as its own task. + * + * Having the caller execute the parent when possible reduces + * locking on the zio taskq's, reduces context switch + * overhead, and has no recursion penalty. Note that one + * read from disk typically causes at least 3 zio's: a + * zio_null(), the logical zio_read(), and then a physical + * zio. When the physical ZIO completes, we are able to call + * zio_done() on all 3 of these zio's from one invocation of + * zio_execute() by returning the parent back to + * zio_execute(). Since the parent isn't executed until this + * thread returns back to zio_execute(), the caller should do + * so promptly. + * + * In other cases, dispatching the parent prevents + * overflowing the stack when we have deeply nested + * parent-child relationships, as we do with the "mega zio" + * of writes for spa_sync(), and the chain of ZIL blocks. */ - zio_taskq_dispatch(pio, type, B_FALSE); + if (next_to_executep != NULL && *next_to_executep == NULL) { + *next_to_executep = pio; + } else { + zio_taskq_dispatch(pio, type, B_FALSE); + } } else { mutex_exit(&pio->io_lock); } @@ -1275,7 +1293,7 @@ zio_shrink(zio_t *zio, uint64_t size) * ========================================================================== */ -static int +static zio_t * zio_read_bp_init(zio_t *zio) { blkptr_t *bp = zio->io_bp; @@ -1312,14 +1330,14 @@ zio_read_bp_init(zio_t *zio) if (BP_GET_DEDUP(bp) && zio->io_child_type == ZIO_CHILD_LOGICAL) zio->io_pipeline = ZIO_DDT_READ_PIPELINE; - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_write_bp_init(zio_t *zio) { if (!IO_IS_ALLOCATING(zio)) - return (ZIO_PIPELINE_CONTINUE); + return (zio); ASSERT(zio->io_child_type != ZIO_CHILD_DDT); @@ -1334,7 +1352,7 @@ zio_write_bp_init(zio_t *zio) zio->io_pipeline = ZIO_INTERLOCK_PIPELINE; if (BP_IS_EMBEDDED(bp)) - return (ZIO_PIPELINE_CONTINUE); + return (zio); /* * If we've been overridden and nopwrite is set then @@ -1345,13 +1363,13 @@ zio_write_bp_init(zio_t *zio) ASSERT(!zp->zp_dedup); ASSERT3U(BP_GET_CHECKSUM(bp), ==, zp->zp_checksum); zio->io_flags |= ZIO_FLAG_NOPWRITE; - return (ZIO_PIPELINE_CONTINUE); + return (zio); } ASSERT(!zp->zp_nopwrite); if (BP_IS_HOLE(bp) || !zp->zp_dedup) - return (ZIO_PIPELINE_CONTINUE); + return (zio); ASSERT((zio_checksum_table[zp->zp_checksum].ci_flags & ZCHECKSUM_FLAG_DEDUP) || zp->zp_dedup_verify); @@ -1359,7 +1377,7 @@ zio_write_bp_init(zio_t *zio) if (BP_GET_CHECKSUM(bp) == zp->zp_checksum) { BP_SET_DEDUP(bp, 1); zio->io_pipeline |= ZIO_STAGE_DDT_WRITE; - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -1371,10 +1389,10 @@ zio_write_bp_init(zio_t *zio) zio->io_pipeline = zio->io_orig_pipeline; } - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_write_compress(zio_t *zio) { spa_t *spa = zio->io_spa; @@ -1393,11 +1411,11 @@ zio_write_compress(zio_t *zio) */ if (zio_wait_for_children(zio, ZIO_CHILD_LOGICAL_BIT | ZIO_CHILD_GANG_BIT, ZIO_WAIT_READY)) { - return (ZIO_PIPELINE_STOP); + return (NULL); } if (!IO_IS_ALLOCATING(zio)) - return (ZIO_PIPELINE_CONTINUE); + return (zio); if (zio->io_children_ready != NULL) { /* @@ -1456,7 +1474,7 @@ zio_write_compress(zio_t *zio) zio->io_pipeline = ZIO_INTERLOCK_PIPELINE; ASSERT(spa_feature_is_active(spa, SPA_FEATURE_EMBEDDED_DATA)); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } else { /* * Round up compressed size up to the ashift @@ -1544,10 +1562,10 @@ zio_write_compress(zio_t *zio) zio->io_pipeline |= ZIO_STAGE_NOP_WRITE; } } - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_free_bp_init(zio_t *zio) { blkptr_t *bp = zio->io_bp; @@ -1559,7 +1577,7 @@ zio_free_bp_init(zio_t *zio) ASSERT3P(zio->io_bp, ==, &zio->io_bp_copy); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -1633,12 +1651,12 @@ zio_taskq_member(zio_t *zio, zio_taskq_type_t q) return (B_FALSE); } -static int +static zio_t * zio_issue_async(zio_t *zio) { zio_taskq_dispatch(zio, ZIO_TASKQ_ISSUE, B_FALSE); - return (ZIO_PIPELINE_STOP); + return (NULL); } void @@ -1720,15 +1738,14 @@ static zio_pipe_stage_t *zio_pipeline[]; void zio_execute(zio_t *zio) { - zio->io_executor = curthread; - ASSERT3U(zio->io_queued_timestamp, >, 0); while (zio->io_stage < ZIO_STAGE_DONE) { enum zio_stage pipeline = zio->io_pipeline; enum zio_stage stage = zio->io_stage; - int rv; + zio->io_executor = curthread; + ASSERT(!MUTEX_HELD(&zio->io_lock)); ASSERT(ISP2(stage)); ASSERT(zio->io_stall == NULL); @@ -1758,12 +1775,16 @@ zio_execute(zio_t *zio) zio->io_stage = stage; zio->io_pipeline_trace |= zio->io_stage; - rv = zio_pipeline[highbit64(stage) - 1](zio); - if (rv == ZIO_PIPELINE_STOP) - return; + /* + * The zio pipeline stage returns the next zio to execute + * (typically the same as this one), or NULL if we should + * stop. + */ + zio = zio_pipeline[highbit64(stage) - 1](zio); - ASSERT(rv == ZIO_PIPELINE_CONTINUE); + if (zio == NULL) + return; } } @@ -2226,7 +2247,7 @@ zio_gang_tree_issue(zio_t *pio, zio_gang_node_t *gn, b zio_nowait(zio); } -static int +static zio_t * zio_gang_assemble(zio_t *zio) { blkptr_t *bp = zio->io_bp; @@ -2238,16 +2259,16 @@ zio_gang_assemble(zio_t *zio) zio_gang_tree_assemble(zio, bp, &zio->io_gang_tree); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_gang_issue(zio_t *zio) { blkptr_t *bp = zio->io_bp; if (zio_wait_for_children(zio, ZIO_CHILD_GANG_BIT, ZIO_WAIT_DONE)) { - return (ZIO_PIPELINE_STOP); + return (NULL); } ASSERT(BP_IS_GANG(bp) && zio->io_gang_leader == zio); @@ -2261,7 +2282,7 @@ zio_gang_issue(zio_t *zio) zio->io_pipeline = ZIO_INTERLOCK_PIPELINE; - return (ZIO_PIPELINE_CONTINUE); + return (zio); } static void @@ -2300,7 +2321,7 @@ zio_write_gang_done(zio_t *zio) abd_put(zio->io_abd); } -static int +static zio_t * zio_write_gang_block(zio_t *pio) { spa_t *spa = pio->io_spa; @@ -2359,7 +2380,7 @@ zio_write_gang_block(zio_t *pio) gbh_copies - copies, pio->io_allocator, pio); } pio->io_error = error; - return (ZIO_PIPELINE_CONTINUE); + return (pio); } if (pio == gio) { @@ -2426,7 +2447,7 @@ zio_write_gang_block(zio_t *pio) zio_nowait(zio); - return (ZIO_PIPELINE_CONTINUE); + return (pio); } /* @@ -2447,7 +2468,7 @@ zio_write_gang_block(zio_t *pio) * used for nopwrite, assuming that the salt and the checksums * themselves remain secret. */ -static int +static zio_t * zio_nop_write(zio_t *zio) { blkptr_t *bp = zio->io_bp; @@ -2474,7 +2495,7 @@ zio_nop_write(zio_t *zio) BP_GET_COMPRESS(bp) != BP_GET_COMPRESS(bp_orig) || BP_GET_DEDUP(bp) != BP_GET_DEDUP(bp_orig) || zp->zp_copies != BP_GET_NDVAS(bp_orig)) - return (ZIO_PIPELINE_CONTINUE); + return (zio); /* * If the checksums match then reset the pipeline so that we @@ -2494,7 +2515,7 @@ zio_nop_write(zio_t *zio) zio->io_flags |= ZIO_FLAG_NOPWRITE; } - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -2522,7 +2543,7 @@ zio_ddt_child_read_done(zio_t *zio) mutex_exit(&pio->io_lock); } -static int +static zio_t * zio_ddt_read_start(zio_t *zio) { blkptr_t *bp = zio->io_bp; @@ -2542,7 +2563,7 @@ zio_ddt_read_start(zio_t *zio) zio->io_vsd = dde; if (ddp_self == NULL) - return (ZIO_PIPELINE_CONTINUE); + return (zio); for (int p = 0; p < DDT_PHYS_TYPES; p++, ddp++) { if (ddp->ddp_phys_birth == 0 || ddp == ddp_self) @@ -2555,23 +2576,23 @@ zio_ddt_read_start(zio_t *zio) zio->io_priority, ZIO_DDT_CHILD_FLAGS(zio) | ZIO_FLAG_DONT_PROPAGATE, &zio->io_bookmark)); } - return (ZIO_PIPELINE_CONTINUE); + return (zio); } zio_nowait(zio_read(zio, zio->io_spa, bp, zio->io_abd, zio->io_size, NULL, NULL, zio->io_priority, ZIO_DDT_CHILD_FLAGS(zio), &zio->io_bookmark)); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_ddt_read_done(zio_t *zio) { blkptr_t *bp = zio->io_bp; if (zio_wait_for_children(zio, ZIO_CHILD_DDT_BIT, ZIO_WAIT_DONE)) { - return (ZIO_PIPELINE_STOP); + return (NULL); } ASSERT(BP_GET_DEDUP(bp)); @@ -2583,12 +2604,12 @@ zio_ddt_read_done(zio_t *zio) ddt_entry_t *dde = zio->io_vsd; if (ddt == NULL) { ASSERT(spa_load_state(zio->io_spa) != SPA_LOAD_NONE); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } if (dde == NULL) { zio->io_stage = ZIO_STAGE_DDT_READ_START >> 1; zio_taskq_dispatch(zio, ZIO_TASKQ_ISSUE, B_FALSE); - return (ZIO_PIPELINE_STOP); + return (NULL); } if (dde->dde_repair_abd != NULL) { abd_copy(zio->io_abd, dde->dde_repair_abd, @@ -2601,7 +2622,7 @@ zio_ddt_read_done(zio_t *zio) ASSERT(zio->io_vsd == NULL); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } static boolean_t @@ -2759,7 +2780,7 @@ zio_ddt_ditto_write_done(zio_t *zio) ddt_exit(ddt); } -static int +static zio_t * zio_ddt_write(zio_t *zio) { spa_t *spa = zio->io_spa; @@ -2803,7 +2824,7 @@ zio_ddt_write(zio_t *zio) ASSERT(!BP_GET_DEDUP(bp)); zio->io_pipeline = ZIO_WRITE_PIPELINE; ddt_exit(ddt); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } ditto_copies = ddt_ditto_copies_needed(ddt, dde, ddp); @@ -2829,7 +2850,7 @@ zio_ddt_write(zio_t *zio) zio->io_bp_override = NULL; BP_ZERO(bp); ddt_exit(ddt); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } dio = zio_write(zio, spa, txg, bp, zio->io_orig_abd, @@ -2871,12 +2892,12 @@ zio_ddt_write(zio_t *zio) if (dio) zio_nowait(dio); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } ddt_entry_t *freedde; /* for debugging */ -static int +static zio_t * zio_ddt_free(zio_t *zio) { spa_t *spa = zio->io_spa; @@ -2894,7 +2915,7 @@ zio_ddt_free(zio_t *zio) ddt_phys_decref(ddp); ddt_exit(ddt); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -2932,7 +2953,7 @@ zio_io_to_allocate(spa_t *spa, int allocator) return (zio); } -static int +static zio_t * zio_dva_throttle(zio_t *zio) { spa_t *spa = zio->io_spa; @@ -2942,7 +2963,7 @@ zio_dva_throttle(zio_t *zio) !spa_normal_class(zio->io_spa)->mc_alloc_throttle_enabled || zio->io_child_type == ZIO_CHILD_GANG || zio->io_flags & ZIO_FLAG_NODATA) { - return (ZIO_PIPELINE_CONTINUE); + return (zio); } ASSERT(zio->io_child_type > ZIO_CHILD_GANG); @@ -2968,22 +2989,7 @@ zio_dva_throttle(zio_t *zio) nio = zio_io_to_allocate(zio->io_spa, zio->io_allocator); mutex_exit(&spa->spa_alloc_locks[zio->io_allocator]); - if (nio == zio) - return (ZIO_PIPELINE_CONTINUE); - - if (nio != NULL) { - ASSERT(nio->io_stage == ZIO_STAGE_DVA_THROTTLE); - /* - * We are passing control to a new zio so make sure that - * it is processed by a different thread. We do this to - * avoid stack overflows that can occur when parents are - * throttled and children are making progress. We allow - * it to go to the head of the taskq since it's already - * been waiting. - */ - zio_taskq_dispatch(nio, ZIO_TASKQ_ISSUE, B_TRUE); - } - return (ZIO_PIPELINE_STOP); + return (nio); } void @@ -3002,7 +3008,7 @@ zio_allocate_dispatch(spa_t *spa, int allocator) zio_taskq_dispatch(zio, ZIO_TASKQ_ISSUE, B_TRUE); } -static int +static zio_t * zio_dva_allocate(zio_t *zio) { spa_t *spa = zio->io_spa; @@ -3045,18 +3051,18 @@ zio_dva_allocate(zio_t *zio) zio->io_error = error; } - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_dva_free(zio_t *zio) { metaslab_free(zio->io_spa, zio->io_bp, zio->io_txg, B_FALSE); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_dva_claim(zio_t *zio) { int error; @@ -3065,7 +3071,7 @@ zio_dva_claim(zio_t *zio) if (error) zio->io_error = error; - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -3160,7 +3166,7 @@ zio_alloc_zil(spa_t *spa, uint64_t objset, uint64_t tx * force the underlying vdev layers to call either zio_execute() or * zio_interrupt() to ensure that the pipeline continues with the correct I/O. */ -static int +static zio_t * zio_vdev_io_start(zio_t *zio) { vdev_t *vd = zio->io_vd; @@ -3179,13 +3185,13 @@ zio_vdev_io_start(zio_t *zio) * The mirror_ops handle multiple DVAs in a single BP. */ vdev_mirror_ops.vdev_op_io_start(zio); - return (ZIO_PIPELINE_STOP); + return (NULL); } if (vd->vdev_ops->vdev_op_leaf && zio->io_type == ZIO_TYPE_FREE && zio->io_priority == ZIO_PRIORITY_NOW) { trim_map_free(vd, zio->io_offset, zio->io_size, zio->io_txg); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } ASSERT3P(zio->io_logical, !=, zio); @@ -3299,24 +3305,24 @@ zio_vdev_io_start(zio_t *zio) !vdev_dtl_contains(vd, DTL_PARTIAL, zio->io_txg, 1)) { ASSERT(zio->io_type == ZIO_TYPE_WRITE); zio_vdev_io_bypass(zio); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } if (vd->vdev_ops->vdev_op_leaf) { switch (zio->io_type) { case ZIO_TYPE_READ: if (vdev_cache_read(zio)) - return (ZIO_PIPELINE_CONTINUE); + return (zio); /* FALLTHROUGH */ case ZIO_TYPE_WRITE: case ZIO_TYPE_FREE: if ((zio = vdev_queue_io(zio)) == NULL) - return (ZIO_PIPELINE_STOP); + return (NULL); if (!vdev_accessible(vd, zio)) { zio->io_error = SET_ERROR(ENXIO); zio_interrupt(zio); - return (ZIO_PIPELINE_STOP); + return (NULL); } break; } @@ -3328,14 +3334,14 @@ zio_vdev_io_start(zio_t *zio) if (zio->io_type == ZIO_TYPE_WRITE && !(zio->io_flags & ZIO_FLAG_IO_REPAIR) && !trim_map_write_start(zio)) - return (ZIO_PIPELINE_STOP); + return (NULL); } vd->vdev_ops->vdev_op_io_start(zio); - return (ZIO_PIPELINE_STOP); + return (NULL); } -static int +static zio_t * zio_vdev_io_done(zio_t *zio) { vdev_t *vd = zio->io_vd; @@ -3343,7 +3349,7 @@ zio_vdev_io_done(zio_t *zio) boolean_t unexpected_error = B_FALSE; if (zio_wait_for_children(zio, ZIO_CHILD_VDEV_BIT, ZIO_WAIT_DONE)) { - return (ZIO_PIPELINE_STOP); + return (NULL); } ASSERT(zio->io_type == ZIO_TYPE_READ || @@ -3386,7 +3392,7 @@ zio_vdev_io_done(zio_t *zio) if (unexpected_error) VERIFY(vdev_probe(vd, zio) == NULL); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -3444,13 +3450,13 @@ zio_vsd_default_cksum_report(zio_t *zio, zio_cksum_rep zcr->zcr_free = zio_buf_free; } -static int +static zio_t * zio_vdev_io_assess(zio_t *zio) { vdev_t *vd = zio->io_vd; if (zio_wait_for_children(zio, ZIO_CHILD_VDEV_BIT, ZIO_WAIT_DONE)) { - return (ZIO_PIPELINE_STOP); + return (NULL); } if (vd == NULL && !(zio->io_flags & ZIO_FLAG_CONFIG_WRITER)) @@ -3496,7 +3502,7 @@ zio_vdev_io_assess(zio_t *zio) zio->io_stage = ZIO_STAGE_VDEV_IO_START >> 1; zio_taskq_dispatch(zio, ZIO_TASKQ_ISSUE, zio_requeue_io_start_cut_in_line); - return (ZIO_PIPELINE_STOP); + return (NULL); } /* @@ -3536,7 +3542,7 @@ zio_vdev_io_assess(zio_t *zio) zio->io_physdone(zio->io_logical); } - return (ZIO_PIPELINE_CONTINUE); + return (zio); } void @@ -3571,7 +3577,7 @@ zio_vdev_io_bypass(zio_t *zio) * Generate and verify checksums * ========================================================================== */ -static int +static zio_t * zio_checksum_generate(zio_t *zio) { blkptr_t *bp = zio->io_bp; @@ -3585,7 +3591,7 @@ zio_checksum_generate(zio_t *zio) checksum = zio->io_prop.zp_checksum; if (checksum == ZIO_CHECKSUM_OFF) - return (ZIO_PIPELINE_CONTINUE); + return (zio); ASSERT(checksum == ZIO_CHECKSUM_LABEL); } else { @@ -3599,10 +3605,10 @@ zio_checksum_generate(zio_t *zio) zio_checksum_compute(zio, checksum, zio->io_abd, zio->io_size); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } -static int +static zio_t * zio_checksum_verify(zio_t *zio) { zio_bad_cksum_t info; @@ -3617,7 +3623,7 @@ zio_checksum_verify(zio_t *zio) * We're either verifying a label checksum, or nothing at all. */ if (zio->io_prop.zp_checksum == ZIO_CHECKSUM_OFF) - return (ZIO_PIPELINE_CONTINUE); + return (zio); ASSERT(zio->io_prop.zp_checksum == ZIO_CHECKSUM_LABEL); } @@ -3632,7 +3638,7 @@ zio_checksum_verify(zio_t *zio) } } - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -3675,7 +3681,7 @@ zio_worst_error(int e1, int e2) * I/O completion * ========================================================================== */ -static int +static zio_t * zio_ready(zio_t *zio) { blkptr_t *bp = zio->io_bp; @@ -3684,7 +3690,7 @@ zio_ready(zio_t *zio) if (zio_wait_for_children(zio, ZIO_CHILD_GANG_BIT | ZIO_CHILD_DDT_BIT, ZIO_WAIT_READY)) { - return (ZIO_PIPELINE_STOP); + return (NULL); } if (zio->io_ready) { @@ -3730,7 +3736,7 @@ zio_ready(zio_t *zio) */ for (; pio != NULL; pio = pio_next) { pio_next = zio_walk_parents(zio, &zl); - zio_notify_parent(pio, zio, ZIO_WAIT_READY); + zio_notify_parent(pio, zio, ZIO_WAIT_READY, NULL); } if (zio->io_flags & ZIO_FLAG_NODATA) { @@ -3746,7 +3752,7 @@ zio_ready(zio_t *zio) zio->io_spa->spa_syncing_txg == zio->io_txg) zio_handle_ignored_writes(zio); - return (ZIO_PIPELINE_CONTINUE); + return (zio); } /* @@ -3810,7 +3816,7 @@ zio_dva_throttle_done(zio_t *zio) zio_allocate_dispatch(zio->io_spa, pio->io_allocator); } -static int +static zio_t * zio_done(zio_t *zio) { spa_t *spa = zio->io_spa; @@ -3827,7 +3833,7 @@ zio_done(zio_t *zio) * wait for them and then repeat this pipeline stage. */ if (zio_wait_for_children(zio, ZIO_CHILD_ALL_BITS, ZIO_WAIT_DONE)) { - return (ZIO_PIPELINE_STOP); + return (NULL); } /* @@ -4041,7 +4047,12 @@ zio_done(zio_t *zio) if ((pio->io_flags & ZIO_FLAG_GODFATHER) && (zio->io_reexecute & ZIO_REEXECUTE_SUSPEND)) { zio_remove_child(pio, zio, remove_zl); - zio_notify_parent(pio, zio, ZIO_WAIT_DONE); + /* + * This is a rare code path, so we don't + * bother with "next_to_execute". + */ + zio_notify_parent(pio, zio, ZIO_WAIT_DONE, + NULL); } } @@ -4053,7 +4064,11 @@ zio_done(zio_t *zio) */ ASSERT(!(zio->io_flags & ZIO_FLAG_GODFATHER)); zio->io_flags |= ZIO_FLAG_DONT_PROPAGATE; - zio_notify_parent(pio, zio, ZIO_WAIT_DONE); + /* + * This is a rare code path, so we don't bother with + * "next_to_execute". + */ + zio_notify_parent(pio, zio, ZIO_WAIT_DONE, NULL); } else if (zio->io_reexecute & ZIO_REEXECUTE_SUSPEND) { /* * We'd fail again if we reexecuted now, so suspend @@ -4074,7 +4089,7 @@ zio_done(zio_t *zio) ZIO_TASKQ_ISSUE, (task_func_t *)zio_reexecute, zio, 0, &zio->io_tqent); } - return (ZIO_PIPELINE_STOP); + return (NULL); } ASSERT(zio->io_child_count == 0); @@ -4104,12 +4119,17 @@ zio_done(zio_t *zio) zio->io_state[ZIO_WAIT_DONE] = 1; mutex_exit(&zio->io_lock); + /* + * We are done executing this zio. We may want to execute a parent + * next. See the comment in zio_notify_parent(). + */ + zio_t *next_to_execute = NULL; zl = NULL; for (pio = zio_walk_parents(zio, &zl); pio != NULL; pio = pio_next) { zio_link_t *remove_zl = zl; pio_next = zio_walk_parents(zio, &zl); zio_remove_child(pio, zio, remove_zl); - zio_notify_parent(pio, zio, ZIO_WAIT_DONE); + zio_notify_parent(pio, zio, ZIO_WAIT_DONE, &next_to_execute); } if (zio->io_waiter != NULL) { @@ -4121,7 +4141,7 @@ zio_done(zio_t *zio) zio_destroy(zio); } - return (ZIO_PIPELINE_STOP); + return (next_to_execute); } /* From owner-svn-src-all@freebsd.org Fri Aug 3 02:51:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E095C105A2E8; Fri, 3 Aug 2018 02:51:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 95D6D8C94A; Fri, 3 Aug 2018 02:51:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 768201A9FF; Fri, 3 Aug 2018 02:51:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w732pbJV017912; Fri, 3 Aug 2018 02:51:37 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w732pbxx017911; Fri, 3 Aug 2018 02:51:37 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808030251.w732pbxx017911@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Fri, 3 Aug 2018 02:51:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337230 - head/sys/x86/x86 X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/x86/x86 X-SVN-Commit-Revision: 337230 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 02:51:38 -0000 Author: markj Date: Fri Aug 3 02:51:37 2018 New Revision: 337230 URL: https://svnweb.freebsd.org/changeset/base/337230 Log: Verify that each frame pointer lies within the thread's kstack. Previously, this check was omitted for the first frame pointer. Reported by: pho Reviewed by: kib MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D16572 Modified: head/sys/x86/x86/stack_machdep.c Modified: head/sys/x86/x86/stack_machdep.c ============================================================================== --- head/sys/x86/x86/stack_machdep.c Fri Aug 3 02:16:45 2018 (r337229) +++ head/sys/x86/x86/stack_machdep.c Fri Aug 3 02:51:37 2018 (r337230) @@ -82,16 +82,16 @@ stack_capture(struct thread *td, struct stack *st, reg stack_zero(st); frame = (x86_frame_t)fp; while (1) { - if (!INKERNEL((long)frame)) + if ((vm_offset_t)frame < td->td_kstack || + (vm_offset_t)frame >= td->td_kstack + + td->td_kstack_pages * PAGE_SIZE) break; callpc = frame->f_retaddr; if (!INKERNEL(callpc)) break; if (stack_put(st, callpc) == -1) break; - if (frame->f_frame <= frame || - (vm_offset_t)frame->f_frame >= td->td_kstack + - td->td_kstack_pages * PAGE_SIZE) + if (frame->f_frame <= frame) break; frame = frame->f_frame; } @@ -106,7 +106,7 @@ stack_nmi_handler(struct trapframe *tf) if (nmi_stack == NULL || curthread != nmi_pending) return (0); - if (INKERNEL(TF_PC(tf)) && (TF_FLAGS(tf) & PSL_I) != 0) + if (!TRAPF_USERMODE(tf) && (TF_FLAGS(tf) & PSL_I) != 0) stack_capture(curthread, nmi_stack, TF_FP(tf)); else /* We were running in usermode or had interrupts disabled. */ From owner-svn-src-all@freebsd.org Fri Aug 3 07:59:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50BBB1060E68; Fri, 3 Aug 2018 07:59:30 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 033767581F; Fri, 3 Aug 2018 07:59:30 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D55531DA98; Fri, 3 Aug 2018 07:59:29 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w737xTMF075765; Fri, 3 Aug 2018 07:59:29 GMT (envelope-from tsoome@FreeBSD.org) Received: (from tsoome@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w737xTwO075764; Fri, 3 Aug 2018 07:59:29 GMT (envelope-from tsoome@FreeBSD.org) Message-Id: <201808030759.w737xTwO075764@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tsoome set sender to tsoome@FreeBSD.org using -f From: Toomas Soome Date: Fri, 3 Aug 2018 07:59:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337231 - head/stand/efi/loader X-SVN-Group: head X-SVN-Commit-Author: tsoome X-SVN-Commit-Paths: head/stand/efi/loader X-SVN-Commit-Revision: 337231 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 07:59:30 -0000 Author: tsoome Date: Fri Aug 3 07:59:29 2018 New Revision: 337231 URL: https://svnweb.freebsd.org/changeset/base/337231 Log: loader.efi: clean up misleading noise from missing block devices If there are no block devices, there is no need to printout error (ENOENT). In case of netboot, our image path has no block device, no need to make noise about it. Modified: head/stand/efi/loader/main.c Modified: head/stand/efi/loader/main.c ============================================================================== --- head/stand/efi/loader/main.c Fri Aug 3 02:51:37 2018 (r337230) +++ head/stand/efi/loader/main.c Fri Aug 3 07:59:29 2018 (r337231) @@ -545,8 +545,6 @@ find_currdev(EFI_LOADED_IMAGE *img, bool do_bootmgr, b return (0); } } - } else { - printf("Can't find device by handle\n"); } /* @@ -862,9 +860,9 @@ main(int argc, CHAR16 *argv[]) * march through the device switch probing for things. */ i = efipart_inithandles(); - if (i != 0) { + if (i != 0 && i != ENOENT) { printf("efipart_inithandles failed with ERRNO %d, expect " - "failures", i); + "failures\n", i); } for (i = 0; devsw[i] != NULL; i++) From owner-svn-src-all@freebsd.org Fri Aug 3 08:32:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81C191061D8F for ; Fri, 3 Aug 2018 08:32:25 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AA5476A47 for ; Fri, 3 Aug 2018 08:32:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x231.google.com with SMTP id g141-v6so7296295ita.4 for ; Fri, 03 Aug 2018 01:32:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=cfmM4oxTBQ+Y1kummHlCOFD2vx+QhjOtuyMMwfy+wxo=; b=j8chzhQcXDOjcMPahLKeVybEFaPtOP+f+c4b6KZ8j06QxmCP2MuH3OMcD7u21xHCnH bRFRu1GXhEFve8Dr0D6+QfGcFLzz6+x7xa7dGJVRI9xKl73y0FrnL+k+dKWVoexvugs0 bKlmWt/E222suIcr5VT9eRmCT16j4OROXFoN/LScIYBRW03TQgOzTbh6NnMax0UsW99p UqgI4FtJOwLYMQrBJofShiOdemtQYZnLbLVovQSwDxbwKnuGIpLK/gjeo8ueye8Judk4 MqWLgutG2Glk4BOWHkJDedCK+Lkgp2TRYidA7ubvhrV+rprIyT5/k6ZxC3pYuP/pYNTU H8bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=cfmM4oxTBQ+Y1kummHlCOFD2vx+QhjOtuyMMwfy+wxo=; b=fnSM1Hz9kGb4QcIPtjCEwCKQnA5w6gHvUkORIMihIdiqYyN8lhs9XxbQxiobD7iFmM DqpGySgZtKVIHfi6ucqxM5hoTus9MvmXTSSSsCMgFqOv2G91r/5b5QMfNH4Vw7zWWgc3 lz/OCZ1tdKabzfvbo9DMWSv68PPoTFVe0G0BcSZCA+c7fvJBTaDChSDG/YUiogmaLMaN wmxxyDyx4MeObo5k3joY5Lt2CFA4ojq2PQ0g19jiOyUhyCpkM+h4WSzvvC+wi896Oj7P JSx3v3muyp5GiIY/RWhgmbEOomWNUMeIRen3Dpm2l4607eDzlkfz3hMavvH3CXVo72yu H1dQ== X-Gm-Message-State: AOUpUlENbgQ0oHH36MQY6sUoYcE4LE8q5p9GrK9s3Iij8odrP3ehGvVg qa1NUzSUKh5CsvPkv+Wtgept2xOR16U/mzZrOemHyv6Q6d2nQw== X-Google-Smtp-Source: AAOMgpehZSAAUrNdCUZCBIA5kc3EZDE1qQgSgH4iGnUk0Typd28ClXkVU7KMp0iZKsTbi93oINSOvRXqxBfY9GmDEik= X-Received: by 2002:a24:3ec3:: with SMTP id s186-v6mr5507879its.73.1533285144255; Fri, 03 Aug 2018 01:32:24 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:4485:0:0:0:0:0 with HTTP; Fri, 3 Aug 2018 01:32:23 -0700 (PDT) X-Originating-IP: [86.153.210.77] In-Reply-To: <201808030759.w737xTwO075764@repo.freebsd.org> References: <201808030759.w737xTwO075764@repo.freebsd.org> From: Warner Losh Date: Fri, 3 Aug 2018 02:32:23 -0600 X-Google-Sender-Auth: wwjR0FQWIRJOqgISWoWlTU1Q2PU Message-ID: Subject: Re: svn commit: r337231 - head/stand/efi/loader To: Toomas Soome Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 08:32:25 -0000 Any reason why efipart_inithandles() should even return an error in this case? Seems to cause us nothing but trouble so we have to filter some, but not all errors which strikes me as a bad design. We should only return errors for real errors, like not having enough memory for the arrays we need. Warner On Fri, Aug 3, 2018 at 1:59 AM, Toomas Soome wrote: > Author: tsoome > Date: Fri Aug 3 07:59:29 2018 > New Revision: 337231 > URL: https://svnweb.freebsd.org/changeset/base/337231 > > Log: > loader.efi: clean up misleading noise from missing block devices > > If there are no block devices, there is no need to printout > error (ENOENT). > > In case of netboot, our image path has no block device, no need to make > noise about it. > > Modified: > head/stand/efi/loader/main.c > > Modified: head/stand/efi/loader/main.c > ============================================================ > ================== > --- head/stand/efi/loader/main.c Fri Aug 3 02:51:37 2018 > (r337230) > +++ head/stand/efi/loader/main.c Fri Aug 3 07:59:29 2018 > (r337231) > @@ -545,8 +545,6 @@ find_currdev(EFI_LOADED_IMAGE *img, bool do_bootmgr, b > return (0); > } > } > - } else { > - printf("Can't find device by handle\n"); > } > > /* > @@ -862,9 +860,9 @@ main(int argc, CHAR16 *argv[]) > * march through the device switch probing for things. > */ > i = efipart_inithandles(); > - if (i != 0) { > + if (i != 0 && i != ENOENT) { > printf("efipart_inithandles failed with ERRNO %d, expect " > - "failures", i); > + "failures\n", i); > } > > for (i = 0; devsw[i] != NULL; i++) > > From owner-svn-src-all@freebsd.org Fri Aug 3 08:47:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D1A41062639; Fri, 3 Aug 2018 08:47:12 +0000 (UTC) (envelope-from tsoome@me.com) Received: from mr11p00im-asmtp002.me.com (mr11p00im-asmtp002.me.com [17.110.69.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77A88774B5; Fri, 3 Aug 2018 08:47:11 +0000 (UTC) (envelope-from tsoome@me.com) Received: from process-dkim-sign-daemon.mr11p00im-asmtp002.me.com by mr11p00im-asmtp002.me.com (Oracle Communications Messaging Server 8.0.2.2.20180531 64bit (built May 31 2018)) id <0PCV00B00M3TPH00@mr11p00im-asmtp002.me.com>; Fri, 03 Aug 2018 08:46:46 +0000 (GMT) Received: from icloud.com ([127.0.0.1]) by mr11p00im-asmtp002.me.com (Oracle Communications Messaging Server 8.0.2.2.20180531 64bit (built May 31 2018)) with ESMTPSA id <0PCV00BSIN1ABD10@mr11p00im-asmtp002.me.com>; Fri, 03 Aug 2018 08:46:29 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-03_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1015 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1808030098 From: Toomas Soome Message-id: <53D5CC0A-16D1-4538-B818-0B7F6E6CAB69@me.com> MIME-version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: svn commit: r337231 - head/stand/efi/loader Date: Fri, 03 Aug 2018 11:46:21 +0300 In-reply-to: Cc: Toomas Soome , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org To: Warner Losh References: <201808030759.w737xTwO075764@repo.freebsd.org> X-Mailer: Apple Mail (2.3445.9.1) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 08:47:12 -0000 > On 3 Aug 2018, at 11:32, Warner Losh wrote: >=20 > Any reason why efipart_inithandles() should even return an error in = this case? Seems to cause us nothing but trouble so we have to filter = some, but not all errors which strikes me as a bad design. We should = only return errors for real errors, like not having enough memory for = the arrays we need. >=20 > Warner The current code does return the efi error translated to errno and does = leave the filtering/notification for caller. I guess it is good idea to = filter cases like no devices in-place, however, the fun is with out of = memory =E2=80=94 at that stage (we even haven't started to probe for = disks, so we do not have even interpreter) it basically means loader.efi = failure. rgds, toomas >=20 > On Fri, Aug 3, 2018 at 1:59 AM, Toomas Soome > wrote: > Author: tsoome > Date: Fri Aug 3 07:59:29 2018 > New Revision: 337231 > URL: https://svnweb.freebsd.org/changeset/base/337231 = >=20 > Log: > loader.efi: clean up misleading noise from missing block devices >=20 > If there are no block devices, there is no need to printout > error (ENOENT). >=20 > In case of netboot, our image path has no block device, no need to = make > noise about it. >=20 > Modified: > head/stand/efi/loader/main.c >=20 > Modified: head/stand/efi/loader/main.c > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/stand/efi/loader/main.c Fri Aug 3 02:51:37 2018 = (r337230) > +++ head/stand/efi/loader/main.c Fri Aug 3 07:59:29 2018 = (r337231) > @@ -545,8 +545,6 @@ find_currdev(EFI_LOADED_IMAGE *img, bool = do_bootmgr, b > return (0); > } > } > - } else { > - printf("Can't find device by handle\n"); > } >=20 > /* > @@ -862,9 +860,9 @@ main(int argc, CHAR16 *argv[]) > * march through the device switch probing for things. > */ > i =3D efipart_inithandles(); > - if (i !=3D 0) { > + if (i !=3D 0 && i !=3D ENOENT) { > printf("efipart_inithandles failed with ERRNO %d, = expect " > - "failures", i); > + "failures\n", i); > } >=20 > for (i =3D 0; devsw[i] !=3D NULL; i++) >=20 >=20 From owner-svn-src-all@freebsd.org Fri Aug 3 09:02:58 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B3BA1062F40; Fri, 3 Aug 2018 09:02:58 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AED3678067; Fri, 3 Aug 2018 09:02:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8ECB11E5AD; Fri, 3 Aug 2018 09:02:57 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7392v7h010809; Fri, 3 Aug 2018 09:02:57 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7392vWX010808; Fri, 3 Aug 2018 09:02:57 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201808030902.w7392vWX010808@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Fri, 3 Aug 2018 09:02:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337232 - head/sys/compat/linuxkpi/common/include/linux X-SVN-Group: head X-SVN-Commit-Author: hselasky X-SVN-Commit-Paths: head/sys/compat/linuxkpi/common/include/linux X-SVN-Commit-Revision: 337232 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 09:02:58 -0000 Author: hselasky Date: Fri Aug 3 09:02:57 2018 New Revision: 337232 URL: https://svnweb.freebsd.org/changeset/base/337232 Log: Implement ktime_add_ms() and ktime_before() in the LinuxKPI. Submitted by: Johannes Lundberg MFC after: 1 week Sponsored by: Mellanox Technologies Modified: head/sys/compat/linuxkpi/common/include/linux/ktime.h Modified: head/sys/compat/linuxkpi/common/include/linux/ktime.h ============================================================================== --- head/sys/compat/linuxkpi/common/include/linux/ktime.h Fri Aug 3 07:59:29 2018 (r337231) +++ head/sys/compat/linuxkpi/common/include/linux/ktime.h Fri Aug 3 09:02:57 2018 (r337232) @@ -86,6 +86,13 @@ ktime_add_ns(ktime_t kt, int64_t ns) } static inline ktime_t +ktime_add_ms(ktime_t kt, int64_t ms) +{ + + return (ktime_add_ns(kt, ms * NSEC_PER_MSEC)); +} + +static inline ktime_t ktime_sub_ns(ktime_t kt, int64_t ns) { return (kt - ns); @@ -144,6 +151,13 @@ ktime_after(const ktime_t cmp1, const ktime_t cmp2) { return (ktime_compare(cmp1, cmp2) > 0); +} + +static inline bool +ktime_before(const ktime_t cmp1, const ktime_t cmp2) +{ + + return (ktime_compare(cmp1, cmp2) < 0); } static inline ktime_t From owner-svn-src-all@freebsd.org Fri Aug 3 10:58:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A3911065B35; Fri, 3 Aug 2018 10:58:14 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DD8C57BECB; Fri, 3 Aug 2018 10:58:13 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BA5EF1F7A1; Fri, 3 Aug 2018 10:58:13 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73AwDlo067077; Fri, 3 Aug 2018 10:58:13 GMT (envelope-from trasz@FreeBSD.org) Received: (from trasz@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73AwDqH067076; Fri, 3 Aug 2018 10:58:13 GMT (envelope-from trasz@FreeBSD.org) Message-Id: <201808031058.w73AwDqH067076@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: trasz set sender to trasz@FreeBSD.org using -f From: Edward Tomasz Napierala Date: Fri, 3 Aug 2018 10:58:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337233 - head/libexec/getty X-SVN-Group: head X-SVN-Commit-Author: trasz X-SVN-Commit-Paths: head/libexec/getty X-SVN-Commit-Revision: 337233 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 10:58:14 -0000 Author: trasz Date: Fri Aug 3 10:58:13 2018 New Revision: 337233 URL: https://svnweb.freebsd.org/changeset/base/337233 Log: .Xr pam_securetty(8) from ttys(5), to hint the users as to what actually enforces the 'secure' tty flag. MFC after: 2 weeks Sponsored by: DARPA, AFRL Modified: head/libexec/getty/ttys.5 Modified: head/libexec/getty/ttys.5 ============================================================================== --- head/libexec/getty/ttys.5 Fri Aug 3 09:02:57 2018 (r337232) +++ head/libexec/getty/ttys.5 Fri Aug 3 10:58:13 2018 (r337233) @@ -28,7 +28,7 @@ .\" from: @(#)ttys.5 8.1 (Berkeley) 6/4/93 .\" $FreeBSD$ .\" " -.Dd March 2, 2018 +.Dd August 3, 2018 .Dt TTYS 5 .Os .Sh NAME @@ -167,6 +167,7 @@ ttyv0 "/usr/local/bin/xterm -display :0" xterm on wind .Xr termcap 5 , .Xr getty 8 , .Xr init 8 , +.Xr pam_securetty 8 , .Xr pstat 8 .\".Xr init 8 , .\".Xr ttyflags 8 From owner-svn-src-all@freebsd.org Fri Aug 3 10:59:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D9841065BCB; Fri, 3 Aug 2018 10:59:06 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 332FC7C05E; Fri, 3 Aug 2018 10:59:06 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 145D61F7A2; Fri, 3 Aug 2018 10:59:06 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73Ax5Lg067158; Fri, 3 Aug 2018 10:59:05 GMT (envelope-from trasz@FreeBSD.org) Received: (from trasz@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73Ax5P6067157; Fri, 3 Aug 2018 10:59:05 GMT (envelope-from trasz@FreeBSD.org) Message-Id: <201808031059.w73Ax5P6067157@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: trasz set sender to trasz@FreeBSD.org using -f From: Edward Tomasz Napierala Date: Fri, 3 Aug 2018 10:59:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337234 - head/libexec/getty X-SVN-Group: head X-SVN-Commit-Author: trasz X-SVN-Commit-Paths: head/libexec/getty X-SVN-Commit-Revision: 337234 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 10:59:06 -0000 Author: trasz Date: Fri Aug 3 10:59:05 2018 New Revision: 337234 URL: https://svnweb.freebsd.org/changeset/base/337234 Log: Remove pointless comment. MFC after: 2 weeks Sponsored by: DARPA, AFRL Modified: head/libexec/getty/ttys.5 Modified: head/libexec/getty/ttys.5 ============================================================================== --- head/libexec/getty/ttys.5 Fri Aug 3 10:58:13 2018 (r337233) +++ head/libexec/getty/ttys.5 Fri Aug 3 10:59:05 2018 (r337234) @@ -169,8 +169,6 @@ ttyv0 "/usr/local/bin/xterm -display :0" xterm on wind .Xr init 8 , .Xr pam_securetty 8 , .Xr pstat 8 -.\".Xr init 8 , -.\".Xr ttyflags 8 .Sh HISTORY A .Nm From owner-svn-src-all@freebsd.org Fri Aug 3 11:36:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E4131066FA9; Fri, 3 Aug 2018 11:36:38 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3059B7D6BB; Fri, 3 Aug 2018 11:36:38 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 102671FDF6; Fri, 3 Aug 2018 11:36:38 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73Bab4H087442; Fri, 3 Aug 2018 11:36:37 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73Babg0087440; Fri, 3 Aug 2018 11:36:37 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808031136.w73Babg0087440@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 3 Aug 2018 11:36:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337235 - in stable/11/sys: amd64/amd64 x86/x86 X-SVN-Group: stable-11 X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in stable/11/sys: amd64/amd64 x86/x86 X-SVN-Commit-Revision: 337235 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 11:36:38 -0000 Author: kib Date: Fri Aug 3 11:36:37 2018 New Revision: 337235 URL: https://svnweb.freebsd.org/changeset/base/337235 Log: MFC r336763: Add workarounds for several Ryzen erratas, on amd64. Modified: stable/11/sys/amd64/amd64/initcpu.c stable/11/sys/x86/x86/cpu_machdep.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/amd64/initcpu.c ============================================================================== --- stable/11/sys/amd64/amd64/initcpu.c Fri Aug 3 10:59:05 2018 (r337234) +++ stable/11/sys/amd64/amd64/initcpu.c Fri Aug 3 11:36:37 2018 (r337235) @@ -128,6 +128,30 @@ init_amd(void) } } + /* Ryzen erratas. */ + if (CPUID_TO_FAMILY(cpu_id) == 0x17 && CPUID_TO_MODEL(cpu_id) == 0x1 && + (cpu_feature2 & CPUID2_HV) == 0) { + /* 1021 */ + msr = rdmsr(0xc0011029); + msr |= 0x2000; + wrmsr(0xc0011029, msr); + + /* 1033 */ + msr = rdmsr(0xc0011020); + msr |= 0x10; + wrmsr(0xc0011020, msr); + + /* 1049 */ + msr = rdmsr(0xc0011028); + msr |= 0x10; + wrmsr(0xc0011028, msr); + + /* 1095 */ + msr = rdmsr(0xc0011020); + msr |= 0x200000000000000; + wrmsr(0xc0011020, msr); + } + /* * Work around a problem on Ryzen that is triggered by executing * code near the top of user memory, in our case the signal Modified: stable/11/sys/x86/x86/cpu_machdep.c ============================================================================== --- stable/11/sys/x86/x86/cpu_machdep.c Fri Aug 3 10:59:05 2018 (r337234) +++ stable/11/sys/x86/x86/cpu_machdep.c Fri Aug 3 11:36:37 2018 (r337235) @@ -739,6 +739,13 @@ cpu_idle_tun(void *unused __unused) if (TUNABLE_STR_FETCH("machdep.idle", tunvar, sizeof(tunvar))) cpu_idle_selector(tunvar); + else if (cpu_vendor_id == CPU_VENDOR_AMD && + CPUID_TO_FAMILY(cpu_id) == 0x17 && CPUID_TO_MODEL(cpu_id) == 0x1) { + /* Ryzen erratas 1057, 1109. */ + cpu_idle_selector("hlt"); + idle_mwait = 0; + } + if (cpu_vendor_id == CPU_VENDOR_INTEL && cpu_id == 0x506c9) { /* * Apollo Lake errata APL31 (public errata APL30). From owner-svn-src-all@freebsd.org Fri Aug 3 12:14:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2D0910688F7; Fri, 3 Aug 2018 12:14:30 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 891317F422; Fri, 3 Aug 2018 12:14:30 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6671D2046C; Fri, 3 Aug 2018 12:14:30 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73CEUZ3007932; Fri, 3 Aug 2018 12:14:30 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73CEUtV007931; Fri, 3 Aug 2018 12:14:30 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808031214.w73CEUtV007931@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 3 Aug 2018 12:14:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337236 - head/share/man/man9 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/share/man/man9 X-SVN-Commit-Revision: 337236 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 12:14:31 -0000 Author: kib Date: Fri Aug 3 12:14:29 2018 New Revision: 337236 URL: https://svnweb.freebsd.org/changeset/base/337236 Log: Some updates to vm_map(9). Remove min_offset, max_offset. Mention the busy flag. Prodded by: https://reviews.freebsd.org/D16105. Sponsored by: The FreeBSD Foundation MFC after: 1 week Modified: head/share/man/man9/vm_map.9 Modified: head/share/man/man9/vm_map.9 ============================================================================== --- head/share/man/man9/vm_map.9 Fri Aug 3 11:36:37 2018 (r337235) +++ head/share/man/man9/vm_map.9 Fri Aug 3 12:14:29 2018 (r337236) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd July 9, 2011 +.Dd July 3, 2018 .Dt VM_MAP 9 .Os .Sh NAME @@ -62,8 +62,7 @@ struct vm_map { vm_flags_t flags; vm_map_entry_t root; pmap_t pmap; -#define min_offset header.start -#define max_offset header.end + int busy; }; .Ed .Pp @@ -99,29 +98,16 @@ Root node of a binary search tree used for fast lookup .It Va pmap Pointer to the underlying physical map with which this virtual map is associated. -.It Va min_offset -The minimum -.Vt vm_offset_t -in this map. -Programs should never use -.Va header.start -or -.Va header.end -directly, use -.Va min_offset -and -.Va max_offset -instead. -.It Va max_offset -The maximum -.Vt vm_offset_t -in this map. +.It Va busy +Map busy counter, prevents forks. .El .Pp -There is one possible map flag: +Possible map flags: .Bl -tag -width ".Dv MAP_PREFAULT_MADVISE" .It Dv MAP_WIREFUTURE Wire all future pages in this map. +.It Dv MAP_BUSY_WAKEUP +There are waiters for the map busy status. .El .Pp The following flags can be passed to From owner-svn-src-all@freebsd.org Fri Aug 3 12:16:03 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6125510689AE; Fri, 3 Aug 2018 12:16:03 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1761F7F5B3; Fri, 3 Aug 2018 12:16:03 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ECBB12046E; Fri, 3 Aug 2018 12:16:02 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73CG2NE008042; Fri, 3 Aug 2018 12:16:02 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73CG2Ug008041; Fri, 3 Aug 2018 12:16:02 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808031216.w73CG2Ug008041@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Fri, 3 Aug 2018 12:16:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337237 - head/tests/sys/geom/class/nop X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/tests/sys/geom/class/nop X-SVN-Commit-Revision: 337237 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 12:16:03 -0000 Author: br Date: Fri Aug 3 12:16:02 2018 New Revision: 337237 URL: https://svnweb.freebsd.org/changeset/base/337237 Log: Increase timeout for nop_test:stripesize. It takes 49s to complete this test in QEMU/RISC-V. Sponsored by: DARPA, AFRL Modified: head/tests/sys/geom/class/nop/nop_test.sh Modified: head/tests/sys/geom/class/nop/nop_test.sh ============================================================================== --- head/tests/sys/geom/class/nop/nop_test.sh Fri Aug 3 12:14:29 2018 (r337236) +++ head/tests/sys/geom/class/nop/nop_test.sh Fri Aug 3 12:16:02 2018 (r337237) @@ -201,7 +201,7 @@ stripesize_head() { atf_set "descr" "Test gnop's -p and -P options" atf_set "require.user" "root" - atf_set "timeout" 15 + atf_set "timeout" 120 } stripesize_body() { From owner-svn-src-all@freebsd.org Fri Aug 3 12:47:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8B4D106932A; Fri, 3 Aug 2018 12:47:55 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 802E2802B3; Fri, 3 Aug 2018 12:47:55 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 53BC720947; Fri, 3 Aug 2018 12:47:55 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73Clt7q023261; Fri, 3 Aug 2018 12:47:55 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73CltUv023260; Fri, 3 Aug 2018 12:47:55 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808031247.w73CltUv023260@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Fri, 3 Aug 2018 12:47:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337238 - head/contrib/netbsd-tests/lib/libc/sys X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/contrib/netbsd-tests/lib/libc/sys X-SVN-Commit-Revision: 337238 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 12:47:56 -0000 Author: br Date: Fri Aug 3 12:47:54 2018 New Revision: 337238 URL: https://svnweb.freebsd.org/changeset/base/337238 Log: Replace __riscv__ with __riscv. __riscv__ is not pre-defined anymore by latest version of GNU compiler. Sponsored by: DARPA, AFRL Modified: head/contrib/netbsd-tests/lib/libc/sys/t_getcontext.c Modified: head/contrib/netbsd-tests/lib/libc/sys/t_getcontext.c ============================================================================== --- head/contrib/netbsd-tests/lib/libc/sys/t_getcontext.c Fri Aug 3 12:16:02 2018 (r337237) +++ head/contrib/netbsd-tests/lib/libc/sys/t_getcontext.c Fri Aug 3 12:47:54 2018 (r337238) @@ -54,7 +54,7 @@ run(int n, ...) #ifdef __FreeBSD__ #if defined(__amd64__) || defined(__sparc64__) for (i = 0; i < 5; i++) { -#elif defined(__aarch64__) || defined(__riscv__) +#elif defined(__aarch64__) || defined(__riscv) for (i = 0; i < 7; i++) { #else for (i = 0; i < 9; i++) { @@ -126,7 +126,7 @@ ATF_TC_BODY(setcontext_link, tc) */ makecontext(&uc[i], (void *)run, 6, i, 0, 1, 2, 3, 4); -#elif defined(__aarch64__) || defined(__riscv__) +#elif defined(__aarch64__) || defined(__riscv) /* * FreeBSD/arm64 and FreeBSD/riscv64 only permit up to * 8 arguments. From owner-svn-src-all@freebsd.org Fri Aug 3 14:01:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60FAE106AB29; Fri, 3 Aug 2018 14:01:33 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1424882740; Fri, 3 Aug 2018 14:01:33 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CF698215AE; Fri, 3 Aug 2018 14:01:32 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73E1WhV061065; Fri, 3 Aug 2018 14:01:32 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73E1WJY061063; Fri, 3 Aug 2018 14:01:32 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031401.w73E1WJY061063@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:01:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337239 - in stable/11: sys/kern tests/sys/aio X-SVN-Group: stable-11 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/11: sys/kern tests/sys/aio X-SVN-Commit-Revision: 337239 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:01:33 -0000 Author: asomers Date: Fri Aug 3 14:01:32 2018 New Revision: 337239 URL: https://svnweb.freebsd.org/changeset/base/337239 Log: MFC r332631: lio_listio: return EAGAIN instead of EIO when out of resources This behavior is already documented by the man page, and suggested by POSIX. Reviewed by: jhb Differential Revision: https://reviews.freebsd.org/D15099 Modified: stable/11/sys/kern/vfs_aio.c stable/11/tests/sys/aio/Makefile stable/11/tests/sys/aio/lio_test.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/kern/vfs_aio.c ============================================================================== --- stable/11/sys/kern/vfs_aio.c Fri Aug 3 12:47:54 2018 (r337238) +++ stable/11/sys/kern/vfs_aio.c Fri Aug 3 14:01:32 2018 (r337239) @@ -2135,7 +2135,7 @@ kern_lio_listio(struct thread *td, int mode, struct ai struct aioliojob *lj; struct kevent kev; int error; - int nerror; + int nagain, nerror; int i; if ((mode != LIO_NOWAIT) && (mode != LIO_WAIT)) @@ -2204,12 +2204,15 @@ kern_lio_listio(struct thread *td, int mode, struct ai /* * Get pointers to the list of I/O requests. */ + nagain = 0; nerror = 0; for (i = 0; i < nent; i++) { job = acb_list[i]; if (job != NULL) { error = aio_aqueue(td, job, lj, LIO_NOP, ops); - if (error != 0) + if (error == EAGAIN) + nagain++; + else if (error != 0) nerror++; } } @@ -2256,7 +2259,10 @@ kern_lio_listio(struct thread *td, int mode, struct ai if (nerror) return (EIO); - return (error); + else if (nagain) + return (EAGAIN); + else + return (error); } /* syscall - list directed I/O (REALTIME) */ Modified: stable/11/tests/sys/aio/Makefile ============================================================================== --- stable/11/tests/sys/aio/Makefile Fri Aug 3 12:47:54 2018 (r337238) +++ stable/11/tests/sys/aio/Makefile Fri Aug 3 14:01:32 2018 (r337239) @@ -6,6 +6,8 @@ ATF_TESTS_C+= aio_test ATF_TESTS_C+= lio_test TEST_METADATA.aio_test+= timeout="30" TEST_METADATA.lio_test+= timeout="10" +# Some lio_test testcases involve system resource limitations, so cannot run concurrently +TEST_METADATA.lio_test+= is_exclusive=true PLAIN_TESTS_C+= aio_kqueue_test PLAIN_TESTS_C+= lio_kqueue_test Modified: stable/11/tests/sys/aio/lio_test.c ============================================================================== --- stable/11/tests/sys/aio/lio_test.c Fri Aug 3 12:47:54 2018 (r337238) +++ stable/11/tests/sys/aio/lio_test.c Fri Aug 3 14:01:32 2018 (r337239) @@ -30,10 +30,13 @@ #include #include +#include #include +#include #include +#include "local.h" #include "freebsd_test_suite/macros.h" static sem_t completions; @@ -51,6 +54,73 @@ thr_handler(union sigval sv __unused) ATF_REQUIRE_EQ(0, sem_post(&completions)); } +/* + * If lio_listio is unable to enqueue any requests at all, it should return + * EAGAIN. + */ +ATF_TC_WITHOUT_HEAD(lio_listio_eagain_kevent); +ATF_TC_BODY(lio_listio_eagain_kevent, tc) +{ + int fd, i, j, kq, max_queue_per_proc, ios_per_call; + size_t max_queue_per_proc_size; + struct aiocb *aiocbs[2]; + struct aiocb **list[2]; + struct sigevent sev[2]; + char *buffer; + const char *path="tempfile"; + void *udata[2]; + + ATF_REQUIRE_KERNEL_MODULE("aio"); + ATF_REQUIRE_UNSAFE_AIO(); + + max_queue_per_proc_size = sizeof(max_queue_per_proc); + ATF_REQUIRE_EQ(sysctlbyname("vfs.aio.max_aio_queue_per_proc", + &max_queue_per_proc, &max_queue_per_proc_size, NULL, 0), 0); + ios_per_call = max_queue_per_proc; + + fd = open(path, O_RDWR|O_CREAT, 0666); + ATF_REQUIRE(fd >= 0); + + kq = kqueue(); + ATF_REQUIRE(kq > 0); + + buffer = calloc(1, 4096); + ATF_REQUIRE(buffer != NULL); + + /* + * Call lio_listio twice, each with the maximum number of operations. + * The first call should succeed and the second should fail. + */ + for (i = 0; i < 2; i++) { + aiocbs[i] = calloc(ios_per_call, sizeof(struct aiocb)); + ATF_REQUIRE(aiocbs[i] != NULL); + list[i] = calloc(ios_per_call, sizeof(struct aiocb*)); + ATF_REQUIRE(list[i] != NULL); + udata[i] = (void*)((caddr_t)0xdead0000 + i); + sev[i].sigev_notify = SIGEV_KEVENT; + sev[i].sigev_notify_kqueue = kq; + sev[i].sigev_value.sival_ptr = udata[i]; + for (j = 0; j < ios_per_call; j++) { + aiocbs[i][j].aio_fildes = fd; + aiocbs[i][j].aio_offset = (i * ios_per_call + j) * 4096; + aiocbs[i][j].aio_buf = buffer; + aiocbs[i][j].aio_nbytes = 4096; + aiocbs[i][j].aio_lio_opcode = LIO_WRITE; + list[i][j] = &aiocbs[i][j]; + } + } + + ATF_REQUIRE_EQ(0, lio_listio(LIO_NOWAIT, list[0], ios_per_call, &sev[0])); + ATF_REQUIRE_EQ(-1, lio_listio(LIO_NOWAIT, list[1], ios_per_call, &sev[1])); + /* + * The second lio_listio call should fail with EAGAIN. Bad timing may + * mean that some requests did get enqueued, but the result should + * still be EAGAIN. + */ + ATF_REQUIRE_EQ(errno, EAGAIN); +} + + /* With LIO_WAIT, an empty lio_listio should return immediately */ ATF_TC_WITHOUT_HEAD(lio_listio_empty_wait); ATF_TC_BODY(lio_listio_empty_wait, tc) @@ -136,6 +206,7 @@ ATF_TC_BODY(lio_listio_empty_nowait_thread, tc) ATF_TP_ADD_TCS(tp) { + ATF_TP_ADD_TC(tp, lio_listio_eagain_kevent); ATF_TP_ADD_TC(tp, lio_listio_empty_nowait_kevent); ATF_TP_ADD_TC(tp, lio_listio_empty_nowait_signal); ATF_TP_ADD_TC(tp, lio_listio_empty_nowait_thread); From owner-svn-src-all@freebsd.org Fri Aug 3 14:03:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB68B106ABD1; Fri, 3 Aug 2018 14:03:51 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70CBA82B2F; Fri, 3 Aug 2018 14:03:51 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 37618215F2; Fri, 3 Aug 2018 14:03:51 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73E3p8X064464; Fri, 3 Aug 2018 14:03:51 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73E3pW5064463; Fri, 3 Aug 2018 14:03:51 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031403.w73E3pW5064463@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:03:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337241 - stable/11/contrib/openbsm/bin/auditd X-SVN-Group: stable-11 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/11/contrib/openbsm/bin/auditd X-SVN-Commit-Revision: 337241 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:03:52 -0000 Author: asomers Date: Fri Aug 3 14:03:50 2018 New Revision: 337241 URL: https://svnweb.freebsd.org/changeset/base/337241 Log: MFC r335899: auditd(8): register signal handlers interrutibly auditd_wait_for_events() relies on read(2) being interrupted by signals, but it registers signal handlers with signal(3), which sets SA_RESTART. That breaks asynchronous signal handling. It means that signals don't actually get handled until after an audit(8) trigger is received. Symptoms include: * Sending SIGTERM to auditd doesn't kill it right away; you must send SIGTERM and then send a trigger with auditon(2). * Same with SIGHUP * Zombie child processes don't get reaped until auditd receives a trigger sent by auditon. This includes children created by expiring audit trails at auditd startup. Fix by using sigaction(2) instead of signal(3). Cherry pick https://github.com/openbsm/openbsm/commit/d060887 PR: 229381 Reviewed by: cem Obtained from: OpenBSM Differential Revision: https://github.com/openbsm/openbsm/pull/36 Modified: stable/11/contrib/openbsm/bin/auditd/auditd.c Directory Properties: stable/11/ (props changed) Modified: stable/11/contrib/openbsm/bin/auditd/auditd.c ============================================================================== --- stable/11/contrib/openbsm/bin/auditd/auditd.c Fri Aug 3 14:02:51 2018 (r337240) +++ stable/11/contrib/openbsm/bin/auditd/auditd.c Fri Aug 3 14:03:50 2018 (r337241) @@ -415,27 +415,35 @@ close_all(void) static int register_daemon(void) { + struct sigaction action; FILE * pidfile; int fd; pid_t pid; /* Set up the signal hander. */ - if (signal(SIGTERM, auditd_relay_signal) == SIG_ERR) { + action.sa_handler = auditd_relay_signal; + /* + * sa_flags must not include SA_RESTART, so that read(2) will be + * interruptible in auditd_wait_for_events + */ + action.sa_flags = 0; + sigemptyset(&action.sa_mask); + if (sigaction(SIGTERM, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGTERM"); fail_exit(); } - if (signal(SIGCHLD, auditd_relay_signal) == SIG_ERR) { + if (sigaction(SIGCHLD, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGCHLD"); fail_exit(); } - if (signal(SIGHUP, auditd_relay_signal) == SIG_ERR) { + if (sigaction(SIGHUP, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGHUP"); fail_exit(); } - if (signal(SIGALRM, auditd_relay_signal) == SIG_ERR) { + if (sigaction(SIGALRM, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGALRM"); fail_exit(); From owner-svn-src-all@freebsd.org Fri Aug 3 14:05:23 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA6F4106AC57; Fri, 3 Aug 2018 14:05:23 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E2AD82C94; Fri, 3 Aug 2018 14:05:23 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3EA5C215F4; Fri, 3 Aug 2018 14:05:23 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73E5NfW064612; Fri, 3 Aug 2018 14:05:23 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73E5MrJ064610; Fri, 3 Aug 2018 14:05:22 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031405.w73E5MrJ064610@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:05:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337242 - stable/11/sys/kern X-SVN-Group: stable-11 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/11/sys/kern X-SVN-Commit-Revision: 337242 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:05:23 -0000 Author: asomers Date: Fri Aug 3 14:05:22 2018 New Revision: 337242 URL: https://svnweb.freebsd.org/changeset/base/337242 Log: MFC r336205: Don't acquire evclass_lock with a spinlock held When the "pc" audit class is enabled and auditd is running, witness will panic during thread exit because au_event_class tries to lock an rwlock while holding a spinlock acquired upstack by thread_exit. To fix this, move AUDIT_SYSCALL_EXIT futher upstack, before the spinlock is acquired. Of thread_exit's 16 callers, it's only necessary to call AUDIT_SYSCALL_EXIT from two, exit1 (for exiting processes) and kern_thr_exit (for exiting threads). The other callers are all kernel threads, which needen't call AUDIT_SYSCALL_EXIT because since they can't make syscalls there will be nothing to audit. And exit1 already does call AUDIT_SYSCALL_EXIT, making the second call in thread_exit redundant for that case. PR: 228444 Reported by: aniketp Reviewed by: aniketp, kib Differential Revision: https://reviews.freebsd.org/D16210 Modified: stable/11/sys/kern/kern_thr.c stable/11/sys/kern/kern_thread.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/kern/kern_thr.c ============================================================================== --- stable/11/sys/kern/kern_thr.c Fri Aug 3 14:03:50 2018 (r337241) +++ stable/11/sys/kern/kern_thr.c Fri Aug 3 14:05:22 2018 (r337242) @@ -375,6 +375,11 @@ kern_thr_exit(struct thread *td) KASSERT(p->p_numthreads > 1, ("too few threads")); racct_sub(p, RACCT_NTHR, 1); tdsigcleanup(td); + +#ifdef AUDIT + AUDIT_SYSCALL_EXIT(0, td); +#endif + PROC_SLOCK(p); thread_stopped(p); thread_exit(); Modified: stable/11/sys/kern/kern_thread.c ============================================================================== --- stable/11/sys/kern/kern_thread.c Fri Aug 3 14:03:50 2018 (r337241) +++ stable/11/sys/kern/kern_thread.c Fri Aug 3 14:05:22 2018 (r337242) @@ -533,9 +533,6 @@ thread_exit(void) SDT_PROBE0(proc, , , lwp__exit); KASSERT(TAILQ_EMPTY(&td->td_sigqueue.sq_list), ("signal pending")); -#ifdef AUDIT - AUDIT_SYSCALL_EXIT(0, td); -#endif /* * drop FPU & debug register state storage, or any other * architecture specific resources that From owner-svn-src-all@freebsd.org Fri Aug 3 14:06:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA5DE106ACB7; Fri, 3 Aug 2018 14:06:09 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CA3D82DD4; Fri, 3 Aug 2018 14:06:09 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3D7E7215F7; Fri, 3 Aug 2018 14:06:09 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73E69Va064716; Fri, 3 Aug 2018 14:06:09 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73E69b9064715; Fri, 3 Aug 2018 14:06:09 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031406.w73E69b9064715@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:06:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337243 - stable/11/sys/security/audit X-SVN-Group: stable-11 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/11/sys/security/audit X-SVN-Commit-Revision: 337243 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:06:09 -0000 Author: asomers Date: Fri Aug 3 14:06:08 2018 New Revision: 337243 URL: https://svnweb.freebsd.org/changeset/base/337243 Log: MFC r336319: auditon(2): fix A_SETPOLICY with 64-bit values A_SETPOLICY is supposed to work with either 64 or 32-bit values, but due to a typo the 64-bit version has never worked correctly. Submitted by: aniketp Reviewed by: asomers, cem Sponsored by: Google, Inc. (GSoC 2018) Differential Revision: https://reviews.freebsd.org/D16222 Modified: stable/11/sys/security/audit/audit_syscalls.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/security/audit/audit_syscalls.c ============================================================================== --- stable/11/sys/security/audit/audit_syscalls.c Fri Aug 3 14:05:22 2018 (r337242) +++ stable/11/sys/security/audit/audit_syscalls.c Fri Aug 3 14:06:08 2018 (r337243) @@ -241,7 +241,7 @@ sys_auditon(struct thread *td, struct auditon_args *ua case A_OLDSETPOLICY: case A_SETPOLICY: if (uap->length == sizeof(udata.au_policy64)) { - if (udata.au_policy & (~AUDIT_CNT|AUDIT_AHLT| + if (udata.au_policy & ~(AUDIT_CNT|AUDIT_AHLT| AUDIT_ARGV|AUDIT_ARGE)) return (EINVAL); audit_fail_stop = ((udata.au_policy64 & AUDIT_CNT) == From owner-svn-src-all@freebsd.org Fri Aug 3 14:08:40 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8394B106AD64; Fri, 3 Aug 2018 14:08:40 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3520E82F63; Fri, 3 Aug 2018 14:08:40 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E9064215F8; Fri, 3 Aug 2018 14:08:39 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73E8dKu064868; Fri, 3 Aug 2018 14:08:39 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73E8dEW064866; Fri, 3 Aug 2018 14:08:39 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031408.w73E8dEW064866@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:08:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337244 - in stable/10: contrib/netbsd-tests/lib/libc/sys lib/libc/sys X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/10: contrib/netbsd-tests/lib/libc/sys lib/libc/sys X-SVN-Commit-Revision: 337244 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:08:40 -0000 Author: asomers Date: Fri Aug 3 14:08:39 2018 New Revision: 337244 URL: https://svnweb.freebsd.org/changeset/base/337244 Log: MFC r328266: mlock(2): correct documentation for error conditions. The man page is years out of date regarding errors. Our implementation _does_ allow unaligned addresses, and it _does_not_ check for negative lengths, because the length is unsigned. It checks for overflow instead. Update the tests accordingly. Reviewed by: bcr Differential Revision: https://reviews.freebsd.org/D13826 Modified: stable/10/contrib/netbsd-tests/lib/libc/sys/t_mlock.c stable/10/lib/libc/sys/mlock.2 Directory Properties: stable/10/ (props changed) Modified: stable/10/contrib/netbsd-tests/lib/libc/sys/t_mlock.c ============================================================================== --- stable/10/contrib/netbsd-tests/lib/libc/sys/t_mlock.c Fri Aug 3 14:06:08 2018 (r337243) +++ stable/10/contrib/netbsd-tests/lib/libc/sys/t_mlock.c Fri Aug 3 14:08:39 2018 (r337244) @@ -131,33 +131,38 @@ ATF_TC_BODY(mlock_err, tc) ATF_REQUIRE_ERRNO(ENOMEM, mlock((char *)0, page) == -1); errno = 0; - ATF_REQUIRE_ERRNO(ENOMEM, mlock((char *)-1, page) == -1); - - errno = 0; ATF_REQUIRE_ERRNO(ENOMEM, munlock(NULL, page) == -1); errno = 0; ATF_REQUIRE_ERRNO(ENOMEM, munlock((char *)0, page) == -1); +#ifdef __FreeBSD__ + /* Wrap around should return EINVAL */ errno = 0; + ATF_REQUIRE_ERRNO(EINVAL, mlock((char *)-1, page) == -1); + errno = 0; + ATF_REQUIRE_ERRNO(EINVAL, munlock((char *)-1, page) == -1); +#else + errno = 0; + ATF_REQUIRE_ERRNO(ENOMEM, mlock((char *)-1, page) == -1); + errno = 0; ATF_REQUIRE_ERRNO(ENOMEM, munlock((char *)-1, page) == -1); +#endif - buf = malloc(page); + buf = malloc(page); /* Get a valid address */ ATF_REQUIRE(buf != NULL); - - /* - * unlocking memory that is not locked is an error... - */ - +#ifdef __FreeBSD__ errno = 0; - ATF_REQUIRE_ERRNO(ENOMEM, munlock(buf, page) == -1); - - /* - * These are permitted to fail (EINVAL) but do not on NetBSD - */ - ATF_REQUIRE(mlock((void *)(((uintptr_t)buf) + page/3), page/5) == 0); - ATF_REQUIRE(munlock((void *)(((uintptr_t)buf) + page/3), page/5) == 0); - + /* Wrap around should return EINVAL */ + ATF_REQUIRE_ERRNO(EINVAL, mlock(buf, -page) == -1); + errno = 0; + ATF_REQUIRE_ERRNO(EINVAL, munlock(buf, -page) == -1); +#else + errno = 0; + ATF_REQUIRE_ERRNO(ENOMEM, mlock(buf, -page) == -1); + errno = 0; + ATF_REQUIRE_ERRNO(ENOMEM, munlock(buf, -page) == -1); +#endif (void)free(buf); /* @@ -355,6 +360,80 @@ ATF_TC_CLEANUP(mlock_nested, tc) } #endif +#ifdef __FreeBSD__ +ATF_TC_WITH_CLEANUP(mlock_unaligned); +#else +ATF_TC(mlock_unaligned); +#endif +ATF_TC_HEAD(mlock_unaligned, tc) +{ + atf_tc_set_md_var(tc, "descr", + "Test that mlock(2) can lock page-unaligned memory"); +#ifdef __FreeBSD__ + atf_tc_set_md_var(tc, "require.config", "allow_sysctl_side_effects"); + atf_tc_set_md_var(tc, "require.user", "root"); +#endif +} + +ATF_TC_BODY(mlock_unaligned, tc) +{ + void *buf, *addr; + +#ifdef __FreeBSD__ + /* Set max_wired really really high to avoid EAGAIN */ + set_vm_max_wired(INT_MAX); +#endif + + buf = malloc(page); + ATF_REQUIRE(buf != NULL); + + if ((uintptr_t)buf & ((uintptr_t)page - 1)) + addr = buf; + else + addr = (void *)(((uintptr_t)buf) + page/3); + + ATF_REQUIRE_EQ(mlock(addr, page/5), 0); + ATF_REQUIRE_EQ(munlock(addr, page/5), 0); + + (void)free(buf); +} + +#ifdef __FreeBSD__ +ATF_TC_CLEANUP(mlock_unaligned, tc) +{ + + restore_vm_max_wired(); +} +#endif + +ATF_TC(munlock_unlocked); +ATF_TC_HEAD(munlock_unlocked, tc) +{ + atf_tc_set_md_var(tc, "descr", +#ifdef __FreeBSD__ + "munlock(2) accepts unlocked memory"); +#else + "munlock(2) of unlocked memory is an error"); +#endif + atf_tc_set_md_var(tc, "require.user", "root"); +} + +ATF_TC_BODY(munlock_unlocked, tc) +{ + void *buf; + + buf = malloc(page); + ATF_REQUIRE(buf != NULL); + +#ifdef __FreeBSD__ + ATF_REQUIRE_EQ(munlock(buf, page), 0); +#else + errno = 0; + ATF_REQUIRE_ERRNO(ENOMEM, munlock(buf, page) == -1); +#endif + (void)free(buf); +} + ATF_TP_ADD_TCS(tp) { @@ -366,6 +445,8 @@ ATF_TP_ADD_TCS(tp) ATF_TP_ADD_TC(tp, mlock_limits); ATF_TP_ADD_TC(tp, mlock_mmap); ATF_TP_ADD_TC(tp, mlock_nested); + ATF_TP_ADD_TC(tp, mlock_unaligned); + ATF_TP_ADD_TC(tp, munlock_unlocked); return atf_no_error(); } Modified: stable/10/lib/libc/sys/mlock.2 ============================================================================== --- stable/10/lib/libc/sys/mlock.2 Fri Aug 3 14:06:08 2018 (r337243) +++ stable/10/lib/libc/sys/mlock.2 Fri Aug 3 14:08:39 2018 (r337244) @@ -28,7 +28,7 @@ .\" @(#)mlock.2 8.2 (Berkeley) 12/11/93 .\" $FreeBSD$ .\" -.Dd May 17, 2014 +.Dd Jan 22, 2018 .Dt MLOCK 2 .Os .Sh NAME @@ -125,7 +125,7 @@ will fail if: .Va security.bsd.unprivileged_mlock is set to 0 and the caller is not the super-user. .It Bq Er EINVAL -The address given is not page aligned or the length is negative. +The address range given wraps around zero. .It Bq Er EAGAIN Locking the indicated range would exceed the system limit for locked memory. .It Bq Er ENOMEM @@ -143,7 +143,7 @@ will fail if: .Va security.bsd.unprivileged_mlock is set to 0 and the caller is not the super-user. .It Bq Er EINVAL -The address given is not page aligned or the length is negative. +The address range given wraps around zero. .It Bq Er ENOMEM Some or all of the address range specified by the addr and len arguments does not correspond to valid mapped pages in the address space From owner-svn-src-all@freebsd.org Fri Aug 3 14:12:39 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43A1A106AF70; Fri, 3 Aug 2018 14:12:39 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB76C83368; Fri, 3 Aug 2018 14:12:38 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CC6C12179B; Fri, 3 Aug 2018 14:12:38 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73ECc6O069911; Fri, 3 Aug 2018 14:12:38 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73ECcst069907; Fri, 3 Aug 2018 14:12:38 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808031412.w73ECcst069907@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 3 Aug 2018 14:12:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337245 - in stable/10/sys: amd64/amd64 i386/i386 i386/isa X-SVN-Group: stable-10 X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in stable/10/sys: amd64/amd64 i386/i386 i386/isa X-SVN-Commit-Revision: 337245 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:12:39 -0000 Author: kib Date: Fri Aug 3 14:12:37 2018 New Revision: 337245 URL: https://svnweb.freebsd.org/changeset/base/337245 Log: MFC r336683: Extend ranges of the critical sections to ensure that context switch code never sees FPU pcb flags not consistent with the hardware state. Modified: stable/10/sys/amd64/amd64/fpu.c stable/10/sys/amd64/amd64/machdep.c stable/10/sys/i386/i386/machdep.c stable/10/sys/i386/isa/npx.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/amd64/amd64/fpu.c ============================================================================== --- stable/10/sys/amd64/amd64/fpu.c Fri Aug 3 14:08:39 2018 (r337244) +++ stable/10/sys/amd64/amd64/fpu.c Fri Aug 3 14:12:37 2018 (r337245) @@ -741,22 +741,22 @@ fpugetregs(struct thread *td) int max_ext_n, i, owned; pcb = td->td_pcb; + critical_enter(); if ((pcb->pcb_flags & PCB_USERFPUINITDONE) == 0) { bcopy(fpu_initialstate, get_pcb_user_save_pcb(pcb), cpu_max_ext_state_size); get_pcb_user_save_pcb(pcb)->sv_env.en_cw = pcb->pcb_initial_fpucw; fpuuserinited(td); + critical_exit(); return (_MC_FPOWNED_PCB); } - critical_enter(); if (td == PCPU_GET(fpcurthread) && PCB_USER_FPU(pcb)) { fpusave(get_pcb_user_save_pcb(pcb)); owned = _MC_FPOWNED_FPU; } else { owned = _MC_FPOWNED_PCB; } - critical_exit(); if (use_xsave) { /* * Handle partially saved state. @@ -776,6 +776,7 @@ fpugetregs(struct thread *td) *xstate_bv |= bit; } } + critical_exit(); return (owned); } @@ -784,6 +785,7 @@ fpuuserinited(struct thread *td) { struct pcb *pcb; + CRITICAL_ASSERT(td); pcb = td->td_pcb; if (PCB_USER_FPU(pcb)) set_pcb_flags(pcb, @@ -842,26 +844,25 @@ fpusetregs(struct thread *td, struct savefpu *addr, ch addr->sv_env.en_mxcsr &= cpu_mxcsr_mask; pcb = td->td_pcb; + error = 0; critical_enter(); if (td == PCPU_GET(fpcurthread) && PCB_USER_FPU(pcb)) { error = fpusetxstate(td, xfpustate, xfpustate_size); - if (error != 0) { - critical_exit(); - return (error); + if (error == 0) { + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + fpurestore(get_pcb_user_save_td(td)); + set_pcb_flags(pcb, PCB_FPUINITDONE | + PCB_USERFPUINITDONE); } - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - fpurestore(get_pcb_user_save_td(td)); - critical_exit(); - set_pcb_flags(pcb, PCB_FPUINITDONE | PCB_USERFPUINITDONE); } else { - critical_exit(); error = fpusetxstate(td, xfpustate, xfpustate_size); - if (error != 0) - return (error); - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - fpuuserinited(td); + if (error == 0) { + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + fpuuserinited(td); + } } - return (0); + critical_exit(); + return (error); } /* @@ -1004,6 +1005,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx return (0); } pcb = td->td_pcb; + critical_enter(); KASSERT(!PCB_USER_FPU(pcb) || pcb->pcb_save == get_pcb_user_save_pcb(pcb), ("mangled pcb_save")); ctx->flags = 0; @@ -1014,6 +1016,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx pcb->pcb_save = fpu_kern_ctx_savefpu(ctx); set_pcb_flags(pcb, PCB_KERNFPU); clear_pcb_flags(pcb, PCB_FPUINITDONE); + critical_exit(); return (0); } @@ -1029,7 +1032,6 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx critical_enter(); if (curthread == PCPU_GET(fpcurthread)) fpudrop(); - critical_exit(); pcb->pcb_save = ctx->prev; if (pcb->pcb_save == get_pcb_user_save_pcb(pcb)) { if ((pcb->pcb_flags & PCB_USERFPUINITDONE) != 0) { @@ -1044,6 +1046,7 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx clear_pcb_flags(pcb, PCB_FPUINITDONE); KASSERT(!PCB_USER_FPU(pcb), ("unpaired fpu_kern_leave")); } + critical_exit(); return (0); } Modified: stable/10/sys/amd64/amd64/machdep.c ============================================================================== --- stable/10/sys/amd64/amd64/machdep.c Fri Aug 3 14:08:39 2018 (r337244) +++ stable/10/sys/amd64/amd64/machdep.c Fri Aug 3 14:12:37 2018 (r337245) @@ -2380,8 +2380,10 @@ int set_fpregs(struct thread *td, struct fpreg *fpregs) { + critical_enter(); set_fpregs_xmm(fpregs, get_pcb_user_save_td(td)); fpuuserinited(td); + critical_exit(); return (0); } Modified: stable/10/sys/i386/i386/machdep.c ============================================================================== --- stable/10/sys/i386/i386/machdep.c Fri Aug 3 14:08:39 2018 (r337244) +++ stable/10/sys/i386/i386/machdep.c Fri Aug 3 14:12:37 2018 (r337245) @@ -3780,6 +3780,7 @@ int set_fpregs(struct thread *td, struct fpreg *fpregs) { + critical_enter(); #ifdef CPU_ENABLE_SSE if (cpu_fxsr) set_fpregs_xmm((struct save87 *)fpregs, @@ -3791,6 +3792,7 @@ set_fpregs(struct thread *td, struct fpreg *fpregs) #ifdef DEV_NPX npxuserinited(td); #endif + critical_exit(); return (0); } Modified: stable/10/sys/i386/isa/npx.c ============================================================================== --- stable/10/sys/i386/isa/npx.c Fri Aug 3 14:08:39 2018 (r337244) +++ stable/10/sys/i386/isa/npx.c Fri Aug 3 14:12:37 2018 (r337245) @@ -1037,14 +1037,15 @@ npxgetregs(struct thread *td) return (_MC_FPOWNED_NONE); pcb = td->td_pcb; + critical_enter(); if ((pcb->pcb_flags & PCB_NPXINITDONE) == 0) { bcopy(npx_initialstate, get_pcb_user_save_pcb(pcb), cpu_max_ext_state_size); SET_FPU_CW(get_pcb_user_save_pcb(pcb), pcb->pcb_initial_npxcw); npxuserinited(td); + critical_exit(); return (_MC_FPOWNED_PCB); } - critical_enter(); if (td == PCPU_GET(fpcurthread)) { fpusave(get_pcb_user_save_pcb(pcb)); #ifdef CPU_ENABLE_SSE @@ -1060,7 +1061,6 @@ npxgetregs(struct thread *td) } else { owned = _MC_FPOWNED_PCB; } - critical_exit(); #ifdef CPU_ENABLE_SSE if (use_xsave) { /* @@ -1085,6 +1085,7 @@ npxgetregs(struct thread *td) } } #endif + critical_exit(); return (owned); } @@ -1093,6 +1094,7 @@ npxuserinited(struct thread *td) { struct pcb *pcb; + CRITICAL_ASSERT(td); pcb = td->td_pcb; if (PCB_USER_FPU(pcb)) pcb->pcb_flags |= PCB_NPXINITDONE; @@ -1156,32 +1158,32 @@ npxsetregs(struct thread *td, union savefpu *addr, cha addr->sv_xmm.sv_env.en_mxcsr &= cpu_mxcsr_mask; #endif pcb = td->td_pcb; + error = 0; critical_enter(); if (td == PCPU_GET(fpcurthread) && PCB_USER_FPU(pcb)) { #ifdef CPU_ENABLE_SSE error = npxsetxstate(td, xfpustate, xfpustate_size); - if (error != 0) { - critical_exit(); - return (error); - } - if (!cpu_fxsr) #endif - fnclex(); /* As in npxdrop(). */ - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - fpurstor(get_pcb_user_save_td(td)); - critical_exit(); - pcb->pcb_flags |= PCB_NPXUSERINITDONE | PCB_NPXINITDONE; + if (error == 0) { +#ifdef CPU_ENABLE_SSE + if (!cpu_fxsr) +#endif + fnclex(); /* As in npxdrop(). */ + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + fpurstor(get_pcb_user_save_td(td)); + pcb->pcb_flags |= PCB_NPXUSERINITDONE | PCB_NPXINITDONE; + } } else { - critical_exit(); #ifdef CPU_ENABLE_SSE error = npxsetxstate(td, xfpustate, xfpustate_size); - if (error != 0) - return (error); #endif - bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); - npxuserinited(td); + if (error == 0) { + bcopy(addr, get_pcb_user_save_td(td), sizeof(*addr)); + npxuserinited(td); + } } - return (0); + critical_exit(); + return (error); } static void @@ -1350,6 +1352,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx return (0); } pcb = td->td_pcb; + critical_enter(); KASSERT(!PCB_USER_FPU(pcb) || pcb->pcb_save == get_pcb_user_save_pcb(pcb), ("mangled pcb_save")); ctx->flags = 0; @@ -1360,6 +1363,7 @@ fpu_kern_enter(struct thread *td, struct fpu_kern_ctx pcb->pcb_save = fpu_kern_ctx_savefpu(ctx); pcb->pcb_flags |= PCB_KERNNPX; pcb->pcb_flags &= ~PCB_NPXINITDONE; + critical_exit(); return (0); } @@ -1374,7 +1378,6 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx critical_enter(); if (curthread == PCPU_GET(fpcurthread)) npxdrop(); - critical_exit(); pcb->pcb_save = ctx->prev; if (pcb->pcb_save == get_pcb_user_save_pcb(pcb)) { if ((pcb->pcb_flags & PCB_NPXUSERINITDONE) != 0) @@ -1389,6 +1392,7 @@ fpu_kern_leave(struct thread *td, struct fpu_kern_ctx pcb->pcb_flags &= ~PCB_NPXINITDONE; KASSERT(!PCB_USER_FPU(pcb), ("unpaired fpu_kern_leave")); } + critical_exit(); return (0); } From owner-svn-src-all@freebsd.org Fri Aug 3 14:13:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6202106AFC4; Fri, 3 Aug 2018 14:13:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7AF3283499; Fri, 3 Aug 2018 14:13:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5BDA42179C; Fri, 3 Aug 2018 14:13:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EDGtx069994; Fri, 3 Aug 2018 14:13:16 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EDFkq069991; Fri, 3 Aug 2018 14:13:15 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031413.w73EDFkq069991@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:13:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337246 - in stable/10: etc/mtree libexec/tftpd libexec/tftpd/tests X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/10: etc/mtree libexec/tftpd libexec/tftpd/tests X-SVN-Commit-Revision: 337246 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:13:17 -0000 Author: asomers Date: Fri Aug 3 14:13:15 2018 New Revision: 337246 URL: https://svnweb.freebsd.org/changeset/base/337246 Log: MFC r330696, r330709, r330742, r331358 r330696: Add some functional tests for tftpd(8) tftpd(8) is difficult to test in isolation due to its relationship with inetd. Create a test program that mimics the behavior of tftp(1) and inetd(8) and verifies tftpd's response in several different scenarios. These test cases cover all of the basic TFTP protocol, but not the optional parts. PR: 157700 PR: 225996 PR: 226004 PR: 226005 Differential Revision: https://reviews.freebsd.org/D14310 r330709: Commit missing file from r330696 X-MFC-With: 330696 r330742: tftpd: fix the build of tests on i386 after 330696 It's those darn printf format specifiers again Reported by: cy, kibab X-MFC-With: 330696 r331358: tftpd: misc Coverity cleanup in the tests A bunch of unchecked return values from open(2) and read(2) Reported by: Coverity CID: 1386900, 1386911, 1386926, 1386928, 1386932, 1386942 CID: 1386961, 1386979 X-MFC-With: 330696 Added: stable/10/libexec/tftpd/tests/ - copied from r330696, head/libexec/tftpd/tests/ Modified: stable/10/etc/mtree/BSD.tests.dist stable/10/libexec/tftpd/Makefile stable/10/libexec/tftpd/tests/functional.c Directory Properties: stable/10/ (props changed) Modified: stable/10/etc/mtree/BSD.tests.dist ============================================================================== --- stable/10/etc/mtree/BSD.tests.dist Fri Aug 3 14:12:37 2018 (r337245) +++ stable/10/etc/mtree/BSD.tests.dist Fri Aug 3 14:13:15 2018 (r337246) @@ -350,6 +350,8 @@ atf-sh .. .. + tftpd + .. .. sbin dhclient Modified: stable/10/libexec/tftpd/Makefile ============================================================================== --- stable/10/libexec/tftpd/Makefile Fri Aug 3 14:12:37 2018 (r337245) +++ stable/10/libexec/tftpd/Makefile Fri Aug 3 14:13:15 2018 (r337246) @@ -15,4 +15,7 @@ DPADD+= ${LIBWRAP} LDADD+= -lwrap .endif +HAS_TESTS= +SUBDIR.${MK_TESTS}+= tests + .include Modified: stable/10/libexec/tftpd/tests/functional.c ============================================================================== --- head/libexec/tftpd/tests/functional.c Fri Mar 9 15:30:20 2018 (r330696) +++ stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:13:15 2018 (r337246) @@ -224,19 +224,17 @@ do { \ static void cleanup(void) { - int fd = -1; - char buffer[80] = {0}; + FILE *f; pid_t pid; - fd = open(pidfile, O_RDONLY); - if (fd < 0) + f = fopen(pidfile, "r"); + if (f == NULL) return; - if (read(fd, buffer, sizeof(buffer)) > 0) { - sscanf(buffer, "%d", &pid); + if (fscanf(f, "%d", &pid) == 1) { kill(pid, SIGTERM); waitpid(pid, NULL, 0); } - close(fd); + fclose(f); unlink(pidfile); } @@ -248,10 +246,10 @@ require_bufeq(const char *expected, ssize_t expected_l ssize_t i; ATF_REQUIRE_EQ_MSG(expected_len, len, - "Expected %ld bytes but got %ld", expected_len, len); + "Expected %zd bytes but got %zd", expected_len, len); for (i = 0; i < len; i++) { ATF_REQUIRE_EQ_MSG(actual[i], expected[i], - "Expected %#hhx at position %ld; got %hhx instead", + "Expected %#hhx at position %zd; got %hhx instead", expected[i], i, actual[i]); } } @@ -696,6 +694,7 @@ TFTPD_TC_DEFINE(w_flag,, w_flag = 1;) atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("small.txt", O_RDONLY); + ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); close(fd); require_bufeq(contents, contents_len, buffer, r); @@ -733,6 +732,7 @@ TFTPD_TC_DEFINE(wrq_dropped_ack,) atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("medium.txt", O_RDONLY); + ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); close(fd); require_bufeq((const char*)contents, 768, buffer, r); @@ -766,6 +766,7 @@ TFTPD_TC_DEFINE(wrq_dropped_data,) atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("small.txt", O_RDONLY); + ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); close(fd); require_bufeq(contents, contents_len, buffer, r); @@ -800,6 +801,7 @@ TFTPD_TC_DEFINE(wrq_duped_data,) atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("medium.txt", O_RDONLY); + ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); close(fd); require_bufeq((const char*)contents, 768, buffer, r); @@ -867,6 +869,7 @@ TFTPD_TC_DEFINE(wrq_medium,) atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("medium.txt", O_RDONLY); + ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); close(fd); require_bufeq((const char*)contents, 768, buffer, r); @@ -900,6 +903,7 @@ TFTPD_TC_DEFINE(wrq_netascii,) atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("unix.txt", O_RDONLY); + ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); close(fd); require_bufeq(expected, sizeof(expected), buffer, r); @@ -940,6 +944,7 @@ TFTPD_TC_DEFINE(wrq_small,) atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("small.txt", O_RDONLY); + ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); close(fd); require_bufeq(contents, contents_len, buffer, r); From owner-svn-src-all@freebsd.org Fri Aug 3 14:17:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E724106B13E; Fri, 3 Aug 2018 14:17:14 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF33883EFB; Fri, 3 Aug 2018 14:17:13 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 433B8217A3; Fri, 3 Aug 2018 14:17:13 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EHCmS070284; Fri, 3 Aug 2018 14:17:12 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EHCjw070280; Fri, 3 Aug 2018 14:17:12 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031417.w73EHCjw070280@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:17:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337247 - in stable/10: libexec/tftpd libexec/tftpd/tests usr.bin/tftp X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/10: libexec/tftpd libexec/tftpd/tests usr.bin/tftp X-SVN-Commit-Revision: 337247 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:17:14 -0000 Author: asomers Date: Fri Aug 3 14:17:11 2018 New Revision: 337247 URL: https://svnweb.freebsd.org/changeset/base/337247 Log: MFC r330710: tftpd: Flush files as soon as they are fully received On an RRQ, tftpd doesn't exit as soon as it's finished receiving a file. Instead, it waits five seconds just in case the client didn't receive the server's last ACK and decides to resend the final DATA packet. Unfortunately, this created a 5 second delay from when the client thinks it's done sending the file, and when the file is available for other processes. Fix this bug by closing the file as soon as receipt is finished. PR: 157700 Reported by: Barry Mishler Modified: stable/10/libexec/tftpd/tests/functional.c stable/10/libexec/tftpd/tftp-transfer.c stable/10/libexec/tftpd/tftpd.c stable/10/usr.bin/tftp/tftp.c Modified: stable/10/libexec/tftpd/tests/functional.c ============================================================================== --- stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:13:15 2018 (r337246) +++ stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:17:11 2018 (r337247) @@ -692,7 +692,6 @@ TFTPD_TC_DEFINE(w_flag,, w_flag = 1;) send_data(1, contents, contents_len); recv_ack(1); - atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("small.txt", O_RDONLY); ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); @@ -730,7 +729,6 @@ TFTPD_TC_DEFINE(wrq_dropped_ack,) send_data(2, (const char*)&contents[128], 256); recv_ack(2); - atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("medium.txt", O_RDONLY); ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); @@ -764,7 +762,6 @@ TFTPD_TC_DEFINE(wrq_dropped_data,) send_data(1, contents, contents_len); recv_ack(1); - atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("small.txt", O_RDONLY); ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); @@ -799,7 +796,6 @@ TFTPD_TC_DEFINE(wrq_duped_data,) send_data(2, (const char*)&contents[128], 256); recv_ack(2); - atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("medium.txt", O_RDONLY); ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); @@ -867,7 +863,6 @@ TFTPD_TC_DEFINE(wrq_medium,) send_data(2, (const char*)&contents[128], 256); recv_ack(2); - atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("medium.txt", O_RDONLY); ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); @@ -894,14 +889,13 @@ TFTPD_TC_DEFINE(wrq_netascii,) fd = open("unix.txt", O_RDWR | O_CREAT, 0666); ATF_REQUIRE(fd >= 0); close(fd); - contents_len = strlen(contents) + 1; + contents_len = sizeof(contents); SEND_WRQ("unix.txt", "netascii"); recv_ack(0); send_data(1, contents, contents_len); recv_ack(1); - atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("unix.txt", O_RDONLY); ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); @@ -942,7 +936,6 @@ TFTPD_TC_DEFINE(wrq_small,) send_data(1, contents, contents_len); recv_ack(1); - atf_tc_expect_fail("PR 157700 tftpd expects more data after EOF"); fd = open("small.txt", O_RDONLY); ATF_REQUIRE(fd >= 0); r = read(fd, buffer, sizeof(buffer)); Modified: stable/10/libexec/tftpd/tftp-transfer.c ============================================================================== --- stable/10/libexec/tftpd/tftp-transfer.c Fri Aug 3 14:13:15 2018 (r337246) +++ stable/10/libexec/tftpd/tftp-transfer.c Fri Aug 3 14:17:11 2018 (r337247) @@ -302,6 +302,8 @@ send_ack: gettimeofday(&(ts->tstop), NULL); } while (n_data == segsize); + write_close(); + /* Don't do late packet management for the client implementation */ if (acting_as_client) return; Modified: stable/10/libexec/tftpd/tftpd.c ============================================================================== --- stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:13:15 2018 (r337246) +++ stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:17:11 2018 (r337247) @@ -821,7 +821,6 @@ tftp_recvfile(int peer, const char *mode) block = 0; tftp_receive(peer, &block, &ts, NULL, 0); - write_close(); gettimeofday(&now2, NULL); if (debug&DEBUG_SIMPLE) { Modified: stable/10/usr.bin/tftp/tftp.c ============================================================================== --- stable/10/usr.bin/tftp/tftp.c Fri Aug 3 14:13:15 2018 (r337246) +++ stable/10/usr.bin/tftp/tftp.c Fri Aug 3 14:17:11 2018 (r337247) @@ -263,7 +263,6 @@ recvfile(int peer, char *port, int fd, char *name, cha tftp_receive(peer, &block, &tftp_stats, rp, n); } - write_close(); if (tftp_stats.amount > 0) printstats("Received", verbose, &tftp_stats); return; From owner-svn-src-all@freebsd.org Fri Aug 3 14:18:03 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14F9F106B1E8; Fri, 3 Aug 2018 14:18:03 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BDD3D84232; Fri, 3 Aug 2018 14:18:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A064B217A6; Fri, 3 Aug 2018 14:18:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EI25M070452; Fri, 3 Aug 2018 14:18:02 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EI26n070450; Fri, 3 Aug 2018 14:18:02 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031418.w73EI26n070450@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:18:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337248 - in stable/10/libexec/tftpd: . tests X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/10/libexec/tftpd: . tests X-SVN-Commit-Revision: 337248 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:18:03 -0000 Author: asomers Date: Fri Aug 3 14:18:02 2018 New Revision: 337248 URL: https://svnweb.freebsd.org/changeset/base/337248 Log: MFC r330718: tftpd: Verify world-writability for WRQ when using relative paths tftpd(8) says that files may only be written if they already exist and are publicly writable. tftpd.c verifies that a file is publicly writable if it uses an absolute pathname. However, if the pathname is relative, that check is skipped. Fix it. Note that this is not a security vulnerability, because the transfer ultimately doesn't work unless the file already exists and is owned by user nobody. Also, this bug does not affect the default configuration, because the default uses the "-s" option which makes all pathnames absolute. PR: 226004 Modified: stable/10/libexec/tftpd/tests/functional.c stable/10/libexec/tftpd/tftpd.c Directory Properties: stable/10/ (props changed) Modified: stable/10/libexec/tftpd/tests/functional.c ============================================================================== --- stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:17:11 2018 (r337247) +++ stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:18:02 2018 (r337248) @@ -348,6 +348,10 @@ setup(struct sockaddr_storage *to, uint16_t idx) ATF_REQUIRE((client_s = socket(protocol, SOCK_DGRAM, 0)) > 0); break; } + + /* Clear the client's umask. Test cases will specify exact modes */ + umask(0000); + return (client_s); } @@ -713,7 +717,7 @@ TFTPD_TC_DEFINE(wrq_dropped_ack,) for (i = 0; i < nitems(contents); i++) contents[i] = i; - fd = open("medium.txt", O_RDWR | O_CREAT, 0644); + fd = open("medium.txt", O_RDWR | O_CREAT, 0666); ATF_REQUIRE(fd >= 0); close(fd); @@ -747,7 +751,7 @@ TFTPD_TC_DEFINE(wrq_dropped_data,) size_t contents_len; char buffer[1024]; - fd = open("small.txt", O_RDWR | O_CREAT, 0644); + fd = open("small.txt", O_RDWR | O_CREAT, 0666); ATF_REQUIRE(fd >= 0); close(fd); contents_len = strlen(contents) + 1; @@ -783,7 +787,7 @@ TFTPD_TC_DEFINE(wrq_duped_data,) for (i = 0; i < nitems(contents); i++) contents[i] = i; - fd = open("medium.txt", O_RDWR | O_CREAT, 0644); + fd = open("medium.txt", O_RDWR | O_CREAT, 0666); ATF_REQUIRE(fd >= 0); close(fd); @@ -833,7 +837,8 @@ TFTPD_TC_DEFINE(wrq_eaccess_world_readable,) close(fd); SEND_WRQ("empty.txt", "octet"); - atf_tc_expect_fail("PR 226004 with relative pathnames, tftpd doesn't validate world writability"); + atf_tc_expect_fail("PR 225996 tftpd doesn't abort on a WRQ access " + "violation"); RECV_ERROR(2, "Access violation"); } Modified: stable/10/libexec/tftpd/tftpd.c ============================================================================== --- stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:17:11 2018 (r337247) +++ stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:18:02 2018 (r337248) @@ -741,8 +741,12 @@ validate_access(int peer, char **filep, int mode) dirp->name, filename); if (stat(pathname, &stbuf) == 0 && (stbuf.st_mode & S_IFMT) == S_IFREG) { - if ((stbuf.st_mode & S_IROTH) != 0) { - break; + if (mode == RRQ) { + if ((stbuf.st_mode & S_IROTH) != 0) + break; + } else { + if ((stbuf.st_mode & S_IWOTH) != 0) + break; } err = EACCESS; } @@ -750,6 +754,8 @@ validate_access(int peer, char **filep, int mode) if (dirp->name != NULL) *filep = filename = pathname; else if (mode == RRQ) + return (err); + else if (err != ENOTFOUND || !create_new) return (err); } From owner-svn-src-all@freebsd.org Fri Aug 3 14:19:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CBBC106B2CF; Fri, 3 Aug 2018 14:19:10 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 220E984566; Fri, 3 Aug 2018 14:19:10 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0344D217A8; Fri, 3 Aug 2018 14:19:10 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EJ9Bj070775; Fri, 3 Aug 2018 14:19:09 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EJ9Z8070773; Fri, 3 Aug 2018 14:19:09 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031419.w73EJ9Z8070773@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:19:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337249 - in stable/10/libexec/tftpd: . tests X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/10/libexec/tftpd: . tests X-SVN-Commit-Revision: 337249 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:19:10 -0000 Author: asomers Date: Fri Aug 3 14:19:09 2018 New Revision: 337249 URL: https://svnweb.freebsd.org/changeset/base/337249 Log: MFC r330719: tftpd: Abort on an WRQ access violation On a WRQ (write request) tftpd checks whether the client has access permission for the file in question. If not, then the write is prevented. However, tftpd doesn't reply with an ERROR packet, nor does it abort. Instead, it tries to receive the packet anyway. The symptom is slightly different depending on the nature of the error. If the target file is nonexistent and tftpd lacks permission to create it, then tftpd will willingly receive the file, but not write it anywhere. If the file exists but is not writable, then tftpd will fail to ACK to WRQ. PR: 225996 Modified: stable/10/libexec/tftpd/tests/functional.c stable/10/libexec/tftpd/tftpd.c Directory Properties: stable/10/ (props changed) Modified: stable/10/libexec/tftpd/tests/functional.c ============================================================================== --- stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:18:02 2018 (r337248) +++ stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:19:09 2018 (r337249) @@ -819,8 +819,6 @@ TFTPD_TC_DEFINE(wrq_eaccess,) close(fd); SEND_WRQ("empty.txt", "octet"); - atf_tc_expect_fail("PR 225996 tftpd doesn't abort on a WRQ access " - "violation"); RECV_ERROR(2, "Access violation"); } @@ -837,8 +835,6 @@ TFTPD_TC_DEFINE(wrq_eaccess_world_readable,) close(fd); SEND_WRQ("empty.txt", "octet"); - atf_tc_expect_fail("PR 225996 tftpd doesn't abort on a WRQ access " - "violation"); RECV_ERROR(2, "Access violation"); } @@ -915,8 +911,6 @@ TFTPD_TC_DEFINE(wrq_netascii,) TFTPD_TC_DEFINE(wrq_nonexistent,) { SEND_WRQ("nonexistent.txt", "octet"); - atf_tc_expect_fail("PR 225996 tftpd doesn't abort on a WRQ access " - "violation"); RECV_ERROR(1, "File not found"); } Modified: stable/10/libexec/tftpd/tftpd.c ============================================================================== --- stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:18:02 2018 (r337248) +++ stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:19:09 2018 (r337249) @@ -543,6 +543,10 @@ tftp_wrq(int peer, char *recvbuffer, ssize_t size) filename, errtomsg(ecode)); } + if (ecode) { + send_error(peer, ecode); + exit(1); + } tftp_recvfile(peer, mode); exit(0); } From owner-svn-src-all@freebsd.org Fri Aug 3 14:19:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C9D2106B35C; Fri, 3 Aug 2018 14:19:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 12D8D846C8; Fri, 3 Aug 2018 14:19:57 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E7A70217AA; Fri, 3 Aug 2018 14:19:56 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EJulC071020; Fri, 3 Aug 2018 14:19:56 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EJuAD071018; Fri, 3 Aug 2018 14:19:56 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031419.w73EJuAD071018@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:19:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337250 - in stable/10/libexec/tftpd: . tests X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/10/libexec/tftpd: . tests X-SVN-Commit-Revision: 337250 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:19:57 -0000 Author: asomers Date: Fri Aug 3 14:19:56 2018 New Revision: 337250 URL: https://svnweb.freebsd.org/changeset/base/337250 Log: MFC r330720: tftpd: reject unknown opcodes If tftpd receives a command with an unknown opcode, it simply exits 1. It doesn't send an ERROR packet, and the client will hang waiting for one. Fix it. PR: 226005 Modified: stable/10/libexec/tftpd/tests/functional.c stable/10/libexec/tftpd/tftpd.c Directory Properties: stable/10/ (props changed) Modified: stable/10/libexec/tftpd/tests/functional.c ============================================================================== --- stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:19:09 2018 (r337249) +++ stable/10/libexec/tftpd/tests/functional.c Fri Aug 3 14:19:56 2018 (r337250) @@ -675,7 +675,6 @@ TFTPD_TC_DEFINE(unknown_opcode,) { /* Looks like an RRQ or WRQ request, but with a bad opcode */ SEND_STR("\0\007foo.txt\0octet\0"); - atf_tc_expect_timeout("PR 226005 tftpd ignores bad opcodes but doesn't reject them"); RECV_ERROR(4, "Illegal TFTP operation"); } Modified: stable/10/libexec/tftpd/tftpd.c ============================================================================== --- stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:19:09 2018 (r337249) +++ stable/10/libexec/tftpd/tftpd.c Fri Aug 3 14:19:56 2018 (r337250) @@ -419,8 +419,7 @@ main(int argc, char *argv[]) "%s read access denied", peername); exit(1); } - } - if (tp->th_opcode == WRQ) { + } else if (tp->th_opcode == WRQ) { if (allow_wo) tftp_wrq(peer, tp->th_stuff, n - 1); else { @@ -428,7 +427,8 @@ main(int argc, char *argv[]) "%s write access denied", peername); exit(1); } - } + } else + send_error(peer, EBADOP); exit(1); } From owner-svn-src-all@freebsd.org Fri Aug 3 14:22:17 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73A53106B65F; Fri, 3 Aug 2018 14:22:17 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 267E584D2E; Fri, 3 Aug 2018 14:22:17 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 07B2F2192A; Fri, 3 Aug 2018 14:22:17 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EMG0a076152; Fri, 3 Aug 2018 14:22:16 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EMGC5076151; Fri, 3 Aug 2018 14:22:16 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031422.w73EMGC5076151@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:22:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337251 - stable/10/sys/security/audit X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/10/sys/security/audit X-SVN-Commit-Revision: 337251 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:22:17 -0000 Author: asomers Date: Fri Aug 3 14:22:16 2018 New Revision: 337251 URL: https://svnweb.freebsd.org/changeset/base/337251 Log: MFC r334296: Fix "Bad tailq" panic when auditing auditon(A_SETCLASS, ...) Due to an oversight in r195280, auditon(A_SETCLASS, ...) would cause a tailq element to get added to the tailq twice, resulting in a circular tailq. This panics when INVARIANTS are on. Differential Revision: https://reviews.freebsd.org/D15381 Modified: stable/10/sys/security/audit/audit_bsm.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/security/audit/audit_bsm.c ============================================================================== --- stable/10/sys/security/audit/audit_bsm.c Fri Aug 3 14:19:56 2018 (r337250) +++ stable/10/sys/security/audit/audit_bsm.c Fri Aug 3 14:22:16 2018 (r337251) @@ -418,7 +418,6 @@ audit_sys_auditon(struct audit_record *ar, struct au_r break; case A_SETCLASS: - kau_write(rec, tok); tok = au_to_arg32(2, "setclass:ec_event", ar->ar_arg_auditon.au_evclass.ec_number); kau_write(rec, tok); From owner-svn-src-all@freebsd.org Fri Aug 3 14:23:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8437C106B6F1; Fri, 3 Aug 2018 14:23:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30EE584EE7; Fri, 3 Aug 2018 14:23:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 120952193F; Fri, 3 Aug 2018 14:23:02 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EN1QE076236; Fri, 3 Aug 2018 14:23:01 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EN1ok076235; Fri, 3 Aug 2018 14:23:01 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031423.w73EN1ok076235@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:23:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337252 - stable/10/contrib/openbsm/libbsm X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/10/contrib/openbsm/libbsm X-SVN-Commit-Revision: 337252 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:23:02 -0000 Author: asomers Date: Fri Aug 3 14:23:01 2018 New Revision: 337252 URL: https://svnweb.freebsd.org/changeset/base/337252 Log: MFC r334390: au_read_rec(3): correct return value in man page Submitted by: aniketp Reviewed by: csjp (earlier version) Sponsored by: Google, Inc. (GSoC 2018) Differential Revision: https://reviews.freebsd.org/D15618 Modified: stable/10/contrib/openbsm/libbsm/au_io.3 Directory Properties: stable/10/ (props changed) Modified: stable/10/contrib/openbsm/libbsm/au_io.3 ============================================================================== --- stable/10/contrib/openbsm/libbsm/au_io.3 Fri Aug 3 14:22:16 2018 (r337251) +++ stable/10/contrib/openbsm/libbsm/au_io.3 Fri Aug 3 14:23:01 2018 (r337252) @@ -24,7 +24,7 @@ .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd August 4, 2009 +.Dd May 30, 2018 .Dt AU_IO 3 .Os .Sh NAME @@ -129,14 +129,14 @@ would be used to free the record buffer. Finally, the source stream would be closed by a call to .Xr fclose 3 . .Sh RETURN VALUES -The +On success, .Fn au_fetch_tok -and +returns 0 while .Fn au_read_rec -functions -return 0 on success, or \-1 on failure along with additional error information -returned via -.Va errno . +returns the number of bytes read. +Both functions return \-1 on failure with +.Va errno +set appropriately. .Sh SEE ALSO .Xr free 3 , .Xr libbsm 3 From owner-svn-src-all@freebsd.org Fri Aug 3 14:23:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B329F106B7A7; Fri, 3 Aug 2018 14:23:53 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 388FA8539B; Fri, 3 Aug 2018 14:23:52 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 37AE721940; Fri, 3 Aug 2018 14:23:52 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73ENqH4076326; Fri, 3 Aug 2018 14:23:52 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73ENq64076325; Fri, 3 Aug 2018 14:23:52 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031423.w73ENq64076325@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:23:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337253 - stable/10/share/man/man4 X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/10/share/man/man4 X-SVN-Commit-Revision: 337253 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:23:54 -0000 Author: asomers Date: Fri Aug 3 14:23:51 2018 New Revision: 337253 URL: https://svnweb.freebsd.org/changeset/base/337253 Log: MFC r334394: auditpipe(4): fix some ioctl arguments in the man page Fix the argument types for the AUDITPIPE_[GS]ET_PRESELECT_(NA)?FLAGS ioctls. Also, fix some grammar. [skip ci] PR: 226713 Submitted by: aniketp Sponsored by: Google, Inc. (GSoC 2018) Differential Revision: https://reviews.freebsd.org/D15620 Modified: stable/10/share/man/man4/auditpipe.4 Directory Properties: stable/10/ (props changed) Modified: stable/10/share/man/man4/auditpipe.4 ============================================================================== --- stable/10/share/man/man4/auditpipe.4 Fri Aug 3 14:23:01 2018 (r337252) +++ stable/10/share/man/man4/auditpipe.4 Fri Aug 3 14:23:51 2018 (r337253) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd May 5, 2006 +.Dd May 30, 2018 .Dt AUDITPIPE 4 .Os .Sh NAME @@ -154,7 +154,7 @@ These flags correspond to the field in .Xr audit_control 5 . The ioctl argument should be of type -.Vt u_int . +.Vt au_mask_t . .It Dv AUDITPIPE_SET_PRESELECT_FLAGS Set the current default preselection flags for attributable events on the pipe. @@ -163,7 +163,7 @@ These flags correspond to the field in .Xr audit_control 5 . The ioctl argument should be of type -.Vt u_int . +.Vt au_mask_t . .It Dv AUDITPIPE_GET_PRESELECT_NAFLAGS Retrieve the current default preselection flags for non-attributable events on the pipe. @@ -172,7 +172,7 @@ These flags correspond to the field in .Xr audit_control 5 . The ioctl argument should be of type -.Vt u_int . +.Vt au_mask_t . .It Dv AUDITPIPE_SET_PRESELECT_NAFLAGS Set the current default preselection flags for non-attributable events on the pipe. @@ -181,7 +181,7 @@ These flags correspond to the field in .Xr audit_control 5 . The ioctl argument should be of type -.Vt u_int . +.Vt au_mask_t . .It Dv AUDITPIPE_GET_PRESELECT_AUID Query the current preselection masks for a specific auid on the pipe. The ioctl argument should be of type @@ -245,7 +245,7 @@ manual page for information on audit-related bugs and .Pp The configurable preselection mechanism mirrors the selection model present for the global audit trail. -It might be desirable to provided a more flexible selection model. +It might be desirable to provide a more flexible selection model. .Pp The per-pipe audit event queue is fifo, with drops occurring if either the user thread provides in sufficient for the record on the queue head, or on From owner-svn-src-all@freebsd.org Fri Aug 3 14:25:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9ED86106B8D9; Fri, 3 Aug 2018 14:25:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48CFC8572D; Fri, 3 Aug 2018 14:25:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 295AB21943; Fri, 3 Aug 2018 14:25:16 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EPGkl076460; Fri, 3 Aug 2018 14:25:16 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EPGwl076459; Fri, 3 Aug 2018 14:25:16 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031425.w73EPGwl076459@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:25:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337254 - stable/10/lib/libutil X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/10/lib/libutil X-SVN-Commit-Revision: 337254 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:25:16 -0000 Author: asomers Date: Fri Aug 3 14:25:15 2018 New Revision: 337254 URL: https://svnweb.freebsd.org/changeset/base/337254 Log: MFC r334547: pty.3: Add a HISTORY section These functions were first added in 4.3 BSD-Reno, according to http://unix.superglobalmegacorp.com/ and the CSRG svn repository. Reviewed by: bcr, bjk Differential Revision: https://reviews.freebsd.org/D15652 Modified: stable/10/lib/libutil/pty.3 Directory Properties: stable/10/ (props changed) Modified: stable/10/lib/libutil/pty.3 ============================================================================== --- stable/10/lib/libutil/pty.3 Fri Aug 3 14:23:51 2018 (r337253) +++ stable/10/lib/libutil/pty.3 Fri Aug 3 14:25:15 2018 (r337254) @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" " -.Dd November 11, 2015 +.Dd June 2, 2018 .Dt PTY 3 .Os .Sh NAME @@ -143,3 +143,18 @@ may set it to any value as described for .Xr pty 4 , .Xr termios 4 , .Xr group 5 +.Sh HISTORY +The +.Fn openpty +and +.Fn forkpty +functions first appeared in +.Bx 4.3 Reno. +.Sh BUGS +.Fn openpty +writes the slave terminal's name to +.Fa name , +but does not check that sufficient space is available. +It is advisable to use +.Xr ptsname 3 +instead. From owner-svn-src-all@freebsd.org Fri Aug 3 14:27:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E60C106BA56; Fri, 3 Aug 2018 14:27:30 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B2B268593B; Fri, 3 Aug 2018 14:27:29 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 940402195F; Fri, 3 Aug 2018 14:27:29 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73ERTSM076593; Fri, 3 Aug 2018 14:27:29 GMT (envelope-from avg@FreeBSD.org) Received: (from avg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73ERT7I076591; Fri, 3 Aug 2018 14:27:29 GMT (envelope-from avg@FreeBSD.org) Message-Id: <201808031427.w73ERT7I076591@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: avg set sender to avg@FreeBSD.org using -f From: Andriy Gapon Date: Fri, 3 Aug 2018 14:27:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337255 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: avg X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 337255 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:27:30 -0000 Author: avg Date: Fri Aug 3 14:27:28 2018 New Revision: 337255 URL: https://svnweb.freebsd.org/changeset/base/337255 Log: safer wait-free iteration of shared interrupt handlers The code that iterates a list of interrupt handlers for a (shared) interrupt, whether in the ISR context or in the context of an interrupt thread, does so in a lock-free fashion. Thus, the routines that modify the list need to take special steps to ensure that the iterating code has a consistent view of the list. Previously, those routines tried to play nice only with the code running in the ithread context. The iteration in the ISR context was left to a chance. After commit r336635 atomic operations and memory fences are used to ensure that ie_handlers list is always safe to navigate with respect to inserting and removal of list elements. There is still a question of when it is safe to actually free a removed element. The idea of this change is somewhat similar to the idea of the epoch based reclamation. There are some simplifications comparing to the general epoch based reclamation. All writers are serialized using a mutex, so we do not need to worry about concurrent modifications. Also, all read accesses from the open context are serialized too. So, we can get away just two epochs / phases. When a thread removes an element it switches the global phase from the current phase to the other and then drains the previous phase. Only after the draining the removed element gets actually freed. The code that iterates the list in the ISR context takes a snapshot of the global phase and then increments the use count of that phase before iterating the list. The use count (in the same phase) is decremented after the iteration. This should ensure that there should be no iteration over the removed element when its gets freed. This commit also simplifies the coordination with the interrupt thread context. Now we always schedule the interrupt thread when removing one of handlers for its interrupt. This makes the code both simpler and safer as the interrupt thread masks the interrupt thus ensuring that there is no interaction with the ISR context. P.S. This change matters only for shared interrupts and I realize that those are becoming a thing of the past (and quickly). I also understand that the problem that I am trying to solve is extremely rare. PR: 229106 Reviewed by: cem Discussed with: Samy Al Bahra MFC after: 5 weeks Differential Revision: https://reviews.freebsd.org/D15905 Modified: head/sys/kern/kern_intr.c head/sys/sys/interrupt.h Modified: head/sys/kern/kern_intr.c ============================================================================== --- head/sys/kern/kern_intr.c Fri Aug 3 14:25:15 2018 (r337254) +++ head/sys/kern/kern_intr.c Fri Aug 3 14:27:28 2018 (r337255) @@ -683,6 +683,45 @@ intr_handler_source(void *cookie) } /* + * If intr_event_handle() is running in the ISR context at the time of the call, + * then wait for it to complete. + */ +static void +intr_event_barrier(struct intr_event *ie) +{ + int phase; + + mtx_assert(&ie->ie_lock, MA_OWNED); + phase = ie->ie_phase; + + /* + * Switch phase to direct future interrupts to the other active counter. + * Make sure that any preceding stores are visible before the switch. + */ + KASSERT(ie->ie_active[!phase] == 0, ("idle phase has activity")); + atomic_store_rel_int(&ie->ie_phase, !phase); + + /* + * This code cooperates with wait-free iteration of ie_handlers + * in intr_event_handle. + * Make sure that the removal and the phase update are not reordered + * with the active count check. + * Note that no combination of acquire and release fences can provide + * that guarantee as Store->Load sequences can always be reordered. + */ + atomic_thread_fence_seq_cst(); + + /* + * Now wait on the inactive phase. + * The acquire fence is needed so that that all post-barrier accesses + * are after the check. + */ + while (ie->ie_active[phase] > 0) + cpu_spinwait(); + atomic_thread_fence_acq(); +} + +/* * Sleep until an ithread finishes executing an interrupt handler. * * XXX Doesn't currently handle interrupt filters or fast interrupt @@ -752,44 +791,30 @@ intr_event_remove_handler(void *cookie) } /* - * If there is no ithread, then just remove the handler and return. - * XXX: Note that an INTR_FAST handler might be running on another - * CPU! + * If there is no ithread, then directly remove the handler. Note that + * intr_event_handle() iterates ie_handlers in a lock-less fashion, so + * care needs to be taken to keep ie_handlers consistent and to free + * the removed handler only when ie_handlers is quiescent. */ if (ie->ie_thread == NULL) { CK_SLIST_REMOVE_PREVPTR(prevptr, ih, ih_next); + intr_event_barrier(ie); + intr_event_update(ie); mtx_unlock(&ie->ie_lock); free(handler, M_ITHREAD); return (0); } /* - * If the interrupt thread is already running, then just mark this - * handler as being dead and let the ithread do the actual removal. - * - * During a cold boot while cold is set, msleep() does not sleep, - * so we have to remove the handler here rather than letting the - * thread do it. + * Let the interrupt thread do the job. + * The interrupt source is disabled when the interrupt thread is + * running, so it does not have to worry about interaction with + * intr_event_handle(). */ - thread_lock(ie->ie_thread->it_thread); - if (!TD_AWAITING_INTR(ie->ie_thread->it_thread) && !cold) { - handler->ih_flags |= IH_DEAD; - - /* - * Ensure that the thread will process the handler list - * again and remove this handler if it has already passed - * it on the list. - * - * The release part of the following store ensures - * that the update of ih_flags is ordered before the - * it_need setting. See the comment before - * atomic_cmpset_acq(&ithd->it_need, ...) operation in - * the ithread_execute_handlers(). - */ - atomic_store_rel_int(&ie->ie_thread->it_need, 1); - } else - CK_SLIST_REMOVE_PREVPTR(prevptr, ih, ih_next); - thread_unlock(ie->ie_thread->it_thread); + KASSERT((handler->ih_flags & IH_DEAD) == 0, + ("duplicate handle remove")); + handler->ih_flags |= IH_DEAD; + intr_event_schedule_thread(ie); while (handler->ih_flags & IH_DEAD) msleep(handler, &ie->ie_lock, 0, "iev_rmh", 0); intr_event_update(ie); @@ -1154,6 +1179,7 @@ intr_event_handle(struct intr_event *ie, struct trapfr struct trapframe *oldframe; struct thread *td; int ret, thread; + int phase; td = curthread; @@ -1178,6 +1204,15 @@ intr_event_handle(struct intr_event *ie, struct trapfr oldframe = td->td_intr_frame; td->td_intr_frame = frame; + phase = ie->ie_phase; + atomic_add_int(&ie->ie_active[phase], 1); + + /* + * This fence is required to ensure that no later loads are + * re-ordered before the ie_active store. + */ + atomic_thread_fence_seq_cst(); + CK_SLIST_FOREACH(ih, &ie->ie_handlers, ih_next) { if (ih->ih_filter == NULL) { thread = 1; @@ -1215,6 +1250,8 @@ intr_event_handle(struct intr_event *ie, struct trapfr thread = 1; } } + atomic_add_rel_int(&ie->ie_active[phase], -1); + td->td_intr_frame = oldframe; if (thread) { Modified: head/sys/sys/interrupt.h ============================================================================== --- head/sys/sys/interrupt.h Fri Aug 3 14:25:15 2018 (r337254) +++ head/sys/sys/interrupt.h Fri Aug 3 14:27:28 2018 (r337255) @@ -122,6 +122,8 @@ struct intr_event { struct timeval ie_warntm; int ie_irq; /* Physical irq number if !SOFT. */ int ie_cpu; /* CPU this event is bound to. */ + volatile int ie_phase; /* Switched to establish a barrier. */ + volatile int ie_active[2]; /* Filters in ISR context. */ }; /* Interrupt event flags kept in ie_flags. */ From owner-svn-src-all@freebsd.org Fri Aug 3 14:36:31 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C368106BDA2; Fri, 3 Aug 2018 14:36:31 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E13E860C5; Fri, 3 Aug 2018 14:36:31 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1BBC721B00; Fri, 3 Aug 2018 14:36:31 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EaUs9081485; Fri, 3 Aug 2018 14:36:30 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EaUCi081481; Fri, 3 Aug 2018 14:36:30 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031436.w73EaUCi081481@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:36:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337256 - in stable/10: contrib/openbsm/bin/praudit etc/mtree usr.sbin/praudit usr.sbin/praudit/tests X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: in stable/10: contrib/openbsm/bin/praudit etc/mtree usr.sbin/praudit usr.sbin/praudit/tests X-SVN-Commit-Revision: 337256 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:36:32 -0000 Author: asomers Date: Fri Aug 3 14:36:29 2018 New Revision: 337256 URL: https://svnweb.freebsd.org/changeset/base/337256 Log: MFC r335287, r335290 r335287: praudit(1): return 0 on success Cherry pick https://github.com/openbsm/openbsm/commit/ed83bb3 Submitted by: aniketp Reviewed by: rwatson, 0mp Obtained from: OpenBSM Sponsored by: Google, Inc. (GSoC 2018) Pull Request: https://github.com/openbsm/openbsm/pull/32 r335290: praudit(1): add tests Submitted by: aniketp X-MFC-With: 335287 Sponsored by: Google, Inc. (GSoC 2018) Differential Revision: https://reviews.freebsd.org/D15751 Added: stable/10/usr.sbin/praudit/tests/ - copied from r335290, head/usr.sbin/praudit/tests/ Modified: stable/10/contrib/openbsm/bin/praudit/praudit.1 stable/10/contrib/openbsm/bin/praudit/praudit.c stable/10/etc/mtree/BSD.tests.dist stable/10/usr.sbin/praudit/Makefile Directory Properties: stable/10/ (props changed) Modified: stable/10/contrib/openbsm/bin/praudit/praudit.1 ============================================================================== --- stable/10/contrib/openbsm/bin/praudit/praudit.1 Fri Aug 3 14:27:28 2018 (r337255) +++ stable/10/contrib/openbsm/bin/praudit/praudit.1 Fri Aug 3 14:36:29 2018 (r337256) @@ -25,7 +25,7 @@ .\" IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE .\" POSSIBILITY OF SUCH DAMAGE. .\" -.Dd August 4, 2009 +.Dd June 11, 2018 .Dt PRAUDIT 1 .Os .Sh NAME @@ -88,6 +88,8 @@ Events are displayed as per their descriptions given i .Pa /etc/security/audit_event ; UIDs and GIDs are expanded to their names; dates and times are displayed in human-readable format. +.Sh EXIT STATUS +.Ex -std .Sh FILES .Bl -tag -width ".Pa /etc/security/audit_control" -compact .It Pa /etc/security/audit_class Modified: stable/10/contrib/openbsm/bin/praudit/praudit.c ============================================================================== --- stable/10/contrib/openbsm/bin/praudit/praudit.c Fri Aug 3 14:27:28 2018 (r337255) +++ stable/10/contrib/openbsm/bin/praudit/praudit.c Fri Aug 3 14:36:29 2018 (r337256) @@ -169,5 +169,5 @@ main(int argc, char **argv) if (oflags & AU_OFLAG_XML) au_print_xml_footer(stdout); - return (1); + return (0); } Modified: stable/10/etc/mtree/BSD.tests.dist ============================================================================== --- stable/10/etc/mtree/BSD.tests.dist Fri Aug 3 14:27:28 2018 (r337255) +++ stable/10/etc/mtree/BSD.tests.dist Fri Aug 3 14:36:29 2018 (r337256) @@ -682,6 +682,8 @@ .. nmtree .. + praudit + .. pw .. rpcbind Modified: stable/10/usr.sbin/praudit/Makefile ============================================================================== --- stable/10/usr.sbin/praudit/Makefile Fri Aug 3 14:27:28 2018 (r337255) +++ stable/10/usr.sbin/praudit/Makefile Fri Aug 3 14:36:29 2018 (r337256) @@ -13,4 +13,7 @@ WARNS?= 3 DPADD= ${LIBBSM} LDADD= -lbsm +HAS_TESTS= +SUBDIR.${MK_TESTS}+= tests + .include From owner-svn-src-all@freebsd.org Fri Aug 3 14:37:24 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B391106BE3C; Fri, 3 Aug 2018 14:37:24 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D4F428622D; Fri, 3 Aug 2018 14:37:23 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9CFCF21B01; Fri, 3 Aug 2018 14:37:23 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EbN91081579; Fri, 3 Aug 2018 14:37:23 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EbNXg081578; Fri, 3 Aug 2018 14:37:23 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031437.w73EbNXg081578@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:37:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337257 - stable/10/contrib/openbsm/bin/auditd X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/10/contrib/openbsm/bin/auditd X-SVN-Commit-Revision: 337257 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:37:24 -0000 Author: asomers Date: Fri Aug 3 14:37:23 2018 New Revision: 337257 URL: https://svnweb.freebsd.org/changeset/base/337257 Log: MFC r335899: auditd(8): register signal handlers interrutibly auditd_wait_for_events() relies on read(2) being interrupted by signals, but it registers signal handlers with signal(3), which sets SA_RESTART. That breaks asynchronous signal handling. It means that signals don't actually get handled until after an audit(8) trigger is received. Symptoms include: * Sending SIGTERM to auditd doesn't kill it right away; you must send SIGTERM and then send a trigger with auditon(2). * Same with SIGHUP * Zombie child processes don't get reaped until auditd receives a trigger sent by auditon. This includes children created by expiring audit trails at auditd startup. Fix by using sigaction(2) instead of signal(3). Cherry pick https://github.com/openbsm/openbsm/commit/d060887 PR: 229381 Reviewed by: cem Obtained from: OpenBSM Differential Revision: https://github.com/openbsm/openbsm/pull/36 Modified: stable/10/contrib/openbsm/bin/auditd/auditd.c Directory Properties: stable/10/ (props changed) Modified: stable/10/contrib/openbsm/bin/auditd/auditd.c ============================================================================== --- stable/10/contrib/openbsm/bin/auditd/auditd.c Fri Aug 3 14:36:29 2018 (r337256) +++ stable/10/contrib/openbsm/bin/auditd/auditd.c Fri Aug 3 14:37:23 2018 (r337257) @@ -415,27 +415,35 @@ close_all(void) static int register_daemon(void) { + struct sigaction action; FILE * pidfile; int fd; pid_t pid; /* Set up the signal hander. */ - if (signal(SIGTERM, auditd_relay_signal) == SIG_ERR) { + action.sa_handler = auditd_relay_signal; + /* + * sa_flags must not include SA_RESTART, so that read(2) will be + * interruptible in auditd_wait_for_events + */ + action.sa_flags = 0; + sigemptyset(&action.sa_mask); + if (sigaction(SIGTERM, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGTERM"); fail_exit(); } - if (signal(SIGCHLD, auditd_relay_signal) == SIG_ERR) { + if (sigaction(SIGCHLD, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGCHLD"); fail_exit(); } - if (signal(SIGHUP, auditd_relay_signal) == SIG_ERR) { + if (sigaction(SIGHUP, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGHUP"); fail_exit(); } - if (signal(SIGALRM, auditd_relay_signal) == SIG_ERR) { + if (sigaction(SIGALRM, &action, NULL) != 0) { auditd_log_err( "Could not set signal handler for SIGALRM"); fail_exit(); From owner-svn-src-all@freebsd.org Fri Aug 3 14:45:54 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 907B7106C226; Fri, 3 Aug 2018 14:45:54 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41DE3868F2; Fri, 3 Aug 2018 14:45:54 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2171521CAC; Fri, 3 Aug 2018 14:45:54 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73Ejsg0086935; Fri, 3 Aug 2018 14:45:54 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EjraV086933; Fri, 3 Aug 2018 14:45:53 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031445.w73EjraV086933@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:45:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337258 - stable/10/sys/kern X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/10/sys/kern X-SVN-Commit-Revision: 337258 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:45:54 -0000 Author: asomers Date: Fri Aug 3 14:45:53 2018 New Revision: 337258 URL: https://svnweb.freebsd.org/changeset/base/337258 Log: MFC r336205: Don't acquire evclass_lock with a spinlock held When the "pc" audit class is enabled and auditd is running, witness will panic during thread exit because au_event_class tries to lock an rwlock while holding a spinlock acquired upstack by thread_exit. To fix this, move AUDIT_SYSCALL_EXIT futher upstack, before the spinlock is acquired. Of thread_exit's 16 callers, it's only necessary to call AUDIT_SYSCALL_EXIT from two, exit1 (for exiting processes) and kern_thr_exit (for exiting threads). The other callers are all kernel threads, which needen't call AUDIT_SYSCALL_EXIT because since they can't make syscalls there will be nothing to audit. And exit1 already does call AUDIT_SYSCALL_EXIT, making the second call in thread_exit redundant for that case. PR: 228444 Reported by: aniketp Reviewed by: aniketp, kib Differential Revision: https://reviews.freebsd.org/D16210 Modified: stable/10/sys/kern/kern_thr.c stable/10/sys/kern/kern_thread.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/kern/kern_thr.c ============================================================================== --- stable/10/sys/kern/kern_thr.c Fri Aug 3 14:37:23 2018 (r337257) +++ stable/10/sys/kern/kern_thr.c Fri Aug 3 14:45:53 2018 (r337258) @@ -362,6 +362,11 @@ kern_thr_exit(struct thread *td) racct_sub(p, RACCT_NTHR, 1); tdsigcleanup(td); umtx_thread_exit(td); + +#ifdef AUDIT + AUDIT_SYSCALL_EXIT(0, td); +#endif + PROC_SLOCK(p); thread_stopped(p); thread_exit(); Modified: stable/10/sys/kern/kern_thread.c ============================================================================== --- stable/10/sys/kern/kern_thread.c Fri Aug 3 14:37:23 2018 (r337257) +++ stable/10/sys/kern/kern_thread.c Fri Aug 3 14:45:53 2018 (r337258) @@ -418,9 +418,6 @@ thread_exit(void) SDT_PROBE0(proc, , , lwp__exit); KASSERT(TAILQ_EMPTY(&td->td_sigqueue.sq_list), ("signal pending")); -#ifdef AUDIT - AUDIT_SYSCALL_EXIT(0, td); -#endif /* * drop FPU & debug register state storage, or any other * architecture specific resources that From owner-svn-src-all@freebsd.org Fri Aug 3 14:47:48 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47572106C2A2; Fri, 3 Aug 2018 14:47:48 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED10A86A57; Fri, 3 Aug 2018 14:47:47 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CA5B821CAE; Fri, 3 Aug 2018 14:47:47 +0000 (UTC) (envelope-from asomers@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73EllQ7087100; Fri, 3 Aug 2018 14:47:47 GMT (envelope-from asomers@FreeBSD.org) Received: (from asomers@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73EllVw087099; Fri, 3 Aug 2018 14:47:47 GMT (envelope-from asomers@FreeBSD.org) Message-Id: <201808031447.w73EllVw087099@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: asomers set sender to asomers@FreeBSD.org using -f From: Alan Somers Date: Fri, 3 Aug 2018 14:47:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r337259 - stable/10/sys/security/audit X-SVN-Group: stable-10 X-SVN-Commit-Author: asomers X-SVN-Commit-Paths: stable/10/sys/security/audit X-SVN-Commit-Revision: 337259 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 14:47:48 -0000 Author: asomers Date: Fri Aug 3 14:47:47 2018 New Revision: 337259 URL: https://svnweb.freebsd.org/changeset/base/337259 Log: MFC r336319: auditon(2): fix A_SETPOLICY with 64-bit values A_SETPOLICY is supposed to work with either 64 or 32-bit values, but due to a typo the 64-bit version has never worked correctly. Submitted by: aniketp Reviewed by: asomers, cem Sponsored by: Google, Inc. (GSoC 2018) Differential Revision: https://reviews.freebsd.org/D16222 Modified: stable/10/sys/security/audit/audit_syscalls.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/security/audit/audit_syscalls.c ============================================================================== --- stable/10/sys/security/audit/audit_syscalls.c Fri Aug 3 14:45:53 2018 (r337258) +++ stable/10/sys/security/audit/audit_syscalls.c Fri Aug 3 14:47:47 2018 (r337259) @@ -241,7 +241,7 @@ sys_auditon(struct thread *td, struct auditon_args *ua case A_OLDSETPOLICY: case A_SETPOLICY: if (uap->length == sizeof(udata.au_policy64)) { - if (udata.au_policy & (~AUDIT_CNT|AUDIT_AHLT| + if (udata.au_policy & ~(AUDIT_CNT|AUDIT_AHLT| AUDIT_ARGV|AUDIT_ARGE)) return (EINVAL); audit_fail_stop = ((udata.au_policy64 & AUDIT_CNT) == From owner-svn-src-all@freebsd.org Fri Aug 3 15:35:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 001F3106CF0F; Fri, 3 Aug 2018 15:35:29 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8D2C88BFC; Fri, 3 Aug 2018 15:35:29 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 86A49224CA; Fri, 3 Aug 2018 15:35:29 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73FZTF0012846; Fri, 3 Aug 2018 15:35:29 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73FZTVw012845; Fri, 3 Aug 2018 15:35:29 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808031535.w73FZTVw012845@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 3 Aug 2018 15:35:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337261 - head/sys/amd64/amd64 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/amd64/amd64 X-SVN-Commit-Revision: 337261 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 15:35:30 -0000 Author: kib Date: Fri Aug 3 15:35:29 2018 New Revision: 337261 URL: https://svnweb.freebsd.org/changeset/base/337261 Log: Fix typo in copyinstr_smap, resulting in mis-handling of too long strings. Reported and tested by: pho PR: 230286 Sponsored by: The FreeBSD Foundation Modified: head/sys/amd64/amd64/support.S Modified: head/sys/amd64/amd64/support.S ============================================================================== --- head/sys/amd64/amd64/support.S Fri Aug 3 15:30:32 2018 (r337260) +++ head/sys/amd64/amd64/support.S Fri Aug 3 15:35:29 2018 (r337261) @@ -922,7 +922,7 @@ ENTRY(copyinstr_smap) 2: decq %rdx - jz copyinstr_succ + jz copyinstr_toolong stac lodsb From owner-svn-src-all@freebsd.org Fri Aug 3 15:42:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C9CC106D258; Fri, 3 Aug 2018 15:42:41 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 12453891C0; Fri, 3 Aug 2018 15:42:41 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E7EF622661; Fri, 3 Aug 2018 15:42:40 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73FgeQj017600; Fri, 3 Aug 2018 15:42:40 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73FgdLE017593; Fri, 3 Aug 2018 15:42:39 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808031542.w73FgdLE017593@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Fri, 3 Aug 2018 15:42:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337262 - in stable/11/sys: amd64/amd64 i386/i386 kern vm X-SVN-Group: stable-11 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in stable/11/sys: amd64/amd64 i386/i386 kern vm X-SVN-Commit-Revision: 337262 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 15:42:41 -0000 Author: markj Date: Fri Aug 3 15:42:39 2018 New Revision: 337262 URL: https://svnweb.freebsd.org/changeset/base/337262 Log: MFC r336505, r336764 Have preload_delete_name() free pages backing preloaded data. Modified: stable/11/sys/amd64/amd64/machdep.c stable/11/sys/amd64/amd64/pmap.c stable/11/sys/i386/i386/machdep.c stable/11/sys/i386/i386/pmap.c stable/11/sys/kern/subr_module.c stable/11/sys/vm/vm_extern.h stable/11/sys/vm/vm_kern.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/amd64/machdep.c ============================================================================== --- stable/11/sys/amd64/amd64/machdep.c Fri Aug 3 15:35:29 2018 (r337261) +++ stable/11/sys/amd64/amd64/machdep.c Fri Aug 3 15:42:39 2018 (r337262) @@ -100,6 +100,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #ifdef DDB #ifndef KDB @@ -1229,6 +1230,12 @@ getmemsize(caddr_t kmdp, u_int64_t first) pt_entry_t *pte; quad_t dcons_addr, dcons_size; int page_counter; + + /* + * Tell the physical memory allocator about pages used to store + * the kernel and preloaded data. See kmem_bootstrap_free(). + */ + vm_phys_add_seg((vm_paddr_t)kernphys, trunc_page(first)); bzero(physmap, sizeof(physmap)); physmap_idx = 0; Modified: stable/11/sys/amd64/amd64/pmap.c ============================================================================== --- stable/11/sys/amd64/amd64/pmap.c Fri Aug 3 15:35:29 2018 (r337261) +++ stable/11/sys/amd64/amd64/pmap.c Fri Aug 3 15:42:39 2018 (r337262) @@ -373,6 +373,8 @@ static u_int64_t DMPDphys; /* phys addr of direct mapp static u_int64_t DMPDPphys; /* phys addr of direct mapped level 3 */ static int ndmpdpphys; /* number of DMPDPphys pages */ +static vm_paddr_t KERNend; /* phys addr of end of bootstrap data */ + /* * pmap_mapdev support pre initialization (i.e. console) */ @@ -938,8 +940,9 @@ create_pagetables(vm_paddr_t *firstaddr) /* Map from zero to end of allocations under 2M pages */ /* This replaces some of the KPTphys entries above */ for (i = 0; (i << PDRSHIFT) < *firstaddr; i++) + /* Preset PG_M and PG_A because demotion expects it. */ pd_p[i] = (i << PDRSHIFT) | X86_PG_RW | X86_PG_V | PG_PS | - pg_g; + X86_PG_M | X86_PG_A | pg_g; /* * Because we map the physical blocks in 2M pages, adjust firstaddr @@ -1015,6 +1018,8 @@ pmap_bootstrap(vm_paddr_t *firstaddr) pt_entry_t *pte; int i; + KERNend = *firstaddr; + if (!pti) pg_g = X86_PG_G; @@ -1247,6 +1252,7 @@ pmap_init(void) * Initialize the vm page array entries for the kernel pmap's * page table pages. */ + PMAP_LOCK(kernel_pmap); for (i = 0; i < nkpt; i++) { mpte = PHYS_TO_VM_PAGE(KPTphys + (i << PAGE_SHIFT)); KASSERT(mpte >= vm_page_array && @@ -1255,7 +1261,11 @@ pmap_init(void) mpte->pindex = pmap_pde_pindex(KERNBASE) + i; mpte->phys_addr = KPTphys + (i << PAGE_SHIFT); mpte->wire_count = 1; + if (i << PDRSHIFT < KERNend && + pmap_insert_pt_page(kernel_pmap, mpte)) + panic("pmap_init: pmap_insert_pt_page failed"); } + PMAP_UNLOCK(kernel_pmap); atomic_add_int(&vm_cnt.v_wire_count, nkpt); /* Modified: stable/11/sys/i386/i386/machdep.c ============================================================================== --- stable/11/sys/i386/i386/machdep.c Fri Aug 3 15:35:29 2018 (r337261) +++ stable/11/sys/i386/i386/machdep.c Fri Aug 3 15:42:39 2018 (r337262) @@ -99,6 +99,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #ifdef DDB #ifndef KDB @@ -2074,6 +2075,12 @@ getmemsize(int first) bzero(&vmf, sizeof(vmf)); bzero(physmap, sizeof(physmap)); basemem = 0; + + /* + * Tell the physical memory allocator about pages used to store + * the kernel and preloaded data. See kmem_bootstrap_free(). + */ + vm_phys_add_seg((vm_paddr_t)KERNLOAD, trunc_page(first)); /* * Check if the loader supplied an SMAP memory map. If so, Modified: stable/11/sys/i386/i386/pmap.c ============================================================================== --- stable/11/sys/i386/i386/pmap.c Fri Aug 3 15:35:29 2018 (r337261) +++ stable/11/sys/i386/i386/pmap.c Fri Aug 3 15:42:39 2018 (r337262) @@ -787,6 +787,7 @@ pmap_init(void) * Initialize the vm page array entries for the kernel pmap's * page table pages. */ + PMAP_LOCK(kernel_pmap); for (i = 0; i < NKPT; i++) { mpte = PHYS_TO_VM_PAGE(KPTphys + (i << PAGE_SHIFT)); KASSERT(mpte >= vm_page_array && @@ -794,7 +795,14 @@ pmap_init(void) ("pmap_init: page table page is out of range")); mpte->pindex = i + KPTDI; mpte->phys_addr = KPTphys + (i << PAGE_SHIFT); + mpte->wire_count = 1; + if (pseflag != 0 && + KERNBASE <= i << PDRSHIFT && i << PDRSHIFT < KERNend && + pmap_insert_pt_page(kernel_pmap, mpte)) + panic("pmap_init: pmap_insert_pt_page failed"); } + PMAP_UNLOCK(kernel_pmap); + vm_wire_add(NKPT); /* * Initialize the address space (zone) for the pv entries. Set a Modified: stable/11/sys/kern/subr_module.c ============================================================================== --- stable/11/sys/kern/subr_module.c Fri Aug 3 15:35:29 2018 (r337261) +++ stable/11/sys/kern/subr_module.c Fri Aug 3 15:42:39 2018 (r337262) @@ -31,6 +31,9 @@ __FBSDID("$FreeBSD$"); #include #include +#include +#include + /* * Preloaded module support */ @@ -202,29 +205,42 @@ preload_search_info(caddr_t mod, int inf) void preload_delete_name(const char *name) { - caddr_t curp; - uint32_t *hdr; + caddr_t addr, curp; + uint32_t *hdr, sz; int next; int clearing; + + addr = 0; + sz = 0; if (preload_metadata != NULL) { - + clearing = 0; curp = preload_metadata; for (;;) { hdr = (uint32_t *)curp; - if (hdr[0] == 0 && hdr[1] == 0) - break; + if (hdr[0] == MODINFO_NAME || (hdr[0] == 0 && hdr[1] == 0)) { + /* Free memory used to store the file. */ + if (addr != 0 && sz != 0) + kmem_bootstrap_free((vm_offset_t)addr, sz); + addr = 0; + sz = 0; - /* Search for a MODINFO_NAME field */ - if (hdr[0] == MODINFO_NAME) { + if (hdr[0] == 0) + break; if (!strcmp(name, curp + sizeof(uint32_t) * 2)) clearing = 1; /* got it, start clearing */ - else if (clearing) + else if (clearing) { clearing = 0; /* at next one now.. better stop */ + } } - if (clearing) + if (clearing) { + if (hdr[0] == MODINFO_ADDR) + addr = *(caddr_t *)(curp + sizeof(uint32_t) * 2); + else if (hdr[0] == MODINFO_SIZE) + sz = *(uint32_t *)(curp + sizeof(uint32_t) * 2); hdr[0] = MODINFO_EMPTY; + } /* skip to next field */ next = sizeof(uint32_t) * 2 + hdr[1]; Modified: stable/11/sys/vm/vm_extern.h ============================================================================== --- stable/11/sys/vm/vm_extern.h Fri Aug 3 15:35:29 2018 (r337261) +++ stable/11/sys/vm/vm_extern.h Fri Aug 3 15:42:39 2018 (r337262) @@ -65,6 +65,7 @@ int kmem_back(vm_object_t, vm_offset_t, vm_size_t, int void kmem_unback(vm_object_t, vm_offset_t, vm_size_t); /* Bootstrapping. */ +void kmem_bootstrap_free(vm_offset_t, vm_size_t); vm_map_t kmem_suballoc(vm_map_t, vm_offset_t *, vm_offset_t *, vm_size_t, boolean_t); void kmem_init(vm_offset_t, vm_offset_t); Modified: stable/11/sys/vm/vm_kern.c ============================================================================== --- stable/11/sys/vm/vm_kern.c Fri Aug 3 15:35:29 2018 (r337261) +++ stable/11/sys/vm/vm_kern.c Fri Aug 3 15:42:39 2018 (r337262) @@ -84,6 +84,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -535,6 +536,37 @@ kmem_init(vm_offset_t start, vm_offset_t end) start, VM_PROT_ALL, VM_PROT_ALL, MAP_NOFAULT); /* ... and ending with the completion of the above `insert' */ vm_map_unlock(m); +} + +/* + * kmem_bootstrap_free: + * + * Free pages backing preloaded data (e.g., kernel modules) to the + * system. Currently only supported on platforms that create a + * vm_phys segment for preloaded data. + */ +void +kmem_bootstrap_free(vm_offset_t start, vm_size_t size) +{ +#if defined(__i386__) || defined(__amd64__) + vm_offset_t end, va; + vm_paddr_t pa; + vm_page_t m; + + end = trunc_page(start + size); + start = round_page(start); + + for (va = start; va < end; va += PAGE_SIZE) { + pa = pmap_kextract(va); + m = PHYS_TO_VM_PAGE(pa); + + mtx_lock(&vm_page_queue_free_mtx); + vm_phys_free_pages(m, 0); + mtx_unlock(&vm_page_queue_free_mtx); + } + pmap_remove(kernel_pmap, start, end); + (void)vmem_add(kernel_arena, start, end - start, M_WAITOK); +#endif } #ifdef DIAGNOSTIC From owner-svn-src-all@freebsd.org Fri Aug 3 15:43:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C0D4106D2FA; Fri, 3 Aug 2018 15:43:38 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6C7989349; Fri, 3 Aug 2018 15:43:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9855F2266D; Fri, 3 Aug 2018 15:43:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73Fhbmv017693; Fri, 3 Aug 2018 15:43:37 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73Fhb40017692; Fri, 3 Aug 2018 15:43:37 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808031543.w73Fhb40017692@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Fri, 3 Aug 2018 15:43:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337263 - stable/11/sys/amd64/conf X-SVN-Group: stable-11 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: stable/11/sys/amd64/conf X-SVN-Commit-Revision: 337263 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 15:43:38 -0000 Author: markj Date: Fri Aug 3 15:43:37 2018 New Revision: 337263 URL: https://svnweb.freebsd.org/changeset/base/337263 Log: MFC r337015: COMPAT_LINUX32 has not depended on COMPAT_43 in some time. Modified: stable/11/sys/amd64/conf/NOTES Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/amd64/conf/NOTES ============================================================================== --- stable/11/sys/amd64/conf/NOTES Fri Aug 3 15:42:39 2018 (r337262) +++ stable/11/sys/amd64/conf/NOTES Fri Aug 3 15:43:37 2018 (r337263) @@ -651,7 +651,7 @@ options COMPAT_CLOUDABI64 # Enable Linux ABI emulation #XXX#options COMPAT_LINUX -# Enable 32-bit Linux ABI emulation (requires COMPAT_43 and COMPAT_FREEBSD32) +# Enable 32-bit Linux ABI emulation (requires COMPAT_FREEBSD32). options COMPAT_LINUX32 # Enable the linux-like proc filesystem support (requires COMPAT_LINUX32 From owner-svn-src-all@freebsd.org Fri Aug 3 16:05:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4901106DACB; Fri, 3 Aug 2018 16:05:04 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B0028A008; Fri, 3 Aug 2018 16:05:04 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 38967229E9; Fri, 3 Aug 2018 16:05:04 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73G542M028672; Fri, 3 Aug 2018 16:05:04 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73G53hJ028670; Fri, 3 Aug 2018 16:05:03 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808031605.w73G53hJ028670@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Fri, 3 Aug 2018 16:05:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337264 - head/lib/libc/sys X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/lib/libc/sys X-SVN-Commit-Revision: 337264 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 16:05:04 -0000 Author: br Date: Fri Aug 3 16:05:03 2018 New Revision: 337264 URL: https://svnweb.freebsd.org/changeset/base/337264 Log: MAXLOGNAME changed to 33 in r243023. Update man pages. Sponsored by: DARPA, AFRL Modified: head/lib/libc/sys/getlogin.2 head/lib/libc/sys/getloginclass.2 Modified: head/lib/libc/sys/getlogin.2 ============================================================================== --- head/lib/libc/sys/getlogin.2 Fri Aug 3 15:43:37 2018 (r337263) +++ head/lib/libc/sys/getlogin.2 Fri Aug 3 16:05:03 2018 (r337264) @@ -161,7 +161,7 @@ Login names are limited to .Dv MAXLOGNAME (from .In sys/param.h ) -characters, currently 17 including null. +characters, currently 33 including null. .It Bq Er EPERM The caller tried to set the login name and was not the super-user. .It Bq Er ERANGE Modified: head/lib/libc/sys/getloginclass.2 ============================================================================== --- head/lib/libc/sys/getloginclass.2 Fri Aug 3 15:43:37 2018 (r337263) +++ head/lib/libc/sys/getloginclass.2 Fri Aug 3 16:05:03 2018 (r337264) @@ -80,7 +80,7 @@ Login class names are limited to .Dv MAXLOGNAME (from .In sys/param.h ) -characters, currently 17 including null. +characters, currently 33 including null. .It Bq Er EPERM The caller tried to set the login class and was not the super-user. .It Bq Er ENAMETOOLONG From owner-svn-src-all@freebsd.org Fri Aug 3 16:35:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47FA0106E6BF; Fri, 3 Aug 2018 16:35:38 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F3B558B684; Fri, 3 Aug 2018 16:35:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D645B22EFE; Fri, 3 Aug 2018 16:35:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73GZbIO045037; Fri, 3 Aug 2018 16:35:37 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73GZbmC045036; Fri, 3 Aug 2018 16:35:37 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808031635.w73GZbmC045036@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Fri, 3 Aug 2018 16:35:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337265 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 337265 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 16:35:38 -0000 Author: markj Date: Fri Aug 3 16:35:37 2018 New Revision: 337265 URL: https://svnweb.freebsd.org/changeset/base/337265 Log: Add the required page accounting to kmem_bootstrap_free(). Reviewed by: alc, kib MFC after: 3 days Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D16581 Modified: head/sys/vm/vm_kern.c Modified: head/sys/vm/vm_kern.c ============================================================================== --- head/sys/vm/vm_kern.c Fri Aug 3 16:05:03 2018 (r337264) +++ head/sys/vm/vm_kern.c Fri Aug 3 16:35:37 2018 (r337265) @@ -714,7 +714,11 @@ kmem_bootstrap_free(vm_offset_t start, vm_size_t size) vmd = vm_pagequeue_domain(m); vm_domain_free_lock(vmd); vm_phys_free_pages(m, 0); + vmd->vmd_page_count++; vm_domain_free_unlock(vmd); + + vm_domain_freecnt_inc(vmd, 1); + vm_cnt.v_page_count++; } pmap_remove(kernel_pmap, start, end); (void)vmem_add(kernel_arena, start, end - start, M_WAITOK); From owner-svn-src-all@freebsd.org Fri Aug 3 16:44:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A8BC106EAD1; Fri, 3 Aug 2018 16:44:57 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17B548BD15; Fri, 3 Aug 2018 16:44:57 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EA32523098; Fri, 3 Aug 2018 16:44:56 +0000 (UTC) (envelope-from br@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73GiuRb050009; Fri, 3 Aug 2018 16:44:56 GMT (envelope-from br@FreeBSD.org) Received: (from br@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73Giui2050008; Fri, 3 Aug 2018 16:44:56 GMT (envelope-from br@FreeBSD.org) Message-Id: <201808031644.w73Giui2050008@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: br set sender to br@FreeBSD.org using -f From: Ruslan Bukin Date: Fri, 3 Aug 2018 16:44:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337266 - head/sys/riscv/riscv X-SVN-Group: head X-SVN-Commit-Author: br X-SVN-Commit-Paths: head/sys/riscv/riscv X-SVN-Commit-Revision: 337266 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 16:44:57 -0000 Author: br Date: Fri Aug 3 16:44:56 2018 New Revision: 337266 URL: https://svnweb.freebsd.org/changeset/base/337266 Log: Return ENAMETOOLONG if the latest copied character is not null terminator. Sponsored by: DARPA, AFRL Modified: head/sys/riscv/riscv/copyinout.S Modified: head/sys/riscv/riscv/copyinout.S ============================================================================== --- head/sys/riscv/riscv/copyinout.S Fri Aug 3 16:35:37 2018 (r337265) +++ head/sys/riscv/riscv/copyinout.S Fri Aug 3 16:44:56 2018 (r337266) @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2015 Ruslan Bukin + * Copyright (c) 2015-2018 Ruslan Bukin * All rights reserved. * * Portions of this software were developed by SRI International and the @@ -133,5 +133,8 @@ ENTRY(copyinstr) sd a5, 0(a3) /* done = count */ 4: mv a0, x0 /* return 0 */ + beqz a4, 5f + li a0, ENAMETOOLONG +5: ret END(copyinstr) From owner-svn-src-all@freebsd.org Fri Aug 3 17:14:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C70A0106F234; Fri, 3 Aug 2018 17:14:53 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7D4AD8CE81; Fri, 3 Aug 2018 17:14:53 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5EFC923585; Fri, 3 Aug 2018 17:14:53 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73HErWE065643; Fri, 3 Aug 2018 17:14:53 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73HErNw065642; Fri, 3 Aug 2018 17:14:53 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808031714.w73HErNw065642@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Fri, 3 Aug 2018 17:14:53 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337267 - stable/11/sys/i386/i386 X-SVN-Group: stable-11 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: stable/11/sys/i386/i386 X-SVN-Commit-Revision: 337267 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 17:14:54 -0000 Author: markj Date: Fri Aug 3 17:14:52 2018 New Revision: 337267 URL: https://svnweb.freebsd.org/changeset/base/337267 Log: Fix a mismerge in r337262. This is a direct commit to stable/11. Modified: stable/11/sys/i386/i386/pmap.c Modified: stable/11/sys/i386/i386/pmap.c ============================================================================== --- stable/11/sys/i386/i386/pmap.c Fri Aug 3 16:44:56 2018 (r337266) +++ stable/11/sys/i386/i386/pmap.c Fri Aug 3 17:14:52 2018 (r337267) @@ -802,7 +802,7 @@ pmap_init(void) panic("pmap_init: pmap_insert_pt_page failed"); } PMAP_UNLOCK(kernel_pmap); - vm_wire_add(NKPT); + atomic_add_int(&vm_cnt.v_wire_count, NKPT); /* * Initialize the address space (zone) for the pv entries. Set a From owner-svn-src-all@freebsd.org Fri Aug 3 18:30:03 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7116D104AF81; Fri, 3 Aug 2018 18:30:03 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 24FF28F92C; Fri, 3 Aug 2018 18:30:03 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 06CB82410C; Fri, 3 Aug 2018 18:30:03 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73IU2nj002236; Fri, 3 Aug 2018 18:30:02 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73IU27D002235; Fri, 3 Aug 2018 18:30:02 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808031830.w73IU27D002235@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 3 Aug 2018 18:30:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337268 - head/usr.sbin/pciconf X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/usr.sbin/pciconf X-SVN-Commit-Revision: 337268 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 18:30:03 -0000 Author: kib Date: Fri Aug 3 18:30:02 2018 New Revision: 337268 URL: https://svnweb.freebsd.org/changeset/base/337268 Log: Fix word-size dumping. Sponsored by: The FreeBSD Foundation MFC after: 12 days Modified: head/usr.sbin/pciconf/pciconf.c Modified: head/usr.sbin/pciconf/pciconf.c ============================================================================== --- head/usr.sbin/pciconf/pciconf.c Fri Aug 3 17:14:52 2018 (r337267) +++ head/usr.sbin/pciconf/pciconf.c Fri Aug 3 18:30:02 2018 (r337268) @@ -1137,7 +1137,7 @@ dump_bar(const char *name, const char *reg, const char case 4: dd = (uint32_t *)(uintptr_t)((uintptr_t)pbm.pbm_map_base + pbm.pbm_bar_off + start * width); - for (a = 0; a < count; a += width) { + for (a = 0; a < count; a += width, dd++) { res = fwrite(dd, width, 1, stdout); if (res != 1) { errx(1, "error writing to stdout"); From owner-svn-src-all@freebsd.org Fri Aug 3 18:35:21 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59C92104C1EC; Fri, 3 Aug 2018 18:35:21 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F15C98FD8C; Fri, 3 Aug 2018 18:35:20 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CD188242C7; Fri, 3 Aug 2018 18:35:20 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73IZKxG006952; Fri, 3 Aug 2018 18:35:20 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73IZKRO006950; Fri, 3 Aug 2018 18:35:20 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808031835.w73IZKRO006950@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 3 Aug 2018 18:35:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337269 - in head: sys/dev/pci usr.sbin/pciconf X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head: sys/dev/pci usr.sbin/pciconf X-SVN-Commit-Revision: 337269 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 18:35:21 -0000 Author: kib Date: Fri Aug 3 18:35:20 2018 New Revision: 337269 URL: https://svnweb.freebsd.org/changeset/base/337269 Log: Require write access when mmapping BAR. This actually makes the rights requirements for accessing PCI config space and BARs using /dev/pci same. Since unchanged /dev/pci mode only allows write open for root, default configuration de-facto limits the BAR read to root only. In particular, state-changing reads of the registers are limited to root. Discussed with: se Suggested and reviewed by: jhb (kernel part) Sponsored by: The FreeBSD Foundation MFC after: 12 days Differential revision: https://reviews.freebsd.org/D16580 Modified: head/sys/dev/pci/pci_user.c head/usr.sbin/pciconf/pciconf.c Modified: head/sys/dev/pci/pci_user.c ============================================================================== --- head/sys/dev/pci/pci_user.c Fri Aug 3 18:30:02 2018 (r337268) +++ head/sys/dev/pci/pci_user.c Fri Aug 3 18:35:20 2018 (r337269) @@ -809,7 +809,6 @@ pci_ioctl(struct cdev *dev, u_long cmd, caddr_t data, #endif case PCIOCGETBAR: case PCIOCLISTVPD: - case PCIOCBARMMAP: break; default: return (EPERM); Modified: head/usr.sbin/pciconf/pciconf.c ============================================================================== --- head/usr.sbin/pciconf/pciconf.c Fri Aug 3 18:30:02 2018 (r337268) +++ head/usr.sbin/pciconf/pciconf.c Fri Aug 3 18:35:20 2018 (r337269) @@ -1085,7 +1085,7 @@ dump_bar(const char *name, const char *reg, const char pbm.pbm_flags = 0; pbm.pbm_memattr = VM_MEMATTR_UNCACHEABLE; /* XXX */ - fd = open(_PATH_DEVPCI, O_RDONLY, 0); + fd = open(_PATH_DEVPCI, O_RDWR, 0); if (fd < 0) err(1, "%s", _PATH_DEVPCI); From owner-svn-src-all@freebsd.org Fri Aug 3 18:52:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5F0D9104C8F8; Fri, 3 Aug 2018 18:52:52 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15F7570680; Fri, 3 Aug 2018 18:52:52 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EC13E245F4; Fri, 3 Aug 2018 18:52:51 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73IqpHe017009; Fri, 3 Aug 2018 18:52:51 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73IqpYr017008; Fri, 3 Aug 2018 18:52:51 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201808031852.w73IqpYr017008@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Fri, 3 Aug 2018 18:52:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337270 - in head: . lib/libclang_rt X-SVN-Group: head X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: in head: . lib/libclang_rt X-SVN-Commit-Revision: 337270 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 18:52:52 -0000 Author: jhb Date: Fri Aug 3 18:52:51 2018 New Revision: 337270 URL: https://svnweb.freebsd.org/changeset/base/337270 Log: Install the 32-bit compat sanitizer libraries. The lib32 build was already building the i386 version of the clang sanitizers (libclang_rt) but they were not being installed. This enables the installation. MK_TOOLCHAIN=no was originally added to the install make environment to disable includes so that NO_INCS could be removed. The MK_TOOLCHAIN in bsd.incs.mk was subsequently renamed to MK_INCLUDES, but bsd.lib.mk doesn't even include bsd.incs.mk when LIBRARIES_ONLY is defined which the install make environment for compat libs now defines. However, setting MK_TOOLCHAIN=no forced MK_CLANG=no which disabled libclang_rt during the install32 phase. Remove MK_TOOLCHAIN=no since LIBRARIES_ONLY is now sufficient. Since the libcompat environment overrides both LIBDIR and SHLIBDIR, libclang_rt/Makefile.inc has to set both variables to force the libraries to be installed to the location expected by the compiler. Reviewed by: bdrewery, dim MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D16574 Modified: head/Makefile.libcompat head/lib/libclang_rt/Makefile.inc Modified: head/Makefile.libcompat ============================================================================== --- head/Makefile.libcompat Fri Aug 3 18:35:20 2018 (r337269) +++ head/Makefile.libcompat Fri Aug 3 18:52:51 2018 (r337270) @@ -126,7 +126,7 @@ LIBCOMPATWMAKE+= ${LIBCOMPATWMAKEENV} ${MAKE} ${LIBCOM MAKEOBJDIRPREFIX= \ MK_MAN=no MK_HTML=no LIBCOMPATIMAKE+= ${LIBCOMPATWMAKE:NINSTALL=*:NDESTDIR=*} \ - MK_TOOLCHAIN=no ${IMAKE_INSTALL} \ + ${IMAKE_INSTALL} \ -DLIBRARIES_ONLY _LC_LIBDIRS.yes= lib Modified: head/lib/libclang_rt/Makefile.inc ============================================================================== --- head/lib/libclang_rt/Makefile.inc Fri Aug 3 18:35:20 2018 (r337269) +++ head/lib/libclang_rt/Makefile.inc Fri Aug 3 18:52:51 2018 (r337270) @@ -16,6 +16,7 @@ CRTSRC= ${SRCTOP}/contrib/compiler-rt CLANGDIR= /usr/lib/clang/6.0.1 LIBDIR= ${CLANGDIR}/lib/freebsd +SHLIBDIR= ${LIBDIR} NO_PIC= MK_PROFILE= no From owner-svn-src-all@freebsd.org Fri Aug 3 19:11:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 760FE104D270; Fri, 3 Aug 2018 19:11:01 +0000 (UTC) (envelope-from cy@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CF3671016; Fri, 3 Aug 2018 19:11:01 +0000 (UTC) (envelope-from cy@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0DA84247B1; Fri, 3 Aug 2018 19:11:01 +0000 (UTC) (envelope-from cy@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73JB0th025165; Fri, 3 Aug 2018 19:11:00 GMT (envelope-from cy@FreeBSD.org) Received: (from cy@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73JB0WK025164; Fri, 3 Aug 2018 19:11:00 GMT (envelope-from cy@FreeBSD.org) Message-Id: <201808031911.w73JB0WK025164@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cy set sender to cy@FreeBSD.org using -f From: Cy Schubert Date: Fri, 3 Aug 2018 19:11:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337271 - head/stand/i386/libi386 X-SVN-Group: head X-SVN-Commit-Author: cy X-SVN-Commit-Paths: head/stand/i386/libi386 X-SVN-Commit-Revision: 337271 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 19:11:01 -0000 Author: cy Date: Fri Aug 3 19:11:00 2018 New Revision: 337271 URL: https://svnweb.freebsd.org/changeset/base/337271 Log: Some drives report a geometry that is inconsisetent with the total number of sectors reported through the BIOS. Cylinders * heads * sectors may not necessarily be equal to the total number of sectors reported through int13h function 48h. An example of this is when a Mediasonic HD3-U2B PATA to USB enclosure with a 80 GB disk is attached. Loader hangs at line 506 of stand/i386/libi386/biosdisk.c while attempting to read sectors beyond the end of the disk, sector 156906855. I discovered that the Mediasonic enclosure was reporting the disk with 9767 cylinders, 255 heads, 63 sectors/track. That's 156906855 sectors. However camcontrol and Windows 10 both report report the disk having 156301488 sectors, not the calculated value. At line 280 biosdisk.c sets the sectors to the higher of either bd->bd_sectors or the total calculated at line 276 (156906855) instead of the lower and correct value of 156301488 reported by int 13h 48h. This was tested on all three of my Mediasonic HD3-U2B PATA to USB enclosures. Instead of using the higher of bd_sectors (returned by int13h) or the calculated value, this patch uses the lower and safer of the values. Reviewed by: tsoome@ Differential Revision: https://reviews.freebsd.org/D16577 Modified: head/stand/i386/libi386/biosdisk.c Modified: head/stand/i386/libi386/biosdisk.c ============================================================================== --- head/stand/i386/libi386/biosdisk.c Fri Aug 3 18:52:51 2018 (r337270) +++ head/stand/i386/libi386/biosdisk.c Fri Aug 3 19:11:00 2018 (r337271) @@ -275,7 +275,7 @@ bd_int13probe(struct bdinfo *bd) total = (uint64_t)params.cylinders * params.heads * params.sectors_per_track; - if (bd->bd_sectors < total) + if (bd->bd_sectors > total) bd->bd_sectors = total; ret = 1; From owner-svn-src-all@freebsd.org Fri Aug 3 19:24:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBB47104D94F; Fri, 3 Aug 2018 19:24:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9861971A46; Fri, 3 Aug 2018 19:24:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7A55E24AEC; Fri, 3 Aug 2018 19:24:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73JO5Yh032771; Fri, 3 Aug 2018 19:24:05 GMT (envelope-from bdrewery@FreeBSD.org) Received: (from bdrewery@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73JO5iM032770; Fri, 3 Aug 2018 19:24:05 GMT (envelope-from bdrewery@FreeBSD.org) Message-Id: <201808031924.w73JO5iM032770@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: bdrewery set sender to bdrewery@FreeBSD.org using -f From: Bryan Drewery Date: Fri, 3 Aug 2018 19:24:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337272 - head/sys/dev/filemon X-SVN-Group: head X-SVN-Commit-Author: bdrewery X-SVN-Commit-Paths: head/sys/dev/filemon X-SVN-Commit-Revision: 337272 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 19:24:06 -0000 Author: bdrewery Date: Fri Aug 3 19:24:04 2018 New Revision: 337272 URL: https://svnweb.freebsd.org/changeset/base/337272 Log: Fix some filemon path logging issues. - Properly handle snprintf return value for truncation and avoid overflowing the later write with the bogus length. - Increase the msgbufr size to handle a rename of 2 full files. The larger allocation causes a slight performance hit which will be mitigated in the future. A rewrite with sbufs will likely be done as well. Reported by: Ilja Van Sprundel MFC after: 2 weeks Approved by: so (gtetlow) Reviewed by: kib Sponsored by: Dell EMC Differential Revision: https://reviews.freebsd.org/D16098 Modified: head/sys/dev/filemon/filemon.c head/sys/dev/filemon/filemon_wrapper.c Modified: head/sys/dev/filemon/filemon.c ============================================================================== --- head/sys/dev/filemon/filemon.c Fri Aug 3 19:11:00 2018 (r337271) +++ head/sys/dev/filemon/filemon.c Fri Aug 3 19:24:04 2018 (r337272) @@ -88,7 +88,7 @@ struct filemon { struct ucred *cred; /* Credential of tracer. */ char fname1[MAXPATHLEN]; /* Temporary filename buffer. */ char fname2[MAXPATHLEN]; /* Temporary filename buffer. */ - char msgbufr[1024]; /* Output message buffer. */ + char msgbufr[2*MAXPATHLEN + 100]; /* Output message buffer. */ int error; /* Log write error, returned on close(2). */ u_int refcnt; /* Pointer reference count. */ u_int proccnt; /* Process count. */ @@ -200,8 +200,8 @@ filemon_write_header(struct filemon *filemon) "# filemon version %d\n# Target pid %d\n# Start %ju.%06ju\nV %d\n", FILEMON_VERSION, curproc->p_pid, (uintmax_t)now.tv_sec, (uintmax_t)now.tv_usec, FILEMON_VERSION); - - filemon_output(filemon, filemon->msgbufr, len); + if (len < sizeof(filemon->msgbufr)) + filemon_output(filemon, filemon->msgbufr, len); } /* @@ -268,7 +268,8 @@ filemon_close_log(struct filemon *filemon) "# Stop %ju.%06ju\n# Bye bye\n", (uintmax_t)now.tv_sec, (uintmax_t)now.tv_usec); - filemon_output(filemon, filemon->msgbufr, len); + if (len < sizeof(filemon->msgbufr)) + filemon_output(filemon, filemon->msgbufr, len); fp = filemon->fp; filemon->fp = NULL; Modified: head/sys/dev/filemon/filemon_wrapper.c ============================================================================== --- head/sys/dev/filemon/filemon_wrapper.c Fri Aug 3 19:11:00 2018 (r337271) +++ head/sys/dev/filemon/filemon_wrapper.c Fri Aug 3 19:24:04 2018 (r337272) @@ -39,6 +39,11 @@ __FBSDID("$FreeBSD$"); #include #include +#include + +static void filemon_output_event(struct filemon *filemon, const char *fmt, ...) + __printflike(2, 3); + static eventhandler_tag filemon_exec_tag; static eventhandler_tag filemon_exit_tag; static eventhandler_tag filemon_fork_tag; @@ -71,11 +76,25 @@ filemon_output(struct filemon *filemon, char *msg, siz filemon->error = error; } +static void +filemon_output_event(struct filemon *filemon, const char *fmt, ...) +{ + va_list ap; + size_t len; + + va_start(ap, fmt); + len = vsnprintf(filemon->msgbufr, sizeof(filemon->msgbufr), fmt, ap); + va_end(ap); + /* The event is truncated but still worth logging. */ + if (len >= sizeof(filemon->msgbufr)) + len = sizeof(filemon->msgbufr) - 1; + filemon_output(filemon, filemon->msgbufr, len); +} + static int filemon_wrapper_chdir(struct thread *td, struct chdir_args *uap) { int error, ret; - size_t len; struct filemon *filemon; if ((ret = sys_chdir(td, uap)) == 0) { @@ -86,11 +105,8 @@ filemon_wrapper_chdir(struct thread *td, struct chdir_ goto copyfail; } - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "C %d %s\n", + filemon_output_event(filemon, "C %d %s\n", curproc->p_pid, filemon->fname1); - - filemon_output(filemon, filemon->msgbufr, len); copyfail: filemon_drop(filemon); } @@ -104,16 +120,12 @@ filemon_event_process_exec(void *arg __unused, struct struct image_params *imgp) { struct filemon *filemon; - size_t len; if ((filemon = filemon_proc_get(p)) != NULL) { - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "E %d %s\n", + filemon_output_event(filemon, "E %d %s\n", p->p_pid, imgp->execpath != NULL ? imgp->execpath : ""); - filemon_output(filemon, filemon->msgbufr, len); - /* If the credentials changed then cease tracing. */ if (imgp->newcred != NULL && imgp->credential_setid && @@ -140,7 +152,6 @@ static void _filemon_wrapper_openat(struct thread *td, char *upath, int flags, int fd) { int error; - size_t len; struct file *fp; struct filemon *filemon; char *atpath, *freepath; @@ -166,10 +177,8 @@ _filemon_wrapper_openat(struct thread *td, char *upath * XXX: This may be able to come out with * the namecache lookup now. */ - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "A %d %s\n", + filemon_output_event(filemon, "A %d %s\n", curproc->p_pid, filemon->fname1); - filemon_output(filemon, filemon->msgbufr, len); /* * Try to resolve the path from the vnode using the * namecache. It may be inaccurate, but better @@ -187,19 +196,15 @@ _filemon_wrapper_openat(struct thread *td, char *upath * to also output an R to distinguish from * O_WRONLY. */ - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "R %d %s%s%s\n", + filemon_output_event(filemon, "R %d %s%s%s\n", curproc->p_pid, atpath, atpath[0] != '\0' ? "/" : "", filemon->fname1); - filemon_output(filemon, filemon->msgbufr, len); } - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "%c %d %s%s%s\n", + filemon_output_event(filemon, "%c %d %s%s%s\n", (flags & O_ACCMODE) ? 'W':'R', curproc->p_pid, atpath, atpath[0] != '\0' ? "/" : "", filemon->fname1); - filemon_output(filemon, filemon->msgbufr, len); copyfail: filemon_drop(filemon); if (fp != NULL) @@ -234,7 +239,6 @@ static int filemon_wrapper_rename(struct thread *td, struct rename_args *uap) { int error, ret; - size_t len; struct filemon *filemon; if ((ret = sys_rename(td, uap)) == 0) { @@ -247,11 +251,8 @@ filemon_wrapper_rename(struct thread *td, struct renam goto copyfail; } - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "M %d '%s' '%s'\n", + filemon_output_event(filemon, "M %d '%s' '%s'\n", curproc->p_pid, filemon->fname1, filemon->fname2); - - filemon_output(filemon, filemon->msgbufr, len); copyfail: filemon_drop(filemon); } @@ -264,7 +265,6 @@ static void _filemon_wrapper_link(struct thread *td, char *upath1, char *upath2) { struct filemon *filemon; - size_t len; int error; if ((filemon = filemon_proc_get(curproc)) != NULL) { @@ -276,11 +276,8 @@ _filemon_wrapper_link(struct thread *td, char *upath1, goto copyfail; } - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "L %d '%s' '%s'\n", + filemon_output_event(filemon, "L %d '%s' '%s'\n", curproc->p_pid, filemon->fname1, filemon->fname2); - - filemon_output(filemon, filemon->msgbufr, len); copyfail: filemon_drop(filemon); } @@ -322,15 +319,12 @@ filemon_wrapper_linkat(struct thread *td, struct linka static void filemon_event_process_exit(void *arg __unused, struct proc *p) { - size_t len; struct filemon *filemon; if ((filemon = filemon_proc_get(p)) != NULL) { - len = snprintf(filemon->msgbufr, sizeof(filemon->msgbufr), - "X %d %d %d\n", p->p_pid, p->p_xexit, p->p_xsig); + filemon_output_event(filemon, "X %d %d %d\n", + p->p_pid, p->p_xexit, p->p_xsig); - filemon_output(filemon, filemon->msgbufr, len); - /* * filemon_untrack_processes() may have dropped this p_filemon * already while in filemon_proc_get() before acquiring the @@ -350,7 +344,6 @@ static int filemon_wrapper_unlink(struct thread *td, struct unlink_args *uap) { int error, ret; - size_t len; struct filemon *filemon; if ((ret = sys_unlink(td, uap)) == 0) { @@ -361,11 +354,8 @@ filemon_wrapper_unlink(struct thread *td, struct unlin goto copyfail; } - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "D %d %s\n", + filemon_output_event(filemon, "D %d %s\n", curproc->p_pid, filemon->fname1); - - filemon_output(filemon, filemon->msgbufr, len); copyfail: filemon_drop(filemon); } @@ -378,15 +368,11 @@ static void filemon_event_process_fork(void *arg __unused, struct proc *p1, struct proc *p2, int flags __unused) { - size_t len; struct filemon *filemon; if ((filemon = filemon_proc_get(p1)) != NULL) { - len = snprintf(filemon->msgbufr, - sizeof(filemon->msgbufr), "F %d %d\n", + filemon_output_event(filemon, "F %d %d\n", p1->p_pid, p2->p_pid); - - filemon_output(filemon, filemon->msgbufr, len); /* * filemon_untrack_processes() or From owner-svn-src-all@freebsd.org Fri Aug 3 19:39:26 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87B26104DC6B; Fri, 3 Aug 2018 19:39:26 +0000 (UTC) (envelope-from dab0816@gmail.com) Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E97A271FC1; Fri, 3 Aug 2018 19:39:25 +0000 (UTC) (envelope-from dab0816@gmail.com) Received: by mail-io0-x235.google.com with SMTP id r15-v6so5947356ioa.3; Fri, 03 Aug 2018 12:39:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=y3fuwpAObwC5Ylk3dHVy5MLiGU67+qj5Va/nnbPQpzE=; b=pkxdKuQzs88tJkEtd9k5cqkpUohOAb4Yq7Kq1uRzNu9FJn+VqZVu+pZ7eSuVEa5ftj Rzfba9MOQHo+fZf9G1Xaysgg9YCziAmSICx3N8ixSjsL837o6Pii/4XnomLiRfwxnEbe 2y4WvxNnUAMz2Hko9Sb7hEb7hwl4eaQ6xcK3tRGVZ7zFG/Z2PRVwF5slftIMeMTo7D5F BFCXagPBunFkYz51vy3k7p3uyxkZMP/1YK16q2kktD7gNAt4tOyNkM8ARhihU3xB3sCu uy3ygGXnjLeiyED9YM8amVk5RURLe/5OWqDd1uGNJeuDMuDY/Fh4V/bYlwIYnpOHyaQE NMjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:openpgp :autocrypt:message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding:content-language; bh=y3fuwpAObwC5Ylk3dHVy5MLiGU67+qj5Va/nnbPQpzE=; b=cPy+mC2xJnmYZnoiKoHz9q0nMPNmRbihNm/LWpZ7aIa6o0U0jFJm1GFFlnoeBnB43E VFXAK7yOPdZjCe66DOKrpYX7AiduV+4fsfGdAunqbqZpbENqNCtZ01YL6gMYRmZV4/bv pHIZVhLb3G41QFJFTXd0TIdxhhUYwp1ZbzLnvTduCwrfuL1Db5ukfGXht2uWj1pfdLdk dKS6yucdAtQiH0qHSh1HmOkpQGIjtN9sOke+KbxlMgglbEH9qlhvqmf/SOtjqPSBQ4/e h82cj5NJMHM2BKSY9yD6TghaPWhYzpBy+ewCLUsLGpT7USgUNj+6zRJM7JDXtLx7twRt 2bDQ== X-Gm-Message-State: AOUpUlE09dvxJ4LFBuQksu+ot08bMpdD+3fei91DyO12icpKMSRn8Wwg 5a+36C2M3hiPy8atiXMJUGxw60s+ X-Google-Smtp-Source: AAOMgpesJKUt53xfCh1i58TXJWoaDJifxA4N05W6LMFV/ucE5UED067+VgUcSCxCn1b/89EFCkhaQg== X-Received: by 2002:a5e:c60c:: with SMTP id f12-v6mr7676526iok.108.1533325165262; Fri, 03 Aug 2018 12:39:25 -0700 (PDT) Received: from dab-xps-13-9370.localdomain ([73.242.52.255]) by smtp.gmail.com with ESMTPSA id 130-v6sm95462itv.32.2018.08.03.12.39.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Aug 2018 12:39:24 -0700 (PDT) Sender: David Bright From: "David A. Bright" X-Google-Original-From: "David A. Bright" Subject: Re: svn commit: r336802 - head/tests/sys/kqueue/libkqueue To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201807280253.w6S2rax2078727@repo.freebsd.org> Openpgp: preference=signencrypt Autocrypt: addr=dab@FreeBSD.org; prefer-encrypt=mutual; keydata= xsBNBFgqQsIBCADfDUZ79qW2MKVRQl9l9zMYzppdHn+eS7KGjUP9hjmYUuX3vM0Sc5DlJ9Py 59Byucg8zo6eow1iv6NdiEPuUEYB2ujDP/kbmC397zEt6p9ldlL4PXXSJumpdZCzwz4ACYB5 ke7OCljNqbigzoZgX5DXjyjx87NfG6eWkbnEweDPmae7DGj5bwQmJPoTqvJR7Ot/R/73ix8l hph1vdl83Iqu+IDVdlXTI2Afz4UaCRdfDiXgMQIo/HABXJ90+TXG/vH37Jb0kBflql9sEaUs CcTvMrye0Wjgv3VSPZm3ZEgmR7UKR71p2MJVlVVZUH3FHQ2jDqKpCMi7F3ERlvRDktm/ABEB AAHNHkRhdmlkIEJyaWdodCA8ZGFiQGZyZWVic2Qub3JnPsLAfQQTAQoAJwUCWCpCwgIbAwUJ BaOagAULCQgHAwUVCgkICwUWAwIBAAIeAQIXgAAKCRAzeMsxoQO+aqagCADDt4FbZBz3VFev TEtAjuyzCDPmPom1MBwsh6KB2J84wKOsXPmZ7vqaZ2G70fCPNowqmHq56cAbp0BD1Lf6gD+3 imeacKc/E6n+L49LOhOtQ8EUyHs/nAUmlE1PrOS1w5wmCgRjPXa5k+zxj/YMsAEFdMn6UzOl 5MIJN/zhBB5tU2bkP5hF4rVvpQ1zlnSldoBLfPlG/w1RzvOSJfunZUTSNVrKYlSjEQ7q56ud xKoLPEdr+X8kqwZilh5WBiTt7mMw7lrP6dMV/GGmDg893J6conQ1l2RN6Un1gKlc3rgdz3sh ncr786WF3cV2dFSIrO/Pn3iNRR3uJouRLdLhU0kMzsBNBFgqQsIBCAC5ABwcVerpyp5NzHBD FSbxEyPab6HFH1E4zksEGyeypU8UW5acc2ko5Ovz7CE0n21uhBtIt/Lz5Koxz0LSqqnk9jIQ L/4XVvne+8YnT+Tf9DZv9We5K0tFit/DFgCzIpxacoxLGBRNSkSzFrAepvVinTk905igdYZY fxtHrAREU4BdqBMQtbgGaewHtOLlRRr+N4wkAZlJLa7RaLn5s/EakpmCM1eR8z0fOBNJSXlW Ozqh1cvwWZ00qj0rYXwXu0fgzpREmJFS01GByFuB4xDyqfAoynxKNBJH/qAcJTp7jST8PLWe 3Tblm4acsIAPHUcABgMnZYdMQnSwwa9/W1DlABEBAAHCwGUEGAEKAA8FAlgqQsICGwwFCQWj moAACgkQM3jLMaEDvmp8hQgAzKTYmU2cf35NdrtT9C7jBDWSVSDdyKHYSAKoP+vAkC3A1HH6 wohE5JvOZ6245vgBQwFHvzh2v7jDUWmB3CrcSSMUoUIwl15mWlCdCy/C3SAknxQx+zHYmx9v uHPS1yF2l0KPS30Kc1le1GqVYi5wxnWPo+gE3MH4DDh5LdDExYveMuBgybdNxV8Qvr25UghJ HrQCT+FCISo0FActO1Z5HzdOKxQe6KTn+zKr4yKCOe4kb7GUaAX3Pt82J7bMtwRzAKLyjMJV QueL/cr+geerKKrd8FyVAgQMhdBYD3lyjK6nGssgr4Og9+QtsE2RAVH9qqhcF3kI7nxJ7EDb /d3xag== Message-ID: <3a51a8e8-630d-29ae-7985-a64ee6967679@FreeBSD.org> Date: Fri, 3 Aug 2018 14:39:24 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <201807280253.w6S2rax2078727@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 19:39:26 -0000 On 07/27/2018 09:53 PM, David Bright wrote: > Author: dab > Date: Sat Jul 28 02:53:36 2018 > New Revision: 336802 > URL: https://svnweb.freebsd.org/changeset/base/336802 > > Log: > Fix compilation error on some arches after r336761 & r336781. > > Another cast for printing an intmax_t was needed in a kqueue test for > some arches. > > Pointy-hat: me (twice) > MFC after: 1 week It turns out that this won't be MFC'd as the change doesn't apply to stable/11 or stable/10. The particular test corrected by this change doesn't exist in those branches. -- David Bright dab@FreeBSD.org From owner-svn-src-all@freebsd.org Fri Aug 3 19:40:28 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3349A104DCDA; Fri, 3 Aug 2018 19:40:28 +0000 (UTC) (envelope-from dab0816@gmail.com) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A9316720F4; Fri, 3 Aug 2018 19:40:27 +0000 (UTC) (envelope-from dab0816@gmail.com) Received: by mail-it0-x22a.google.com with SMTP id s7-v6so9806784itb.4; Fri, 03 Aug 2018 12:40:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:openpgp:autocrypt:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=nnpo7tvcPhTnwCjy+no0C58dPtLPteOSPTz374MNkDo=; b=OcP8DRP082KaIXeBHzpUdbHBXi1HbzRFMcHd2PosDgXF2NX7GPzAYpEEzehZlrZVSW mdV+SF8oeiVYp4IJDIaec4VFrL0U6ZHO5SdRLwIqUft5DJw33egIz0uE10cSTDplyYHw Uzik2sU/HVJLdcRc5PtWuwy4wJ+zSsQD80QZRG12ptb+FBfaBMmStsKtkP+3BZaU+m/d UfDdFByos2mvOAb6Np+dhSz0iAVu3Oy3S42mUBgZ9mDWsnrSFiNasstE6LEihVMduARP ufqyZS2eAjS0rJrBOb6UTq93IZ/vtPyg7XHLtjnrId+PFOXiZYT/1ZmhNgTT9XNiwLkr Pd2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:openpgp :autocrypt:message-id:date:user-agent:mime-version:in-reply-to :content-transfer-encoding:content-language; bh=nnpo7tvcPhTnwCjy+no0C58dPtLPteOSPTz374MNkDo=; b=okxXIJaH0r694dmAlOteXFsdWrHVQc3AAsjlUoze/RAgRlUIuGS3epXSVHqgZofDlZ 4VL+peMFQ/1FjBj/nu4LubWaLvOCdDHs86+PRrH0Qt27gQRvCkPCke/7pg/23ualHX/K mWL9E69gVSxsxGbnuiAsvZcMMfJgYtwf/iFC8ZIir7SKjggu1Do6oNJHY3CGvDEiddn1 +hvmXQNJEcUdYO/TFyqtNwTbRVIbTToDmeU/7MNVE8q52vUPgjkAzrbZj7IGoouovmdP lFg7wuy6nsNIFoeIAoVkr/URK1LoAeucqMUadkx1L4wA8sFnV16triK/EBMiWKEGXM+g XwwQ== X-Gm-Message-State: AOUpUlGie2zBVQo7k/0EUfP7uiy8MOPNhd/XixXvYV/Pxm2XjwEKW83W V/T1173WwHoFsAXJ7XhaNcu5nfEa X-Google-Smtp-Source: AAOMgpdDArnUjm2l3wzPb/sqboLjPASBzsnbO+Bo/SJkAtoo+Ol+XrPC0gH2+g09qcFiL97eAElTjw== X-Received: by 2002:a24:d681:: with SMTP id o123-v6mr7699063itg.115.1533325226956; Fri, 03 Aug 2018 12:40:26 -0700 (PDT) Received: from dab-xps-13-9370.localdomain ([73.242.52.255]) by smtp.gmail.com with ESMTPSA id o18-v6sm394433ioa.83.2018.08.03.12.40.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Aug 2018 12:40:26 -0700 (PDT) Sender: David Bright From: "David A. Bright" X-Google-Original-From: "David A. Bright" Subject: Re: svn commit: r336905 - head/tests/sys/kqueue/libkqueue To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201807301421.w6UELnWf076212@repo.freebsd.org> Openpgp: preference=signencrypt Autocrypt: addr=dab@FreeBSD.org; prefer-encrypt=mutual; keydata= xsBNBFgqQsIBCADfDUZ79qW2MKVRQl9l9zMYzppdHn+eS7KGjUP9hjmYUuX3vM0Sc5DlJ9Py 59Byucg8zo6eow1iv6NdiEPuUEYB2ujDP/kbmC397zEt6p9ldlL4PXXSJumpdZCzwz4ACYB5 ke7OCljNqbigzoZgX5DXjyjx87NfG6eWkbnEweDPmae7DGj5bwQmJPoTqvJR7Ot/R/73ix8l hph1vdl83Iqu+IDVdlXTI2Afz4UaCRdfDiXgMQIo/HABXJ90+TXG/vH37Jb0kBflql9sEaUs CcTvMrye0Wjgv3VSPZm3ZEgmR7UKR71p2MJVlVVZUH3FHQ2jDqKpCMi7F3ERlvRDktm/ABEB AAHNHkRhdmlkIEJyaWdodCA8ZGFiQGZyZWVic2Qub3JnPsLAfQQTAQoAJwUCWCpCwgIbAwUJ BaOagAULCQgHAwUVCgkICwUWAwIBAAIeAQIXgAAKCRAzeMsxoQO+aqagCADDt4FbZBz3VFev TEtAjuyzCDPmPom1MBwsh6KB2J84wKOsXPmZ7vqaZ2G70fCPNowqmHq56cAbp0BD1Lf6gD+3 imeacKc/E6n+L49LOhOtQ8EUyHs/nAUmlE1PrOS1w5wmCgRjPXa5k+zxj/YMsAEFdMn6UzOl 5MIJN/zhBB5tU2bkP5hF4rVvpQ1zlnSldoBLfPlG/w1RzvOSJfunZUTSNVrKYlSjEQ7q56ud xKoLPEdr+X8kqwZilh5WBiTt7mMw7lrP6dMV/GGmDg893J6conQ1l2RN6Un1gKlc3rgdz3sh ncr786WF3cV2dFSIrO/Pn3iNRR3uJouRLdLhU0kMzsBNBFgqQsIBCAC5ABwcVerpyp5NzHBD FSbxEyPab6HFH1E4zksEGyeypU8UW5acc2ko5Ovz7CE0n21uhBtIt/Lz5Koxz0LSqqnk9jIQ L/4XVvne+8YnT+Tf9DZv9We5K0tFit/DFgCzIpxacoxLGBRNSkSzFrAepvVinTk905igdYZY fxtHrAREU4BdqBMQtbgGaewHtOLlRRr+N4wkAZlJLa7RaLn5s/EakpmCM1eR8z0fOBNJSXlW Ozqh1cvwWZ00qj0rYXwXu0fgzpREmJFS01GByFuB4xDyqfAoynxKNBJH/qAcJTp7jST8PLWe 3Tblm4acsIAPHUcABgMnZYdMQnSwwa9/W1DlABEBAAHCwGUEGAEKAA8FAlgqQsICGwwFCQWj moAACgkQM3jLMaEDvmp8hQgAzKTYmU2cf35NdrtT9C7jBDWSVSDdyKHYSAKoP+vAkC3A1HH6 wohE5JvOZ6245vgBQwFHvzh2v7jDUWmB3CrcSSMUoUIwl15mWlCdCy/C3SAknxQx+zHYmx9v uHPS1yF2l0KPS30Kc1le1GqVYi5wxnWPo+gE3MH4DDh5LdDExYveMuBgybdNxV8Qvr25UghJ HrQCT+FCISo0FActO1Z5HzdOKxQe6KTn+zKr4yKCOe4kb7GUaAX3Pt82J7bMtwRzAKLyjMJV QueL/cr+geerKKrd8FyVAgQMhdBYD3lyjK6nGssgr4Og9+QtsE2RAVH9qqhcF3kI7nxJ7EDb /d3xag== Message-ID: Date: Fri, 3 Aug 2018 14:40:26 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <201807301421.w6UELnWf076212@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 19:40:28 -0000 On 07/30/2018 09:21 AM, David Bright wrote: > Author: dab > Date: Mon Jul 30 14:21:49 2018 > New Revision: 336905 > URL: https://svnweb.freebsd.org/changeset/base/336905 > > Log: > Correct possible misleading error message in kqtest. > > ian@ pointed out that in the test_abstime() function time(NULL) is > used twice; once in an "if" test and again in the enclosed error > message. If the true branch was taken and the process got preempted > before the second time(NULL) call, by the time the error message was > generated enough time could have elapsed that the message could claim > that the event came "too early" but print an event time that was after > the expected timeout. Correct by making the time(NULL) call only once > and using that returned time in both the "if" test and the error > message. > > Reported by: ian@ > MFC after: 4 days > X-MFC-with: r336761, r336781, r336802 It turns out that this won't be MFC'd as the change doesn't apply to stable/11 or stable/10. The particular test corrected by this change doesn't exist in those branches. -- David Bright dab@FreeBSD.org From owner-svn-src-all@freebsd.org Fri Aug 3 20:04:07 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E37B5104E687; Fri, 3 Aug 2018 20:04:06 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A09D72CDF; Fri, 3 Aug 2018 20:04:06 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7B27625141; Fri, 3 Aug 2018 20:04:06 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73K46m2053250; Fri, 3 Aug 2018 20:04:06 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73K46XJ053249; Fri, 3 Aug 2018 20:04:06 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201808032004.w73K46XJ053249@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Fri, 3 Aug 2018 20:04:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337273 - head/sys/dev/nvme X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/dev/nvme X-SVN-Commit-Revision: 337273 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 20:04:07 -0000 Author: jhibbits Date: Fri Aug 3 20:04:06 2018 New Revision: 337273 URL: https://svnweb.freebsd.org/changeset/base/337273 Log: nvme(4): Add bus_dmamap_sync() at the end of the request path Summary: Some architectures, in this case powerpc64, need explicit synchronization barriers vs device accesses. Prior to this change, when running 'make buildworld -j72' on a 18-core (72-thread) POWER9, I would see controller resets often. With this change, I don't see these resets messages, though another tester still does, for yet to be determined reasons, so this may not be a complete fix. Additionally, I see a ~5-10% speed up in buildworld times, likely due to not needing to reset the controller. Reviewed By: jimharris Differential Revision: https://reviews.freebsd.org/D16570 Modified: head/sys/dev/nvme/nvme_qpair.c Modified: head/sys/dev/nvme/nvme_qpair.c ============================================================================== --- head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 19:24:04 2018 (r337272) +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 20:04:06 2018 (r337273) @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct nvme_qpair *qpair, req->retries++; nvme_qpair_submit_tracker(qpair, tr); } else { - if (req->type != NVME_REQUEST_NULL) + if (req->type != NVME_REQUEST_NULL) { + bus_dmamap_sync(qpair->dma_tag_payload, + tr->payload_dma_map, + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); bus_dmamap_unload(qpair->dma_tag_payload, tr->payload_dma_map); + } nvme_free_request(req); tr->req = NULL; @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct nvme_qpair *qpai */ return (false); + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); while (1) { cpl = qpair->cpl[qpair->cq_head]; @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair *qpair, st if (++qpair->sq_tail == qpair->num_entries) qpair->sq_tail = 0; + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); +#ifndef __powerpc__ + /* + * powerpc's bus_dmamap_sync() already includes a heavyweight sync, but + * no other archs do. + */ wmb(); +#endif + nvme_mmio_write_4(qpair->ctrlr, doorbell[qpair->id].sq_tdbl, qpair->sq_tail); @@ -879,6 +894,8 @@ nvme_payload_map(void *arg, bus_dma_segment_t *seg, in tr->req->cmd.prp2 = 0; } + bus_dmamap_sync(tr->qpair->dma_tag_payload, tr->payload_dma_map, + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); nvme_qpair_submit_tracker(tr->qpair, tr); } From owner-svn-src-all@freebsd.org Fri Aug 3 21:59:03 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93C901052939; Fri, 3 Aug 2018 21:59:03 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C4C777201; Fri, 3 Aug 2018 21:59:03 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 107A926308; Fri, 3 Aug 2018 21:59:03 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73Lx2kQ009674; Fri, 3 Aug 2018 21:59:02 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73Lx1p5009667; Fri, 3 Aug 2018 21:59:01 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201808032159.w73Lx1p5009667@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Fri, 3 Aug 2018 21:59:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337274 - stable/11/release/arm X-SVN-Group: stable-11 X-SVN-Commit-Author: manu X-SVN-Commit-Paths: stable/11/release/arm X-SVN-Commit-Revision: 337274 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 21:59:03 -0000 Author: manu Date: Fri Aug 3 21:59:01 2018 New Revision: 337274 URL: https://svnweb.freebsd.org/changeset/base/337274 Log: MFC r336997: release: Restore copy of boot.scr for some board This is not a problem for 12-CURRENT as EFI boot works but it doesn't for 11. While here some board arm_install_uboot also copy ubldr.bin et create firstboot files but it's already done in arm_install_boot Reviewed by: gjb Differential Revision: https://reviews.freebsd.org/D16481 Modified: stable/11/release/arm/BANANAPI.conf stable/11/release/arm/BEAGLEBONE.conf stable/11/release/arm/CUBIEBOARD.conf stable/11/release/arm/CUBIEBOARD2.conf stable/11/release/arm/CUBOX-HUMMINGBOARD.conf stable/11/release/arm/PANDABOARD.conf stable/11/release/arm/WANDBOARD.conf Directory Properties: stable/11/ (props changed) Modified: stable/11/release/arm/BANANAPI.conf ============================================================================== --- stable/11/release/arm/BANANAPI.conf Fri Aug 3 20:04:06 2018 (r337273) +++ stable/11/release/arm/BANANAPI.conf Fri Aug 3 21:59:01 2018 (r337274) @@ -20,20 +20,15 @@ export BOARDNAME="BANANAPI" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-bananapi" UBOOT_FILES="u-boot-sunxi-with-spl.bin" + FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: stable/11/release/arm/BEAGLEBONE.conf ============================================================================== --- stable/11/release/arm/BEAGLEBONE.conf Fri Aug 3 20:04:06 2018 (r337273) +++ stable/11/release/arm/BEAGLEBONE.conf Fri Aug 3 21:59:01 2018 (r337274) @@ -30,6 +30,6 @@ arm_install_uboot() { sync umount_loop ${CHROOTDIR}/${FATMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - + return 0 } Modified: stable/11/release/arm/CUBIEBOARD.conf ============================================================================== --- stable/11/release/arm/CUBIEBOARD.conf Fri Aug 3 20:04:06 2018 (r337273) +++ stable/11/release/arm/CUBIEBOARD.conf Fri Aug 3 21:59:01 2018 (r337274) @@ -20,21 +20,14 @@ arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard" UBOOT_FILES="u-boot-sunxi-with-spl.bin" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: stable/11/release/arm/CUBIEBOARD2.conf ============================================================================== --- stable/11/release/arm/CUBIEBOARD2.conf Fri Aug 3 20:04:06 2018 (r337273) +++ stable/11/release/arm/CUBIEBOARD2.conf Fri Aug 3 21:59:01 2018 (r337274) @@ -21,23 +21,14 @@ arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubieboard2" UBOOT_FILES="u-boot-sunxi-with-spl.bin" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=1k seek=8 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} ln ${UFSMOUNT}/boot/dtb/cubieboard2.dtb \ - ${UFSMOUNT}/boot/dtb/sun7i-a20-cubieboard2.dtb - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: stable/11/release/arm/CUBOX-HUMMINGBOARD.conf ============================================================================== --- stable/11/release/arm/CUBOX-HUMMINGBOARD.conf Fri Aug 3 20:04:06 2018 (r337273) +++ stable/11/release/arm/CUBOX-HUMMINGBOARD.conf Fri Aug 3 21:59:01 2018 (r337274) @@ -21,21 +21,14 @@ arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-cubox-hummingboard" UBOOT_FILES="u-boot.imx" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=512 seek=2 conv=sync - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: stable/11/release/arm/PANDABOARD.conf ============================================================================== --- stable/11/release/arm/PANDABOARD.conf Fri Aug 3 20:04:06 2018 (r337273) +++ stable/11/release/arm/PANDABOARD.conf Fri Aug 3 21:59:01 2018 (r337274) @@ -19,21 +19,14 @@ WORLD_FLAGS="${WORLD_FLAGS} UBLDR_LOADADDR=0x88000000" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-pandaboard" FATMOUNT="${DESTDIR%${KERNEL}}/fat" - UFSMOUNT="${DESTDIR%${KERNEL}}/ufs" - chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" "${UFSMOUNT}" + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} - chroot ${CHROOTDIR} mount /dev/${mddev}s2a ${UFSMOUNT} chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/MLO ${FATMOUNT}/MLO chroot ${CHROOTDIR} cp -p ${UBOOT_DIR}/u-boot.img ${FATMOUNT}/u-boot.img chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr - chroot ${CHROOTDIR} cp -p ${UFSMOUNT}/boot/ubldr.bin \ - ${FATMOUNT}/ubldr.bin - chroot ${CHROOTDIR} touch ${UFSMOUNT}/firstboot sync umount_loop ${CHROOTDIR}/${FATMOUNT} - umount_loop ${CHROOTDIR}/${UFSMOUNT} chroot ${CHROOTDIR} rmdir ${FATMOUNT} - chroot ${CHROOTDIR} rmdir ${UFSMOUNT} - + return 0 } Modified: stable/11/release/arm/WANDBOARD.conf ============================================================================== --- stable/11/release/arm/WANDBOARD.conf Fri Aug 3 20:04:06 2018 (r337273) +++ stable/11/release/arm/WANDBOARD.conf Fri Aug 3 21:59:01 2018 (r337274) @@ -20,8 +20,15 @@ export BOARDNAME="WANDBOARD" arm_install_uboot() { UBOOT_DIR="/usr/local/share/u-boot/u-boot-wandboard" UBOOT_FILES="u-boot.imx" + FATMOUNT="${DESTDIR%${KERNEL}}/fat" chroot ${CHROOTDIR} dd if=${UBOOT_DIR}/${UBOOT_FILES} \ of=/dev/${mddev} bs=512 seek=2 conv=sync - + chroot ${CHROOTDIR} mkdir -p "${FATMOUNT}" + chroot ${CHROOTDIR} mount_msdosfs /dev/${mddev}s1 ${FATMOUNT} + chroot ${CHROOTDIR} cp ${UBOOT_DIR}/boot.scr ${FATMOUNT}/boot.scr + sync + umount_loop ${CHROOTDIR}/${FATMOUNT} + chroot ${CHROOTDIR} rmdir ${FATMOUNT} + return 0 } From owner-svn-src-all@freebsd.org Fri Aug 3 22:04:01 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55FF61052B6B; Fri, 3 Aug 2018 22:04:01 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0B170775FA; Fri, 3 Aug 2018 22:04:01 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E1D8726495; Fri, 3 Aug 2018 22:04:00 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73M40sI014576; Fri, 3 Aug 2018 22:04:00 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73M40RW014575; Fri, 3 Aug 2018 22:04:00 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201808032204.w73M40RW014575@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Fri, 3 Aug 2018 22:04:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337275 - head/sys/modules/dtb/am335x X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/sys/modules/dtb/am335x X-SVN-Commit-Revision: 337275 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 22:04:01 -0000 Author: manu Date: Fri Aug 3 22:04:00 2018 New Revision: 337275 URL: https://svnweb.freebsd.org/changeset/base/337275 Log: dtb: am335x: Remove links and add more dts The links were to cope with the switch to upstream dts. We don't need them anymore. While here add the rest of the beaglebone family dts as u-boot is common on all those boards and load the dtb based on the product name. This just miss the pocketbeagle variant as it's not yet in sys/gnu/dts but will be with the Linux 4.18 dts import. Modified: head/sys/modules/dtb/am335x/Makefile Modified: head/sys/modules/dtb/am335x/Makefile ============================================================================== --- head/sys/modules/dtb/am335x/Makefile Fri Aug 3 21:59:01 2018 (r337274) +++ head/sys/modules/dtb/am335x/Makefile Fri Aug 3 22:04:00 2018 (r337275) @@ -3,11 +3,10 @@ DTS= \ am335x-bone.dts \ am335x-boneblack.dts \ + am335x-boneblack-wireless.dts \ am335x-bonegreen.dts \ + am335x-bonegreen-wireless.dts \ + am335x-boneblue.dts \ ufw.dts - -LINKS= \ - ${DTBDIR}/am335x-bone.dtb ${DTBDIR}/beaglebone.dtb \ - ${DTBDIR}/am335x-boneblack.dtb ${DTBDIR}/beaglebone-black.dtb .include From owner-svn-src-all@freebsd.org Fri Aug 3 22:06:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 464291052C55; Fri, 3 Aug 2018 22:06:16 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED6D2777B0; Fri, 3 Aug 2018 22:06:15 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CE0EA26497; Fri, 3 Aug 2018 22:06:15 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73M6F1t014718; Fri, 3 Aug 2018 22:06:15 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73M6FSh014717; Fri, 3 Aug 2018 22:06:15 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201808032206.w73M6FSh014717@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Fri, 3 Aug 2018 22:06:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337276 - head/sys/modules/dtb/rpi X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/sys/modules/dtb/rpi X-SVN-Commit-Revision: 337276 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 22:06:16 -0000 Author: manu Date: Fri Aug 3 22:06:15 2018 New Revision: 337276 URL: https://svnweb.freebsd.org/changeset/base/337276 Log: dtb: rpi: Only compile and copy the DTSO The DTB is now loaded via the firmware, passed to u-boot then to loader.efi Only compile and copy the dts overlays. Modified: head/sys/modules/dtb/rpi/Makefile Modified: head/sys/modules/dtb/rpi/Makefile ============================================================================== --- head/sys/modules/dtb/rpi/Makefile Fri Aug 3 22:04:00 2018 (r337275) +++ head/sys/modules/dtb/rpi/Makefile Fri Aug 3 22:06:15 2018 (r337276) @@ -1,16 +1,7 @@ # $FreeBSD$ # DTS files for the Raspberry Pi-B -DTS=rpi.dts rpi2.dts - DTSO= \ spigen-rpi-b.dtso \ spigen-rpi2.dtso \ - -LINKS= \ - ${DTBDIR}/rpi.dtb ${DTBDIR}/bcm2835-rpi-b.dtb \ - ${DTBDIR}/rpi.dtb ${DTBDIR}/bcm2835-rpi-b-rev2.dtb \ - ${DTBDIR}/rpi.dtb ${DTBDIR}/bcm2835-rpi-b-plus.dtb \ - ${DTBDIR}/rpi.dtb ${DTBDIR}/bcm2835-rpi-zero.dtb \ - ${DTBDIR}/rpi2.dtb ${DTBDIR}/bcm2836-rpi-2-b.dtb .include From owner-svn-src-all@freebsd.org Fri Aug 3 22:15:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34705105303E; Fri, 3 Aug 2018 22:15:59 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCBAB77D4C; Fri, 3 Aug 2018 22:15:58 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9F6212663E; Fri, 3 Aug 2018 22:15:58 +0000 (UTC) (envelope-from manu@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73MFwOo019814; Fri, 3 Aug 2018 22:15:58 GMT (envelope-from manu@FreeBSD.org) Received: (from manu@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73MFw6R019813; Fri, 3 Aug 2018 22:15:58 GMT (envelope-from manu@FreeBSD.org) Message-Id: <201808032215.w73MFw6R019813@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: manu set sender to manu@FreeBSD.org using -f From: Emmanuel Vadot Date: Fri, 3 Aug 2018 22:15:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337277 - head/sys/arm/conf X-SVN-Group: head X-SVN-Commit-Author: manu X-SVN-Commit-Paths: head/sys/arm/conf X-SVN-Commit-Revision: 337277 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 22:15:59 -0000 Author: manu Date: Fri Aug 3 22:15:58 2018 New Revision: 337277 URL: https://svnweb.freebsd.org/changeset/base/337277 Log: arm: Remove ALLWINNER_UP kernel config This was needed when we GENERIC couldn't boot on UP system. Deleted: head/sys/arm/conf/ALLWINNER_UP From owner-svn-src-all@freebsd.org Fri Aug 3 22:27:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABAF1105363C; Fri, 3 Aug 2018 22:27:41 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [198.45.61.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E768784B9; Fri, 3 Aug 2018 22:27:41 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id w73MRdNR062807 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 3 Aug 2018 15:27:39 -0700 (PDT) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id w73MRdTT062806; Fri, 3 Aug 2018 15:27:39 -0700 (PDT) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Fri, 3 Aug 2018 15:27:39 -0700 From: Gleb Smirnoff To: Konstantin Belousov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336047 - in head/sys: amd64/include i386/include sys Message-ID: <20180803222739.GH420@FreeBSD.org> References: <201807061950.w66Joibw081390@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201807061950.w66Joibw081390@repo.freebsd.org> User-Agent: Mutt/1.10.0 (2018-05-17) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 22:27:41 -0000 Hi Konstantin, On Fri, Jul 06, 2018 at 07:50:44PM +0000, Konstantin Belousov wrote: K> Author: kib K> Date: Fri Jul 6 19:50:44 2018 K> New Revision: 336047 K> URL: https://svnweb.freebsd.org/changeset/base/336047 K> K> Log: K> Expand x86 struct pcpus to UMA_PCPU_ALLOC_SIZE AKA PAGE_SIZE. K> K> This restores counters(9) operation. K> Revert r336024. Improve assert of pcpu size on x86. K> K> Reviewed by: mmacy K> Sponsored by: The FreeBSD Foundation K> Differential revision: https://reviews.freebsd.org/D16163 Since this has been broken already several times as people who edit surrounding code don't understand the magic, and since now we round the struct pcpu size to PAGE_SIZE anyways, what about stop carefully craft the padding and just change alignment of struct pcpu to PAGE_SIZE? -- Gleb Smirnoff From owner-svn-src-all@freebsd.org Fri Aug 3 22:30:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16ECE1053732; Fri, 3 Aug 2018 22:30:32 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [198.45.61.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 99A9478645; Fri, 3 Aug 2018 22:30:31 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id w73MEfCS032258 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 3 Aug 2018 15:14:41 -0700 (PDT) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id w73MEf9d032254; Fri, 3 Aug 2018 15:14:41 -0700 (PDT) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Fri, 3 Aug 2018 15:14:41 -0700 From: Gleb Smirnoff To: Brooks Davis Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r335979 - in head: . lib/libkvm sys/kern sys/netinet sys/sys usr.bin/netstat usr.bin/sockstat Message-ID: <20180803221441.GG420@FreeBSD.org> References: <201807051313.w65DDnQJ041281@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201807051313.w65DDnQJ041281@repo.freebsd.org> User-Agent: Mutt/1.10.0 (2018-05-17) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 22:30:32 -0000 Hi Brooks, On Thu, Jul 05, 2018 at 01:13:49PM +0000, Brooks Davis wrote: B> Author: brooks B> Date: Thu Jul 5 13:13:48 2018 B> New Revision: 335979 B> URL: https://svnweb.freebsd.org/changeset/base/335979 B> B> Log: B> Make struct xinpcb and friends word-size independent. B> B> Replace size_t members with ksize_t (uint64_t) and pointer members B> (never used as pointers in userspace, but instead as unique B> idenitifiers) with kvaddr_t (uint64_t). This makes the structs B> identical between 32-bit and 64-bit ABIs. B> B> On 64-bit bit systems, the ABI is maintained. On 32-bit systems, B> this is an ABI breaking change. The ABI of most of these structs B> was previously broken in r315662. This also imposes a small API B> change on userspace consumers who must handle kernel pointers B> becoming virtual addresses. Thanks for this change. After it the anonymous unions in xinpcb, unpxb are no longer needed. Do you agree? I added them to make sizes on 32-bit and 64-bit ABIs to be identical. I prefer your new code, but didn't the unions work? Anyway, now they can be removed. -- Gleb Smirnoff From owner-svn-src-all@freebsd.org Fri Aug 3 22:55:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8552610544FC; Fri, 3 Aug 2018 22:55:59 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BB3679B1C; Fri, 3 Aug 2018 22:55:59 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0A1DF26C9B; Fri, 3 Aug 2018 22:55:59 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w73MtwJ0041056; Fri, 3 Aug 2018 22:55:58 GMT (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w73MtwWb041054; Fri, 3 Aug 2018 22:55:58 GMT (envelope-from glebius@FreeBSD.org) Message-Id: <201808032255.w73MtwWb041054@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: glebius set sender to glebius@FreeBSD.org using -f From: Gleb Smirnoff Date: Fri, 3 Aug 2018 22:55:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337278 - head/sys/netgraph X-SVN-Group: head X-SVN-Commit-Author: glebius X-SVN-Commit-Paths: head/sys/netgraph X-SVN-Commit-Revision: 337278 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 22:55:59 -0000 Author: glebius Date: Fri Aug 3 22:55:58 2018 New Revision: 337278 URL: https://svnweb.freebsd.org/changeset/base/337278 Log: Use if_tunnel_check_nesting() for ng_iface(4). Modified: head/sys/netgraph/ng_iface.c head/sys/netgraph/ng_iface.h Modified: head/sys/netgraph/ng_iface.c ============================================================================== --- head/sys/netgraph/ng_iface.c Fri Aug 3 22:15:58 2018 (r337277) +++ head/sys/netgraph/ng_iface.c Fri Aug 3 22:55:58 2018 (r337278) @@ -344,7 +344,6 @@ static int ng_iface_output(struct ifnet *ifp, struct mbuf *m, const struct sockaddr *dst, struct route *ro) { - struct m_tag *mtag; uint32_t af; int error; @@ -356,22 +355,11 @@ ng_iface_output(struct ifnet *ifp, struct mbuf *m, } /* Protect from deadly infinite recursion. */ - mtag = NULL; - while ((mtag = m_tag_locate(m, MTAG_NGIF, MTAG_NGIF_CALLED, mtag))) { - if (*(struct ifnet **)(mtag + 1) == ifp) { - log(LOG_NOTICE, "Loop detected on %s\n", ifp->if_xname); - m_freem(m); - return (EDEADLK); - } - } - mtag = m_tag_alloc(MTAG_NGIF, MTAG_NGIF_CALLED, sizeof(struct ifnet *), - M_NOWAIT); - if (mtag == NULL) { + error = if_tunnel_check_nesting(ifp, m, NGM_IFACE_COOKIE, 1); + if (error) { m_freem(m); - return (ENOMEM); + return (error); } - *(struct ifnet **)(mtag + 1) = ifp; - m_tag_prepend(m, mtag); /* BPF writes need to be handled specially. */ if (dst->sa_family == AF_UNSPEC) Modified: head/sys/netgraph/ng_iface.h ============================================================================== --- head/sys/netgraph/ng_iface.h Fri Aug 3 22:15:58 2018 (r337277) +++ head/sys/netgraph/ng_iface.h Fri Aug 3 22:55:58 2018 (r337278) @@ -70,7 +70,4 @@ enum { NGM_IFACE_GET_IFINDEX, }; -#define MTAG_NGIF NGM_IFACE_COOKIE -#define MTAG_NGIF_CALLED 0 | MTAG_PERSISTENT - #endif /* _NETGRAPH_NG_IFACE_H_ */ From owner-svn-src-all@freebsd.org Fri Aug 3 23:04:11 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7BF271054787; Fri, 3 Aug 2018 23:04:11 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [198.45.61.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 07C5379F9C; Fri, 3 Aug 2018 23:04:10 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id w73N46XJ066562 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 3 Aug 2018 16:04:06 -0700 (PDT) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id w73N45q9066561; Fri, 3 Aug 2018 16:04:05 -0700 (PDT) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Fri, 3 Aug 2018 16:04:05 -0700 From: Gleb Smirnoff To: Kristof Provost Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336221 - head/sys/net Message-ID: <20180803230405.GI420@FreeBSD.org> References: <201807121635.w6CGZZAN046919@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201807121635.w6CGZZAN046919@repo.freebsd.org> User-Agent: Mutt/1.10.0 (2018-05-17) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Aug 2018 23:04:11 -0000 Kristof, On Thu, Jul 12, 2018 at 04:35:35PM +0000, Kristof Provost wrote: K> Author: kp K> Date: Thu Jul 12 16:35:35 2018 K> New Revision: 336221 K> URL: https://svnweb.freebsd.org/changeset/base/336221 K> K> Log: K> pf: Increate default state table size K> K> The typical system now has a lot more memory than when pf was new, and is also K> expected to handle more connections. Increase the default size of the state K> table. K> Note that users can overrule this using 'set limit states' in pf.conf. K> K> From OpenBSD: K> The year is 2018. K> Mercury, Bowie, Cash, Motorola and DEC all left us. K> Just pf still has a default state table limit of 10000. K> Had! Now it's a tiny little bit more, 100k. K> lead guitar: me K> ok chorus: phessler theo claudio benno K> background school girl laughing: bob For FreeBSD it would also make sense to bump hash sizes along with this change. -- Gleb Smirnoff From owner-svn-src-all@freebsd.org Sat Aug 4 00:03:23 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B1931055AEE; Sat, 4 Aug 2018 00:03:23 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A88757B738; Sat, 4 Aug 2018 00:03:22 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 83E86277E4; Sat, 4 Aug 2018 00:03:22 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w7403Mek076446; Sat, 4 Aug 2018 00:03:22 GMT (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w7403M1m076445; Sat, 4 Aug 2018 00:03:22 GMT (envelope-from glebius@FreeBSD.org) Message-Id: <201808040003.w7403M1m076445@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: glebius set sender to glebius@FreeBSD.org using -f From: Gleb Smirnoff Date: Sat, 4 Aug 2018 00:03:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337279 - in head/sys: netinet sys X-SVN-Group: head X-SVN-Commit-Author: glebius X-SVN-Commit-Paths: in head/sys: netinet sys X-SVN-Commit-Revision: 337279 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 00:03:23 -0000 Author: glebius Date: Sat Aug 4 00:03:21 2018 New Revision: 337279 URL: https://svnweb.freebsd.org/changeset/base/337279 Log: Now that after r335979 the kernel addresses in API structures are fixed size, there is no reason left for the unions. Discussed with: brooks Modified: head/sys/netinet/in_pcb.h head/sys/sys/socketvar.h Modified: head/sys/netinet/in_pcb.h ============================================================================== --- head/sys/netinet/in_pcb.h Fri Aug 3 22:55:58 2018 (r337278) +++ head/sys/netinet/in_pcb.h Sat Aug 4 00:03:21 2018 (r337279) @@ -370,10 +370,7 @@ struct xinpcb { struct xsocket xi_socket; /* (s,p) */ struct in_conninfo inp_inc; /* (s,p) */ uint64_t inp_gencnt; /* (s,p) */ - union { - kvaddr_t inp_ppcb; /* (s) netstat(1) */ - int64_t ph_ppcb; - }; + kvaddr_t inp_ppcb; /* (s) netstat(1) */ int64_t inp_spare64[4]; uint32_t inp_flow; /* (s) */ uint32_t inp_flowid; /* (s) */ Modified: head/sys/sys/socketvar.h ============================================================================== --- head/sys/sys/socketvar.h Fri Aug 3 22:55:58 2018 (r337278) +++ head/sys/sys/socketvar.h Sat Aug 4 00:03:21 2018 (r337279) @@ -475,14 +475,8 @@ int accept_filt_generic_mod_event(module_t mod, int ev */ struct xsocket { ksize_t xso_len; /* length of this structure */ - union { - kvaddr_t xso_so; /* kernel address of struct socket */ - int64_t ph_so; - }; - union { - kvaddr_t so_pcb; /* kernel address of struct inpcb */ - int64_t ph_pcb; - }; + kvaddr_t xso_so; /* kernel address of struct socket */ + kvaddr_t so_pcb; /* kernel address of struct inpcb */ uint64_t so_oobmark; int64_t so_spare64[8]; int32_t xso_protocol; From owner-svn-src-all@freebsd.org Sat Aug 4 00:12:36 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23FAB1055D6A; Sat, 4 Aug 2018 00:12:36 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BDDC87BBB4; Sat, 4 Aug 2018 00:12:35 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 0232210AFD2; Fri, 3 Aug 2018 20:12:33 -0400 (EDT) Subject: Re: svn commit: r336892 - head/sys/arm/arm To: Andrew Turner , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201807300950.w6U9oQtq034833@repo.freebsd.org> From: John Baldwin Message-ID: <56f85a74-0ca7-86f2-df11-a4ad177935cc@FreeBSD.org> Date: Fri, 3 Aug 2018 17:12:33 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <201807300950.w6U9oQtq034833@repo.freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Fri, 03 Aug 2018 20:12:34 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 00:12:36 -0000 On 7/30/18 2:50 AM, Andrew Turner wrote: > Author: andrew > Date: Mon Jul 30 09:50:26 2018 > New Revision: 336892 > URL: https://svnweb.freebsd.org/changeset/base/336892 > > Log: > Require ARMv5 for arm. All current kernels are for ARMv5 or later, and it > will allow us to clean out old ARMv4 (and earlier) specific assembly. > > Relnotes: yes FYI, arm's LINT config hasn't built since this commit breaking make tinderbox. -- John Baldwin From owner-svn-src-all@freebsd.org Sat Aug 4 00:34:17 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E4A210563C5; Sat, 4 Aug 2018 00:34:17 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B312E7C5E7; Sat, 4 Aug 2018 00:34:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7136627CC5; Sat, 4 Aug 2018 00:34:16 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w740YGiV092120; Sat, 4 Aug 2018 00:34:16 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w740YGEM092119; Sat, 4 Aug 2018 00:34:16 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201808040034.w740YGEM092119@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Sat, 4 Aug 2018 00:34:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337280 - stable/11/sys/cam X-SVN-Group: stable-11 X-SVN-Commit-Author: mav X-SVN-Commit-Paths: stable/11/sys/cam X-SVN-Commit-Revision: 337280 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 00:34:17 -0000 Author: mav Date: Sat Aug 4 00:34:15 2018 New Revision: 337280 URL: https://svnweb.freebsd.org/changeset/base/337280 Log: MFC r336590: Stop further SCSI recovery attempts after one has failed. We've got a set of probably damaged hard disks, reporting 0x04,0x02 ("Logical unit not ready, initializing command required") in response to READ CAPACITY(16), where attempts to use START STOP UNIT for recovery results in 0x44,0x00 ("Internal target failure") after ~1 second delay. As result of all recovery retries, device open attempt took ~3 seconds before finally reporting to GEOM that device is opened, but has no media. If the open was for writing and since it hasn't formally failed, following close triggered GEOM retaste, opening device few more times with respective delays. This change reduces whole time of this cycle from ~12 seconds to ~3 by giving up on recovery after the first failure. Modified: stable/11/sys/cam/cam_periph.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/cam/cam_periph.c ============================================================================== --- stable/11/sys/cam/cam_periph.c Sat Aug 4 00:03:21 2018 (r337279) +++ stable/11/sys/cam/cam_periph.c Sat Aug 4 00:34:15 2018 (r337280) @@ -1244,7 +1244,7 @@ camperiphdone(struct cam_periph *periph, union ccb *do union ccb *saved_ccb; cam_status status; struct scsi_start_stop_unit *scsi_cmd; - int error_code, sense_key, asc, ascq; + int error = 0, error_code, sense_key, asc, ascq; scsi_cmd = (struct scsi_start_stop_unit *) &done_ccb->csio.cdb_io.cdb_bytes; @@ -1276,8 +1276,9 @@ camperiphdone(struct cam_periph *periph, union ccb *do goto out; } } - if (cam_periph_error(done_ccb, - 0, SF_RETRY_UA | SF_NO_PRINT, NULL) == ERESTART) + error = cam_periph_error(done_ccb, 0, + SF_RETRY_UA | SF_NO_PRINT, NULL); + if (error == ERESTART) goto out; if (done_ccb->ccb_h.status & CAM_DEV_QFRZN) { cam_release_devq(done_ccb->ccb_h.path, 0, 0, 0, 0); @@ -1296,14 +1297,21 @@ camperiphdone(struct cam_periph *periph, union ccb *do } /* - * Perform the final retry with the original CCB so that final - * error processing is performed by the owner of the CCB. + * After recovery action(s) completed, return to the original CCB. + * If the recovery CCB has failed, considering its own possible + * retries and recovery, assume we are back in state where we have + * been originally, but without recovery hopes left. In such case, + * after the final attempt below, we cancel any further retries, + * blocking by that also any new recovery attempts for this CCB, + * and the result will be the final one returned to the CCB owher. */ saved_ccb = (union ccb *)done_ccb->ccb_h.saved_ccb_ptr; bcopy(saved_ccb, done_ccb, sizeof(*done_ccb)); xpt_free_ccb(saved_ccb); if (done_ccb->ccb_h.cbfcnp != camperiphdone) periph->flags &= ~CAM_PERIPH_RECOVERY_INPROG; + if (error != 0) + done_ccb->ccb_h.retry_count = 0; xpt_action(done_ccb); out: From owner-svn-src-all@freebsd.org Sat Aug 4 01:45:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0970A10589CD; Sat, 4 Aug 2018 01:45:18 +0000 (UTC) (envelope-from pkelsey@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB0F07F1EB; Sat, 4 Aug 2018 01:45:17 +0000 (UTC) (envelope-from pkelsey@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 89ED5986; Sat, 4 Aug 2018 01:45:17 +0000 (UTC) (envelope-from pkelsey@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w741jHwx027791; Sat, 4 Aug 2018 01:45:17 GMT (envelope-from pkelsey@FreeBSD.org) Received: (from pkelsey@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w741jHLC027790; Sat, 4 Aug 2018 01:45:17 GMT (envelope-from pkelsey@FreeBSD.org) Message-Id: <201808040145.w741jHLC027790@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: pkelsey set sender to pkelsey@FreeBSD.org using -f From: Patrick Kelsey Date: Sat, 4 Aug 2018 01:45:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337281 - head/sys/net X-SVN-Group: head X-SVN-Commit-Author: pkelsey X-SVN-Commit-Paths: head/sys/net X-SVN-Commit-Revision: 337281 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 01:45:18 -0000 Author: pkelsey Date: Sat Aug 4 01:45:17 2018 New Revision: 337281 URL: https://svnweb.freebsd.org/changeset/base/337281 Log: Mark the send queue ready so ALTQ is available. Modified: head/sys/net/iflib.c Modified: head/sys/net/iflib.c ============================================================================== --- head/sys/net/iflib.c Sat Aug 4 00:34:15 2018 (r337280) +++ head/sys/net/iflib.c Sat Aug 4 01:45:17 2018 (r337281) @@ -5234,6 +5234,7 @@ iflib_register(if_ctx_t ctx) #ifdef ALTQ if_setstartfn(ifp, iflib_altq_if_start); if_settransmitfn(ifp, iflib_altq_if_transmit); + if_setsendqready(ifp); #else if_settransmitfn(ifp, iflib_if_transmit); #endif From owner-svn-src-all@freebsd.org Sat Aug 4 02:30:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D139105AC0B; Sat, 4 Aug 2018 02:30:52 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 43F7080C90; Sat, 4 Aug 2018 02:30:52 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 252F3106C; Sat, 4 Aug 2018 02:30:52 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w742UqYv048547; Sat, 4 Aug 2018 02:30:52 GMT (envelope-from alc@FreeBSD.org) Received: (from alc@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w742UpiR048545; Sat, 4 Aug 2018 02:30:51 GMT (envelope-from alc@FreeBSD.org) Message-Id: <201808040230.w742UpiR048545@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: alc set sender to alc@FreeBSD.org using -f From: Alan Cox Date: Sat, 4 Aug 2018 02:30:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337282 - head/contrib/llvm/tools/lld/ELF/Arch X-SVN-Group: head X-SVN-Commit-Author: alc X-SVN-Commit-Paths: head/contrib/llvm/tools/lld/ELF/Arch X-SVN-Commit-Revision: 337282 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 02:30:52 -0000 Author: alc Date: Sat Aug 4 02:30:51 2018 New Revision: 337282 URL: https://svnweb.freebsd.org/changeset/base/337282 Log: Set the default image base on arm64 and i386 to a superpage-aligned address. Reviewed by: emaste, markj Discussed with: dim Differential Revision: https://reviews.freebsd.org/D16385 Modified: head/contrib/llvm/tools/lld/ELF/Arch/AArch64.cpp head/contrib/llvm/tools/lld/ELF/Arch/X86.cpp Modified: head/contrib/llvm/tools/lld/ELF/Arch/AArch64.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Arch/AArch64.cpp Sat Aug 4 01:45:17 2018 (r337281) +++ head/contrib/llvm/tools/lld/ELF/Arch/AArch64.cpp Sat Aug 4 02:30:51 2018 (r337282) @@ -66,6 +66,10 @@ AArch64::AArch64() { PltHeaderSize = 32; DefaultMaxPageSize = 65536; + // Align to the 2 MiB page size (known as a superpage or huge page). + // FreeBSD automatically promotes 2 MiB-aligned allocations. + DefaultImageBase = 0x200000; + // It doesn't seem to be documented anywhere, but tls on aarch64 uses variant // 1 of the tls structures and the tcb size is 16. TcbSize = 16; Modified: head/contrib/llvm/tools/lld/ELF/Arch/X86.cpp ============================================================================== --- head/contrib/llvm/tools/lld/ELF/Arch/X86.cpp Sat Aug 4 01:45:17 2018 (r337281) +++ head/contrib/llvm/tools/lld/ELF/Arch/X86.cpp Sat Aug 4 02:30:51 2018 (r337282) @@ -61,6 +61,10 @@ X86::X86() { PltHeaderSize = 16; TlsGdRelaxSkip = 2; TrapInstr = 0xcccccccc; // 0xcc = INT3 + + // Align to the non-PAE large page size (known as a superpage or huge page). + // FreeBSD automatically promotes large, superpage-aligned allocations. + DefaultImageBase = 0x400000; } static bool hasBaseReg(uint8_t ModRM) { return (ModRM & 0xc7) != 0x5; } From owner-svn-src-all@freebsd.org Sat Aug 4 06:40:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4309105FEF7; Sat, 4 Aug 2018 06:40:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 639A9883A2; Sat, 4 Aug 2018 06:40:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 40AB73E2E; Sat, 4 Aug 2018 06:40:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w746eJu0076446; Sat, 4 Aug 2018 06:40:19 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w746eJw4076445; Sat, 4 Aug 2018 06:40:19 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201808040640.w746eJw4076445@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Sat, 4 Aug 2018 06:40:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337285 - head/stand/efi/loader X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: head/stand/efi/loader X-SVN-Commit-Revision: 337285 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 06:40:19 -0000 Author: kevans Date: Sat Aug 4 06:40:18 2018 New Revision: 337285 URL: https://svnweb.freebsd.org/changeset/base/337285 Log: efi-autoresizecons: Don't fail the boot w/o GOP or UGA efi-autoresizecons is currently executed for every boot. If it fails, we risk failing the boot, and we really shouldn't do that unless we absolutely must. Not being able to locate GOP or UGA is not a significant enough failure to kill the boot. We always have the option to fall back to resizing ConOut to a higher text mode resolution (if available), so do that. This was detected by Doug [1] while attempting a bhyve + UEFI + PXE boot. This patch was effectively also submitted by Doug, but I expanded the comment he had originally sent me a little bit to indicate why this is an OK idea. Reported by: Doug Ambrisko [1] Modified: head/stand/efi/loader/framebuffer.c Modified: head/stand/efi/loader/framebuffer.c ============================================================================== --- head/stand/efi/loader/framebuffer.c Sat Aug 4 06:29:46 2018 (r337284) +++ head/stand/efi/loader/framebuffer.c Sat Aug 4 06:40:18 2018 (r337285) @@ -648,7 +648,14 @@ command_autoresize(int argc, char *argv[]) snprintf(command_errbuf, sizeof(command_errbuf), "%s: Neither Graphics Output Protocol nor Universal Graphics Adapter present", argv[0]); - return (CMD_ERROR); + + /* + * Default to text_autoresize if we have neither GOP or UGA. This won't + * give us the most ideal resolution, but it will at least leave us + * functional rather than failing the boot for an objectively bad + * reason. + */ + return (text_autoresize()); } COMMAND_SET(gop, "gop", "graphics output protocol", command_gop); From owner-svn-src-all@freebsd.org Sat Aug 4 07:48:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D4631061413; Sat, 4 Aug 2018 07:48:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C5F6189DC4; Sat, 4 Aug 2018 07:48:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w747mHkm059067 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 4 Aug 2018 10:48:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w747mHkm059067 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w747mHc5059066; Sat, 4 Aug 2018 10:48:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 4 Aug 2018 10:48:17 +0300 From: Konstantin Belousov To: Gleb Smirnoff Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336047 - in head/sys: amd64/include i386/include sys Message-ID: <20180804074817.GH6049@kib.kiev.ua> References: <201807061950.w66Joibw081390@repo.freebsd.org> <20180803222739.GH420@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180803222739.GH420@FreeBSD.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 07:48:29 -0000 On Fri, Aug 03, 2018 at 03:27:39PM -0700, Gleb Smirnoff wrote: > Hi Konstantin, > > On Fri, Jul 06, 2018 at 07:50:44PM +0000, Konstantin Belousov wrote: > K> Author: kib > K> Date: Fri Jul 6 19:50:44 2018 > K> New Revision: 336047 > K> URL: https://svnweb.freebsd.org/changeset/base/336047 > K> > K> Log: > K> Expand x86 struct pcpus to UMA_PCPU_ALLOC_SIZE AKA PAGE_SIZE. > K> > K> This restores counters(9) operation. > K> Revert r336024. Improve assert of pcpu size on x86. > K> > K> Reviewed by: mmacy > K> Sponsored by: The FreeBSD Foundation > K> Differential revision: https://reviews.freebsd.org/D16163 > > Since this has been broken already several times as people who edit > surrounding code don't understand the magic, and since now we round > the struct pcpu size to PAGE_SIZE anyways, what about stop carefully > craft the padding and just change alignment of struct pcpu to PAGE_SIZE? I do not see how it would avoided that breakage. Also, I like the explicitness in the padding, it is useful when new pcpu members are added. I have to do that more than once this year. From owner-svn-src-all@freebsd.org Sat Aug 4 08:08:52 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1C1F1061E7A; Sat, 4 Aug 2018 08:08:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D6078A9EA; Sat, 4 Aug 2018 08:08:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w7488eKj063566 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 4 Aug 2018 11:08:44 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w7488eKj063566 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w7488eO0063565; Sat, 4 Aug 2018 11:08:40 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 4 Aug 2018 11:08:40 +0300 From: Konstantin Belousov To: Justin Hibbits Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r337273 - head/sys/dev/nvme Message-ID: <20180804080840.GI6049@kib.kiev.ua> References: <201808032004.w73K46XJ053249@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201808032004.w73K46XJ053249@repo.freebsd.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 08:08:52 -0000 On Fri, Aug 03, 2018 at 08:04:06PM +0000, Justin Hibbits wrote: > Author: jhibbits > Date: Fri Aug 3 20:04:06 2018 > New Revision: 337273 > URL: https://svnweb.freebsd.org/changeset/base/337273 > > Log: > nvme(4): Add bus_dmamap_sync() at the end of the request path > > Summary: > Some architectures, in this case powerpc64, need explicit synchronization > barriers vs device accesses. > > Prior to this change, when running 'make buildworld -j72' on a 18-core > (72-thread) POWER9, I would see controller resets often. With this change, I > don't see these resets messages, though another tester still does, for yet to be > determined reasons, so this may not be a complete fix. Additionally, I see a > ~5-10% speed up in buildworld times, likely due to not needing to reset the > controller. > > Reviewed By: jimharris > Differential Revision: https://reviews.freebsd.org/D16570 > > Modified: > head/sys/dev/nvme/nvme_qpair.c > > Modified: head/sys/dev/nvme/nvme_qpair.c > ============================================================================== > --- head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 19:24:04 2018 (r337272) > +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 20:04:06 2018 (r337273) > @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct nvme_qpair *qpair, > req->retries++; > nvme_qpair_submit_tracker(qpair, tr); > } else { > - if (req->type != NVME_REQUEST_NULL) > + if (req->type != NVME_REQUEST_NULL) { > + bus_dmamap_sync(qpair->dma_tag_payload, > + tr->payload_dma_map, > + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > bus_dmamap_unload(qpair->dma_tag_payload, > tr->payload_dma_map); > + } > > nvme_free_request(req); > tr->req = NULL; > @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct nvme_qpair *qpai > */ > return (false); > > + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > while (1) { > cpl = qpair->cpl[qpair->cq_head]; > > @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair *qpair, st > if (++qpair->sq_tail == qpair->num_entries) > qpair->sq_tail = 0; > > + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); > +#ifndef __powerpc__ > + /* > + * powerpc's bus_dmamap_sync() already includes a heavyweight sync, but > + * no other archs do. > + */ > wmb(); > +#endif What is the purpose of this call ? It is useless without paired read barrier. So where is the reciprocal rmb() ? This call is almost certainly wrong on x86. > + > nvme_mmio_write_4(qpair->ctrlr, doorbell[qpair->id].sq_tdbl, > qpair->sq_tail); > > @@ -879,6 +894,8 @@ nvme_payload_map(void *arg, bus_dma_segment_t *seg, in > tr->req->cmd.prp2 = 0; > } > > + bus_dmamap_sync(tr->qpair->dma_tag_payload, tr->payload_dma_map, > + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); > nvme_qpair_submit_tracker(tr->qpair, tr); > } > From owner-svn-src-all@freebsd.org Sat Aug 4 08:54:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D596010633CB; Sat, 4 Aug 2018 08:54:26 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "anubis.delphij.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31CDB8C277; Sat, 4 Aug 2018 08:54:26 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from odin.corp.delphij.net (unknown [IPv6:2601:646:8882:37a:3dd6:e226:c80f:febd]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id 1CD8F2A5F5; Sat, 4 Aug 2018 01:54:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=m7e2; t=1533372865; x=1533387265; bh=ZbwStxn5JA4TQUdoOyzI8pedb+YjMcY1K2vNzaT9yHo=; h=Reply-To:To:References:From:Subject:Date:In-Reply-To; b=GA7ul4sF+wPmU4YBoWmWVf+p3Kn7HnjRSHxdZoCVl5ofMsLLs8xRoxlpdpkXGyOk3 VGrmox/cqs4YxmmbqDFhsgVYl6Jf0mqeZbDzURV7xiisoeqL/8PeG3aOWho2qHG6gn syXTVpxS5SVCAIXzeELNfhm3AgMaAz9yOscPmg1AiJc88g+alhvOGQ6tPj1tKEkbB2 EOSbWxBmiSys2A993s1AX9Vj8WuRYRDF4Wbepb3+4tuNJfm2QmwrfFB/jgC3TShsyK 8btmgFDRp8IbYimMH+JjiV+9FPD9s6GvsfOEJieQFDRet/B1V5hcRvXkq4nj4qUVa4 iB5lshkl+10tA== Reply-To: d@delphij.net To: Cy Schubert , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, tsoome@freebsd.org References: <201808031911.w73JB0WK025164@repo.freebsd.org> From: Xin Li Openpgp: preference=signencrypt Autocrypt: addr=delphij@delphij.net; prefer-encrypt=mutual; keydata= xsFNBFJNzwQBEACuPNSJjL/AD8oHFuG72vtx5P7Q6dpiEbFABgw/IohS65yDZDd3qFH9ssQv AsFafwB/ofsk6t7dx6zIC05dv5qjhGIOKSJxFC4U1HAot9+QpeUG+8boTKZiiycrMruItj2U JANlv+gN5h0mAsL5f9eNzhRM43kdjN8cQnBIujhO54Derjnrnqz6cQtoonV6SvvVJZUQGxHK 5R1XYJ6wiTuvoEuRYnNObJmPFWZyYOaGZz0qqD6Qe1BhkZuRzv2bZxwJc3Raap/GF6Pm9J/c hlYHUmm2QLaXvmoP8WNosNjla1fup0tgYQE+7MTtHFVxmVj9ZTihN3rEL5IkeEKjQAqcpe1n Db8X2o4K262LRpFl8WtVMW2TfN5Avpj+knZMl3tkYGvYK/nfadCr6Af4co9mkhX6QYgkerg2 mXEGaQzSD/omnsxHCfqMgdphaX3B3eoY2Fv36BMpjSdHmm0rmwqjqZaqlZn89vQ/I6ATvLyx JsdHwTbrj57audl/RKC+OpREOJPaVULp1L+9zdBXslILO8MJaT6YEw1T29bEj5jvLm03Y4rF u/YTruHcMPpsGbpJckDKiy6ISAbMtPvz7/KR91xPHS6KExGiIakIX9xpIXIDKgq+ecEWwkFK PogoKqO6K0/GYkTRoKdXGzsILvIurtbPqSFqWzbRIyNOa82jowARAQABzRZYaW4gTGkgPGRA ZGVscGhpai5uZXQ+wsF9BBMBCgAnBQJTQvBFAhsjBQkJZgGABQsJCAcDBRUKCQgLBRYCAwEA Ah4BAheAAAoJEJW2GBstM+nsha4P/2Roa/REjZLZlIG1TKOxEDqmwc3fynX4w2g7/FXA7f7Z YO5N4vnnnQdJbDZDt4TJtiP1NHHdheQ5+loJrrCXVlU31LuJv1ebM2Ajsuo/0l3tfulEf6Ki GoozmaNZAhwiGJkQVg9DSKsea5xIA31lPnFH4T0SKn8Q6F4HYienmJJtlKVTADvYXA+DRmv0 rNOyVe+V/AuTFuelKg3Ua5a+dY3oqtrQQvFS4n7iIrNjEMUBVx0XTrYLddnF+YjXDg5Phf0D pV/2yJOXiTGiZMK6i7vwHZkJvarACoTSrUrr6OBuZv5Gf87VgifZKLr2Fuf+FePiVCoZTQiL 0hPQyABMzeWa32P6BY2LBMMMFvFiyL5pN5k6nJ0nx4skl8UxZ5ay4yyVg2u3f4aI3+m0XlZ+ iixrjmCTGi1s+d/n6E3eFXdJUUbSOXLZaU4qrbXRzTYCZmZViryv7ibtOHXnG6oWy7BFEHuT rUW6OBvsQDTp5iQ6opENJ5/ZzSA3c5p1WS9Ezv4Bpdqcm7LTQX2j6kXikj8YqICtDF2rkKZ2 Ynjm9se9B0h/T1SOaSpbtRg05UKjsinDq2x8EeX21yFs3UyvwePLrGoNKL45EJM0xwxrnlfr M0ayKJNLoYysY78d54hg7XMmkQD/oZz9I+k4fN6CmZ2i5WGH2BgYs0313JMHxSg7zsFNBFJN zwQBEADPtS+nfTKM6PwgSWLDGVgUYQ/RLaKzCcpQAf4ryLBugXpx3s2BBT1bixX7CpsLXKQi +RRETgSFzDaBL9SEs2ZDV2YT+zGp08aijK/Yl9+RIeezAukI3c+XMHuo8ktUWJmo5/1DX07q G30ckG7uFuTnt31sFzwhh/ZeSuLFyel/fWF48KExLDIVa8DyEUJaYvE9Vfph4T/3LkKuzVTy +iwUBLiSLj5G5N70A+4usbL3eKyYrJqCSaLfrP99/nlgBhMAHVcKcv0uqSuiaH9OMqg1VjQs N8j6NDQug9QrbBTM6U7oZWF/AK+CdFoe+leq5MZfzwCevs0BQgxWm4SHMpXL2vtly67QSPMY dl96fOzw8YbKHv1o0ixhCvc37cI9oUVuSJLXKhEEAvWvLuusiuNeoz+6aPlELvD8h5txJqui tVOzctvJ7ktGZTNiz73tKYVdkKaQVyo8QJFLCNLnUulrQ5wXwteYPg6mrpBxu9VqgDrMp7eB T2kaZ4GRBoMWXXPYSIEe5PM5hhNCsSUfqrKj34UZPijPe+HiWoFJ4S5vIpzutiae11Ctki7u XzeLAhOJQB2raraIqDlFP9I9Zj9JOAZhmiKSEWKfOooCNxQYGiUdPrdYnAe+m7FXRomjF0OO gSepNIESt2gOEIbE5cMxQ0gAueNJc58eHCjWhsNJIwARAQABwsFlBBgBCgAPBQJSTc8EAhsM BQkJZgGAAAoJEJW2GBstM+nsh8EP/1sxZpkJelu+smmqaqdrGHlNrFVLOmeN5yr2IGHBUbmF htjr7fVoU8T0mUnlUU724aKPla4nWhMb4NMu+VxRRFGaT2TYpyR6VIxaStycyUdMGjdXV0Pz TGmxFXhNZXKEITXH9sIxuONBp1czl4AgwN7AAl1MKyV13AaLIyajs58mYmuXtyFn/O+4lxh5 nl2Fa3L9YkL9O7QU2p6WAnDky+L3PgUWp1AzJGfYlLZ8XXCi+KK+pnta+f9yKHt/Oqd/s7OC W4mXgFkBrfuSZZofa4eZckh5u0yBYW3OnEJhClgxRbuOhyYwqQr5oxPrQtjtbMiBzbrOkHhy NnrVCFd9EqlojREGDefHo3V+ZlUOc6OoN3CAYnNa2uLEOm5DCuqOE4z5atBCih5EyITPp7JP J2disEP6ddipcilqbnJdP+TyRQwSv5qRNy8cHahD1Cg9XJJHiC3qr+W3eOtqPkJxhU5biPEr 7dljaLS1Ij771brzqO/x5zW1L9py7muXzYBsW8+keKj8LOYs2242KgjI5Og9YhIJGBFBNddQ wxKBKQpytKQOiXwjhk4Nj77U796bsCd/jIS0r0ZUKBEptPyKso7ncfrm163aEmSaDUkiIjyp 9CEOVT87D+VAVh9PyLGP1niQzWEWFSK36tRGZlF0odP1ZB6wub9zq2DxFouSjHgH Subject: Re: svn commit: r337271 - head/stand/i386/libi386 Message-ID: <87cc3ae5-cbef-0d04-071e-cb7f7a410ce8@delphij.net> Date: Sat, 4 Aug 2018 01:54:22 -0700 User-Agent: Thunderbird MIME-Version: 1.0 In-Reply-To: <201808031911.w73JB0WK025164@repo.freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="sLvrY7cl1RRdAnsYccFihl14kpNsTv5Fa" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 08:54:27 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --sLvrY7cl1RRdAnsYccFihl14kpNsTv5Fa Content-Type: multipart/mixed; boundary="ZGRV3EdwnFbod01qzj7SedSQINKX01EmT"; protected-headers="v1" From: Xin Li Reply-To: d@delphij.net To: Cy Schubert , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, tsoome@freebsd.org Message-ID: <87cc3ae5-cbef-0d04-071e-cb7f7a410ce8@delphij.net> Subject: Re: svn commit: r337271 - head/stand/i386/libi386 References: <201808031911.w73JB0WK025164@repo.freebsd.org> In-Reply-To: <201808031911.w73JB0WK025164@repo.freebsd.org> --ZGRV3EdwnFbod01qzj7SedSQINKX01EmT Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi, Cy, On 8/3/18 12:11, Cy Schubert wrote: > Author: cy > Date: Fri Aug 3 19:11:00 2018 > New Revision: 337271 > URL: https://svnweb.freebsd.org/changeset/base/337271 >=20 > Log: > Some drives report a geometry that is inconsisetent with the total > number of sectors reported through the BIOS. Cylinders * heads * > sectors may not necessarily be equal to the total number of sectors > reported through int13h function 48h. > =20 > An example of this is when a Mediasonic HD3-U2B PATA to USB enclosure= > with a 80 GB disk is attached. Loader hangs at line 506 of > stand/i386/libi386/biosdisk.c while attempting to read sectors beyond= > the end of the disk, sector 156906855. I discovered that the Mediason= ic > enclosure was reporting the disk with 9767 cylinders, 255 heads, 63 > sectors/track. That's 156906855 sectors. However camcontrol and > Windows 10 both report report the disk having 156301488 sectors, not > the calculated value. At line 280 biosdisk.c sets the sectors to the > higher of either bd->bd_sectors or the total calculated at line 276 > (156906855) instead of the lower and correct value of 156301488 repor= ted > by int 13h 48h. > =20 > This was tested on all three of my Mediasonic HD3-U2B PATA to USB > enclosures. > =20 > Instead of using the higher of bd_sectors (returned by int13h) or the= > calculated value, this patch uses the lower and safer of the values. > =20 > Reviewed by: tsoome@ > Differential Revision: https://reviews.freebsd.org/D16577 >=20 > Modified: > head/stand/i386/libi386/biosdisk.c >=20 > Modified: head/stand/i386/libi386/biosdisk.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > --- head/stand/i386/libi386/biosdisk.c Fri Aug 3 18:52:51 2018 (r33727= 0) > +++ head/stand/i386/libi386/biosdisk.c Fri Aug 3 19:11:00 2018 (r33727= 1) > @@ -275,7 +275,7 @@ bd_int13probe(struct bdinfo *bd) > =20 > total =3D (uint64_t)params.cylinders * > params.heads * params.sectors_per_track; > - if (bd->bd_sectors < total) > + if (bd->bd_sectors > total) > bd->bd_sectors =3D total; > =20 > ret =3D 1; >=20 This broke loader on my system, but I think your reasoning was valid so I took a deeper look and discovered that on my system, INT 13h, function 48h would give zeros in EDD parameters' CHS fields. With that, the calculated CHS based total would be 0, and your change would cause bd_sectors be zeroed. Could you please let me know if https://reviews.freebsd.org/D16588 makes sense to you? (I'm not 100% certain if I have followed the code). It allowed my Asrock C2750D4I based board to boot from ZFS. Cheers, --ZGRV3EdwnFbod01qzj7SedSQINKX01EmT-- --sLvrY7cl1RRdAnsYccFihl14kpNsTv5Fa Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJbZWnAAAoJEJW2GBstM+nstZkQAImlL2dAeCmOkNZ903AKK1pa YuDbZowlg3P+GC3OtWjHVzf4YDbWY1JQUS0IGqfmOUIAX/2J/MaQGZl9hk2kbc9p 6aJtt6a3H7PVVX8qyzi27eNrwUpZhaI4DA7aXE3i3ZeeUtHvlN0L1xOqT9x8vpcH qAn7XHFaR841PvhgGoySx/T6EGKIRHy9la8+6TVoIyZJbJ7QTVg6jgxA0swa3Kow ZWCj0ANQEG2beNHdUmqZqSGaVLPtGW9AGxLkxGc79a0WfYgk0BJkH40OVnucx+3v M9sFN0RBCsQVzIuwIEkq1PFbLLR3k61jqSGO9ctj3+1AxvXgQnpM/QPF+Mse51HE 0xN/EMw/ZI7xZySaycX+alpSxMWEXA5TBDBJ9UsnNhLt6J/E1LrNXKDTyjE2jNyw JMLNVpq1Lwm3wa6RrWrRVL//7GNphrksy3IHBHq+LtVa//BNrO/EfRIn+4SijbFn k9WFgXXouUvEl1zBH3DYzCYH08YdIjo1PjhZTSAdaJn8DT/VsFnOuuYDUgNEE11K fc69d+xG2AhhfQL1AYjKFcMfKQEuf+J9csbP3HjZfeltsiTLAuAcZq0A9dK/hTjW yd55Pm+3xqOALh6+OR4s47PlxPFWn5pFlKiSxJPhIwDgLmxmrA4Ao0as7/VjF+pD DLlqJzgYRP41sfPEXM6j =6FnU -----END PGP SIGNATURE----- --sLvrY7cl1RRdAnsYccFihl14kpNsTv5Fa-- From owner-svn-src-all@freebsd.org Sat Aug 4 10:49:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 296A11065E82; Sat, 4 Aug 2018 10:49:41 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D36C68F357; Sat, 4 Aug 2018 10:49:40 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 6A260173B6; Sat, 4 Aug 2018 10:49:40 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8rgnodtw3ikd82geexy.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240b:b802:2085:aca2:f9e7:5b26]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id D6CCF3F0B0; Sat, 4 Aug 2018 12:49:37 +0200 (CEST) From: "Kristof Provost" To: "Gleb Smirnoff" Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336221 - head/sys/net Date: Sat, 04 Aug 2018 12:49:36 +0200 X-Mailer: MailMate (2.0BETAr6116) Message-ID: <25795E0A-A362-44B2-AC5A-573442FC256D@FreeBSD.org> In-Reply-To: <20180803230405.GI420@FreeBSD.org> References: <201807121635.w6CGZZAN046919@repo.freebsd.org> <20180803230405.GI420@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed; markup=markdown Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 10:49:41 -0000 On 4 Aug 2018, at 1:04, Gleb Smirnoff wrote: > On Thu, Jul 12, 2018 at 04:35:35PM +0000, Kristof Provost wrote: > K> Author: kp > K> Date: Thu Jul 12 16:35:35 2018 > K> New Revision: 336221 > K> URL: https://svnweb.freebsd.org/changeset/base/336221 > K> > K> Log: > K> pf: Increate default state table size > K> > K> The typical system now has a lot more memory than when pf was = > new, and is also > K> expected to handle more connections. Increase the default size of = > the state > K> table. > K> Note that users can overrule this using 'set limit states' in = > pf.conf. > K> > K> From OpenBSD: > K> The year is 2018. > K> Mercury, Bowie, Cash, Motorola and DEC all left us. > K> Just pf still has a default state table limit of 10000. > K> Had! Now it's a tiny little bit more, 100k. > K> lead guitar: me > K> ok chorus: phessler theo claudio benno > K> background school girl laughing: bob > > For FreeBSD it would also make sense to bump hash sizes along with = > this change. > Yeah, Olivier did some benchmarking work around this: https://github.com/ocochard/netbenches/blob/master/Atom_C2758_8Cores-Chel= sio_T540-CR/pf-states_hashsize/results/fbsd12-head.r332390/README.md He found a roughly 10% increase in throughput by increasing the hash = size to 256K. I=E2=80=99ve been thinking about making the hash size dynamically configu= rable = (so we could calculate it based in the state limit, rather than expect = users to tune this manually), but that=E2=80=99s probably not going to ge= t = done any time soon. I=E2=80=99ll see about increasing the default (although perhaps not to 25= 6K, = because if my math is right that=E2=80=99s cost us ~20MB of memory) soon,= = because that=E2=80=99s mostly a quick win. Regards, Kristof From owner-svn-src-all@freebsd.org Sat Aug 4 10:58:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 182581066279; Sat, 4 Aug 2018 10:58:46 +0000 (UTC) (envelope-from tsoome@me.com) Received: from mr11p00im-asmtp004.me.com (mr11p00im-asmtp004.me.com [17.110.69.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A9E8C8F95F; Sat, 4 Aug 2018 10:58:45 +0000 (UTC) (envelope-from tsoome@me.com) Received: from process-dkim-sign-daemon.mr11p00im-asmtp004.me.com by mr11p00im-asmtp004.me.com (Oracle Communications Messaging Server 8.0.2.2.20180531 64bit (built May 31 2018)) id <0PCX00F00N46B100@mr11p00im-asmtp004.me.com>; Sat, 04 Aug 2018 10:58:38 +0000 (GMT) Received: from icloud.com ([127.0.0.1]) by mr11p00im-asmtp004.me.com (Oracle Communications Messaging Server 8.0.2.2.20180531 64bit (built May 31 2018)) with ESMTPSA id <0PCX003F1NTDXE30@mr11p00im-asmtp004.me.com>; Sat, 04 Aug 2018 10:58:30 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-04_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1808040123 Content-type: text/plain; charset=utf-8 MIME-version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: svn commit: r337271 - head/stand/i386/libi386 From: Toomas Soome In-reply-to: <87cc3ae5-cbef-0d04-071e-cb7f7a410ce8@delphij.net> Date: Sat, 04 Aug 2018 13:58:24 +0300 Cc: Cy Schubert , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org, tsoome@freebsd.org Content-transfer-encoding: quoted-printable Message-id: <7212F80A-9BA8-4D64-AB66-B8FA9F08409F@me.com> References: <201808031911.w73JB0WK025164@repo.freebsd.org> <87cc3ae5-cbef-0d04-071e-cb7f7a410ce8@delphij.net> To: d@delphij.net X-Mailer: Apple Mail (2.3445.9.1) X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 10:58:46 -0000 > On 4 Aug 2018, at 11:54, Xin Li wrote: >=20 > Hi, Cy, >=20 > On 8/3/18 12:11, Cy Schubert wrote: >> Author: cy >> Date: Fri Aug 3 19:11:00 2018 >> New Revision: 337271 >> URL: https://svnweb.freebsd.org/changeset/base/337271 >>=20 >> Log: >> Some drives report a geometry that is inconsisetent with the total >> number of sectors reported through the BIOS. Cylinders * heads * >> sectors may not necessarily be equal to the total number of sectors >> reported through int13h function 48h. >>=20 >> An example of this is when a Mediasonic HD3-U2B PATA to USB = enclosure >> with a 80 GB disk is attached. Loader hangs at line 506 of >> stand/i386/libi386/biosdisk.c while attempting to read sectors = beyond >> the end of the disk, sector 156906855. I discovered that the = Mediasonic >> enclosure was reporting the disk with 9767 cylinders, 255 heads, 63 >> sectors/track. That's 156906855 sectors. However camcontrol and >> Windows 10 both report report the disk having 156301488 sectors, not >> the calculated value. At line 280 biosdisk.c sets the sectors to the >> higher of either bd->bd_sectors or the total calculated at line 276 >> (156906855) instead of the lower and correct value of 156301488 = reported >> by int 13h 48h. >>=20 >> This was tested on all three of my Mediasonic HD3-U2B PATA to USB >> enclosures. >>=20 >> Instead of using the higher of bd_sectors (returned by int13h) or = the >> calculated value, this patch uses the lower and safer of the values. >>=20 >> Reviewed by: tsoome@ >> Differential Revision: https://reviews.freebsd.org/D16577 >>=20 >> Modified: >> head/stand/i386/libi386/biosdisk.c >>=20 >> Modified: head/stand/i386/libi386/biosdisk.c >> = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D >> --- head/stand/i386/libi386/biosdisk.c Fri Aug 3 18:52:51 2018 = (r337270) >> +++ head/stand/i386/libi386/biosdisk.c Fri Aug 3 19:11:00 2018 = (r337271) >> @@ -275,7 +275,7 @@ bd_int13probe(struct bdinfo *bd) >>=20 >> total =3D (uint64_t)params.cylinders * >> params.heads * params.sectors_per_track; >> - if (bd->bd_sectors < total) >> + if (bd->bd_sectors > total) >> bd->bd_sectors =3D total; >>=20 >> ret =3D 1; >>=20 >=20 > This broke loader on my system, but I think your reasoning was valid = so > I took a deeper look and discovered that on my system, INT 13h, = function > 48h would give zeros in EDD parameters' CHS fields. With that, the > calculated CHS based total would be 0, and your change would cause > bd_sectors be zeroed. >=20 > Could you please let me know if https://reviews.freebsd.org/D16588 = makes > sense to you? (I'm not 100% certain if I have followed the code). It > allowed my Asrock C2750D4I based board to boot from ZFS. >=20 I have in mind something a bit different for some time, but haven=E2=80=99= t had chance to complete it because I have no =E2=80=9Cweird=E2=80=9D = systems to validate the idea:D I=E2=80=99ll try to get a bit of time and = post an phabricator soon. rgds, toomas From owner-svn-src-all@freebsd.org Sat Aug 4 11:11:46 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE31A1066752 for ; Sat, 4 Aug 2018 11:11:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x243.google.com (mail-io0-x243.google.com [IPv6:2607:f8b0:4001:c06::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32E7F700A5 for ; Sat, 4 Aug 2018 11:11:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x243.google.com with SMTP id z20-v6so7188185iol.0 for ; Sat, 04 Aug 2018 04:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3z2HHJ5UnGqCaLWPIKLfdssqD6nFSRfG50+C16+kUIE=; b=YvCAF3SdX9dCjukanIo6w8tCr7DDnGo5JNYwn7i4vXHjBxuEWAKDuvSagjG1Fev9Wm 7BThI7JFaOgQTLy/QVgRETjgKcxUk3Xcau7iaG95ulf1Q2sgsybHQUDWNm6M9CUQG7rC igKrpBaYD1i4X6WewDG/ly92HyJuKrDgaRgMsboyRbA797cRkvRAqVqkxrs1SqIaxFmX q5W8ExY4UGmjfHJMxxUcLpwG9C6FaHbjelNYBefiJg/2qV66YU+ZLht7PEumy/hH0oo8 i5MfNCYFhrHcy4yNQpo89dYDoeKKPeadwMokewueL8Dh7gcZowlFpylr4hDLWuDJ4Nve vBhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3z2HHJ5UnGqCaLWPIKLfdssqD6nFSRfG50+C16+kUIE=; b=Vxo6FuEId6iI46OKo1vWo3e/f0tVctogJCm0u6Sjnke12n65MoVSq+m2ococdwsPP6 1LlRQ5sdCdWD4aLXSUdg5cwb0qGxRvQw9Ziq9QQYydXdza2kmfu/34AerNaBgVPIfW/P 1s5Kp/W9sqWxEHZrJgXIRXvhcg9NoltgoDamF4/KOen6a1hgLFQ4DnqwPJo1L1awl68v R4D6bdBWz66J8lK5tds9kiD3ABQcpTiYRgGc08v1oKb3lB7xhx9f+1u9UUbTohE2zp+q LwgDfy3GX7l4Rgl7ZNLEo5iJoBPnccfDG4rPCTR4t1uKxpcRHKCW2PtWdNt3KtIScPYo /WHg== X-Gm-Message-State: AOUpUlH4/NKtzAK5hg2Pojk4WzA4cyeuWVN4HbBuHYQ7X8fEmodu+uUg kumyNG1i7jacDS5uLaCu20eonFDkPLE1FTXebLL0n5lJaY4= X-Google-Smtp-Source: AA+uWPzvNeIiPrZnV8tMGj9ugQI7hH9GimAjL/0uS1+gKoVb8k/cHh5iT3s41cxId6R4fF6unSl02lkGl5FRNmID0Os= X-Received: by 2002:a6b:d004:: with SMTP id x4-v6mr8822133ioa.299.1533381105397; Sat, 04 Aug 2018 04:11:45 -0700 (PDT) MIME-Version: 1.0 References: <201808031911.w73JB0WK025164@repo.freebsd.org> <87cc3ae5-cbef-0d04-071e-cb7f7a410ce8@delphij.net> <7212F80A-9BA8-4D64-AB66-B8FA9F08409F@me.com> In-Reply-To: <7212F80A-9BA8-4D64-AB66-B8FA9F08409F@me.com> From: Warner Losh Date: Sat, 4 Aug 2018 12:11:34 +0100 Message-ID: Subject: Re: svn commit: r337271 - head/stand/i386/libi386 To: Toomas Soome Cc: Xin LI , Cy Schubert , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org, tsoome@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 11:11:46 -0000 On Sat, Aug 4, 2018, 11:58 AM Toomas Soome wrote: > > > > On 4 Aug 2018, at 11:54, Xin Li wrote: > > > > Hi, Cy, > > > > On 8/3/18 12:11, Cy Schubert wrote: > >> Author: cy > >> Date: Fri Aug 3 19:11:00 2018 > >> New Revision: 337271 > >> URL: https://svnweb.freebsd.org/changeset/base/337271 > >> > >> Log: > >> Some drives report a geometry that is inconsisetent with the total > >> number of sectors reported through the BIOS. Cylinders * heads * > >> sectors may not necessarily be equal to the total number of sectors > >> reported through int13h function 48h. > >> > >> An example of this is when a Mediasonic HD3-U2B PATA to USB enclosure > >> with a 80 GB disk is attached. Loader hangs at line 506 of > >> stand/i386/libi386/biosdisk.c while attempting to read sectors beyond > >> the end of the disk, sector 156906855. I discovered that the Mediason= ic > >> enclosure was reporting the disk with 9767 cylinders, 255 heads, 63 > >> sectors/track. That's 156906855 sectors. However camcontrol and > >> Windows 10 both report report the disk having 156301488 sectors, not > >> the calculated value. At line 280 biosdisk.c sets the sectors to the > >> higher of either bd->bd_sectors or the total calculated at line 276 > >> (156906855) instead of the lower and correct value of 156301488 > reported > >> by int 13h 48h. > >> > >> This was tested on all three of my Mediasonic HD3-U2B PATA to USB > >> enclosures. > >> > >> Instead of using the higher of bd_sectors (returned by int13h) or the > >> calculated value, this patch uses the lower and safer of the values. > >> > >> Reviewed by: tsoome@ > >> Differential Revision: https://reviews.freebsd.org/D16577 > >> > >> Modified: > >> head/stand/i386/libi386/biosdisk.c > >> > >> Modified: head/stand/i386/libi386/biosdisk.c > >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D > >> --- head/stand/i386/libi386/biosdisk.c Fri Aug 3 18:52:51 2018 > (r337270) > >> +++ head/stand/i386/libi386/biosdisk.c Fri Aug 3 19:11:00 2018 > (r337271) > >> @@ -275,7 +275,7 @@ bd_int13probe(struct bdinfo *bd) > >> > >> total =3D (uint64_t)params.cylinders * > >> params.heads * params.sectors_per_track; > >> - if (bd->bd_sectors < total) > >> + if (bd->bd_sectors > total) > >> bd->bd_sectors =3D total; > >> > >> ret =3D 1; > >> > > > > This broke loader on my system, but I think your reasoning was valid so > > I took a deeper look and discovered that on my system, INT 13h, functio= n > > 48h would give zeros in EDD parameters' CHS fields. With that, the > > calculated CHS based total would be 0, and your change would cause > > bd_sectors be zeroed. > > > > Could you please let me know if https://reviews.freebsd.org/D16588 make= s > > sense to you? (I'm not 100% certain if I have followed the code). It > > allowed my Asrock C2750D4I based board to boot from ZFS. > > > > I have in mind something a bit different for some time, but haven=E2=80= =99t had > chance to complete it because I have no =E2=80=9Cweird=E2=80=9D systems t= o validate the > idea:D I=E2=80=99ll try to get a bit of time and post an phabricator soon= . > I think the phab looks good, but I am only on my phone so can't say so there... Warner > From owner-svn-src-all@freebsd.org Sat Aug 4 12:14:23 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7DF31068B93; Sat, 4 Aug 2018 12:14:22 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E9A372803; Sat, 4 Aug 2018 12:14:22 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (unknown [IPv6:2601:648:8880:1e30:219f:c5f:b5d2:8746]) by mail.baldwin.cx (Postfix) with ESMTPSA id B4D7110AFCD; Sat, 4 Aug 2018 08:14:20 -0400 (EDT) Subject: Re: svn commit: r337273 - head/sys/dev/nvme To: Konstantin Belousov , Justin Hibbits References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org From: John Baldwin Message-ID: Date: Sat, 4 Aug 2018 05:14:31 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20180804080840.GI6049@kib.kiev.ua> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Sat, 04 Aug 2018 08:14:21 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:14:23 -0000 On 8/4/18 1:08 AM, Konstantin Belousov wrote: > On Fri, Aug 03, 2018 at 08:04:06PM +0000, Justin Hibbits wrote: >> Author: jhibbits >> Date: Fri Aug 3 20:04:06 2018 >> New Revision: 337273 >> URL: https://svnweb.freebsd.org/changeset/base/337273 >> >> Log: >> nvme(4): Add bus_dmamap_sync() at the end of the request path >> >> Summary: >> Some architectures, in this case powerpc64, need explicit synchronization >> barriers vs device accesses. >> >> Prior to this change, when running 'make buildworld -j72' on a 18-core >> (72-thread) POWER9, I would see controller resets often. With this change, I >> don't see these resets messages, though another tester still does, for yet to be >> determined reasons, so this may not be a complete fix. Additionally, I see a >> ~5-10% speed up in buildworld times, likely due to not needing to reset the >> controller. >> >> Reviewed By: jimharris >> Differential Revision: https://reviews.freebsd.org/D16570 >> >> Modified: >> head/sys/dev/nvme/nvme_qpair.c >> >> Modified: head/sys/dev/nvme/nvme_qpair.c >> ============================================================================== >> --- head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 19:24:04 2018 (r337272) >> +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 20:04:06 2018 (r337273) >> @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct nvme_qpair *qpair, >> req->retries++; >> nvme_qpair_submit_tracker(qpair, tr); >> } else { >> - if (req->type != NVME_REQUEST_NULL) >> + if (req->type != NVME_REQUEST_NULL) { >> + bus_dmamap_sync(qpair->dma_tag_payload, >> + tr->payload_dma_map, >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); >> bus_dmamap_unload(qpair->dma_tag_payload, >> tr->payload_dma_map); >> + } >> >> nvme_free_request(req); >> tr->req = NULL; >> @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct nvme_qpair *qpai >> */ >> return (false); >> >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); >> while (1) { >> cpl = qpair->cpl[qpair->cq_head]; >> >> @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair *qpair, st >> if (++qpair->sq_tail == qpair->num_entries) >> qpair->sq_tail = 0; >> >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, >> + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); >> +#ifndef __powerpc__ >> + /* >> + * powerpc's bus_dmamap_sync() already includes a heavyweight sync, but >> + * no other archs do. >> + */ >> wmb(); >> +#endif > What is the purpose of this call ? It is useless without paired read > barrier. So where is the reciprocal rmb() ? For DMA, the rmb is in the device controller. However, architectures that need this kind of ordering should do it in their bus_dmmap_sync op, and this explicit one needs to be removed. (Alpha had a wmb in its bus_dmamap_sync op for this reason.) -- John Baldwin From owner-svn-src-all@freebsd.org Sat Aug 4 12:24:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFED71069558; Sat, 4 Aug 2018 12:24:38 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F4F072FBC; Sat, 4 Aug 2018 12:24:38 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3DF15773A; Sat, 4 Aug 2018 12:24:38 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74COcTO056139; Sat, 4 Aug 2018 12:24:38 GMT (envelope-from wulf@FreeBSD.org) Received: (from wulf@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74COcOm056138; Sat, 4 Aug 2018 12:24:38 GMT (envelope-from wulf@FreeBSD.org) Message-Id: <201808041224.w74COcOm056138@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wulf set sender to wulf@FreeBSD.org using -f From: Vladimir Kondratyev Date: Sat, 4 Aug 2018 12:24:38 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337287 - head/sys/dev/usb/input X-SVN-Group: head X-SVN-Commit-Author: wulf X-SVN-Commit-Paths: head/sys/dev/usb/input X-SVN-Commit-Revision: 337287 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:24:38 -0000 Author: wulf Date: Sat Aug 4 12:24:37 2018 New Revision: 337287 URL: https://svnweb.freebsd.org/changeset/base/337287 Log: wmt(4): Read 'Contact count maximum' usage value from feature report rather than from HID descriptor to match Microsoft documentation. Fall back to HID descriptor provided value if 'Get Report' request failed. MFC after: 2 weeks Modified: head/sys/dev/usb/input/wmt.c Modified: head/sys/dev/usb/input/wmt.c ============================================================================== --- head/sys/dev/usb/input/wmt.c Sat Aug 4 11:53:41 2018 (r337286) +++ head/sys/dev/usb/input/wmt.c Sat Aug 4 12:24:37 2018 (r337287) @@ -203,6 +203,10 @@ struct wmt_softc uint32_t nconts_max; uint8_t report_id; + struct hid_location cont_max_loc; + uint32_t cont_max_rlen; + uint8_t cont_max_rid; + uint8_t buf[WMT_BSIZE] __aligned(4); }; @@ -212,6 +216,7 @@ struct wmt_softc if (USAGE_SUPPORTED((caps), (usage))) static bool wmt_hid_parse(struct wmt_softc *, const void *, uint16_t); +static void wmt_cont_max_parse(struct wmt_softc *, const void *, uint16_t); static usb_callback_t wmt_intr_callback; @@ -291,15 +296,36 @@ wmt_attach(device_t dev) return (ENXIO); } - mtx_init(&sc->mtx, "wmt lock", NULL, MTX_DEF); + if (!wmt_hid_parse(sc, d_ptr, d_len)) { + DPRINTF("multi-touch HID descriptor not found\n"); + free(d_ptr, M_TEMP); + return (ENXIO); + } /* Get HID report length */ sc->isize = hid_report_size(d_ptr, d_len, hid_input, NULL); + free(d_ptr, M_TEMP); if (sc->isize <= 0 || sc->isize > WMT_BSIZE) { DPRINTF("Input size invalid or too large: %d\n", sc->isize); - goto detach; + return (ENXIO); } + /* Fetch and parse "Contact count maximum" feature report */ + if (sc->cont_max_rlen > 0 && sc->cont_max_rlen <= WMT_BSIZE) { + err = usbd_req_get_report(uaa->device, NULL, sc->buf, + sc->cont_max_rlen, uaa->info.bIfaceIndex, + UHID_FEATURE_REPORT, sc->cont_max_rid); + if (err == USB_ERR_NORMAL_COMPLETION) + wmt_cont_max_parse(sc, sc->buf, sc->cont_max_rlen); + else + DPRINTF("usbd_req_get_report error=(%s)\n", + usbd_errstr(err)); + } else + DPRINTF("Feature report %hhu size invalid or too large: %u\n", + sc->cont_max_rid, sc->cont_max_rlen); + + mtx_init(&sc->mtx, "wmt lock", NULL, MTX_DEF); + err = usbd_transfer_setup(uaa->device, &uaa->info.bIfaceIndex, sc->xfer, wmt_config, WMT_N_TRANSFER, sc, &sc->mtx); if (err) { @@ -307,9 +333,6 @@ wmt_attach(device_t dev) goto detach; } - if (!wmt_hid_parse(sc, d_ptr, d_len)) - goto detach; - sc->evdev = evdev_alloc(); evdev_set_name(sc->evdev, device_get_desc(dev)); evdev_set_phys(sc->evdev, device_get_nameunit(dev)); @@ -334,7 +357,6 @@ wmt_attach(device_t dev) return (0); detach: - free(d_ptr, M_TEMP); wmt_detach(dev); return (ENXIO); } @@ -513,6 +535,46 @@ wmt_ev_open(struct evdev_dev *evdev, void *ev_softc) return (0); } +/* port of userland hid_report_size() from usbhid(3) to kernel */ +static int +wmt_hid_report_size(const void *buf, uint16_t len, enum hid_kind k, uint8_t id) +{ + struct hid_data *d; + struct hid_item h; + uint32_t temp; + uint32_t hpos; + uint32_t lpos; + int report_id = 0; + + hpos = 0; + lpos = 0xFFFFFFFF; + + for (d = hid_start_parse(buf, len, 1 << k); hid_get_item(d, &h);) { + if (h.kind == k && h.report_ID == id) { + /* compute minimum */ + if (lpos > h.loc.pos) + lpos = h.loc.pos; + /* compute end position */ + temp = h.loc.pos + (h.loc.size * h.loc.count); + /* compute maximum */ + if (hpos < temp) + hpos = temp; + if (h.report_ID != 0) + report_id = 1; + } + } + hid_end_parse(d); + + /* safety check - can happen in case of currupt descriptors */ + if (lpos > hpos) + temp = 0; + else + temp = hpos - lpos; + + /* return length in bytes rounded up */ + return ((temp + 7) / 8 + report_id); +} + static bool wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, uint16_t d_len) { @@ -523,6 +585,7 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, uint32_t caps = 0; int32_t cont_count_max = 0; uint8_t report_id = 0; + uint8_t cont_max_rid = 0; bool touch_coll = false; bool finger_coll = false; bool cont_count_found = false; @@ -547,8 +610,12 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, case hid_feature: if (hi.collevel == 1 && touch_coll && WMT_HI_ABSOLUTE(hi) && hi.usage == - HID_USAGE2(HUP_DIGITIZERS, HUD_CONTACT_MAX)) + HID_USAGE2(HUP_DIGITIZERS, HUD_CONTACT_MAX)) { cont_count_max = hi.logical_maximum; + cont_max_rid = hi.report_ID; + if (sc != NULL) + sc->cont_max_loc = hi.loc; + } break; default: break; @@ -557,7 +624,7 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, hid_end_parse(hd); /* Maximum contact count is required usage */ - if (cont_count_max < 1) + if (cont_max_rid == 0) return (false); touch_coll = false; @@ -668,12 +735,17 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, if (sc == NULL) return (true); + /* + * According to specifications 'Contact Count Maximum' should be read + * from Feature Report rather than from HID descriptor. Set sane + * default value now to handle the case of 'Get Report' request failure + */ + if (cont_count_max < 1) + cont_count_max = cont; + /* Cap contact count maximum to MAX_MT_SLOTS */ - if (cont_count_max > MAX_MT_SLOTS) { - DPRINTF("Hardware reported %d contacts while only %d is " - "supported\n", (int)cont_count_max, MAX_MT_SLOTS); + if (cont_count_max > MAX_MT_SLOTS) cont_count_max = MAX_MT_SLOTS; - } /* Set number of MT protocol type B slots */ sc->ai[WMT_SLOT] = (struct wmt_absinfo) { @@ -689,9 +761,13 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, sc->ai[WMT_ORIENTATION].max = 1; } + sc->cont_max_rlen = wmt_hid_report_size(d_ptr, d_len, hid_feature, + cont_max_rid); + sc->report_id = report_id; sc->caps = caps; sc->nconts_max = cont; + sc->cont_max_rid = cont_max_rid; /* Announce information about the touch device */ device_printf(sc->dev, @@ -705,6 +781,27 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, (int)sc->ai[WMT_X].min, (int)sc->ai[WMT_Y].min, (int)sc->ai[WMT_X].max, (int)sc->ai[WMT_Y].max); return (true); +} + +static void +wmt_cont_max_parse(struct wmt_softc *sc, const void *r_ptr, uint16_t r_len) +{ + uint32_t cont_count_max; + + cont_count_max = hid_get_data_unsigned((const uint8_t *)r_ptr + 1, + r_len - 1, &sc->cont_max_loc); + if (cont_count_max > MAX_MT_SLOTS) { + DPRINTF("Hardware reported %d contacts while only %d is " + "supported\n", (int)cont_count_max, MAX_MT_SLOTS); + cont_count_max = MAX_MT_SLOTS; + } + /* Feature report is a primary source of 'Contact Count Maximum' */ + if (cont_count_max > 0 && + cont_count_max != sc->ai[WMT_SLOT].max + 1) { + sc->ai[WMT_SLOT].max = cont_count_max - 1; + device_printf(sc->dev, "%d feature report contacts", + cont_count_max); + } } static devclass_t wmt_devclass; From owner-svn-src-all@freebsd.org Sat Aug 4 12:29:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D14D10696DD; Sat, 4 Aug 2018 12:29:10 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C70977319C; Sat, 4 Aug 2018 12:29:09 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 886E7773F; Sat, 4 Aug 2018 12:29:09 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CT9xj056366; Sat, 4 Aug 2018 12:29:09 GMT (envelope-from wulf@FreeBSD.org) Received: (from wulf@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CT9Pa056365; Sat, 4 Aug 2018 12:29:09 GMT (envelope-from wulf@FreeBSD.org) Message-Id: <201808041229.w74CT9Pa056365@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wulf set sender to wulf@FreeBSD.org using -f From: Vladimir Kondratyev Date: Sat, 4 Aug 2018 12:29:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337288 - head/sys/dev/usb/input X-SVN-Group: head X-SVN-Commit-Author: wulf X-SVN-Commit-Paths: head/sys/dev/usb/input X-SVN-Commit-Revision: 337288 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:29:10 -0000 Author: wulf Date: Sat Aug 4 12:29:08 2018 New Revision: 337288 URL: https://svnweb.freebsd.org/changeset/base/337288 Log: wmt(4): Read Microsoft's "Touch Hardware Quality Assurance" certificate blob if present to enable some devices like WaveShare touchscreens. Unlike Windows we discard content of the blob. We try mimic Windows driver behaviour from the USB device point of view. Submitted by: glebius (initial version) Modified: head/sys/dev/usb/input/wmt.c Modified: head/sys/dev/usb/input/wmt.c ============================================================================== --- head/sys/dev/usb/input/wmt.c Sat Aug 4 12:24:37 2018 (r337287) +++ head/sys/dev/usb/input/wmt.c Sat Aug 4 12:29:08 2018 (r337288) @@ -206,6 +206,8 @@ struct wmt_softc struct hid_location cont_max_loc; uint32_t cont_max_rlen; uint8_t cont_max_rid; + uint32_t thqa_cert_rlen; + uint8_t thqa_cert_rid; uint8_t buf[WMT_BSIZE] __aligned(4); }; @@ -324,6 +326,13 @@ wmt_attach(device_t dev) DPRINTF("Feature report %hhu size invalid or too large: %u\n", sc->cont_max_rid, sc->cont_max_rlen); + /* Fetch THQA certificate to enable some devices like WaveShare */ + if (sc->thqa_cert_rlen > 0 && sc->thqa_cert_rlen <= WMT_BSIZE && + sc->thqa_cert_rid != sc->cont_max_rid) + (void)usbd_req_get_report(uaa->device, NULL, sc->buf, + sc->thqa_cert_rlen, uaa->info.bIfaceIndex, + UHID_FEATURE_REPORT, sc->thqa_cert_rid); + mtx_init(&sc->mtx, "wmt lock", NULL, MTX_DEF); err = usbd_transfer_setup(uaa->device, &uaa->info.bIfaceIndex, @@ -586,6 +595,7 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, int32_t cont_count_max = 0; uint8_t report_id = 0; uint8_t cont_max_rid = 0; + uint8_t thqa_cert_rid = 0; bool touch_coll = false; bool finger_coll = false; bool cont_count_found = false; @@ -593,6 +603,7 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, #define WMT_HI_ABSOLUTE(hi) \ (((hi).flags & (HIO_CONST|HIO_VARIABLE|HIO_RELATIVE)) == HIO_VARIABLE) +#define HUMS_THQA_CERT 0xC5 /* Parse features for maximum contact count */ hd = hid_start_parse(d_ptr, d_len, 1 << hid_feature); @@ -608,6 +619,11 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, touch_coll = false; break; case hid_feature: + if (hi.collevel == 1 && touch_coll && hi.usage == + HID_USAGE2(HUP_MICROSOFT, HUMS_THQA_CERT)) { + thqa_cert_rid = hi.report_ID; + break; + } if (hi.collevel == 1 && touch_coll && WMT_HI_ABSOLUTE(hi) && hi.usage == HID_USAGE2(HUP_DIGITIZERS, HUD_CONTACT_MAX)) { @@ -763,11 +779,15 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, sc->cont_max_rlen = wmt_hid_report_size(d_ptr, d_len, hid_feature, cont_max_rid); + if (thqa_cert_rid > 0) + sc->thqa_cert_rlen = wmt_hid_report_size(d_ptr, d_len, + hid_feature, thqa_cert_rid); sc->report_id = report_id; sc->caps = caps; sc->nconts_max = cont; sc->cont_max_rid = cont_max_rid; + sc->thqa_cert_rid = thqa_cert_rid; /* Announce information about the touch device */ device_printf(sc->dev, From owner-svn-src-all@freebsd.org Sat Aug 4 12:31:20 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D66F610698F6; Sat, 4 Aug 2018 12:31:19 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C651733D0; Sat, 4 Aug 2018 12:31:19 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6D835786E; Sat, 4 Aug 2018 12:31:19 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CVJZ5057328; Sat, 4 Aug 2018 12:31:19 GMT (envelope-from wulf@FreeBSD.org) Received: (from wulf@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CVJX0057327; Sat, 4 Aug 2018 12:31:19 GMT (envelope-from wulf@FreeBSD.org) Message-Id: <201808041231.w74CVJX0057327@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wulf set sender to wulf@FreeBSD.org using -f From: Vladimir Kondratyev Date: Sat, 4 Aug 2018 12:31:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337289 - head/sys/dev/usb/input X-SVN-Group: head X-SVN-Commit-Author: wulf X-SVN-Commit-Paths: head/sys/dev/usb/input X-SVN-Commit-Revision: 337289 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:31:20 -0000 Author: wulf Date: Sat Aug 4 12:31:19 2018 New Revision: 337289 URL: https://svnweb.freebsd.org/changeset/base/337289 Log: wmt(4): Use internal function to calculate input report size Usbhid's hid_report_size() calculates integral size of all reports of given kind found in the HID descriptor rather then exact size of report with given ID as its userland counterpart does. As all input data processed by the driver is located within the same report, calculate required driver's buffer size with userland version, imported in one of the previous commits. This allows us to skip zeroing of buffer on processing of each report. While here do some minor refactoring. MFC after: 2 weeks Modified: head/sys/dev/usb/input/wmt.c Modified: head/sys/dev/usb/input/wmt.c ============================================================================== --- head/sys/dev/usb/input/wmt.c Sat Aug 4 12:29:08 2018 (r337288) +++ head/sys/dev/usb/input/wmt.c Sat Aug 4 12:31:19 2018 (r337289) @@ -286,6 +286,7 @@ wmt_attach(device_t dev) uint16_t d_len; size_t i; int err; + bool hid_ok; device_set_usb_desc(dev); sc->dev = dev; @@ -298,15 +299,14 @@ wmt_attach(device_t dev) return (ENXIO); } - if (!wmt_hid_parse(sc, d_ptr, d_len)) { + hid_ok = wmt_hid_parse(sc, d_ptr, d_len); + free(d_ptr, M_TEMP); + if (!hid_ok) { DPRINTF("multi-touch HID descriptor not found\n"); - free(d_ptr, M_TEMP); return (ENXIO); } - /* Get HID report length */ - sc->isize = hid_report_size(d_ptr, d_len, hid_input, NULL); - free(d_ptr, M_TEMP); + /* Check HID report length */ if (sc->isize <= 0 || sc->isize > WMT_BSIZE) { DPRINTF("Input size invalid or too large: %d\n", sc->isize); return (ENXIO); @@ -337,7 +337,7 @@ wmt_attach(device_t dev) err = usbd_transfer_setup(uaa->device, &uaa->info.bIfaceIndex, sc->xfer, wmt_config, WMT_N_TRANSFER, sc, &sc->mtx); - if (err) { + if (err != USB_ERR_NORMAL_COMPLETION) { DPRINTF("usbd_transfer_setup error=%s\n", usbd_errstr(err)); goto detach; } @@ -777,6 +777,7 @@ wmt_hid_parse(struct wmt_softc *sc, const void *d_ptr, sc->ai[WMT_ORIENTATION].max = 1; } + sc->isize = wmt_hid_report_size(d_ptr, d_len, hid_input, report_id); sc->cont_max_rlen = wmt_hid_report_size(d_ptr, d_len, hid_feature, cont_max_rid); if (thqa_cert_rid > 0) From owner-svn-src-all@freebsd.org Sat Aug 4 12:41:00 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE0C61069E96; Sat, 4 Aug 2018 12:41:00 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63A8273DDB; Sat, 4 Aug 2018 12:41:00 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 45CB2790A; Sat, 4 Aug 2018 12:41:00 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74Cf0S7065349; Sat, 4 Aug 2018 12:41:00 GMT (envelope-from wulf@FreeBSD.org) Received: (from wulf@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CexgP065348; Sat, 4 Aug 2018 12:40:59 GMT (envelope-from wulf@FreeBSD.org) Message-Id: <201808041240.w74CexgP065348@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wulf set sender to wulf@FreeBSD.org using -f From: Vladimir Kondratyev Date: Sat, 4 Aug 2018 12:40:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337290 - stable/11/sys/dev/evdev X-SVN-Group: stable-11 X-SVN-Commit-Author: wulf X-SVN-Commit-Paths: stable/11/sys/dev/evdev X-SVN-Commit-Revision: 337290 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:41:00 -0000 Author: wulf Date: Sat Aug 4 12:40:59 2018 New Revision: 337290 URL: https://svnweb.freebsd.org/changeset/base/337290 Log: MFC r334555: [evdev] Sync event codes with Linux kernel 4.16 Modified: stable/11/sys/dev/evdev/input-event-codes.h stable/11/sys/dev/evdev/input.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/evdev/input-event-codes.h ============================================================================== --- stable/11/sys/dev/evdev/input-event-codes.h Sat Aug 4 12:31:19 2018 (r337289) +++ stable/11/sys/dev/evdev/input-event-codes.h Sat Aug 4 12:40:59 2018 (r337290) @@ -421,6 +421,7 @@ #define BTN_TOOL_MOUSE 0x146 #define BTN_TOOL_LENS 0x147 #define BTN_TOOL_QUINTTAP 0x148 /* Five fingers on trackpad */ +#define BTN_STYLUS3 0x149 #define BTN_TOUCH 0x14a #define BTN_STYLUS 0x14b #define BTN_STYLUS2 0x14c @@ -607,6 +608,7 @@ #define BTN_DPAD_RIGHT 0x223 #define KEY_ALS_TOGGLE 0x230 /* Ambient light sensor */ +#define KEY_ROTATE_LOCK_TOGGLE 0x231 /* Display rotation lock */ #define KEY_BUTTONCONFIG 0x240 /* AL Button Configuration */ #define KEY_TASKMANAGER 0x241 /* AL Task/Project Manager */ @@ -615,6 +617,7 @@ #define KEY_APPSELECT 0x244 /* AL Select Task/Application */ #define KEY_SCREENSAVER 0x245 /* AL Screen Saver */ #define KEY_VOICECOMMAND 0x246 /* Listening Voice Command */ +#define KEY_ASSISTANT 0x247 /* AL Context-aware desktop assistant */ #define KEY_BRIGHTNESS_MIN 0x250 /* Set Brightness to Minimum */ #define KEY_BRIGHTNESS_MAX 0x251 /* Set Brightness to Maximum */ @@ -626,6 +629,38 @@ #define KEY_KBDINPUTASSIST_ACCEPT 0x264 #define KEY_KBDINPUTASSIST_CANCEL 0x265 +/* Diagonal movement keys */ +#define KEY_RIGHT_UP 0x266 +#define KEY_RIGHT_DOWN 0x267 +#define KEY_LEFT_UP 0x268 +#define KEY_LEFT_DOWN 0x269 + +#define KEY_ROOT_MENU 0x26a /* Show Device's Root Menu */ +/* Show Top Menu of the Media (e.g. DVD) */ +#define KEY_MEDIA_TOP_MENU 0x26b +#define KEY_NUMERIC_11 0x26c +#define KEY_NUMERIC_12 0x26d +/* + * Toggle Audio Description: refers to an audio service that helps blind and + * visually impaired consumers understand the action in a program. Note: in + * some countries this is referred to as "Video Description". + */ +#define KEY_AUDIO_DESC 0x26e +#define KEY_3D_MODE 0x26f +#define KEY_NEXT_FAVORITE 0x270 +#define KEY_STOP_RECORD 0x271 +#define KEY_PAUSE_RECORD 0x272 +#define KEY_VOD 0x273 /* Video on Demand */ +#define KEY_UNMUTE 0x274 +#define KEY_FASTREVERSE 0x275 +#define KEY_SLOWREVERSE 0x276 +/* + * Control a data application associated with the currently viewed channel, + * e.g. teletext or data broadcast application (MHEG, MHP, HbbTV, etc.) + */ +#define KEY_DATA 0x277 +#define KEY_ONSCREEN_KEYBOARD 0x278 + #define BTN_TRIGGER_HAPPY 0x2c0 #define BTN_TRIGGER_HAPPY1 0x2c0 #define BTN_TRIGGER_HAPPY2 0x2c1 @@ -763,6 +798,7 @@ #define SW_ROTATE_LOCK 0x0c /* set = rotate locked/disabled */ #define SW_LINEIN_INSERT 0x0d /* set = inserted */ #define SW_MUTE_DEVICE 0x0e /* set = device disabled */ +#define SW_PEN_INSERTED 0x0f /* set = pen inserted */ #define SW_MAX 0x0f #define SW_CNT (SW_MAX+1) Modified: stable/11/sys/dev/evdev/input.h ============================================================================== --- stable/11/sys/dev/evdev/input.h Sat Aug 4 12:31:19 2018 (r337289) +++ stable/11/sys/dev/evdev/input.h Sat Aug 4 12:40:59 2018 (r337290) @@ -38,6 +38,9 @@ #include "input-event-codes.h" +#define input_event_sec time.tv_sec +#define input_event_usec time.tv_usec + struct input_event { struct timeval time; uint16_t type; @@ -138,6 +141,9 @@ struct input_keymap_entry { #define BUS_GSC 0x1A #define BUS_ATARI 0x1B #define BUS_SPI 0x1C +#define BUS_RMI 0x1D +#define BUS_CEC 0x1E +#define BUS_INTEL_ISHTP 0x1F /* * MT_TOOL types From owner-svn-src-all@freebsd.org Sat Aug 4 12:45:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F880106A19A; Sat, 4 Aug 2018 12:45:38 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33EB3743A4; Sat, 4 Aug 2018 12:45:38 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EADD77A7A; Sat, 4 Aug 2018 12:45:37 +0000 (UTC) (envelope-from wulf@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CjbQZ067370; Sat, 4 Aug 2018 12:45:37 GMT (envelope-from wulf@FreeBSD.org) Received: (from wulf@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74Cjbb9067369; Sat, 4 Aug 2018 12:45:37 GMT (envelope-from wulf@FreeBSD.org) Message-Id: <201808041245.w74Cjbb9067369@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: wulf set sender to wulf@FreeBSD.org using -f From: Vladimir Kondratyev Date: Sat, 4 Aug 2018 12:45:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337291 - stable/11/sys/dev/usb X-SVN-Group: stable-11 X-SVN-Commit-Author: wulf X-SVN-Commit-Paths: stable/11/sys/dev/usb X-SVN-Commit-Revision: 337291 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:45:38 -0000 Author: wulf Date: Sat Aug 4 12:45:37 2018 New Revision: 337291 URL: https://svnweb.freebsd.org/changeset/base/337291 Log: MFC r336577: Raise "Report Count" value limit in HID descriptor parser. This fixes usage/report size calculation of Microsoft`s "Touch Hardware Quality Assurance" certificate blob found in many touchscreens. While here, join several "c->flags = dval" lines in to single line. Reviewed by: hselasky Differential Revision: https://reviews.freebsd.org/D16357 Modified: stable/11/sys/dev/usb/usb_hid.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/usb/usb_hid.c ============================================================================== --- stable/11/sys/dev/usb/usb_hid.c Sat Aug 4 12:40:59 2018 (r337290) +++ stable/11/sys/dev/usb/usb_hid.c Sat Aug 4 12:45:37 2018 (r337291) @@ -72,6 +72,7 @@ static uint8_t hid_get_byte(struct hid_data *s, const #define MAXUSAGE 64 #define MAXPUSH 4 #define MAXID 16 +#define MAXLOCCNT 1024 struct hid_pos_data { int32_t rid; @@ -89,10 +90,10 @@ struct hid_data { int32_t usage_last; /* last seen usage */ uint32_t loc_size; /* last seen size */ uint32_t loc_count; /* last seen count */ + uint32_t ncount; /* end usage item count */ + uint32_t icount; /* current usage item count */ uint8_t kindset; /* we have 5 kinds so 8 bits are enough */ uint8_t pushlevel; /* current pushlevel */ - uint8_t ncount; /* end usage item count */ - uint8_t icount; /* current usage item count */ uint8_t nusage; /* end "usages_min/max" index */ uint8_t iusage; /* current "usages_min/max" index */ uint8_t ousage; /* current "usages_min/max" offset */ @@ -345,18 +346,19 @@ hid_get_item(struct hid_data *s, struct hid_item *h) switch (bTag) { case 8: /* Input */ c->kind = hid_input; - c->flags = dval; ret: + c->flags = dval; c->loc.count = s->loc_count; c->loc.size = s->loc_size; if (c->flags & HIO_VARIABLE) { /* range check usage count */ - if (c->loc.count > 255) { + if (c->loc.count > MAXLOCCNT) { DPRINTFN(0, "Number of " - "items(%u) truncated to 255\n", - (unsigned)(c->loc.count)); - s->ncount = 255; + "items(%u) truncated to %u\n", + (unsigned)(c->loc.count), + MAXLOCCNT); + s->ncount = MAXLOCCNT; } else s->ncount = c->loc.count; @@ -372,7 +374,6 @@ hid_get_item(struct hid_data *s, struct hid_item *h) case 9: /* Output */ c->kind = hid_output; - c->flags = dval; goto ret; case 10: /* Collection */ c->kind = hid_collection; @@ -383,7 +384,6 @@ hid_get_item(struct hid_data *s, struct hid_item *h) return (1); case 11: /* Feature */ c->kind = hid_feature; - c->flags = dval; goto ret; case 12: /* End collection */ c->kind = hid_endcollection; From owner-svn-src-all@freebsd.org Sat Aug 4 12:52:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 070A1106A45F; Sat, 4 Aug 2018 12:52:16 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF439747EA; Sat, 4 Aug 2018 12:52:15 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 776BA7C03; Sat, 4 Aug 2018 12:52:15 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CqF9T072747; Sat, 4 Aug 2018 12:52:15 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CqFpO072746; Sat, 4 Aug 2018 12:52:15 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041252.w74CqFpO072746@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 12:52:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337292 - vendor/llvm/dist-release_70 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/llvm/dist-release_70 X-SVN-Commit-Revision: 337292 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:52:16 -0000 Author: dim Date: Sat Aug 4 12:52:15 2018 New Revision: 337292 URL: https://svnweb.freebsd.org/changeset/base/337292 Log: Branch vendor/llvm/dist to vendor/llvm/dist-release_70, to allow for independent merges of the upstream trunk and release_70 branches. Added: vendor/llvm/dist-release_70/ - copied from r337291, vendor/llvm/dist/ From owner-svn-src-all@freebsd.org Sat Aug 4 12:52:21 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B5A1106A4A5; Sat, 4 Aug 2018 12:52:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E77A6748C9; Sat, 4 Aug 2018 12:52:20 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CB8AB7C07; Sat, 4 Aug 2018 12:52:18 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CqI0i072793; Sat, 4 Aug 2018 12:52:18 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CqIZG072792; Sat, 4 Aug 2018 12:52:18 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041252.w74CqIZG072792@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 12:52:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337293 - vendor/clang/dist-release_70 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/clang/dist-release_70 X-SVN-Commit-Revision: 337293 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:52:21 -0000 Author: dim Date: Sat Aug 4 12:52:18 2018 New Revision: 337293 URL: https://svnweb.freebsd.org/changeset/base/337293 Log: Branch vendor/clang/dist to vendor/clang/dist-release_70, to allow for independent merges of the upstream trunk and release_70 branches. Added: vendor/clang/dist-release_70/ - copied from r337292, vendor/clang/dist/ From owner-svn-src-all@freebsd.org Sat Aug 4 12:52:22 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E83B106A4AB; Sat, 4 Aug 2018 12:52:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09517748F6; Sat, 4 Aug 2018 12:52:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C0E3A7C0E; Sat, 4 Aug 2018 12:52:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CqLx6072845; Sat, 4 Aug 2018 12:52:21 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CqLPg072844; Sat, 4 Aug 2018 12:52:21 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041252.w74CqLPg072844@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 12:52:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337294 - vendor/compiler-rt/dist-release_70 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/compiler-rt/dist-release_70 X-SVN-Commit-Revision: 337294 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:52:22 -0000 Author: dim Date: Sat Aug 4 12:52:21 2018 New Revision: 337294 URL: https://svnweb.freebsd.org/changeset/base/337294 Log: Branch vendor/compiler-rt/dist to vendor/compiler-rt/dist-release_70, to allow for independent merges of the upstream trunk and release_70 branches. Added: vendor/compiler-rt/dist-release_70/ - copied from r337293, vendor/compiler-rt/dist/ From owner-svn-src-all@freebsd.org Sat Aug 4 12:52:26 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74B06106A4EA; Sat, 4 Aug 2018 12:52:26 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 987CB749B6; Sat, 4 Aug 2018 12:52:25 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EAD7B7C14; Sat, 4 Aug 2018 12:52:24 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CqO92072892; Sat, 4 Aug 2018 12:52:24 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CqOrT072891; Sat, 4 Aug 2018 12:52:24 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041252.w74CqOrT072891@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 12:52:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337295 - vendor/libc++/dist-release_70 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/libc++/dist-release_70 X-SVN-Commit-Revision: 337295 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:52:26 -0000 Author: dim Date: Sat Aug 4 12:52:24 2018 New Revision: 337295 URL: https://svnweb.freebsd.org/changeset/base/337295 Log: Branch vendor/libc++/dist to vendor/libc++/dist-release_70, to allow for independent merges of the upstream trunk and release_70 branches. Added: vendor/libc++/dist-release_70/ - copied from r337294, vendor/libc++/dist/ From owner-svn-src-all@freebsd.org Sat Aug 4 12:52:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EF54106A50D; Sat, 4 Aug 2018 12:52:30 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48DB874A3B; Sat, 4 Aug 2018 12:52:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 471AC7C17; Sat, 4 Aug 2018 12:52:28 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CqRQq072941; Sat, 4 Aug 2018 12:52:27 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CqRDq072940; Sat, 4 Aug 2018 12:52:27 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041252.w74CqRDq072940@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 12:52:27 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337296 - vendor/lld/dist-release_70 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lld/dist-release_70 X-SVN-Commit-Revision: 337296 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:52:30 -0000 Author: dim Date: Sat Aug 4 12:52:27 2018 New Revision: 337296 URL: https://svnweb.freebsd.org/changeset/base/337296 Log: Branch vendor/lld/dist to vendor/lld/dist-release_70, to allow for independent merges of the upstream trunk and release_70 branches. Added: vendor/lld/dist-release_70/ - copied from r337295, vendor/lld/dist/ From owner-svn-src-all@freebsd.org Sat Aug 4 12:52:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D35C106A530; Sat, 4 Aug 2018 12:52:37 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D87A674AE2; Sat, 4 Aug 2018 12:52:31 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 442017C1F; Sat, 4 Aug 2018 12:52:31 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74CqVLc072991; Sat, 4 Aug 2018 12:52:31 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74CqVM2072990; Sat, 4 Aug 2018 12:52:31 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041252.w74CqVM2072990@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 12:52:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337297 - vendor/lldb/dist-release_70 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lldb/dist-release_70 X-SVN-Commit-Revision: 337297 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 12:52:37 -0000 Author: dim Date: Sat Aug 4 12:52:30 2018 New Revision: 337297 URL: https://svnweb.freebsd.org/changeset/base/337297 Log: Branch vendor/lldb/dist to vendor/lldb/dist-release_70, to allow for independent merges of the upstream trunk and release_70 branches. Added: vendor/lldb/dist-release_70/ - copied from r337296, vendor/lldb/dist/ From owner-svn-src-all@freebsd.org Sat Aug 4 13:03:31 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A26B0106A96F; Sat, 4 Aug 2018 13:03:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 20B257539D; Sat, 4 Aug 2018 13:03:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w74D3Fba030919 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 4 Aug 2018 16:03:18 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w74D3Fba030919 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w74D3Fca030918; Sat, 4 Aug 2018 16:03:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 4 Aug 2018 16:03:15 +0300 From: Konstantin Belousov To: John Baldwin Cc: Justin Hibbits , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r337273 - head/sys/dev/nvme Message-ID: <20180804130315.GK6049@kib.kiev.ua> References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:03:32 -0000 On Sat, Aug 04, 2018 at 05:14:31AM -0700, John Baldwin wrote: > On 8/4/18 1:08 AM, Konstantin Belousov wrote: > > On Fri, Aug 03, 2018 at 08:04:06PM +0000, Justin Hibbits wrote: > >> Author: jhibbits > >> Date: Fri Aug 3 20:04:06 2018 > >> New Revision: 337273 > >> URL: https://svnweb.freebsd.org/changeset/base/337273 > >> > >> Log: > >> nvme(4): Add bus_dmamap_sync() at the end of the request path > >> > >> Summary: > >> Some architectures, in this case powerpc64, need explicit synchronization > >> barriers vs device accesses. > >> > >> Prior to this change, when running 'make buildworld -j72' on a 18-core > >> (72-thread) POWER9, I would see controller resets often. With this change, I > >> don't see these resets messages, though another tester still does, for yet to be > >> determined reasons, so this may not be a complete fix. Additionally, I see a > >> ~5-10% speed up in buildworld times, likely due to not needing to reset the > >> controller. > >> > >> Reviewed By: jimharris > >> Differential Revision: https://reviews.freebsd.org/D16570 > >> > >> Modified: > >> head/sys/dev/nvme/nvme_qpair.c > >> > >> Modified: head/sys/dev/nvme/nvme_qpair.c > >> ============================================================================== > >> --- head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 19:24:04 2018 (r337272) > >> +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 20:04:06 2018 (r337273) > >> @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct nvme_qpair *qpair, > >> req->retries++; > >> nvme_qpair_submit_tracker(qpair, tr); > >> } else { > >> - if (req->type != NVME_REQUEST_NULL) > >> + if (req->type != NVME_REQUEST_NULL) { > >> + bus_dmamap_sync(qpair->dma_tag_payload, > >> + tr->payload_dma_map, > >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > >> bus_dmamap_unload(qpair->dma_tag_payload, > >> tr->payload_dma_map); > >> + } > >> > >> nvme_free_request(req); > >> tr->req = NULL; > >> @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct nvme_qpair *qpai > >> */ > >> return (false); > >> > >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > >> while (1) { > >> cpl = qpair->cpl[qpair->cq_head]; > >> > >> @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair *qpair, st > >> if (++qpair->sq_tail == qpair->num_entries) > >> qpair->sq_tail = 0; > >> > >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > >> + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); > >> +#ifndef __powerpc__ > >> + /* > >> + * powerpc's bus_dmamap_sync() already includes a heavyweight sync, but > >> + * no other archs do. > >> + */ > >> wmb(); > >> +#endif > > What is the purpose of this call ? It is useless without paired read > > barrier. So where is the reciprocal rmb() ? > > For DMA, the rmb is in the device controller. However, architectures > that need this kind of ordering should do it in their bus_dmmap_sync op, > and this explicit one needs to be removed. (Alpha had a wmb in its > bus_dmamap_sync op for this reason.) Yes, if something special is needed, it should happen in platform-specific busdma code. Also, if wmb() is needed, then it is not a supposed semantic or wmb(), but a specific side-effects of one of the instruction in the implementation of wmb(). As I noted, on x86 it is not needed and detrimental to the performance. From owner-svn-src-all@freebsd.org Sat Aug 4 13:08:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BED7B106AB38; Sat, 4 Aug 2018 13:08:37 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 74543756B5; Sat, 4 Aug 2018 13:08:37 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 557C67DC5; Sat, 4 Aug 2018 13:08:37 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74D8bEd078224; Sat, 4 Aug 2018 13:08:37 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74D8b9i078223; Sat, 4 Aug 2018 13:08:37 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041308.w74D8b9i078223@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:08:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337298 - vendor/libcxxrt/dist X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/libcxxrt/dist X-SVN-Commit-Revision: 337298 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:08:37 -0000 Author: dim Date: Sat Aug 4 13:08:36 2018 New Revision: 337298 URL: https://svnweb.freebsd.org/changeset/base/337298 Log: Remove FreeBSD keyword properties. Modified: Directory Properties: vendor/libcxxrt/dist/CMakeLists.txt (props changed) vendor/libcxxrt/dist/abi_namespace.h (props changed) vendor/libcxxrt/dist/auxhelper.cc (props changed) vendor/libcxxrt/dist/cxxabi.h (props changed) vendor/libcxxrt/dist/dwarf_eh.h (props changed) vendor/libcxxrt/dist/dynamic_cast.cc (props changed) vendor/libcxxrt/dist/exception.cc (props changed) vendor/libcxxrt/dist/guard.cc (props changed) vendor/libcxxrt/dist/libelftc_dem_gnu3.c (props changed) vendor/libcxxrt/dist/memory.cc (props changed) vendor/libcxxrt/dist/stdexcept.cc (props changed) vendor/libcxxrt/dist/stdexcept.h (props changed) vendor/libcxxrt/dist/terminate.cc (props changed) vendor/libcxxrt/dist/typeinfo.cc (props changed) vendor/libcxxrt/dist/typeinfo.h (props changed) From owner-svn-src-all@freebsd.org Sat Aug 4 13:15:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A105106AE92; Sat, 4 Aug 2018 13:15:12 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B295C75C3C; Sat, 4 Aug 2018 13:15:11 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7CAA17F5C; Sat, 4 Aug 2018 13:15:11 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DFBLQ083305; Sat, 4 Aug 2018 13:15:11 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DFBC6083304; Sat, 4 Aug 2018 13:15:11 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041315.w74DFBC6083304@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:15:11 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337300 - vendor/llvm/llvm-release_70-r338892 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/llvm/llvm-release_70-r338892 X-SVN-Commit-Revision: 337300 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:15:12 -0000 Author: dim Date: Sat Aug 4 13:15:11 2018 New Revision: 337300 URL: https://svnweb.freebsd.org/changeset/base/337300 Log: Tag llvm release_70 branch r338892. Added: vendor/llvm/llvm-release_70-r338892/ - copied from r337299, vendor/llvm/dist-release_70/ From owner-svn-src-all@freebsd.org Sat Aug 4 13:15:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD2FB106AE86; Sat, 4 Aug 2018 13:15:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F20975C2E; Sat, 4 Aug 2018 13:15:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4F5087F5B; Sat, 4 Aug 2018 13:15:08 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DF8bx083258; Sat, 4 Aug 2018 13:15:08 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DF61w083248; Sat, 4 Aug 2018 13:15:06 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041315.w74DF61w083248@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:15:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337299 - in vendor/llvm/dist-release_70: . docs include/llvm/Support lib/Support lib/Target/AArch64 lib/Target/PowerPC lib/Target/X86 test/CodeGen/AArch64 test/CodeGen/PowerPC test/Cod... X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/llvm/dist-release_70: . docs include/llvm/Support lib/Support lib/Target/AArch64 lib/Target/PowerPC lib/Target/X86 test/CodeGen/AArch64 test/CodeGen/PowerPC test/CodeGen/X86 test/tools/llvm-... X-SVN-Commit-Revision: 337299 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:15:09 -0000 Author: dim Date: Sat Aug 4 13:15:05 2018 New Revision: 337299 URL: https://svnweb.freebsd.org/changeset/base/337299 Log: Vendor import of llvm release_70 branch r338892: https://llvm.org/svn/llvm-project/llvm/branches/release_70@338892 Added: vendor/llvm/dist-release_70/test/CodeGen/X86/absolute-bit-mask-fastisel.ll Modified: vendor/llvm/dist-release_70/CMakeLists.txt vendor/llvm/dist-release_70/docs/ReleaseNotes.rst vendor/llvm/dist-release_70/include/llvm/Support/DebugCounter.h vendor/llvm/dist-release_70/lib/Support/DebugCounter.cpp vendor/llvm/dist-release_70/lib/Target/AArch64/AArch64InstrFormats.td vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCISelLowering.cpp vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCInstrVSX.td vendor/llvm/dist-release_70/lib/Target/X86/X86FastISel.cpp vendor/llvm/dist-release_70/test/CodeGen/AArch64/f16-instructions.ll vendor/llvm/dist-release_70/test/CodeGen/PowerPC/build-vector-tests.ll vendor/llvm/dist-release_70/test/tools/llvm-ar/invalid-command-line.test vendor/llvm/dist-release_70/tools/llvm-ar/llvm-ar.cpp vendor/llvm/dist-release_70/tools/llvm-shlib/CMakeLists.txt vendor/llvm/dist-release_70/utils/release/tag.sh Modified: vendor/llvm/dist-release_70/CMakeLists.txt ============================================================================== --- vendor/llvm/dist-release_70/CMakeLists.txt Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/CMakeLists.txt Sat Aug 4 13:15:05 2018 (r337299) @@ -32,7 +32,7 @@ if(NOT DEFINED LLVM_VERSION_PATCH) set(LLVM_VERSION_PATCH 0) endif() if(NOT DEFINED LLVM_VERSION_SUFFIX) - set(LLVM_VERSION_SUFFIX svn) + set(LLVM_VERSION_SUFFIX "") endif() if (NOT PACKAGE_VERSION) @@ -556,8 +556,6 @@ if(LLVM_LINK_LLVM_DYLIB OR LLVM_BUILD_LLVM_C_DYLIB) set(LLVM_BUILD_LLVM_DYLIB_default ON) endif() option(LLVM_BUILD_LLVM_DYLIB "Build libllvm dynamic library" ${LLVM_BUILD_LLVM_DYLIB_default}) - -option(LLVM_DYLIB_SYMBOL_VERSIONING OFF) option(LLVM_OPTIMIZED_TABLEGEN "Force TableGen to be built with optimization" OFF) if(CMAKE_CROSSCOMPILING OR (LLVM_OPTIMIZED_TABLEGEN AND (LLVM_ENABLE_ASSERTIONS OR CMAKE_CONFIGURATION_TYPES))) Modified: vendor/llvm/dist-release_70/docs/ReleaseNotes.rst ============================================================================== --- vendor/llvm/dist-release_70/docs/ReleaseNotes.rst Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/docs/ReleaseNotes.rst Sat Aug 4 13:15:05 2018 (r337299) @@ -65,7 +65,7 @@ Non-comprehensive list of changes in this release results for code that is relying on the undefined behavior of overflowing casts. The optimization can be disabled by specifying a function attribute: "strict-float-cast-overflow"="false". This attribute may be created by the - clang option :option:`-fno-strict-float-cast-overflow`. + clang option ``-fno-strict-float-cast-overflow``. Code sanitizers can be used to detect affected patterns. The option for detecting this problem alone is "-fsanitize=float-cast-overflow": @@ -108,6 +108,12 @@ Non-comprehensive list of changes in this release implementations saddle std::vector with and is better at using ``realloc``, it's now a better choice even on the heap (although when TinyPtrVector works, it's even smaller). + +* Preliminary/experimental support for DWARF v5 debugging information, + including the new .debug_names accelerator table. DWARF emitted at ``-O0`` + should be fully DWARF v5 compliant. Type units and split DWARF are known + not to be compliant, and higher optimization levels will still emit some + information in v4 format. * Note.. Modified: vendor/llvm/dist-release_70/include/llvm/Support/DebugCounter.h ============================================================================== --- vendor/llvm/dist-release_70/include/llvm/Support/DebugCounter.h Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/include/llvm/Support/DebugCounter.h Sat Aug 4 13:15:05 2018 (r337299) @@ -70,10 +70,9 @@ class DebugCounter { (public) return instance().addCounter(Name, Desc); } inline static bool shouldExecute(unsigned CounterName) { -// Compile to nothing when debugging is off -#ifdef NDEBUG - return true; -#else + if (!isCountingEnabled()) + return true; + auto &Us = instance(); auto Result = Us.Counters.find(CounterName); if (Result != Us.Counters.end()) { @@ -93,7 +92,6 @@ class DebugCounter { (public) } // Didn't find the counter, should we warn? return true; -#endif // NDEBUG } // Return true if a given counter had values set (either programatically or on @@ -142,7 +140,23 @@ class DebugCounter { (public) } CounterVector::const_iterator end() const { return RegisteredCounters.end(); } + // Force-enables counting all DebugCounters. + // + // Since DebugCounters are incompatible with threading (not only do they not + // make sense, but we'll also see data races), this should only be used in + // contexts where we're certain we won't spawn threads. + static void enableAllCounters() { instance().Enabled = true; } + private: + static bool isCountingEnabled() { +// Compile to nothing when debugging is off +#ifdef NDEBUG + return false; +#else + return instance().Enabled; +#endif + } + unsigned addCounter(const std::string &Name, const std::string &Desc) { unsigned Result = RegisteredCounters.insert(Name); Counters[Result] = {}; @@ -159,6 +173,10 @@ class DebugCounter { (public) }; DenseMap Counters; CounterVector RegisteredCounters; + + // Whether we should do DebugCounting at all. DebugCounters aren't + // thread-safe, so this should always be false in multithreaded scenarios. + bool Enabled = false; }; #define DEBUG_COUNTER(VARNAME, COUNTERNAME, DESC) \ Modified: vendor/llvm/dist-release_70/lib/Support/DebugCounter.cpp ============================================================================== --- vendor/llvm/dist-release_70/lib/Support/DebugCounter.cpp Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/lib/Support/DebugCounter.cpp Sat Aug 4 13:15:05 2018 (r337299) @@ -82,6 +82,7 @@ void DebugCounter::push_back(const std::string &Val) { << " is not a registered counter\n"; return; } + enableAllCounters(); Counters[CounterID].Skip = CounterVal; Counters[CounterID].IsSet = true; } else if (CounterPair.first.endswith("-count")) { @@ -92,6 +93,7 @@ void DebugCounter::push_back(const std::string &Val) { << " is not a registered counter\n"; return; } + enableAllCounters(); Counters[CounterID].StopAfter = CounterVal; Counters[CounterID].IsSet = true; } else { Modified: vendor/llvm/dist-release_70/lib/Target/AArch64/AArch64InstrFormats.td ============================================================================== --- vendor/llvm/dist-release_70/lib/Target/AArch64/AArch64InstrFormats.td Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/lib/Target/AArch64/AArch64InstrFormats.td Sat Aug 4 13:15:05 2018 (r337299) @@ -4639,7 +4639,9 @@ class BaseFPCondComparison { - def Hrr : BaseFPCondComparison { + def Hrr : BaseFPCondComparison { let Inst{23-22} = 0b11; let Predicates = [HasFullFP16]; } Modified: vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCISelLowering.cpp ============================================================================== --- vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCISelLowering.cpp Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCISelLowering.cpp Sat Aug 4 13:15:05 2018 (r337299) @@ -11761,6 +11761,14 @@ SDValue PPCTargetLowering::DAGCombineExtBoolTrunc(SDNo ShiftCst); } +// Is this an extending load from an f32 to an f64? +static bool isFPExtLoad(SDValue Op) { + if (LoadSDNode *LD = dyn_cast(Op.getNode())) + return LD->getExtensionType() == ISD::EXTLOAD && + Op.getValueType() == MVT::f64; + return false; +} + /// Reduces the number of fp-to-int conversion when building a vector. /// /// If this vector is built out of floating to integer conversions, @@ -11795,11 +11803,18 @@ combineElementTruncationToVectorTruncation(SDNode *N, SmallVector Ops; EVT TargetVT = N->getValueType(0); for (int i = 0, e = N->getNumOperands(); i < e; ++i) { - if (N->getOperand(i).getOpcode() != PPCISD::MFVSR) + SDValue NextOp = N->getOperand(i); + if (NextOp.getOpcode() != PPCISD::MFVSR) return SDValue(); - unsigned NextConversion = N->getOperand(i).getOperand(0).getOpcode(); + unsigned NextConversion = NextOp.getOperand(0).getOpcode(); if (NextConversion != FirstConversion) return SDValue(); + // If we are converting to 32-bit integers, we need to add an FP_ROUND. + // This is not valid if the input was originally double precision. It is + // also not profitable to do unless this is an extending load in which + // case doing this combine will allow us to combine consecutive loads. + if (Is32Bit && !isFPExtLoad(NextOp.getOperand(0).getOperand(0))) + return SDValue(); if (N->getOperand(i) != FirstInput) IsSplat = false; } @@ -11813,8 +11828,9 @@ combineElementTruncationToVectorTruncation(SDNode *N, // Now that we know we have the right type of node, get its operands for (int i = 0, e = N->getNumOperands(); i < e; ++i) { SDValue In = N->getOperand(i).getOperand(0); - // For 32-bit values, we need to add an FP_ROUND node. if (Is32Bit) { + // For 32-bit values, we need to add an FP_ROUND node (if we made it + // here, we know that all inputs are extending loads so this is safe). if (In.isUndef()) Ops.push_back(DAG.getUNDEF(SrcVT)); else { Modified: vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCInstrVSX.td ============================================================================== --- vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCInstrVSX.td Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/lib/Target/PowerPC/PPCInstrVSX.td Sat Aug 4 13:15:05 2018 (r337299) @@ -3494,6 +3494,17 @@ def DblToFlt { dag B1 = (f32 (fpround (f64 (extractelt v2f64:$B, 1)))); } +def ExtDbl { + dag A0S = (i32 (PPCmfvsr (f64 (PPCfctiwz (f64 (extractelt v2f64:$A, 0)))))); + dag A1S = (i32 (PPCmfvsr (f64 (PPCfctiwz (f64 (extractelt v2f64:$A, 1)))))); + dag B0S = (i32 (PPCmfvsr (f64 (PPCfctiwz (f64 (extractelt v2f64:$B, 0)))))); + dag B1S = (i32 (PPCmfvsr (f64 (PPCfctiwz (f64 (extractelt v2f64:$B, 1)))))); + dag A0U = (i32 (PPCmfvsr (f64 (PPCfctiwuz (f64 (extractelt v2f64:$A, 0)))))); + dag A1U = (i32 (PPCmfvsr (f64 (PPCfctiwuz (f64 (extractelt v2f64:$A, 1)))))); + dag B0U = (i32 (PPCmfvsr (f64 (PPCfctiwuz (f64 (extractelt v2f64:$B, 0)))))); + dag B1U = (i32 (PPCmfvsr (f64 (PPCfctiwuz (f64 (extractelt v2f64:$B, 1)))))); +} + def ByteToWord { dag LE_A0 = (i32 (sext_inreg (i32 (vector_extract v16i8:$A, 0)), i8)); dag LE_A1 = (i32 (sext_inreg (i32 (vector_extract v16i8:$A, 4)), i8)); @@ -3571,9 +3582,15 @@ def FltToULong { } def DblToInt { dag A = (i32 (PPCmfvsr (f64 (PPCfctiwz f64:$A)))); + dag B = (i32 (PPCmfvsr (f64 (PPCfctiwz f64:$B)))); + dag C = (i32 (PPCmfvsr (f64 (PPCfctiwz f64:$C)))); + dag D = (i32 (PPCmfvsr (f64 (PPCfctiwz f64:$D)))); } def DblToUInt { dag A = (i32 (PPCmfvsr (f64 (PPCfctiwuz f64:$A)))); + dag B = (i32 (PPCmfvsr (f64 (PPCfctiwuz f64:$B)))); + dag C = (i32 (PPCmfvsr (f64 (PPCfctiwuz f64:$C)))); + dag D = (i32 (PPCmfvsr (f64 (PPCfctiwuz f64:$D)))); } def DblToLong { dag A = (i64 (PPCmfvsr (f64 (PPCfctidz f64:$A)))); @@ -3612,6 +3629,47 @@ def MrgFP { dag BAlToFlt = (XVCVDPSP (XXPERMDI $B, $A, 3)); } +// Word-element merge dags - conversions from f64 to i32 merged into vectors. +def MrgWords { + // For big endian, we merge low and hi doublewords (A, B). + dag A0B0 = (v2f64 (XXPERMDI v2f64:$A, v2f64:$B, 0)); + dag A1B1 = (v2f64 (XXPERMDI v2f64:$A, v2f64:$B, 3)); + dag CVA1B1S = (v4i32 (XVCVDPSXWS A1B1)); + dag CVA0B0S = (v4i32 (XVCVDPSXWS A0B0)); + dag CVA1B1U = (v4i32 (XVCVDPUXWS A1B1)); + dag CVA0B0U = (v4i32 (XVCVDPUXWS A0B0)); + + // For little endian, we merge low and hi doublewords (B, A). + dag B1A1 = (v2f64 (XXPERMDI v2f64:$B, v2f64:$A, 0)); + dag B0A0 = (v2f64 (XXPERMDI v2f64:$B, v2f64:$A, 3)); + dag CVB1A1S = (v4i32 (XVCVDPSXWS B1A1)); + dag CVB0A0S = (v4i32 (XVCVDPSXWS B0A0)); + dag CVB1A1U = (v4i32 (XVCVDPUXWS B1A1)); + dag CVB0A0U = (v4i32 (XVCVDPUXWS B0A0)); + + // For big endian, we merge hi doublewords of (A, C) and (B, D), convert + // then merge. + dag AC = (v2f64 (XXPERMDI (COPY_TO_REGCLASS f64:$A, VSRC), + (COPY_TO_REGCLASS f64:$C, VSRC), 0)); + dag BD = (v2f64 (XXPERMDI (COPY_TO_REGCLASS f64:$B, VSRC), + (COPY_TO_REGCLASS f64:$D, VSRC), 0)); + dag CVACS = (v4i32 (XVCVDPSXWS AC)); + dag CVBDS = (v4i32 (XVCVDPSXWS BD)); + dag CVACU = (v4i32 (XVCVDPUXWS AC)); + dag CVBDU = (v4i32 (XVCVDPUXWS BD)); + + // For little endian, we merge hi doublewords of (D, B) and (C, A), convert + // then merge. + dag DB = (v2f64 (XXPERMDI (COPY_TO_REGCLASS f64:$D, VSRC), + (COPY_TO_REGCLASS f64:$B, VSRC), 0)); + dag CA = (v2f64 (XXPERMDI (COPY_TO_REGCLASS f64:$C, VSRC), + (COPY_TO_REGCLASS f64:$A, VSRC), 0)); + dag CVDBS = (v4i32 (XVCVDPSXWS DB)); + dag CVCAS = (v4i32 (XVCVDPSXWS CA)); + dag CVDBU = (v4i32 (XVCVDPUXWS DB)); + dag CVCAU = (v4i32 (XVCVDPUXWS CA)); +} + // Patterns for BUILD_VECTOR nodes. let AddedComplexity = 400 in { @@ -3679,6 +3737,20 @@ let AddedComplexity = 400 in { def : Pat<(v4f32 (build_vector DblToFlt.A0, DblToFlt.A1, DblToFlt.B0, DblToFlt.B1)), (v4f32 (VMRGEW MrgFP.ABhToFlt, MrgFP.ABlToFlt))>; + + // Convert 4 doubles to a vector of ints. + def : Pat<(v4i32 (build_vector DblToInt.A, DblToInt.B, + DblToInt.C, DblToInt.D)), + (v4i32 (VMRGEW MrgWords.CVACS, MrgWords.CVBDS))>; + def : Pat<(v4i32 (build_vector DblToUInt.A, DblToUInt.B, + DblToUInt.C, DblToUInt.D)), + (v4i32 (VMRGEW MrgWords.CVACU, MrgWords.CVBDU))>; + def : Pat<(v4i32 (build_vector ExtDbl.A0S, ExtDbl.A1S, + ExtDbl.B0S, ExtDbl.B1S)), + (v4i32 (VMRGEW MrgWords.CVA0B0S, MrgWords.CVA1B1S))>; + def : Pat<(v4i32 (build_vector ExtDbl.A0U, ExtDbl.A1U, + ExtDbl.B0U, ExtDbl.B1U)), + (v4i32 (VMRGEW MrgWords.CVA0B0U, MrgWords.CVA1B1U))>; } let Predicates = [IsLittleEndian, HasVSX] in { @@ -3693,6 +3765,20 @@ let AddedComplexity = 400 in { def : Pat<(v4f32 (build_vector DblToFlt.A0, DblToFlt.A1, DblToFlt.B0, DblToFlt.B1)), (v4f32 (VMRGEW MrgFP.BAhToFlt, MrgFP.BAlToFlt))>; + + // Convert 4 doubles to a vector of ints. + def : Pat<(v4i32 (build_vector DblToInt.A, DblToInt.B, + DblToInt.C, DblToInt.D)), + (v4i32 (VMRGEW MrgWords.CVDBS, MrgWords.CVCAS))>; + def : Pat<(v4i32 (build_vector DblToUInt.A, DblToUInt.B, + DblToUInt.C, DblToUInt.D)), + (v4i32 (VMRGEW MrgWords.CVDBU, MrgWords.CVCAU))>; + def : Pat<(v4i32 (build_vector ExtDbl.A0S, ExtDbl.A1S, + ExtDbl.B0S, ExtDbl.B1S)), + (v4i32 (VMRGEW MrgWords.CVB1A1S, MrgWords.CVB0A0S))>; + def : Pat<(v4i32 (build_vector ExtDbl.A0U, ExtDbl.A1U, + ExtDbl.B0U, ExtDbl.B1U)), + (v4i32 (VMRGEW MrgWords.CVB1A1U, MrgWords.CVB0A0U))>; } let Predicates = [HasDirectMove] in { Modified: vendor/llvm/dist-release_70/lib/Target/X86/X86FastISel.cpp ============================================================================== --- vendor/llvm/dist-release_70/lib/Target/X86/X86FastISel.cpp Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/lib/Target/X86/X86FastISel.cpp Sat Aug 4 13:15:05 2018 (r337299) @@ -738,6 +738,10 @@ bool X86FastISel::handleConstantAddresses(const Value if (GV->isThreadLocal()) return false; + // Can't handle !absolute_symbol references yet. + if (GV->isAbsoluteSymbolRef()) + return false; + // RIP-relative addresses can't have additional register operands, so if // we've already folded stuff into the addressing mode, just force the // global value into its own register, which we can use as the basereg. Modified: vendor/llvm/dist-release_70/test/CodeGen/AArch64/f16-instructions.ll ============================================================================== --- vendor/llvm/dist-release_70/test/CodeGen/AArch64/f16-instructions.ll Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/test/CodeGen/AArch64/f16-instructions.ll Sat Aug 4 13:15:05 2018 (r337299) @@ -456,6 +456,36 @@ define i1 @test_fcmp_ord(half %a, half %b) #0 { ret i1 %r } +; CHECK-COMMON-LABEL: test_fccmp: +; CHECK-CVT: fcvt s0, h0 +; CHECK-CVT-NEXT: fmov s1, #8.00000000 +; CHECK-CVT-NEXT: fmov s2, #5.00000000 +; CHECK-CVT-NEXT: fcmp s0, s1 +; CHECK-CVT-NEXT: cset w8, gt +; CHECK-CVT-NEXT: fcmp s0, s2 +; CHECK-CVT-NEXT: cset w9, mi +; CHECK-CVT-NEXT: tst w8, w9 +; CHECK-CVT-NEXT: fcsel s0, s0, s2, ne +; CHECK-CVT-NEXT: fcvt h0, s0 +; CHECK-CVT-NEXT: str h0, [x0] +; CHECK-CVT-NEXT: ret +; CHECK-FP16: fmov h1, #5.00000000 +; CHECK-FP16-NEXT: fcmp h0, h1 +; CHECK-FP16-NEXT: fmov h2, #8.00000000 +; CHECK-FP16-NEXT: fccmp h0, h2, #4, mi +; CHECK-FP16-NEXT: fcsel h0, h0, h1, gt +; CHECK-FP16-NEXT: str h0, [x0] +; CHECK-FP16-NEXT: ret + +define void @test_fccmp(half %in, half* %out) { + %cmp1 = fcmp ogt half %in, 0xH4800 + %cmp2 = fcmp olt half %in, 0xH4500 + %cond = and i1 %cmp1, %cmp2 + %result = select i1 %cond, half %in, half 0xH4500 + store half %result, half* %out + ret void +} + ; CHECK-CVT-LABEL: test_br_cc: ; CHECK-CVT-NEXT: fcvt s1, h1 ; CHECK-CVT-NEXT: fcvt s0, h0 Modified: vendor/llvm/dist-release_70/test/CodeGen/PowerPC/build-vector-tests.ll ============================================================================== --- vendor/llvm/dist-release_70/test/CodeGen/PowerPC/build-vector-tests.ll Sat Aug 4 13:08:36 2018 (r337298) +++ vendor/llvm/dist-release_70/test/CodeGen/PowerPC/build-vector-tests.ll Sat Aug 4 13:15:05 2018 (r337299) @@ -119,8 +119,8 @@ ;vector int spltCnstConvftoi() { // ; return (vector int) 4.74f; // ;} // -;// P8: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // -;// P9: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvdpsxws // +;// P8: 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // ;vector int fromRegsConvftoi(float a, float b, float c, float d) { // ; return (vector int) { a, b, c, d }; // ;} // @@ -139,15 +139,15 @@ ;vector int fromDiffMemConsDConvftoi(float *ptr) { // ; return (vector int) { ptr[3], ptr[2], ptr[1], ptr[0] }; // ;} // -;// P8: 4 x lxsspx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // -;// P9: 4 x lxssp, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // +;// P8: 4 x lxsspx, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: 4 x lxssp, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // ;// Note: if the consecutive loads learns to handle pre-inc, this can be: // ;// sldi 2, load, xvcvspuxws // ;vector int fromDiffMemVarAConvftoi(float *arr, int elem) { // ; return (vector int) { arr[elem], arr[elem+1], arr[elem+2], arr[elem+3] }; // ;} // -;// P8: 4 x lxsspx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // -;// P9: 4 x lxssp, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // +;// P8: 4 x lxsspx, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: 4 x lxssp, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // ;// Note: if the consecutive loads learns to handle pre-inc, this can be: // ;// sldi 2, 2 x load, vperm, xvcvspuxws // ;vector int fromDiffMemVarDConvftoi(float *arr, int elem) { // @@ -168,8 +168,8 @@ ;vector int spltCnstConvdtoi() { // ; return (vector int) 4.74; // ;} // -;// P8: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // -;// P9: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // +;// P8: 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // ;vector int fromRegsConvdtoi(double a, double b, double c, double d) { // ; return (vector int) { a, b, c, d }; // ;} // @@ -178,25 +178,23 @@ ;vector int fromDiffConstsConvdtoi() { // ; return (vector int) { 24.46, 234., 988.19, 422.39 }; // ;} // -;// P8: 2 x lxvd2x, 2 x xxswapd, xxmrgld, xxmrghd, 2 x xvcvdpsp, vmrgew, // -;// xvcvspsxws // -;// P9: 2 x lxvx, 2 x xxswapd, xxmrgld, xxmrghd, 2 x xvcvdpsp, vmrgew, // -;// xvcvspsxws // +;// P8: 2 x lxvd2x, 2 x xxswapd, xxmrgld, xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: 2 x lxvx, 2 x xxswapd, xxmrgld, xxmrghd, 2 x xvcvspsxws, vmrgew // ;vector int fromDiffMemConsAConvdtoi(double *ptr) { // ; return (vector int) { ptr[0], ptr[1], ptr[2], ptr[3] }; // ;} // -;// P8: 4 x lxsdx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // -;// P9: 4 x lfd, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // +;// P8: 4 x lxsdx, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: 4 x lfd, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // ;vector int fromDiffMemConsDConvdtoi(double *ptr) { // ; return (vector int) { ptr[3], ptr[2], ptr[1], ptr[0] }; // ;} // -;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // -;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // +;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // ;vector int fromDiffMemVarAConvdtoi(double *arr, int elem) { // ; return (vector int) { arr[elem], arr[elem+1], arr[elem+2], arr[elem+3] }; // ;} // -;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // -;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspsxws // +;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // +;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvspsxws, vmrgew // ;vector int fromDiffMemVarDConvdtoi(double *arr, int elem) { // ; return (vector int) { arr[elem], arr[elem-1], arr[elem-2], arr[elem-3] }; // ;} // @@ -296,8 +294,8 @@ ;vector unsigned int spltCnstConvftoui() { // ; return (vector unsigned int) 4.74f; // ;} // -;// P8: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // -;// P9: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // ;vector unsigned int fromRegsConvftoui(float a, float b, float c, float d) { // ; return (vector unsigned int) { a, b, c, d }; // ;} // @@ -316,16 +314,16 @@ ;vector unsigned int fromDiffMemConsDConvftoui(float *ptr) { // ; return (vector unsigned int) { ptr[3], ptr[2], ptr[1], ptr[0] }; // ;} // -;// P8: lfsux, 3 x lxsspx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // -;// P9: lfsux, 3 x lfs, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: lfsux, 3 x lxsspx, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: lfsux, 3 x lfs, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // ;// Note: if the consecutive loads learns to handle pre-inc, this can be: // ;// sldi 2, load, xvcvspuxws // ;vector unsigned int fromDiffMemVarAConvftoui(float *arr, int elem) { // ; return (vector unsigned int) { arr[elem], arr[elem+1], // ; arr[elem+2], arr[elem+3] }; // ;} // -;// P8: lfsux, 3 x lxsspx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // -;// P9: lfsux, 3 x lfs, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: lfsux, 3 x lxsspx, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: lfsux, 3 x lfs, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // ;// Note: if the consecutive loads learns to handle pre-inc, this can be: // ;// sldi 2, 2 x load, vperm, xvcvspuxws // ;vector unsigned int fromDiffMemVarDConvftoui(float *arr, int elem) { // @@ -347,8 +345,8 @@ ;vector unsigned int spltCnstConvdtoui() { // ; return (vector unsigned int) 4.74; // ;} // -;// P8: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // -;// P9: 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // ;vector unsigned int fromRegsConvdtoui(double a, double b, // ; double c, double d) { // ; return (vector unsigned int) { a, b, c, d }; // @@ -358,25 +356,24 @@ ;vector unsigned int fromDiffConstsConvdtoui() { // ; return (vector unsigned int) { 24.46, 234., 988.19, 422.39 }; // ;} // -;// P8: 2 x lxvd2x, 2 x xxswapd, xxmrgld, xxmrghd, 2 x xvcvdpsp, vmrgew, // -;// xvcvspuxws // -;// P9: 2 x lxvx, xxmrgld, xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: 2 x lxvd2x, 2 x xxswapd, xxmrgld, xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: 2 x lxvx, xxmrgld, xxmrghd, 2 x xvcvspuxws, vmrgew // ;vector unsigned int fromDiffMemConsAConvdtoui(double *ptr) { // ; return (vector unsigned int) { ptr[0], ptr[1], ptr[2], ptr[3] }; // ;} // -;// P8: 4 x lxsdx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // -;// P9: 4 x lfd, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: 4 x lxsdx, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: 4 x lfd, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // ;vector unsigned int fromDiffMemConsDConvdtoui(double *ptr) { // ; return (vector unsigned int) { ptr[3], ptr[2], ptr[1], ptr[0] }; // ;} // -;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // -;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // ;vector unsigned int fromDiffMemVarAConvdtoui(double *arr, int elem) { // ; return (vector unsigned int) { arr[elem], arr[elem+1], // ; arr[elem+2], arr[elem+3] }; // ;} // -;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // -;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvdpsp, vmrgew, xvcvspuxws // +;// P8: lfdux, 3 x lxsdx, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // +;// P9: lfdux, 3 x lfd, 2 x xxmrghd, 2 x xvcvspuxws, vmrgew // ;vector unsigned int fromDiffMemVarDConvdtoui(double *arr, int elem) { // ; return (vector unsigned int) { arr[elem], arr[elem-1], // ; arr[elem-2], arr[elem-3] }; // @@ -1253,28 +1250,24 @@ entry: ; P8LE-LABEL: fromRegsConvftoi ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P9BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9BE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9BE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9BE: vmrgew v2, [[REG3]], [[REG4]] -; P9BE: xvcvspsxws v2, v2 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P9LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9LE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9LE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9LE: vmrgew v2, [[REG4]], [[REG3]] -; P9LE: xvcvspsxws v2, v2 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P8BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8BE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8BE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8BE: vmrgew v2, [[REG3]], [[REG4]] -; P8BE: xvcvspsxws v2, v2 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P8LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8LE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8LE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8LE: vmrgew v2, [[REG4]], [[REG3]] -; P8LE: xvcvspsxws v2, v2 } ; Function Attrs: norecurse nounwind readnone @@ -1529,28 +1522,24 @@ entry: ; P8LE-LABEL: fromRegsConvdtoi ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P9BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9BE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9BE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9BE: vmrgew v2, [[REG3]], [[REG4]] -; P9BE: xvcvspsxws v2, v2 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P9LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9LE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9LE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9LE: vmrgew v2, [[REG4]], [[REG3]] -; P9LE: xvcvspsxws v2, v2 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P8BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8BE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8BE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8BE: vmrgew v2, [[REG3]], [[REG4]] -; P8BE: xvcvspsxws v2, v2 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P8LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8LE-DAG: xvcvdpsxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8LE-DAG: xvcvdpsxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8LE: vmrgew v2, [[REG4]], [[REG3]] -; P8LE: xvcvspsxws v2, v2 } ; Function Attrs: norecurse nounwind readnone @@ -1592,36 +1581,32 @@ entry: ; P9BE-DAG: lxv [[REG2:[vs0-9]+]], 16(r3) ; P9BE-DAG: xxmrgld [[REG3:[vs0-9]+]], [[REG1]], [[REG2]] ; P9BE-DAG: xxmrghd [[REG4:[vs0-9]+]], [[REG1]], [[REG2]] -; P9BE-DAG: xvcvdpsp [[REG5:[vs0-9]+]], [[REG3]] -; P9BE-DAG: xvcvdpsp [[REG6:[vs0-9]+]], [[REG4]] +; P9BE-DAG: xvcvdpsxws [[REG5:[vs0-9]+]], [[REG3]] +; P9BE-DAG: xvcvdpsxws [[REG6:[vs0-9]+]], [[REG4]] ; P9BE: vmrgew v2, [[REG6]], [[REG5]] -; P9BE: xvcvspsxws v2, v2 ; P9LE-DAG: lxv [[REG1:[vs0-9]+]], 0(r3) ; P9LE-DAG: lxv [[REG2:[vs0-9]+]], 16(r3) ; P9LE-DAG: xxmrgld [[REG3:[vs0-9]+]], [[REG2]], [[REG1]] ; P9LE-DAG: xxmrghd [[REG4:[vs0-9]+]], [[REG2]], [[REG1]] -; P9LE-DAG: xvcvdpsp [[REG5:[vs0-9]+]], [[REG3]] -; P9LE-DAG: xvcvdpsp [[REG6:[vs0-9]+]], [[REG4]] +; P9LE-DAG: xvcvdpsxws [[REG5:[vs0-9]+]], [[REG3]] +; P9LE-DAG: xvcvdpsxws [[REG6:[vs0-9]+]], [[REG4]] ; P9LE: vmrgew v2, [[REG6]], [[REG5]] -; P9LE: xvcvspsxws v2, v2 ; P8BE: lxvd2x [[REG1:[vs0-9]+]], 0, r3 ; P8BE: lxvd2x [[REG2:[vs0-9]+]], r3, r4 ; P8BE-DAG: xxmrgld [[REG3:[vs0-9]+]], [[REG1]], [[REG2]] ; P8BE-DAG: xxmrghd [[REG4:[vs0-9]+]], [[REG1]], [[REG2]] -; P8BE-DAG: xvcvdpsp [[REG5:[vs0-9]+]], [[REG3]] -; P8BE-DAG: xvcvdpsp [[REG6:[vs0-9]+]], [[REG4]] +; P8BE-DAG: xvcvdpsxws [[REG5:[vs0-9]+]], [[REG3]] +; P8BE-DAG: xvcvdpsxws [[REG6:[vs0-9]+]], [[REG4]] ; P8BE: vmrgew v2, [[REG6]], [[REG5]] -; P8BE: xvcvspsxws v2, v2 ; P8LE: lxvd2x [[REG1:[vs0-9]+]], 0, r3 ; P8LE: lxvd2x [[REG2:[vs0-9]+]], r3, r4 ; P8LE-DAG: xxswapd [[REG3:[vs0-9]+]], [[REG1]] ; P8LE-DAG: xxswapd [[REG4:[vs0-9]+]], [[REG2]] ; P8LE-DAG: xxmrgld [[REG5:[vs0-9]+]], [[REG4]], [[REG3]] ; P8LE-DAG: xxmrghd [[REG6:[vs0-9]+]], [[REG4]], [[REG3]] -; P8LE-DAG: xvcvdpsp [[REG7:[vs0-9]+]], [[REG5]] -; P8LE-DAG: xvcvdpsp [[REG8:[vs0-9]+]], [[REG6]] +; P8LE-DAG: xvcvdpsxws [[REG7:[vs0-9]+]], [[REG5]] +; P8LE-DAG: xvcvdpsxws [[REG8:[vs0-9]+]], [[REG6]] ; P8LE: vmrgew v2, [[REG8]], [[REG7]] -; P8LE: xvcvspsxws v2, v2 } ; Function Attrs: norecurse nounwind readonly @@ -1653,40 +1638,36 @@ entry: ; P9BE: lfd ; P9BE: xxmrghd ; P9BE: xxmrghd -; P9BE: xvcvdpsp -; P9BE: xvcvdpsp -; P9BE: vmrgew -; P9BE: xvcvspsxws v2 +; P9BE: xvcvdpsxws +; P9BE: xvcvdpsxws +; P9BE: vmrgew v2 ; P9LE: lfd ; P9LE: lfd ; P9LE: lfd ; P9LE: lfd ; P9LE: xxmrghd ; P9LE: xxmrghd -; P9LE: xvcvdpsp -; P9LE: xvcvdpsp -; P9LE: vmrgew -; P9LE: xvcvspsxws v2 +; P9LE: xvcvdpsxws +; P9LE: xvcvdpsxws +; P9LE: vmrgew v2 ; P8BE: lfdx ; P8BE: lfd ; P8BE: lfd ; P8BE: lfd ; P8BE: xxmrghd ; P8BE: xxmrghd -; P8BE: xvcvdpsp -; P8BE: xvcvdpsp -; P8BE: vmrgew -; P8BE: xvcvspsxws v2 +; P8BE: xvcvdpsxws +; P8BE: xvcvdpsxws +; P8BE: vmrgew v2 ; P8LE: lfdx ; P8LE: lfd ; P8LE: lfd ; P8LE: lfd ; P8LE: xxmrghd ; P8LE: xxmrghd -; P8LE: xvcvdpsp -; P8LE: xvcvdpsp -; P8LE: vmrgew -; P8LE: xvcvspsxws v2 +; P8LE: xvcvdpsxws +; P8LE: xvcvdpsxws +; P8LE: vmrgew v2 } ; Function Attrs: norecurse nounwind readonly @@ -1726,40 +1707,36 @@ entry: ; P9BE: lfd ; P9BE: xxmrghd ; P9BE: xxmrghd -; P9BE: xvcvdpsp -; P9BE: xvcvdpsp -; P9BE: vmrgew -; P9BE: xvcvspsxws v2 +; P9BE: xvcvdpsxws +; P9BE: xvcvdpsxws +; P9BE: vmrgew v2 ; P9LE: lfdux ; P9LE: lfd ; P9LE: lfd ; P9LE: lfd ; P9LE: xxmrghd ; P9LE: xxmrghd -; P9LE: xvcvdpsp -; P9LE: xvcvdpsp -; P9LE: vmrgew -; P9LE: xvcvspsxws v2 +; P9LE: xvcvdpsxws +; P9LE: xvcvdpsxws +; P9LE: vmrgew v2 ; P8BE: lfdux ; P8BE: lfd ; P8BE: lfd ; P8BE: lfd ; P8BE: xxmrghd ; P8BE: xxmrghd -; P8BE: xvcvdpsp -; P8BE: xvcvdpsp -; P8BE: vmrgew -; P8BE: xvcvspsxws v2 +; P8BE: xvcvdpsxws +; P8BE: xvcvdpsxws +; P8BE: vmrgew v2 ; P8LE: lfdux ; P8LE: lfd ; P8LE: lfd ; P8LE: lfd ; P8LE: xxmrghd ; P8LE: xxmrghd -; P8LE: xvcvdpsp -; P8LE: xvcvdpsp -; P8LE: vmrgew -; P8LE: xvcvspsxws v2 +; P8LE: xvcvdpsxws +; P8LE: xvcvdpsxws +; P8LE: vmrgew v2 } ; Function Attrs: norecurse nounwind readonly @@ -1799,40 +1776,36 @@ entry: ; P9BE: lfd ; P9BE: xxmrghd ; P9BE: xxmrghd -; P9BE: xvcvdpsp -; P9BE: xvcvdpsp -; P9BE: vmrgew -; P9BE: xvcvspsxws v2 +; P9BE: xvcvdpsxws +; P9BE: xvcvdpsxws +; P9BE: vmrgew v2 ; P9LE: lfdux ; P9LE: lfd ; P9LE: lfd ; P9LE: lfd ; P9LE: xxmrghd ; P9LE: xxmrghd -; P9LE: xvcvdpsp -; P9LE: xvcvdpsp -; P9LE: vmrgew -; P9LE: xvcvspsxws v2 +; P9LE: xvcvdpsxws +; P9LE: xvcvdpsxws +; P9LE: vmrgew v2 ; P8BE: lfdux ; P8BE: lfd ; P8BE: lfd ; P8BE: lfd ; P8BE: xxmrghd ; P8BE: xxmrghd -; P8BE: xvcvdpsp -; P8BE: xvcvdpsp -; P8BE: vmrgew -; P8BE: xvcvspsxws v2 +; P8BE: xvcvdpsxws +; P8BE: xvcvdpsxws +; P8BE: vmrgew v2 ; P8LE: lfdux ; P8LE: lfd ; P8LE: lfd ; P8LE: lfd ; P8LE: xxmrghd ; P8LE: xxmrghd -; P8LE: xvcvdpsp -; P8LE: xvcvdpsp -; P8LE: vmrgew -; P8LE: xvcvspsxws v2 +; P8LE: xvcvdpsxws +; P8LE: xvcvdpsxws +; P8LE: vmrgew v2 } ; Function Attrs: norecurse nounwind readnone @@ -2413,28 +2386,24 @@ entry: ; P8LE-LABEL: fromRegsConvftoui ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P9BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9BE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9BE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9BE: vmrgew v2, [[REG3]], [[REG4]] -; P9BE: xvcvspuxws v2, v2 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P9LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9LE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9LE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9LE: vmrgew v2, [[REG4]], [[REG3]] -; P9LE: xvcvspuxws v2, v2 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P8BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8BE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8BE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8BE: vmrgew v2, [[REG3]], [[REG4]] -; P8BE: xvcvspuxws v2, v2 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P8LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8LE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8LE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8LE: vmrgew v2, [[REG4]], [[REG3]] -; P8LE: xvcvspuxws v2, v2 } ; Function Attrs: norecurse nounwind readnone @@ -2689,28 +2658,24 @@ entry: ; P8LE-LABEL: fromRegsConvdtoui ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P9BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P9BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9BE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9BE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9BE: vmrgew v2, [[REG3]], [[REG4]] -; P9BE: xvcvspuxws v2, v2 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P9LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P9LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P9LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P9LE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P9LE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P9LE: vmrgew v2, [[REG4]], [[REG3]] -; P9LE: xvcvspuxws v2, v2 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs1, vs3 ; P8BE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs2, vs4 -; P8BE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8BE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8BE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8BE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8BE: vmrgew v2, [[REG3]], [[REG4]] -; P8BE: xvcvspuxws v2, v2 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG1:[0-9]+]], vs3, vs1 ; P8LE-DAG: xxmrghd {{[vs]+}}[[REG2:[0-9]+]], vs4, vs2 -; P8LE-DAG: xvcvdpsp [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] -; P8LE-DAG: xvcvdpsp [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] +; P8LE-DAG: xvcvdpuxws [[REG3:v[0-9]+]], {{[vs]+}}[[REG1]] +; P8LE-DAG: xvcvdpuxws [[REG4:v[0-9]+]], {{[vs]+}}[[REG2]] ; P8LE: vmrgew v2, [[REG4]], [[REG3]] -; P8LE: xvcvspuxws v2, v2 } ; Function Attrs: norecurse nounwind readnone @@ -2752,36 +2717,32 @@ entry: ; P9BE-DAG: lxv [[REG2:[vs0-9]+]], 16(r3) ; P9BE-DAG: xxmrgld [[REG3:[vs0-9]+]], [[REG1]], [[REG2]] ; P9BE-DAG: xxmrghd [[REG4:[vs0-9]+]], [[REG1]], [[REG2]] -; P9BE-DAG: xvcvdpsp [[REG5:[vs0-9]+]], [[REG3]] -; P9BE-DAG: xvcvdpsp [[REG6:[vs0-9]+]], [[REG4]] +; P9BE-DAG: xvcvdpuxws [[REG5:[vs0-9]+]], [[REG3]] +; P9BE-DAG: xvcvdpuxws [[REG6:[vs0-9]+]], [[REG4]] ; P9BE: vmrgew v2, [[REG6]], [[REG5]] -; P9BE: xvcvspuxws v2, v2 ; P9LE-DAG: lxv [[REG1:[vs0-9]+]], 0(r3) ; P9LE-DAG: lxv [[REG2:[vs0-9]+]], 16(r3) -; P9LE-DAG: xxmrgld [[REG3:[vs0-9]+]], [[REG2]], [[REG1]] ; P9LE-DAG: xxmrghd [[REG4:[vs0-9]+]], [[REG2]], [[REG1]] -; P9LE-DAG: xvcvdpsp [[REG5:[vs0-9]+]], [[REG3]] -; P9LE-DAG: xvcvdpsp [[REG6:[vs0-9]+]], [[REG4]] +; P9LE-DAG: xxmrgld [[REG3:[vs0-9]+]], [[REG2]], [[REG1]] +; P9LE-DAG: xvcvdpuxws [[REG5:[vs0-9]+]], [[REG3]] +; P9LE-DAG: xvcvdpuxws [[REG6:[vs0-9]+]], [[REG4]] ; P9LE: vmrgew v2, [[REG6]], [[REG5]] -; P9LE: xvcvspuxws v2, v2 ; P8BE: lxvd2x [[REG1:[vs0-9]+]], 0, r3 ; P8BE: lxvd2x [[REG2:[vs0-9]+]], r3, r4 ; P8BE-DAG: xxmrgld [[REG3:[vs0-9]+]], [[REG1]], [[REG2]] ; P8BE-DAG: xxmrghd [[REG4:[vs0-9]+]], [[REG1]], [[REG2]] -; P8BE-DAG: xvcvdpsp [[REG5:[vs0-9]+]], [[REG3]] -; P8BE-DAG: xvcvdpsp [[REG6:[vs0-9]+]], [[REG4]] +; P8BE-DAG: xvcvdpuxws [[REG5:[vs0-9]+]], [[REG3]] +; P8BE-DAG: xvcvdpuxws [[REG6:[vs0-9]+]], [[REG4]] ; P8BE: vmrgew v2, [[REG6]], [[REG5]] -; P8BE: xvcvspuxws v2, v2 ; P8LE: lxvd2x [[REG1:[vs0-9]+]], 0, r3 ; P8LE: lxvd2x [[REG2:[vs0-9]+]], r3, r4 ; P8LE-DAG: xxswapd [[REG3:[vs0-9]+]], [[REG1]] ; P8LE-DAG: xxswapd [[REG4:[vs0-9]+]], [[REG2]] ; P8LE-DAG: xxmrgld [[REG5:[vs0-9]+]], [[REG4]], [[REG3]] ; P8LE-DAG: xxmrghd [[REG6:[vs0-9]+]], [[REG4]], [[REG3]] -; P8LE-DAG: xvcvdpsp [[REG7:[vs0-9]+]], [[REG5]] -; P8LE-DAG: xvcvdpsp [[REG8:[vs0-9]+]], [[REG6]] +; P8LE-DAG: xvcvdpuxws [[REG7:[vs0-9]+]], [[REG5]] +; P8LE-DAG: xvcvdpuxws [[REG8:[vs0-9]+]], [[REG6]] ; P8LE: vmrgew v2, [[REG8]], [[REG7]] -; P8LE: xvcvspuxws v2, v2 } ; Function Attrs: norecurse nounwind readonly @@ -2813,40 +2774,36 @@ entry: ; P9BE: lfd ; P9BE: xxmrghd ; P9BE: xxmrghd -; P9BE: xvcvdpsp -; P9BE: xvcvdpsp -; P9BE: vmrgew -; P9BE: xvcvspuxws v2 +; P9BE: xvcvdpuxws +; P9BE: xvcvdpuxws +; P9BE: vmrgew v2 ; P9LE: lfd ; P9LE: lfd ; P9LE: lfd ; P9LE: lfd ; P9LE: xxmrghd ; P9LE: xxmrghd -; P9LE: xvcvdpsp -; P9LE: xvcvdpsp -; P9LE: vmrgew -; P9LE: xvcvspuxws v2 +; P9LE: xvcvdpuxws +; P9LE: xvcvdpuxws +; P9LE: vmrgew v2 ; P8BE: lfdx ; P8BE: lfd ; P8BE: lfd ; P8BE: lfd ; P8BE: xxmrghd ; P8BE: xxmrghd -; P8BE: xvcvdpsp -; P8BE: xvcvdpsp -; P8BE: vmrgew -; P8BE: xvcvspuxws v2 +; P8BE: xvcvdpuxws +; P8BE: xvcvdpuxws +; P8BE: vmrgew v2 ; P8LE: lfdx ; P8LE: lfd ; P8LE: lfd ; P8LE: lfd ; P8LE: xxmrghd ; P8LE: xxmrghd -; P8LE: xvcvdpsp -; P8LE: xvcvdpsp -; P8LE: vmrgew -; P8LE: xvcvspuxws v2 +; P8LE: xvcvdpuxws +; P8LE: xvcvdpuxws +; P8LE: vmrgew v2 } ; Function Attrs: norecurse nounwind readonly @@ -2886,40 +2843,36 @@ entry: ; P9BE: lfd ; P9BE: xxmrghd ; P9BE: xxmrghd -; P9BE: xvcvdpsp -; P9BE: xvcvdpsp -; P9BE: vmrgew -; P9BE: xvcvspuxws v2 +; P9BE: xvcvdpuxws +; P9BE: xvcvdpuxws +; P9BE: vmrgew v2 ; P9LE: lfdux ; P9LE: lfd ; P9LE: lfd ; P9LE: lfd ; P9LE: xxmrghd ; P9LE: xxmrghd -; P9LE: xvcvdpsp -; P9LE: xvcvdpsp -; P9LE: vmrgew -; P9LE: xvcvspuxws v2 +; P9LE: xvcvdpuxws +; P9LE: xvcvdpuxws +; P9LE: vmrgew v2 ; P8BE: lfdux ; P8BE: lfd ; P8BE: lfd ; P8BE: lfd ; P8BE: xxmrghd ; P8BE: xxmrghd -; P8BE: xvcvdpsp -; P8BE: xvcvdpsp -; P8BE: vmrgew *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Sat Aug 4 13:15:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CF46106AF1F; Sat, 4 Aug 2018 13:15:53 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 304C675E5C; Sat, 4 Aug 2018 13:15:53 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EA8FE7F5E; Sat, 4 Aug 2018 13:15:52 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DFqlM083440; Sat, 4 Aug 2018 13:15:52 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DFqca083439; Sat, 4 Aug 2018 13:15:52 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041315.w74DFqca083439@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:15:52 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337302 - vendor/clang/clang-release_70-r338892 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/clang/clang-release_70-r338892 X-SVN-Commit-Revision: 337302 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:15:53 -0000 Author: dim Date: Sat Aug 4 13:15:52 2018 New Revision: 337302 URL: https://svnweb.freebsd.org/changeset/base/337302 Log: Tag clang release_70 branch r338892. Added: vendor/clang/clang-release_70-r338892/ - copied from r337301, vendor/clang/dist-release_70/ From owner-svn-src-all@freebsd.org Sat Aug 4 13:15:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B84A106AF19; Sat, 4 Aug 2018 13:15:50 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E321575E55; Sat, 4 Aug 2018 13:15:49 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C57AF7F5D; Sat, 4 Aug 2018 13:15:49 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DFnZm083394; Sat, 4 Aug 2018 13:15:49 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DFnLM083392; Sat, 4 Aug 2018 13:15:49 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041315.w74DFnLM083392@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:15:49 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337301 - in vendor/clang/dist-release_70: docs include/clang/Basic lib/AST lib/Basic lib/Sema test/CodeGen test/Driver test/SemaCXX X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: in vendor/clang/dist-release_70: docs include/clang/Basic lib/AST lib/Basic lib/Sema test/CodeGen test/Driver test/SemaCXX X-SVN-Commit-Revision: 337301 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:15:50 -0000 Author: dim Date: Sat Aug 4 13:15:49 2018 New Revision: 337301 URL: https://svnweb.freebsd.org/changeset/base/337301 Log: Vendor import of clang release_70 branch r338892: https://llvm.org/svn/llvm-project/cfe/branches/release_70@338892 Modified: vendor/clang/dist-release_70/docs/AttributeReference.rst vendor/clang/dist-release_70/docs/ClangCommandLineReference.rst vendor/clang/dist-release_70/docs/ReleaseNotes.rst vendor/clang/dist-release_70/include/clang/Basic/Builtins.def vendor/clang/dist-release_70/include/clang/Basic/DiagnosticASTKinds.td vendor/clang/dist-release_70/lib/AST/ExprConstant.cpp vendor/clang/dist-release_70/lib/Basic/Version.cpp vendor/clang/dist-release_70/lib/Sema/SemaInit.cpp vendor/clang/dist-release_70/test/CodeGen/builtin-memfns.c vendor/clang/dist-release_70/test/Driver/darwin-infer-simulator-sdkroot.c vendor/clang/dist-release_70/test/Driver/integrated-as.c vendor/clang/dist-release_70/test/SemaCXX/constexpr-string.cpp Modified: vendor/clang/dist-release_70/docs/AttributeReference.rst ============================================================================== --- vendor/clang/dist-release_70/docs/AttributeReference.rst Sat Aug 4 13:15:11 2018 (r337300) +++ vendor/clang/dist-release_70/docs/AttributeReference.rst Sat Aug 4 13:15:49 2018 (r337301) @@ -1,13 +1,3899 @@ .. ------------------------------------------------------------------- NOTE: This file is automatically generated by running clang-tblgen - -gen-attr-docs. Do not edit this file by hand!! The contents for - this file are automatically generated by a server-side process. - - Please do not commit this file. The file exists for local testing - purposes only. + -gen-attr-docs. Do not edit this file by hand!! ------------------------------------------------------------------- =================== Attributes in Clang -=================== \ No newline at end of file +=================== +.. contents:: + :local: + +Introduction +============ + +This page lists the attributes currently supported by Clang. + +Function Attributes +=================== + + +#pragma omp declare simd +------------------------ +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "","","","","", "X", "" + +The `declare simd` construct can be applied to a function to enable the creation +of one or more versions that can process multiple arguments using SIMD +instructions from a single invocation in a SIMD loop. The `declare simd` +directive is a declarative directive. There may be multiple `declare simd` +directives for a function. The use of a `declare simd` construct on a function +enables the creation of SIMD versions of the associated function that can be +used to process multiple arguments from a single invocation from a SIMD loop +concurrently. +The syntax of the `declare simd` construct is as follows: + + .. code-block:: none + + #pragma omp declare simd [clause[[,] clause] ...] new-line + [#pragma omp declare simd [clause[[,] clause] ...] new-line] + [...] + function definition or declaration + +where clause is one of the following: + + .. code-block:: none + + simdlen(length) + linear(argument-list[:constant-linear-step]) + aligned(argument-list[:alignment]) + uniform(argument-list) + inbranch + notinbranch + + +#pragma omp declare target +-------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "","","","","", "X", "" + +The `declare target` directive specifies that variables and functions are mapped +to a device for OpenMP offload mechanism. + +The syntax of the declare target directive is as follows: + + .. code-block:: c + + #pragma omp declare target new-line + declarations-definition-seq + #pragma omp end declare target new-line + + +_Noreturn +--------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "","","","","X", "", "" + +A function declared as ``_Noreturn`` shall not return to its caller. The +compiler will generate a diagnostic for a function declared as ``_Noreturn`` +that appears to be capable of returning to its caller. + + +abi_tag (gnu::abi_tag) +---------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "X" + +The ``abi_tag`` attribute can be applied to a function, variable, class or +inline namespace declaration to modify the mangled name of the entity. It gives +the ability to distinguish between different versions of the same entity but +with different ABI versions supported. For example, a newer version of a class +could have a different set of data members and thus have a different size. Using +the ``abi_tag`` attribute, it is possible to have different mangled names for +a global variable of the class type. Therefore, the old code could keep using +the old manged name and the new code will use the new mangled name with tags. + + +acquire_capability (acquire_shared_capability, clang::acquire_capability, clang::acquire_shared_capability) +----------------------------------------------------------------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "" + +Marks a function as acquiring a capability. + + +alloc_align (gnu::alloc_align) +------------------------------ +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "" + +Use ``__attribute__((alloc_align())`` on a function +declaration to specify that the return value of the function (which must be a +pointer type) is at least as aligned as the value of the indicated parameter. The +parameter is given by its index in the list of formal parameters; the first +parameter has index 1 unless the function is a C++ non-static member function, +in which case the first parameter has index 2 to account for the implicit ``this`` +parameter. + +.. code-block:: c++ + + // The returned pointer has the alignment specified by the first parameter. + void *a(size_t align) __attribute__((alloc_align(1))); + + // The returned pointer has the alignment specified by the second parameter. + void *b(void *v, size_t align) __attribute__((alloc_align(2))); + + // The returned pointer has the alignment specified by the second visible + // parameter, however it must be adjusted for the implicit 'this' parameter. + void *Foo::b(void *v, size_t align) __attribute__((alloc_align(3))); + +Note that this attribute merely informs the compiler that a function always +returns a sufficiently aligned pointer. It does not cause the compiler to +emit code to enforce that alignment. The behavior is undefined if the returned +poitner is not sufficiently aligned. + + +alloc_size (gnu::alloc_size) +---------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "X" + +The ``alloc_size`` attribute can be placed on functions that return pointers in +order to hint to the compiler how many bytes of memory will be available at the +returned pointer. ``alloc_size`` takes one or two arguments. + +- ``alloc_size(N)`` implies that argument number N equals the number of + available bytes at the returned pointer. +- ``alloc_size(N, M)`` implies that the product of argument number N and + argument number M equals the number of available bytes at the returned + pointer. + +Argument numbers are 1-based. + +An example of how to use ``alloc_size`` + +.. code-block:: c + + void *my_malloc(int a) __attribute__((alloc_size(1))); + void *my_calloc(int a, int b) __attribute__((alloc_size(1, 2))); + + int main() { + void *const p = my_malloc(100); + assert(__builtin_object_size(p, 0) == 100); + void *const a = my_calloc(20, 5); + assert(__builtin_object_size(a, 0) == 100); + } + +.. Note:: This attribute works differently in clang than it does in GCC. + Specifically, clang will only trace ``const`` pointers (as above); we give up + on pointers that are not marked as ``const``. In the vast majority of cases, + this is unimportant, because LLVM has support for the ``alloc_size`` + attribute. However, this may cause mildly unintuitive behavior when used with + other attributes, such as ``enable_if``. + + +artificial (gnu::artificial) +---------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "" + +The ``artificial`` attribute can be applied to an inline function. If such a +function is inlined, the attribute indicates that debuggers should associate +the resulting instructions with the call site, rather than with the +corresponding line within the inlined callee. + + +assert_capability (assert_shared_capability, clang::assert_capability, clang::assert_shared_capability) +------------------------------------------------------------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "" + +Marks a function that dynamically tests whether a capability is held, and halts +the program if it is not held. + + +assume_aligned (gnu::assume_aligned) +------------------------------------ +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "X" + +Use ``__attribute__((assume_aligned([,]))`` on a function +declaration to specify that the return value of the function (which must be a +pointer type) has the specified offset, in bytes, from an address with the +specified alignment. The offset is taken to be zero if omitted. + +.. code-block:: c++ + + // The returned pointer value has 32-byte alignment. + void *a() __attribute__((assume_aligned (32))); + + // The returned pointer value is 4 bytes greater than an address having + // 32-byte alignment. + void *b() __attribute__((assume_aligned (32, 4))); + +Note that this attribute provides information to the compiler regarding a +condition that the code already ensures is true. It does not cause the compiler +to enforce the provided alignment assumption. + + +availability (clang::availability, clang::availability) +------------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","X","","", "", "X" + +The ``availability`` attribute can be placed on declarations to describe the +lifecycle of that declaration relative to operating system versions. Consider +the function declaration for a hypothetical function ``f``: + +.. code-block:: c++ + + void f(void) __attribute__((availability(macos,introduced=10.4,deprecated=10.6,obsoleted=10.7))); + +The availability attribute states that ``f`` was introduced in macOS 10.4, +deprecated in macOS 10.6, and obsoleted in macOS 10.7. This information +is used by Clang to determine when it is safe to use ``f``: for example, if +Clang is instructed to compile code for macOS 10.5, a call to ``f()`` +succeeds. If Clang is instructed to compile code for macOS 10.6, the call +succeeds but Clang emits a warning specifying that the function is deprecated. +Finally, if Clang is instructed to compile code for macOS 10.7, the call +fails because ``f()`` is no longer available. + +The availability attribute is a comma-separated list starting with the +platform name and then including clauses specifying important milestones in the +declaration's lifetime (in any order) along with additional information. Those +clauses can be: + +introduced=\ *version* + The first version in which this declaration was introduced. + +deprecated=\ *version* + The first version in which this declaration was deprecated, meaning that + users should migrate away from this API. + +obsoleted=\ *version* + The first version in which this declaration was obsoleted, meaning that it + was removed completely and can no longer be used. + +unavailable + This declaration is never available on this platform. + +message=\ *string-literal* + Additional message text that Clang will provide when emitting a warning or + error about use of a deprecated or obsoleted declaration. Useful to direct + users to replacement APIs. + +replacement=\ *string-literal* + Additional message text that Clang will use to provide Fix-It when emitting + a warning about use of a deprecated declaration. The Fix-It will replace + the deprecated declaration with the new declaration specified. + +Multiple availability attributes can be placed on a declaration, which may +correspond to different platforms. Only the availability attribute with the +platform corresponding to the target platform will be used; any others will be +ignored. If no availability attribute specifies availability for the current +target platform, the availability attributes are ignored. Supported platforms +are: + +``ios`` + Apple's iOS operating system. The minimum deployment target is specified by + the ``-mios-version-min=*version*`` or ``-miphoneos-version-min=*version*`` + command-line arguments. + +``macos`` + Apple's macOS operating system. The minimum deployment target is + specified by the ``-mmacosx-version-min=*version*`` command-line argument. + ``macosx`` is supported for backward-compatibility reasons, but it is + deprecated. + +``tvos`` + Apple's tvOS operating system. The minimum deployment target is specified by + the ``-mtvos-version-min=*version*`` command-line argument. + +``watchos`` + Apple's watchOS operating system. The minimum deployment target is specified by + the ``-mwatchos-version-min=*version*`` command-line argument. + +A declaration can typically be used even when deploying back to a platform +version prior to when the declaration was introduced. When this happens, the +declaration is `weakly linked +`_, +as if the ``weak_import`` attribute were added to the declaration. A +weakly-linked declaration may or may not be present a run-time, and a program +can determine whether the declaration is present by checking whether the +address of that declaration is non-NULL. + +The flag ``strict`` disallows using API when deploying back to a +platform version prior to when the declaration was introduced. An +attempt to use such API before its introduction causes a hard error. +Weakly-linking is almost always a better API choice, since it allows +users to query availability at runtime. + +If there are multiple declarations of the same entity, the availability +attributes must either match on a per-platform basis or later +declarations must not have availability attributes for that +platform. For example: + +.. code-block:: c + + void g(void) __attribute__((availability(macos,introduced=10.4))); + void g(void) __attribute__((availability(macos,introduced=10.4))); // okay, matches + void g(void) __attribute__((availability(ios,introduced=4.0))); // okay, adds a new platform + void g(void); // okay, inherits both macos and ios availability from above. + void g(void) __attribute__((availability(macos,introduced=10.5))); // error: mismatch + +When one method overrides another, the overriding method can be more widely available than the overridden method, e.g.,: + +.. code-block:: objc + + @interface A + - (id)method __attribute__((availability(macos,introduced=10.4))); + - (id)method2 __attribute__((availability(macos,introduced=10.4))); + @end + + @interface B : A + - (id)method __attribute__((availability(macos,introduced=10.3))); // okay: method moved into base class later + - (id)method __attribute__((availability(macos,introduced=10.5))); // error: this method was available via the base class in 10.4 + @end + +Starting with the macOS 10.12 SDK, the ``API_AVAILABLE`` macro from +```` can simplify the spelling: + +.. code-block:: objc + + @interface A + - (id)method API_AVAILABLE(macos(10.11))); + - (id)otherMethod API_AVAILABLE(macos(10.11), ios(11.0)); + @end + +Also see the documentation for `@available +`_ + + +carries_dependency +------------------ +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "X" + +The ``carries_dependency`` attribute specifies dependency propagation into and +out of functions. + +When specified on a function or Objective-C method, the ``carries_dependency`` +attribute means that the return value carries a dependency out of the function, +so that the implementation need not constrain ordering upon return from that +function. Implementations of the function and its caller may choose to preserve +dependencies instead of emitting memory ordering instructions such as fences. + +Note, this attribute does not change the meaning of the program, but may result +in generation of more efficient code. + + +code_seg +-------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "","","","X","", "", "" + +The ``__declspec(code_seg)`` attribute enables the placement of code into separate +named segments that can be paged or locked in memory individually. This attribute +is used to control the placement of instantiated templates and compiler-generated +code. See the documentation for `__declspec(code_seg)`_ on MSDN. + +.. _`__declspec(code_seg)`: http://msdn.microsoft.com/en-us/library/dn636922.aspx + + +convergent (clang::convergent, clang::convergent) +------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","X","","", "", "X" + +The ``convergent`` attribute can be placed on a function declaration. It is +translated into the LLVM ``convergent`` attribute, which indicates that the call +instructions of a function with this attribute cannot be made control-dependent +on any additional values. + +In languages designed for SPMD/SIMT programming model, e.g. OpenCL or CUDA, +the call instructions of a function with this attribute must be executed by +all work items or threads in a work group or sub group. + +This attribute is different from ``noduplicate`` because it allows duplicating +function calls if it can be proved that the duplicated function calls are +not made control-dependent on any additional values, e.g., unrolling a loop +executed by all work items. + +Sample usage: +.. code-block:: c + + void convfunc(void) __attribute__((convergent)); + // Setting it as a C++11 attribute is also valid in a C++ program. + // void convfunc(void) [[clang::convergent]]; + + +cpu_dispatch (clang::cpu_dispatch, clang::cpu_dispatch) +------------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","X","","", "", "X" + +The ``cpu_specific`` and ``cpu_dispatch`` attributes are used to define and +resolve multiversioned functions. This form of multiversioning provides a +mechanism for declaring versions across translation units and manually +specifying the resolved function list. A specified CPU defines a set of minimum +features that are required for the function to be called. The result of this is +that future processors execute the most restrictive version of the function the +new processor can execute. + +Function versions are defined with ``cpu_specific``, which takes one or more CPU +names as a parameter. For example: + +.. code-block:: c + + // Declares and defines the ivybridge version of single_cpu. + __attribute__((cpu_specific(ivybridge))) + void single_cpu(void){} + + // Declares and defines the atom version of single_cpu. + __attribute__((cpu_specific(atom))) + void single_cpu(void){} + + // Declares and defines both the ivybridge and atom version of multi_cpu. + __attribute__((cpu_specific(ivybridge, atom))) + void multi_cpu(void){} + +A dispatching (or resolving) function can be declared anywhere in a project's +source code with ``cpu_dispatch``. This attribute takes one or more CPU names +as a parameter (like ``cpu_specific``). Functions marked with ``cpu_dispatch`` +are not expected to be defined, only declared. If such a marked function has a +definition, any side effects of the function are ignored; trivial function +bodies are permissible for ICC compatibility. + +.. code-block:: c + + // Creates a resolver for single_cpu above. + __attribute__((cpu_dispatch(ivybridge, atom))) + void single_cpu(void){} + + // Creates a resolver for multi_cpu, but adds a 3rd version defined in another + // translation unit. + __attribute__((cpu_dispatch(ivybridge, atom, sandybridge))) + void multi_cpu(void){} + +Note that it is possible to have a resolving function that dispatches based on +more or fewer options than are present in the program. Specifying fewer will +result in the omitted options not being considered during resolution. Specifying +a version for resolution that isn't defined in the program will result in a +linking failure. + +It is also possible to specify a CPU name of ``generic`` which will be resolved +if the executing processor doesn't satisfy the features required in the CPU +name. The behavior of a program executing on a processor that doesn't satisfy +any option of a multiversioned function is undefined. + + +cpu_specific (clang::cpu_specific, clang::cpu_specific) +------------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","X","","", "", "X" + +The ``cpu_specific`` and ``cpu_dispatch`` attributes are used to define and +resolve multiversioned functions. This form of multiversioning provides a +mechanism for declaring versions across translation units and manually +specifying the resolved function list. A specified CPU defines a set of minimum +features that are required for the function to be called. The result of this is +that future processors execute the most restrictive version of the function the +new processor can execute. + +Function versions are defined with ``cpu_specific``, which takes one or more CPU +names as a parameter. For example: + +.. code-block:: c + + // Declares and defines the ivybridge version of single_cpu. + __attribute__((cpu_specific(ivybridge))) + void single_cpu(void){} + + // Declares and defines the atom version of single_cpu. + __attribute__((cpu_specific(atom))) + void single_cpu(void){} + + // Declares and defines both the ivybridge and atom version of multi_cpu. + __attribute__((cpu_specific(ivybridge, atom))) + void multi_cpu(void){} + +A dispatching (or resolving) function can be declared anywhere in a project's +source code with ``cpu_dispatch``. This attribute takes one or more CPU names +as a parameter (like ``cpu_specific``). Functions marked with ``cpu_dispatch`` +are not expected to be defined, only declared. If such a marked function has a +definition, any side effects of the function are ignored; trivial function +bodies are permissible for ICC compatibility. + +.. code-block:: c + + // Creates a resolver for single_cpu above. + __attribute__((cpu_dispatch(ivybridge, atom))) + void single_cpu(void){} + + // Creates a resolver for multi_cpu, but adds a 3rd version defined in another + // translation unit. + __attribute__((cpu_dispatch(ivybridge, atom, sandybridge))) + void multi_cpu(void){} + +Note that it is possible to have a resolving function that dispatches based on +more or fewer options than are present in the program. Specifying fewer will +result in the omitted options not being considered during resolution. Specifying +a version for resolution that isn't defined in the program will result in a +linking failure. + +It is also possible to specify a CPU name of ``generic`` which will be resolved +if the executing processor doesn't satisfy the features required in the CPU +name. The behavior of a program executing on a processor that doesn't satisfy +any option of a multiversioned function is undefined. + + +deprecated (gnu::deprecated) +---------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","X","X","", "", "" + +The ``deprecated`` attribute can be applied to a function, a variable, or a +type. This is useful when identifying functions, variables, or types that are +expected to be removed in a future version of a program. + +Consider the function declaration for a hypothetical function ``f``: + +.. code-block:: c++ + + void f(void) __attribute__((deprecated("message", "replacement"))); + +When spelled as `__attribute__((deprecated))`, the deprecated attribute can have +two optional string arguments. The first one is the message to display when +emitting the warning; the second one enables the compiler to provide a Fix-It +to replace the deprecated name with a new name. Otherwise, when spelled as +`[[gnu::deprecated]] or [[deprecated]]`, the attribute can have one optional +string argument which is the message to display when emitting the warning. + + +diagnose_if +----------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","","","","", "", "" + +The ``diagnose_if`` attribute can be placed on function declarations to emit +warnings or errors at compile-time if calls to the attributed function meet +certain user-defined criteria. For example: + +.. code-block:: c + + void abs(int a) + __attribute__((diagnose_if(a >= 0, "Redundant abs call", "warning"))); + void must_abs(int a) + __attribute__((diagnose_if(a >= 0, "Redundant abs call", "error"))); + + int val = abs(1); // warning: Redundant abs call + int val2 = must_abs(1); // error: Redundant abs call + int val3 = abs(val); + int val4 = must_abs(val); // Because run-time checks are not emitted for + // diagnose_if attributes, this executes without + // issue. + + +``diagnose_if`` is closely related to ``enable_if``, with a few key differences: + +* Overload resolution is not aware of ``diagnose_if`` attributes: they're + considered only after we select the best candidate from a given candidate set. +* Function declarations that differ only in their ``diagnose_if`` attributes are + considered to be redeclarations of the same function (not overloads). +* If the condition provided to ``diagnose_if`` cannot be evaluated, no + diagnostic will be emitted. + +Otherwise, ``diagnose_if`` is essentially the logical negation of ``enable_if``. + +As a result of bullet number two, ``diagnose_if`` attributes will stack on the +same function. For example: + +.. code-block:: c + + int foo() __attribute__((diagnose_if(1, "diag1", "warning"))); + int foo() __attribute__((diagnose_if(1, "diag2", "warning"))); + + int bar = foo(); // warning: diag1 + // warning: diag2 + int (*fooptr)(void) = foo; // warning: diag1 + // warning: diag2 + + constexpr int supportsAPILevel(int N) { return N < 5; } + int baz(int a) + __attribute__((diagnose_if(!supportsAPILevel(10), + "Upgrade to API level 10 to use baz", "error"))); + int baz(int a) + __attribute__((diagnose_if(!a, "0 is not recommended.", "warning"))); + + int (*bazptr)(int) = baz; // error: Upgrade to API level 10 to use baz + int v = baz(0); // error: Upgrade to API level 10 to use baz + +Query for this feature with ``__has_attribute(diagnose_if)``. + + +disable_tail_calls (clang::disable_tail_calls, clang::disable_tail_calls) +------------------------------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","X","","", "", "X" + +The ``disable_tail_calls`` attribute instructs the backend to not perform tail call optimization inside the marked function. + +For example: + + .. code-block:: c + + int callee(int); + + int foo(int a) __attribute__((disable_tail_calls)) { + return callee(a); // This call is not tail-call optimized. + } + +Marking virtual functions as ``disable_tail_calls`` is legal. + + .. code-block:: c++ + + int callee(int); + + class Base { + public: + [[clang::disable_tail_calls]] virtual int foo1() { + return callee(); // This call is not tail-call optimized. + } + }; + + class Derived1 : public Base { + public: + int foo1() override { + return callee(); // This call is tail-call optimized. + } + }; + + +enable_if +--------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","","","","", "", "X" + +.. Note:: Some features of this attribute are experimental. The meaning of + multiple enable_if attributes on a single declaration is subject to change in + a future version of clang. Also, the ABI is not standardized and the name + mangling may change in future versions. To avoid that, use asm labels. + +The ``enable_if`` attribute can be placed on function declarations to control +which overload is selected based on the values of the function's arguments. +When combined with the ``overloadable`` attribute, this feature is also +available in C. + +.. code-block:: c++ + + int isdigit(int c); + int isdigit(int c) __attribute__((enable_if(c <= -1 || c > 255, "chosen when 'c' is out of range"))) __attribute__((unavailable("'c' must have the value of an unsigned char or EOF"))); + + void foo(char c) { + isdigit(c); + isdigit(10); + isdigit(-10); // results in a compile-time error. + } + +The enable_if attribute takes two arguments, the first is an expression written +in terms of the function parameters, the second is a string explaining why this +overload candidate could not be selected to be displayed in diagnostics. The +expression is part of the function signature for the purposes of determining +whether it is a redeclaration (following the rules used when determining +whether a C++ template specialization is ODR-equivalent), but is not part of +the type. + +The enable_if expression is evaluated as if it were the body of a +bool-returning constexpr function declared with the arguments of the function +it is being applied to, then called with the parameters at the call site. If the +result is false or could not be determined through constant expression +evaluation, then this overload will not be chosen and the provided string may +be used in a diagnostic if the compile fails as a result. + +Because the enable_if expression is an unevaluated context, there are no global +state changes, nor the ability to pass information from the enable_if +expression to the function body. For example, suppose we want calls to +strnlen(strbuf, maxlen) to resolve to strnlen_chk(strbuf, maxlen, size of +strbuf) only if the size of strbuf can be determined: + +.. code-block:: c++ + + __attribute__((always_inline)) + static inline size_t strnlen(const char *s, size_t maxlen) + __attribute__((overloadable)) + __attribute__((enable_if(__builtin_object_size(s, 0) != -1))), + "chosen when the buffer size is known but 'maxlen' is not"))) + { + return strnlen_chk(s, maxlen, __builtin_object_size(s, 0)); + } + +Multiple enable_if attributes may be applied to a single declaration. In this +case, the enable_if expressions are evaluated from left to right in the +following manner. First, the candidates whose enable_if expressions evaluate to +false or cannot be evaluated are discarded. If the remaining candidates do not +share ODR-equivalent enable_if expressions, the overload resolution is +ambiguous. Otherwise, enable_if overload resolution continues with the next +enable_if attribute on the candidates that have not been discarded and have +remaining enable_if attributes. In this way, we pick the most specific +overload out of a number of viable overloads using enable_if. + +.. code-block:: c++ + + void f() __attribute__((enable_if(true, ""))); // #1 + void f() __attribute__((enable_if(true, ""))) __attribute__((enable_if(true, ""))); // #2 + + void g(int i, int j) __attribute__((enable_if(i, ""))); // #1 + void g(int i, int j) __attribute__((enable_if(j, ""))) __attribute__((enable_if(true))); // #2 + +In this example, a call to f() is always resolved to #2, as the first enable_if +expression is ODR-equivalent for both declarations, but #1 does not have another +enable_if expression to continue evaluating, so the next round of evaluation has +only a single candidate. In a call to g(1, 1), the call is ambiguous even though +#2 has more enable_if attributes, because the first enable_if expressions are +not ODR-equivalent. + +Query for this feature with ``__has_attribute(enable_if)``. + +Note that functions with one or more ``enable_if`` attributes may not have +their address taken, unless all of the conditions specified by said +``enable_if`` are constants that evaluate to ``true``. For example: + +.. code-block:: c + + const int TrueConstant = 1; + const int FalseConstant = 0; + int f(int a) __attribute__((enable_if(a > 0, ""))); + int g(int a) __attribute__((enable_if(a == 0 || a != 0, ""))); + int h(int a) __attribute__((enable_if(1, ""))); + int i(int a) __attribute__((enable_if(TrueConstant, ""))); + int j(int a) __attribute__((enable_if(FalseConstant, ""))); + + void fn() { + int (*ptr)(int); + ptr = &f; // error: 'a > 0' is not always true + ptr = &g; // error: 'a == 0 || a != 0' is not a truthy constant + ptr = &h; // OK: 1 is a truthy constant + ptr = &i; // OK: 'TrueConstant' is a truthy constant + ptr = &j; // error: 'FalseConstant' is a constant, but not truthy + } + +Because ``enable_if`` evaluation happens during overload resolution, +``enable_if`` may give unintuitive results when used with templates, depending +on when overloads are resolved. In the example below, clang will emit a +diagnostic about no viable overloads for ``foo`` in ``bar``, but not in ``baz``: + +.. code-block:: c++ + + double foo(int i) __attribute__((enable_if(i > 0, ""))); + void *foo(int i) __attribute__((enable_if(i <= 0, ""))); + template + auto bar() { return foo(I); } + + template + auto baz() { return foo(T::number); } + + struct WithNumber { constexpr static int number = 1; }; + void callThem() { + bar(); + baz(); + } + +This is because, in ``bar``, ``foo`` is resolved prior to template +instantiation, so the value for ``I`` isn't known (thus, both ``enable_if`` +conditions for ``foo`` fail). However, in ``baz``, ``foo`` is resolved during +template instantiation, so the value for ``T::number`` is known. + + +external_source_symbol (clang::external_source_symbol, clang::external_source_symbol) +------------------------------------------------------------------------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","X","","", "", "X" + +The ``external_source_symbol`` attribute specifies that a declaration originates +from an external source and describes the nature of that source. + +The fact that Clang is capable of recognizing declarations that were defined +externally can be used to provide better tooling support for mixed-language +projects or projects that rely on auto-generated code. For instance, an IDE that +uses Clang and that supports mixed-language projects can use this attribute to +provide a correct 'jump-to-definition' feature. For a concrete example, +consider a protocol that's defined in a Swift file: + +.. code-block:: swift + + @objc public protocol SwiftProtocol { + func method() + } + +This protocol can be used from Objective-C code by including a header file that +was generated by the Swift compiler. The declarations in that header can use +the ``external_source_symbol`` attribute to make Clang aware of the fact +that ``SwiftProtocol`` actually originates from a Swift module: + +.. code-block:: objc + + __attribute__((external_source_symbol(language="Swift",defined_in="module"))) + @protocol SwiftProtocol + @required + - (void) method; + @end + +Consequently, when 'jump-to-definition' is performed at a location that +references ``SwiftProtocol``, the IDE can jump to the original definition in +the Swift source file rather than jumping to the Objective-C declaration in the +auto-generated header file. + +The ``external_source_symbol`` attribute is a comma-separated list that includes +clauses that describe the origin and the nature of the particular declaration. +Those clauses can be: + +language=\ *string-literal* + The name of the source language in which this declaration was defined. + +defined_in=\ *string-literal* + The name of the source container in which the declaration was defined. The + exact definition of source container is language-specific, e.g. Swift's + source containers are modules, so ``defined_in`` should specify the Swift + module name. + +generated_declaration + This declaration was automatically generated by some tool. + +The clauses can be specified in any order. The clauses that are listed above are +all optional, but the attribute has to have at least one clause. + + +flatten (gnu::flatten) +---------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "X" + +The ``flatten`` attribute causes calls within the attributed function to +be inlined unless it is impossible to do so, for example if the body of the +callee is unavailable or if the callee has the ``noinline`` attribute. + + +force_align_arg_pointer (gnu::force_align_arg_pointer) +------------------------------------------------------ +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "" + +Use this attribute to force stack alignment. + +Legacy x86 code uses 4-byte stack alignment. Newer aligned SSE instructions +(like 'movaps') that work with the stack require operands to be 16-byte aligned. +This attribute realigns the stack in the function prologue to make sure the +stack can be used with SSE instructions. + +Note that the x86_64 ABI forces 16-byte stack alignment at the call site. +Because of this, 'force_align_arg_pointer' is not needed on x86_64, except in +rare cases where the caller does not align the stack properly (e.g. flow +jumps from i386 arch code). + + .. code-block:: c + + __attribute__ ((force_align_arg_pointer)) + void f () { + ... + } + + +format (gnu::format) +-------------------- +.. csv-table:: Supported Syntaxes + :header: "GNU", "C++11", "C2x", "__declspec", "Keyword", "Pragma", "Pragma clang attribute" + + "X","X","","","", "", "" + +Clang supports the ``format`` attribute, which indicates that the function +accepts a ``printf`` or ``scanf``-like format string and corresponding +arguments or a ``va_list`` that contains these arguments. + +Please see `GCC documentation about format attribute +`_ to find details +about attribute syntax. + +Clang implements two kinds of checks with this attribute. + +#. Clang checks that the function with the ``format`` attribute is called with + a format string that uses format specifiers that are allowed, and that + arguments match the format string. This is the ``-Wformat`` warning, it is + on by default. + +#. Clang checks that the format string argument is a literal string. This is + the ``-Wformat-nonliteral`` warning, it is off by default. + + Clang implements this mostly the same way as GCC, but there is a difference + for functions that accept a ``va_list`` argument (for example, ``vprintf``). + GCC does not emit ``-Wformat-nonliteral`` warning for calls to such + functions. Clang does not warn if the format string comes from a function + parameter, where the function is annotated with a compatible attribute, + otherwise it warns. For example: + + .. code-block:: c + + __attribute__((__format__ (__scanf__, 1, 3))) + void foo(const char* s, char *buf, ...) { + va_list ap; + va_start(ap, buf); + + vprintf(s, ap); // warning: format string is not a string literal + } + + In this case we warn because ``s`` contains a format string for a + ``scanf``-like function, but it is passed to a ``printf``-like function. + + If the attribute is removed, clang still warns, because the format string is + not a string literal. + + Another example: + + .. code-block:: c + + __attribute__((__format__ (__printf__, 1, 3))) + void foo(const char* s, char *buf, ...) { + va_list ap; + va_start(ap, buf); + + vprintf(s, ap); // warning + } + + In this case Clang does not warn because the format string ``s`` and + the corresponding arguments are annotated. If the arguments are + incorrect, the caller of ``foo`` will receive a warning. *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Sat Aug 4 13:15:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E70B106AF95; Sat, 4 Aug 2018 13:15:59 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A3E3975F47; Sat, 4 Aug 2018 13:15:58 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9AE317F5F; Sat, 4 Aug 2018 13:15:57 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DFvAI083491; Sat, 4 Aug 2018 13:15:57 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DFvSv083489; Sat, 4 Aug 2018 13:15:57 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041315.w74DFvSv083489@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:15:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337303 - vendor/compiler-rt/dist-release_70/lib/sanitizer_common X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/compiler-rt/dist-release_70/lib/sanitizer_common X-SVN-Commit-Revision: 337303 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:15:59 -0000 Author: dim Date: Sat Aug 4 13:15:56 2018 New Revision: 337303 URL: https://svnweb.freebsd.org/changeset/base/337303 Log: Vendor import of compiler-rt release_70 branch r338892: https://llvm.org/svn/llvm-project/compiler-rt/branches/release_70@338892 Modified: vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_common_interceptors.inc vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_linux_libcdep.cc Modified: vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_common_interceptors.inc ============================================================================== --- vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_common_interceptors.inc Sat Aug 4 13:15:52 2018 (r337302) +++ vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_common_interceptors.inc Sat Aug 4 13:15:56 2018 (r337303) @@ -4460,6 +4460,7 @@ INTERCEPTOR(int, random_r, void *buf, u32 *result) { // its metadata. See // https://github.com/google/sanitizers/issues/321. #if SANITIZER_INTERCEPT_PTHREAD_ATTR_GET || \ + SANITIZER_INTERCEPT_PTHREAD_ATTR_GET_SCHED || \ SANITIZER_INTERCEPT_PTHREAD_ATTR_GETINHERITSSCHED || \ SANITIZER_INTERCEPT_PTHREAD_MUTEXATTR_GET || \ SANITIZER_INTERCEPT_PTHREAD_RWLOCKATTR_GET || \ Modified: vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_linux_libcdep.cc ============================================================================== --- vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_linux_libcdep.cc Sat Aug 4 13:15:52 2018 (r337302) +++ vendor/compiler-rt/dist-release_70/lib/sanitizer_common/sanitizer_linux_libcdep.cc Sat Aug 4 13:15:56 2018 (r337303) @@ -292,7 +292,7 @@ uptr ThreadDescriptorSize() { val = FIRST_32_SECOND_64(1168, 1776); else if (minor == 11 || (minor == 12 && patch == 1)) val = FIRST_32_SECOND_64(1168, 2288); - else if (minor <= 13) + else if (minor <= 14) val = FIRST_32_SECOND_64(1168, 2304); else val = FIRST_32_SECOND_64(1216, 2304); From owner-svn-src-all@freebsd.org Sat Aug 4 13:16:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D33E106AFC4; Sat, 4 Aug 2018 13:16:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 507CF75FD5; Sat, 4 Aug 2018 13:16:01 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ADBE47F60; Sat, 4 Aug 2018 13:16:00 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DG0nw083544; Sat, 4 Aug 2018 13:16:00 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DG0xG083543; Sat, 4 Aug 2018 13:16:00 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041316.w74DG0xG083543@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:16:00 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337304 - vendor/compiler-rt/compiler-rt-release_70-r338892 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/compiler-rt/compiler-rt-release_70-r338892 X-SVN-Commit-Revision: 337304 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:16:04 -0000 Author: dim Date: Sat Aug 4 13:16:00 2018 New Revision: 337304 URL: https://svnweb.freebsd.org/changeset/base/337304 Log: Tag compiler-rt release_70 branch r338892. Added: vendor/compiler-rt/compiler-rt-release_70-r338892/ - copied from r337303, vendor/compiler-rt/dist-release_70/ From owner-svn-src-all@freebsd.org Sat Aug 4 13:16:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77AE7106AFF9; Sat, 4 Aug 2018 13:16:09 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8C7F676105; Sat, 4 Aug 2018 13:16:07 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2E3427F61; Sat, 4 Aug 2018 13:16:07 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DG60e083594; Sat, 4 Aug 2018 13:16:06 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DG6FJ083593; Sat, 4 Aug 2018 13:16:06 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041316.w74DG6FJ083593@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:16:06 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337305 - vendor/libc++/dist-release_70 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/libc++/dist-release_70 X-SVN-Commit-Revision: 337305 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:16:09 -0000 Author: dim Date: Sat Aug 4 13:16:06 2018 New Revision: 337305 URL: https://svnweb.freebsd.org/changeset/base/337305 Log: Vendor import of libc++ release_70 branch r338892: https://llvm.org/svn/llvm-project/libcxx/branches/release_70@338892 Modified: vendor/libc++/dist-release_70/CMakeLists.txt Modified: vendor/libc++/dist-release_70/CMakeLists.txt ============================================================================== --- vendor/libc++/dist-release_70/CMakeLists.txt Sat Aug 4 13:16:00 2018 (r337304) +++ vendor/libc++/dist-release_70/CMakeLists.txt Sat Aug 4 13:16:06 2018 (r337305) @@ -27,7 +27,7 @@ if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR project(libcxx CXX C) set(PACKAGE_NAME libcxx) - set(PACKAGE_VERSION 7.0.0svn) + set(PACKAGE_VERSION 7.0.0) set(PACKAGE_STRING "${PACKAGE_NAME} ${PACKAGE_VERSION}") set(PACKAGE_BUGREPORT "llvm-bugs@lists.llvm.org") From owner-svn-src-all@freebsd.org Sat Aug 4 13:16:14 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29E6D106B01B; Sat, 4 Aug 2018 13:16:14 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7DE4E76139; Sat, 4 Aug 2018 13:16:10 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 327C57F62; Sat, 4 Aug 2018 13:16:10 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DGAPY083640; Sat, 4 Aug 2018 13:16:10 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DGA0p083639; Sat, 4 Aug 2018 13:16:10 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041316.w74DGA0p083639@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:16:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337306 - vendor/libc++/libc++-release_70-r338892 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/libc++/libc++-release_70-r338892 X-SVN-Commit-Revision: 337306 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:16:14 -0000 Author: dim Date: Sat Aug 4 13:16:09 2018 New Revision: 337306 URL: https://svnweb.freebsd.org/changeset/base/337306 Log: Tag libc++ release_70 branch r338892. Added: vendor/libc++/libc++-release_70-r338892/ - copied from r337305, vendor/libc++/dist-release_70/ From owner-svn-src-all@freebsd.org Sat Aug 4 13:16:16 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95C50106B02C; Sat, 4 Aug 2018 13:16:16 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0DB8476201; Sat, 4 Aug 2018 13:16:14 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 990317F63; Sat, 4 Aug 2018 13:16:14 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DGEni083689; Sat, 4 Aug 2018 13:16:14 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DGEMD083688; Sat, 4 Aug 2018 13:16:14 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041316.w74DGEMD083688@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:16:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337307 - vendor/lld/lld-release_70-r338892 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lld/lld-release_70-r338892 X-SVN-Commit-Revision: 337307 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:16:16 -0000 Author: dim Date: Sat Aug 4 13:16:14 2018 New Revision: 337307 URL: https://svnweb.freebsd.org/changeset/base/337307 Log: Tag lld release_70 branch r338892. Added: vendor/lld/lld-release_70-r338892/ - copied from r337306, vendor/lld/dist-release_70/ From owner-svn-src-all@freebsd.org Sat Aug 4 13:16:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D37F106B079; Sat, 4 Aug 2018 13:16:25 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B1F676368; Sat, 4 Aug 2018 13:16:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8171C7F64; Sat, 4 Aug 2018 13:16:21 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DGLZj083738; Sat, 4 Aug 2018 13:16:21 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DGLlt083737; Sat, 4 Aug 2018 13:16:21 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041316.w74DGLlt083737@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 13:16:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-vendor@freebsd.org Subject: svn commit: r337308 - vendor/lldb/lldb-release_70-r338892 X-SVN-Group: vendor X-SVN-Commit-Author: dim X-SVN-Commit-Paths: vendor/lldb/lldb-release_70-r338892 X-SVN-Commit-Revision: 337308 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:16:25 -0000 Author: dim Date: Sat Aug 4 13:16:20 2018 New Revision: 337308 URL: https://svnweb.freebsd.org/changeset/base/337308 Log: Tag lldb release_70 branch r338892. Added: vendor/lldb/lldb-release_70-r338892/ - copied from r337307, vendor/lldb/dist-release_70/ From owner-svn-src-all@freebsd.org Sat Aug 4 13:30:04 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC13F106B615; Sat, 4 Aug 2018 13:30:04 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-lj1-f176.google.com (mail-lj1-f176.google.com [209.85.208.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4848076E8C; Sat, 4 Aug 2018 13:30:04 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-lj1-f176.google.com with SMTP id f1-v6so7081316ljc.9; Sat, 04 Aug 2018 06:30:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TMHtv96tmTtFm/sRDodgETRQFo7xuccXDITvZDd9Vyo=; b=G1BisB7EwZ8rTkW8javMB7oNfTmHYINHrJ5uWUuz3ubp5l9R+cAFQxRL8O9/F0G3/D ODe5JGgd8MLVlF9jSVlwOLFIKXLEQP3LO2DXseVd7MhpA5zn2Gm8cbSYM0pdgH/UWdvP 5Rrm49DL3etSHgv17vjesYATpPyZIIuWFdhFftRYbkARd1xNst8u/0IfWwCLFek6sTeF MQgJ2GHyVg4YWNGAeYrfV9oi/RtiimmmXwPnq9AMQlZqOonKqFOAs1pgtY5Lox/x5Vih BYN1uGsXpYUo/fWlART7ojn4/9aAyqeqj/oqajarzONqGehWIvLptiDOT6MPL8um02w/ llYQ== X-Gm-Message-State: AOUpUlHhjHg7AcurwbC2lqTspxqrE0Y9hRYmDYgixzUzvNCj5EEmMaE4 WTu8aLgqqu+TcF1WxQuxxLNejtw6neJoWVUSLMA= X-Google-Smtp-Source: AAOMgpc640xyusMwP/aq3h98cOjZMsmhlGyiPJxwZ4QlyU+fjMQWjW/MkoisLqDKSQxWxtCGhto6+J63qmFEg6qdLaY= X-Received: by 2002:a2e:558c:: with SMTP id g12-v6mr7944443lje.4.1533389397199; Sat, 04 Aug 2018 06:29:57 -0700 (PDT) MIME-Version: 1.0 References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> <20180804130315.GK6049@kib.kiev.ua> In-Reply-To: <20180804130315.GK6049@kib.kiev.ua> From: Justin Hibbits Date: Sat, 4 Aug 2018 08:29:47 -0500 Message-ID: Subject: Re: svn commit: r337273 - head/sys/dev/nvme To: Konstantin Belousov Cc: John Baldwin , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:30:05 -0000 On Sat, Aug 4, 2018, 08:03 Konstantin Belousov wrote: > On Sat, Aug 04, 2018 at 05:14:31AM -0700, John Baldwin wrote: > > On 8/4/18 1:08 AM, Konstantin Belousov wrote: > > > On Fri, Aug 03, 2018 at 08:04:06PM +0000, Justin Hibbits wrote: > > >> Author: jhibbits > > >> Date: Fri Aug 3 20:04:06 2018 > > >> New Revision: 337273 > > >> URL: https://svnweb.freebsd.org/changeset/base/337273 > > >> > > >> Log: > > >> nvme(4): Add bus_dmamap_sync() at the end of the request path > > >> > > >> Summary: > > >> Some architectures, in this case powerpc64, need explicit > synchronization > > >> barriers vs device accesses. > > >> > > >> Prior to this change, when running 'make buildworld -j72' on a > 18-core > > >> (72-thread) POWER9, I would see controller resets often. With this > change, I > > >> don't see these resets messages, though another tester still does, > for yet to be > > >> determined reasons, so this may not be a complete fix. > Additionally, I see a > > >> ~5-10% speed up in buildworld times, likely due to not needing to > reset the > > >> controller. > > >> > > >> Reviewed By: jimharris > > >> Differential Revision: https://reviews.freebsd.org/D16570 > > >> > > >> Modified: > > >> head/sys/dev/nvme/nvme_qpair.c > > >> > > >> Modified: head/sys/dev/nvme/nvme_qpair.c > > >> > ============================================================================== > > >> --- head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 19:24:04 2018 > (r337272) > > >> +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 20:04:06 2018 > (r337273) > > >> @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct nvme_qpair > *qpair, > > >> req->retries++; > > >> nvme_qpair_submit_tracker(qpair, tr); > > >> } else { > > >> - if (req->type != NVME_REQUEST_NULL) > > >> + if (req->type != NVME_REQUEST_NULL) { > > >> + bus_dmamap_sync(qpair->dma_tag_payload, > > >> + tr->payload_dma_map, > > >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > > >> bus_dmamap_unload(qpair->dma_tag_payload, > > >> tr->payload_dma_map); > > >> + } > > >> > > >> nvme_free_request(req); > > >> tr->req = NULL; > > >> @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct nvme_qpair > *qpai > > >> */ > > >> return (false); > > >> > > >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > > >> while (1) { > > >> cpl = qpair->cpl[qpair->cq_head]; > > >> > > >> @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair > *qpair, st > > >> if (++qpair->sq_tail == qpair->num_entries) > > >> qpair->sq_tail = 0; > > >> > > >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > >> + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); > > >> +#ifndef __powerpc__ > > >> + /* > > >> + * powerpc's bus_dmamap_sync() already includes a heavyweight > sync, but > > >> + * no other archs do. > > >> + */ > > >> wmb(); > > >> +#endif > > > What is the purpose of this call ? It is useless without paired read > > > barrier. So where is the reciprocal rmb() ? > > > > For DMA, the rmb is in the device controller. However, architectures > > that need this kind of ordering should do it in their bus_dmmap_sync op, > > and this explicit one needs to be removed. (Alpha had a wmb in its > > bus_dmamap_sync op for this reason.) > Yes, if something special is needed, it should happen in platform-specific > busdma code. > > Also, if wmb() is needed, then it is not a supposed semantic or > wmb(), but a specific side-effects of one of the instruction in the > implementation of wmb(). > > As I noted, on x86 it is not needed and detrimental to the performance. > According to Jim Harris it is needed for x86. I tried to remove it thinking it was unnecessary with the sync in place now. The wmb() was added way back in r240616 by him, when only x86 was supported by the driver. bus_dmamap_sync() for all archs except powerpc lack a barrier, from what I can see. See the review for more discussion that took place. If it isn't needed I will gladly remove it. - Justin > From owner-svn-src-all@freebsd.org Sat Aug 4 13:57:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF88A106C294; Sat, 4 Aug 2018 13:57:51 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 66165781B9; Sat, 4 Aug 2018 13:57:51 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 433B71061E; Sat, 4 Aug 2018 13:57:51 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74DvpVq004862; Sat, 4 Aug 2018 13:57:51 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74DvouN004861; Sat, 4 Aug 2018 13:57:50 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808041357.w74DvouN004861@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Sat, 4 Aug 2018 13:57:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337316 - in head/sys/amd64: linux linux32 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys/amd64: linux linux32 X-SVN-Commit-Revision: 337316 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 13:57:52 -0000 Author: kib Date: Sat Aug 4 13:57:50 2018 New Revision: 337316 URL: https://svnweb.freebsd.org/changeset/base/337316 Log: Add END()s for amd64 linux futex support routines. Sponsored by: The FreeBSD Foundation MFC after: 1 week Modified: head/sys/amd64/linux/linux_support.s head/sys/amd64/linux32/linux32_support.s Modified: head/sys/amd64/linux/linux_support.s ============================================================================== --- head/sys/amd64/linux/linux_support.s Sat Aug 4 13:39:44 2018 (r337315) +++ head/sys/amd64/linux/linux_support.s Sat Aug 4 13:57:50 2018 (r337316) @@ -49,6 +49,7 @@ ENTRY(futex_xchgl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_xchgl) ENTRY(futex_addl) movq PCPU(CURPCB),%r8 @@ -64,6 +65,7 @@ ENTRY(futex_addl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_addl) ENTRY(futex_orl) movq PCPU(CURPCB),%r8 @@ -83,6 +85,7 @@ ENTRY(futex_orl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_orl) ENTRY(futex_andl) movq PCPU(CURPCB),%r8 @@ -102,6 +105,7 @@ ENTRY(futex_andl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_andl) ENTRY(futex_xorl) movq PCPU(CURPCB),%r8 @@ -121,3 +125,4 @@ ENTRY(futex_xorl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_xorl) Modified: head/sys/amd64/linux32/linux32_support.s ============================================================================== --- head/sys/amd64/linux32/linux32_support.s Sat Aug 4 13:39:44 2018 (r337315) +++ head/sys/amd64/linux32/linux32_support.s Sat Aug 4 13:57:50 2018 (r337316) @@ -49,6 +49,7 @@ ENTRY(futex_xchgl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_xchgl) ENTRY(futex_addl) movq PCPU(CURPCB),%r8 @@ -64,6 +65,7 @@ ENTRY(futex_addl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_addl) ENTRY(futex_orl) movq PCPU(CURPCB),%r8 @@ -83,6 +85,7 @@ ENTRY(futex_orl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_orl) ENTRY(futex_andl) movq PCPU(CURPCB),%r8 @@ -102,6 +105,7 @@ ENTRY(futex_andl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_andl) ENTRY(futex_xorl) movq PCPU(CURPCB),%r8 @@ -121,3 +125,4 @@ ENTRY(futex_xorl) xorl %eax,%eax movq %rax,PCB_ONFAULT(%r8) ret +END(futex_xorl) From owner-svn-src-all@freebsd.org Sat Aug 4 14:13:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F1F5106C978; Sat, 4 Aug 2018 14:13:10 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C816779046; Sat, 4 Aug 2018 14:13:09 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A90B810954; Sat, 4 Aug 2018 14:13:09 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74ED9i3015021; Sat, 4 Aug 2018 14:13:09 GMT (envelope-from delphij@FreeBSD.org) Received: (from delphij@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74ED9Yd015020; Sat, 4 Aug 2018 14:13:09 GMT (envelope-from delphij@FreeBSD.org) Message-Id: <201808041413.w74ED9Yd015020@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: delphij set sender to delphij@FreeBSD.org using -f From: Xin LI Date: Sat, 4 Aug 2018 14:13:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337317 - head/stand/i386/libi386 X-SVN-Group: head X-SVN-Commit-Author: delphij X-SVN-Commit-Paths: head/stand/i386/libi386 X-SVN-Commit-Revision: 337317 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 14:13:10 -0000 Author: delphij Date: Sat Aug 4 14:13:09 2018 New Revision: 337317 URL: https://svnweb.freebsd.org/changeset/base/337317 Log: In r337271, we limited the sector number to the lower of calculated number and CHS based number. However, on some systems, BIOS would report 0 in CHS fields, making the system to think there is 0 sectors. Add a check before comparing the calculated total with bd_sectors. Reviewed by: tsoome, cy Differential Revision: https://reviews.freebsd.org/D16577 Modified: head/stand/i386/libi386/biosdisk.c Modified: head/stand/i386/libi386/biosdisk.c ============================================================================== --- head/stand/i386/libi386/biosdisk.c Sat Aug 4 13:57:50 2018 (r337316) +++ head/stand/i386/libi386/biosdisk.c Sat Aug 4 14:13:09 2018 (r337317) @@ -275,7 +275,7 @@ bd_int13probe(struct bdinfo *bd) total = (uint64_t)params.cylinders * params.heads * params.sectors_per_track; - if (bd->bd_sectors > total) + if (total > 0 && bd->bd_sectors > total) bd->bd_sectors = total; ret = 1; From owner-svn-src-all@freebsd.org Sat Aug 4 14:14:55 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E0D09106CA5B; Sat, 4 Aug 2018 14:14:54 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FFCA7928D; Sat, 4 Aug 2018 14:14:54 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Received: from odin.corp.delphij.net (unknown [IPv6:2601:646:8882:37a:3dd6:e226:c80f:febd]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: delphij/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id EAF31188E4; Sat, 4 Aug 2018 14:14:53 +0000 (UTC) (envelope-from delphij@FreeBSD.org) Subject: Re: svn commit: r337317 - head/stand/i386/libi386 From: Xin Li To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201808041413.w74ED9Yd015020@repo.freebsd.org> Openpgp: preference=signencrypt Autocrypt: addr=delphij@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFJNzwQBEACuPNSJjL/AD8oHFuG72vtx5P7Q6dpiEbFABgw/IohS65yDZDd3qFH9ssQv AsFafwB/ofsk6t7dx6zIC05dv5qjhGIOKSJxFC4U1HAot9+QpeUG+8boTKZiiycrMruItj2U JANlv+gN5h0mAsL5f9eNzhRM43kdjN8cQnBIujhO54Derjnrnqz6cQtoonV6SvvVJZUQGxHK 5R1XYJ6wiTuvoEuRYnNObJmPFWZyYOaGZz0qqD6Qe1BhkZuRzv2bZxwJc3Raap/GF6Pm9J/c hlYHUmm2QLaXvmoP8WNosNjla1fup0tgYQE+7MTtHFVxmVj9ZTihN3rEL5IkeEKjQAqcpe1n Db8X2o4K262LRpFl8WtVMW2TfN5Avpj+knZMl3tkYGvYK/nfadCr6Af4co9mkhX6QYgkerg2 mXEGaQzSD/omnsxHCfqMgdphaX3B3eoY2Fv36BMpjSdHmm0rmwqjqZaqlZn89vQ/I6ATvLyx JsdHwTbrj57audl/RKC+OpREOJPaVULp1L+9zdBXslILO8MJaT6YEw1T29bEj5jvLm03Y4rF u/YTruHcMPpsGbpJckDKiy6ISAbMtPvz7/KR91xPHS6KExGiIakIX9xpIXIDKgq+ecEWwkFK PogoKqO6K0/GYkTRoKdXGzsILvIurtbPqSFqWzbRIyNOa82jowARAQABzRZYaW4gTGkgPGRA ZGVscGhpai5uZXQ+wsF9BBMBCgAnBQJTQvBFAhsjBQkJZgGABQsJCAcDBRUKCQgLBRYCAwEA Ah4BAheAAAoJEJW2GBstM+nsha4P/2Roa/REjZLZlIG1TKOxEDqmwc3fynX4w2g7/FXA7f7Z YO5N4vnnnQdJbDZDt4TJtiP1NHHdheQ5+loJrrCXVlU31LuJv1ebM2Ajsuo/0l3tfulEf6Ki GoozmaNZAhwiGJkQVg9DSKsea5xIA31lPnFH4T0SKn8Q6F4HYienmJJtlKVTADvYXA+DRmv0 rNOyVe+V/AuTFuelKg3Ua5a+dY3oqtrQQvFS4n7iIrNjEMUBVx0XTrYLddnF+YjXDg5Phf0D pV/2yJOXiTGiZMK6i7vwHZkJvarACoTSrUrr6OBuZv5Gf87VgifZKLr2Fuf+FePiVCoZTQiL 0hPQyABMzeWa32P6BY2LBMMMFvFiyL5pN5k6nJ0nx4skl8UxZ5ay4yyVg2u3f4aI3+m0XlZ+ iixrjmCTGi1s+d/n6E3eFXdJUUbSOXLZaU4qrbXRzTYCZmZViryv7ibtOHXnG6oWy7BFEHuT rUW6OBvsQDTp5iQ6opENJ5/ZzSA3c5p1WS9Ezv4Bpdqcm7LTQX2j6kXikj8YqICtDF2rkKZ2 Ynjm9se9B0h/T1SOaSpbtRg05UKjsinDq2x8EeX21yFs3UyvwePLrGoNKL45EJM0xwxrnlfr M0ayKJNLoYysY78d54hg7XMmkQD/oZz9I+k4fN6CmZ2i5WGH2BgYs0313JMHxSg7zsFNBFJN zwQBEADPtS+nfTKM6PwgSWLDGVgUYQ/RLaKzCcpQAf4ryLBugXpx3s2BBT1bixX7CpsLXKQi +RRETgSFzDaBL9SEs2ZDV2YT+zGp08aijK/Yl9+RIeezAukI3c+XMHuo8ktUWJmo5/1DX07q G30ckG7uFuTnt31sFzwhh/ZeSuLFyel/fWF48KExLDIVa8DyEUJaYvE9Vfph4T/3LkKuzVTy +iwUBLiSLj5G5N70A+4usbL3eKyYrJqCSaLfrP99/nlgBhMAHVcKcv0uqSuiaH9OMqg1VjQs N8j6NDQug9QrbBTM6U7oZWF/AK+CdFoe+leq5MZfzwCevs0BQgxWm4SHMpXL2vtly67QSPMY dl96fOzw8YbKHv1o0ixhCvc37cI9oUVuSJLXKhEEAvWvLuusiuNeoz+6aPlELvD8h5txJqui tVOzctvJ7ktGZTNiz73tKYVdkKaQVyo8QJFLCNLnUulrQ5wXwteYPg6mrpBxu9VqgDrMp7eB T2kaZ4GRBoMWXXPYSIEe5PM5hhNCsSUfqrKj34UZPijPe+HiWoFJ4S5vIpzutiae11Ctki7u XzeLAhOJQB2raraIqDlFP9I9Zj9JOAZhmiKSEWKfOooCNxQYGiUdPrdYnAe+m7FXRomjF0OO gSepNIESt2gOEIbE5cMxQ0gAueNJc58eHCjWhsNJIwARAQABwsFlBBgBCgAPBQJSTc8EAhsM BQkJZgGAAAoJEJW2GBstM+nsh8EP/1sxZpkJelu+smmqaqdrGHlNrFVLOmeN5yr2IGHBUbmF htjr7fVoU8T0mUnlUU724aKPla4nWhMb4NMu+VxRRFGaT2TYpyR6VIxaStycyUdMGjdXV0Pz TGmxFXhNZXKEITXH9sIxuONBp1czl4AgwN7AAl1MKyV13AaLIyajs58mYmuXtyFn/O+4lxh5 nl2Fa3L9YkL9O7QU2p6WAnDky+L3PgUWp1AzJGfYlLZ8XXCi+KK+pnta+f9yKHt/Oqd/s7OC W4mXgFkBrfuSZZofa4eZckh5u0yBYW3OnEJhClgxRbuOhyYwqQr5oxPrQtjtbMiBzbrOkHhy NnrVCFd9EqlojREGDefHo3V+ZlUOc6OoN3CAYnNa2uLEOm5DCuqOE4z5atBCih5EyITPp7JP J2disEP6ddipcilqbnJdP+TyRQwSv5qRNy8cHahD1Cg9XJJHiC3qr+W3eOtqPkJxhU5biPEr 7dljaLS1Ij771brzqO/x5zW1L9py7muXzYBsW8+keKj8LOYs2242KgjI5Og9YhIJGBFBNddQ wxKBKQpytKQOiXwjhk4Nj77U796bsCd/jIS0r0ZUKBEptPyKso7ncfrm163aEmSaDUkiIjyp 9CEOVT87D+VAVh9PyLGP1niQzWEWFSK36tRGZlF0odP1ZB6wub9zq2DxFouSjHgH Organization: The FreeBSD Project Message-ID: <0b5d21a1-00e6-2264-c03e-f0d82ae2d6d0@FreeBSD.org> Date: Sat, 4 Aug 2018 07:14:49 -0700 User-Agent: Thunderbird MIME-Version: 1.0 In-Reply-To: <201808041413.w74ED9Yd015020@repo.freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Yb07RtaDpQgkK9NzFUzgqXY686Potv85g" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 14:14:55 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Yb07RtaDpQgkK9NzFUzgqXY686Potv85g Content-Type: multipart/mixed; boundary="PSQU60lD1VMEs1l8Dl40hZ733F19v43Gm"; protected-headers="v1" From: Xin Li To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Message-ID: <0b5d21a1-00e6-2264-c03e-f0d82ae2d6d0@FreeBSD.org> Subject: Re: svn commit: r337317 - head/stand/i386/libi386 References: <201808041413.w74ED9Yd015020@repo.freebsd.org> In-Reply-To: <201808041413.w74ED9Yd015020@repo.freebsd.org> --PSQU60lD1VMEs1l8Dl40hZ733F19v43Gm Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 8/4/18 07:13, Xin LI wrote: > Author: delphij > Date: Sat Aug 4 14:13:09 2018 > New Revision: 337317 > URL: https://svnweb.freebsd.org/changeset/base/337317 >=20 > Log: > In r337271, we limited the sector number to the lower of calculated > number and CHS based number. However, on some systems, BIOS would > report 0 in CHS fields, making the system to think there is 0 sectors= =2E > =20 > Add a check before comparing the calculated total with bd_sectors. > =20 > Reviewed by: tsoome, cy > Differential Revision: https://reviews.freebsd.org/D16577 Ooops. Should be https://reviews.freebsd.org/D16588 --PSQU60lD1VMEs1l8Dl40hZ733F19v43Gm-- --Yb07RtaDpQgkK9NzFUzgqXY686Potv85g Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJbZbTcAAoJEJW2GBstM+nsKa4QAKxutNDybItQ9dhnITWi9El7 zte5fVzrUIqiguB3Zn+fpTp/yKfCQTIWsAP5SfXNHQHDgDo8YDG0W/if03MgSbhj 59UjGdrnm8LuRgEC6ua1clryQWDTWBb+Lm4JF0u88RFT7pcC+4BLLrljZEQotf+L BxZnLFONTigoNINivPoErpP7PCVa1U9z1RQSgMeWDIRh1go+cwYV6x10YSYbRCXB ar+EX2OxewDX700phOIqcop+xa49UUsNaQIaK8L86Icb+LtrHPJlER5hqjms/0SQ VJfrMbHXXU81VhgYJppnvXbFUdHlOBBvYXhTqCFWYD1kYTFiQmhdXv2nPP/ORxTw Pvl7USAiAaa3BJfzCnM3keKk0RVsQmQJw11EeybfI1KFdLynGTZ7S7L7N46zsfHA U6phQPeHu3ZLLkOvwqtVnJS/fEXaRYWny93yo/Asn9jc5sAI+45Dvx/l8YXtzZWP lqSZy4YNHEiW71QMNViu+0tEkFeApHBykp92l0P1Gqjxc2WAzf9KqMxTgKpgpj8m VRevVrfKrxcXIfHJMvHT0eOBqUP7oA9ugp57DTTfQ+zE2T+72nEOdVOQ5r1qga3j PBewVnNn01Gq4tMDVlK2BasHHpPl7c/cYKxadQWSZfSOvsoacRu8DCPjJ1SEk5Gj C5FGDYQQSLbzbxRLJtLk =Zmm3 -----END PGP SIGNATURE----- --Yb07RtaDpQgkK9NzFUzgqXY686Potv85g-- From owner-svn-src-all@freebsd.org Sat Aug 4 14:26:40 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 99C4B106CF56; Sat, 4 Aug 2018 14:26:40 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E0E0E79B7E; Sat, 4 Aug 2018 14:26:39 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id lxVlf8gBgwyxUlxVmf58mc; Sat, 04 Aug 2018 08:26:32 -0600 X-Authority-Analysis: v=2.3 cv=NPJhBHyg c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=dapMudl6Dx4A:10 a=xfDLHkLGAAAA:8 a=HHGDD-5mAAAA:8 a=SWg00rOMAAAA:8 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=NxvbkCmBGjDIeVatmJsA:9 a=XYqoIGHLEpixj2xu:21 a=Ca4R-PxvlJ3HifTr:21 a=CjuIK1q_8ugA:10 a=IfaqVvZgccqrtc8gcwf2:22 a=nWvTgx2JuP7DHgfbJPXu:22 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id BFE0519AD; Sat, 4 Aug 2018 07:26:46 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id w74EQS1X028532; Sat, 4 Aug 2018 07:26:28 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id w74EQSHd028096; Sat, 4 Aug 2018 07:26:28 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201808041426.w74EQSHd028096@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Warner Losh cc: Toomas Soome , Xin LI , Cy Schubert , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org, tsoome@freebsd.org Subject: Re: svn commit: r337271 - head/stand/i386/libi386 In-Reply-To: Message from Warner Losh of "Sat, 04 Aug 2018 12:11:34 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 04 Aug 2018 07:26:28 -0700 X-CMAE-Envelope: MS4wfCXOyzuaLNyy0nDHS20Z6I7QZUbFvFVN6Er6i/SM2Or5MewQv6At9gDT6IOLxkUXWLF9eKNYpRDVD89N8DpblGFETFUfG0+nrX8o4AWrGvddXiwWNHdn AHEFnCMkvG6OfFsnfaJgu7PG+RB43zyieRUzQrEeHtN4r4e0biTklCnmHBttrLWeULJj6zix24xnn9GVJ/nEgobrcsQpDnOJwDoqqtDnRORTgTq2Nkfd7VM1 31OkRzbZW9FjtaVZ5ija3sgpHI0wE0OC5wSr84ot3lCwNPBXB8FLNphXCl4r0u3fUydFotZwoWs+hZ/kHYSiCQselKMqphAH0w8r26xnI04= X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 14:26:40 -0000 In message , Warner Losh writes: > --00000000000017c64d05729a1cbf > Content-Type: text/plain; charset="UTF-8" > Content-Transfer-Encoding: quoted-printable > > On Sat, Aug 4, 2018, 11:58 AM Toomas Soome wrote: > > > > > > > > On 4 Aug 2018, at 11:54, Xin Li wrote: > > > > > > Hi, Cy, > > > > > > On 8/3/18 12:11, Cy Schubert wrote: > > >> Author: cy > > >> Date: Fri Aug 3 19:11:00 2018 > > >> New Revision: 337271 > > >> URL: https://svnweb.freebsd.org/changeset/base/337271 > > >> > > >> Log: > > >> Some drives report a geometry that is inconsisetent with the total > > >> number of sectors reported through the BIOS. Cylinders * heads * > > >> sectors may not necessarily be equal to the total number of sectors > > >> reported through int13h function 48h. > > >> > > >> An example of this is when a Mediasonic HD3-U2B PATA to USB enclosure > > >> with a 80 GB disk is attached. Loader hangs at line 506 of > > >> stand/i386/libi386/biosdisk.c while attempting to read sectors beyond > > >> the end of the disk, sector 156906855. I discovered that the Mediason= > ic > > >> enclosure was reporting the disk with 9767 cylinders, 255 heads, 63 > > >> sectors/track. That's 156906855 sectors. However camcontrol and > > >> Windows 10 both report report the disk having 156301488 sectors, not > > >> the calculated value. At line 280 biosdisk.c sets the sectors to the > > >> higher of either bd->bd_sectors or the total calculated at line 276 > > >> (156906855) instead of the lower and correct value of 156301488 > > reported > > >> by int 13h 48h. > > >> > > >> This was tested on all three of my Mediasonic HD3-U2B PATA to USB > > >> enclosures. > > >> > > >> Instead of using the higher of bd_sectors (returned by int13h) or the > > >> calculated value, this patch uses the lower and safer of the values. > > >> > > >> Reviewed by: tsoome@ > > >> Differential Revision: https://reviews.freebsd.org/D16577 > > >> > > >> Modified: > > >> head/stand/i386/libi386/biosdisk.c > > >> > > >> Modified: head/stand/i386/libi386/biosdisk.c > > >> > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D=3D=3D > > >> --- head/stand/i386/libi386/biosdisk.c Fri Aug 3 18:52:51 2018 > > (r337270) > > >> +++ head/stand/i386/libi386/biosdisk.c Fri Aug 3 19:11:00 2018 > > (r337271) > > >> @@ -275,7 +275,7 @@ bd_int13probe(struct bdinfo *bd) > > >> > > >> total =3D (uint64_t)params.cylinders * > > >> params.heads * params.sectors_per_track; > > >> - if (bd->bd_sectors < total) > > >> + if (bd->bd_sectors > total) > > >> bd->bd_sectors =3D total; > > >> > > >> ret =3D 1; > > >> > > > > > > This broke loader on my system, but I think your reasoning was valid so > > > I took a deeper look and discovered that on my system, INT 13h, functio= > n > > > 48h would give zeros in EDD parameters' CHS fields. With that, the > > > calculated CHS based total would be 0, and your change would cause > > > bd_sectors be zeroed. > > > > > > Could you please let me know if https://reviews.freebsd.org/D16588 make= > s > > > sense to you? (I'm not 100% certain if I have followed the code). It > > > allowed my Asrock C2750D4I based board to boot from ZFS. > > > > > > > I have in mind something a bit different for some time, but haven=E2=80= > =99t had > > chance to complete it because I have no =E2=80=9Cweird=E2=80=9D systems t= > o validate the > > idea:D I=E2=80=99ll try to get a bit of time and post an phabricator soon= > . > > > > I think the phab looks good, but I am only on my phone so can't say so > there... > > Warner It looks good. The only case that hasn't been considered so far is if bd_sectors is arbitrarily low but not zero and the calculated value is the correct one. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-svn-src-all@freebsd.org Sat Aug 4 14:47:44 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6940F106D93B; Sat, 4 Aug 2018 14:47:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E906C7A7DB; Sat, 4 Aug 2018 14:47:43 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w74ElXJX054970 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 4 Aug 2018 17:47:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w74ElXJX054970 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w74ElXjX054969; Sat, 4 Aug 2018 17:47:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 4 Aug 2018 17:47:33 +0300 From: Konstantin Belousov To: Justin Hibbits Cc: John Baldwin , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r337273 - head/sys/dev/nvme Message-ID: <20180804144733.GO6049@kib.kiev.ua> References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> <20180804130315.GK6049@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 14:47:44 -0000 On Sat, Aug 04, 2018 at 08:29:47AM -0500, Justin Hibbits wrote: > On Sat, Aug 4, 2018, 08:03 Konstantin Belousov wrote: > > > On Sat, Aug 04, 2018 at 05:14:31AM -0700, John Baldwin wrote: > > > On 8/4/18 1:08 AM, Konstantin Belousov wrote: > > > > On Fri, Aug 03, 2018 at 08:04:06PM +0000, Justin Hibbits wrote: > > > >> Author: jhibbits > > > >> Date: Fri Aug 3 20:04:06 2018 > > > >> New Revision: 337273 > > > >> URL: https://svnweb.freebsd.org/changeset/base/337273 > > > >> > > > >> Log: > > > >> nvme(4): Add bus_dmamap_sync() at the end of the request path > > > >> > > > >> Summary: > > > >> Some architectures, in this case powerpc64, need explicit > > synchronization > > > >> barriers vs device accesses. > > > >> > > > >> Prior to this change, when running 'make buildworld -j72' on a > > 18-core > > > >> (72-thread) POWER9, I would see controller resets often. With this > > change, I > > > >> don't see these resets messages, though another tester still does, > > for yet to be > > > >> determined reasons, so this may not be a complete fix. > > Additionally, I see a > > > >> ~5-10% speed up in buildworld times, likely due to not needing to > > reset the > > > >> controller. > > > >> > > > >> Reviewed By: jimharris > > > >> Differential Revision: https://reviews.freebsd.org/D16570 > > > >> > > > >> Modified: > > > >> head/sys/dev/nvme/nvme_qpair.c > > > >> > > > >> Modified: head/sys/dev/nvme/nvme_qpair.c > > > >> > > ============================================================================== > > > >> --- head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 19:24:04 2018 > > (r337272) > > > >> +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 20:04:06 2018 > > (r337273) > > > >> @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct nvme_qpair > > *qpair, > > > >> req->retries++; > > > >> nvme_qpair_submit_tracker(qpair, tr); > > > >> } else { > > > >> - if (req->type != NVME_REQUEST_NULL) > > > >> + if (req->type != NVME_REQUEST_NULL) { > > > >> + bus_dmamap_sync(qpair->dma_tag_payload, > > > >> + tr->payload_dma_map, > > > >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > > > >> bus_dmamap_unload(qpair->dma_tag_payload, > > > >> tr->payload_dma_map); > > > >> + } > > > >> > > > >> nvme_free_request(req); > > > >> tr->req = NULL; > > > >> @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct nvme_qpair > > *qpai > > > >> */ > > > >> return (false); > > > >> > > > >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > > >> + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > > > >> while (1) { > > > >> cpl = qpair->cpl[qpair->cq_head]; > > > >> > > > >> @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair > > *qpair, st > > > >> if (++qpair->sq_tail == qpair->num_entries) > > > >> qpair->sq_tail = 0; > > > >> > > > >> + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > > >> + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); > > > >> +#ifndef __powerpc__ > > > >> + /* > > > >> + * powerpc's bus_dmamap_sync() already includes a heavyweight > > sync, but > > > >> + * no other archs do. > > > >> + */ > > > >> wmb(); > > > >> +#endif > > > > What is the purpose of this call ? It is useless without paired read > > > > barrier. So where is the reciprocal rmb() ? > > > > > > For DMA, the rmb is in the device controller. However, architectures > > > that need this kind of ordering should do it in their bus_dmmap_sync op, > > > and this explicit one needs to be removed. (Alpha had a wmb in its > > > bus_dmamap_sync op for this reason.) > > Yes, if something special is needed, it should happen in platform-specific > > busdma code. > > > > Also, if wmb() is needed, then it is not a supposed semantic or > > wmb(), but a specific side-effects of one of the instruction in the > > implementation of wmb(). > > > > As I noted, on x86 it is not needed and detrimental to the performance. > > > > According to Jim Harris it is needed for x86. I tried to remove it thinking > it was unnecessary with the sync in place now. The wmb() was added way > back in r240616 by him, when only x86 was supported by the driver. > bus_dmamap_sync() for all archs except powerpc lack a barrier, from what I > can see. See the review for more discussion that took place. If it isn't > needed I will gladly remove it. I am very curious why x86 would need MFENCE there. I do not see an explanation in the review. From owner-svn-src-all@freebsd.org Sat Aug 4 14:52:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 144AC106DBE6; Sat, 4 Aug 2018 14:52:33 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB4647ACEA; Sat, 4 Aug 2018 14:52:32 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9D9DF10FB6; Sat, 4 Aug 2018 14:52:32 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74EqWZ4035653; Sat, 4 Aug 2018 14:52:32 GMT (envelope-from trasz@FreeBSD.org) Received: (from trasz@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74EqWZB035652; Sat, 4 Aug 2018 14:52:32 GMT (envelope-from trasz@FreeBSD.org) Message-Id: <201808041452.w74EqWZB035652@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: trasz set sender to trasz@FreeBSD.org using -f From: Edward Tomasz Napierala Date: Sat, 4 Aug 2018 14:52:32 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337321 - head/sbin/init X-SVN-Group: head X-SVN-Commit-Author: trasz X-SVN-Commit-Paths: head/sbin/init X-SVN-Commit-Revision: 337321 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 14:52:33 -0000 Author: trasz Date: Sat Aug 4 14:52:32 2018 New Revision: 337321 URL: https://svnweb.freebsd.org/changeset/base/337321 Log: Make it possible for init to execute any executable, not just sh(1) scripts. This means one should be able to eg rewrite their /etc/rc in Python. Reviewed by: kib MFC after: 2 weeks Relnotes: yes Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D16565 Modified: head/sbin/init/init.c Modified: head/sbin/init/init.c ============================================================================== --- head/sbin/init/init.c Sat Aug 4 14:39:45 2018 (r337320) +++ head/sbin/init/init.c Sat Aug 4 14:52:32 2018 (r337321) @@ -1057,7 +1057,7 @@ static state_func_t run_script(const char *script) { pid_t pid, wpid; - int status; + int error, status; char *argv[4]; const char *shell; struct sigaction sa; @@ -1086,6 +1086,21 @@ run_script(const char *script) #ifdef LOGIN_CAP setprocresources(RESOURCE_RC); #endif + + /* + * Try to directly execute the script first. If it + * fails, try the old method of passing the script path + * to sh(1). Don't complain if it fails because of + * the missing execute bit. + */ + error = access(script, X_OK); + if (error == 0) { + execv(script, argv + 1); + warning("can't exec %s: %m", script); + } else if (errno != EACCES) { + warning("can't access %s: %m", script); + } + execv(shell, argv); stall("can't exec %s for %s: %m", shell, script); _exit(1); /* force single user mode */ @@ -1854,7 +1869,7 @@ static int runshutdown(void) { pid_t pid, wpid; - int status; + int error, status; int shutdowntimeout; size_t len; char *argv[4]; @@ -1897,6 +1912,21 @@ runshutdown(void) #ifdef LOGIN_CAP setprocresources(RESOURCE_RC); #endif + + /* + * Try to directly execute the script first. If it + * fails, try the old method of passing the script path + * to sh(1). Don't complain if it fails because of + * the missing execute bit. + */ + error = access(_path_rundown, X_OK); + if (error == 0) { + execv(_path_rundown, argv + 1); + warning("can't exec %s: %m", _path_rundown); + } else if (errno != EACCES) { + warning("can't access %s: %m", _path_rundown); + } + execv(shell, argv); warning("can't exec %s for %s: %m", shell, _PATH_RUNDOWN); _exit(1); /* force single user mode */ From owner-svn-src-all@freebsd.org Sat Aug 4 14:57:24 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 821E1106DDF0; Sat, 4 Aug 2018 14:57:24 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 365DB7AF5E; Sat, 4 Aug 2018 14:57:24 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 16BBD10FC0; Sat, 4 Aug 2018 14:57:24 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74EvNRj035917; Sat, 4 Aug 2018 14:57:23 GMT (envelope-from dim@FreeBSD.org) Received: (from dim@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74EvNHc035916; Sat, 4 Aug 2018 14:57:23 GMT (envelope-from dim@FreeBSD.org) Message-Id: <201808041457.w74EvNHc035916@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dim set sender to dim@FreeBSD.org using -f From: Dimitry Andric Date: Sat, 4 Aug 2018 14:57:23 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337322 - head/sys/dev/hyperv/pcib X-SVN-Group: head X-SVN-Commit-Author: dim X-SVN-Commit-Paths: head/sys/dev/hyperv/pcib X-SVN-Commit-Revision: 337322 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 14:57:24 -0000 Author: dim Date: Sat Aug 4 14:57:23 2018 New Revision: 337322 URL: https://svnweb.freebsd.org/changeset/base/337322 Log: Fix build of hyperv with base gcc on i386 Summary: Base gcc fails to compile `sys/dev/hyperv/pcib/vmbus_pcib.c` for i386, with the following -Werror warnings: cc1: warnings being treated as errors /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c: In function 'new_pcichild_device': /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c:567: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c: In function 'vmbus_pcib_on_channel_callback': /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c:940: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c: In function 'hv_pci_protocol_negotiation': /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c:1012: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c: In function 'hv_pci_enter_d0': /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c:1073: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c: In function 'hv_send_resources_allocated': /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c:1125: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c: In function 'vmbus_pcib_map_msi': /usr/src/sys/dev/hyperv/pcib/vmbus_pcib.c:1730: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] This is because on i386, several casts from `uint64_t` to a pointer reduce the value from 64 bit to 32 bit. For gcc, this can be fixed by an intermediate cast to uintptr_t. Note that I am assuming the incoming values will always fit into 32 bit! Differential Revision: https://reviews.freebsd.org/D15753 MFC after: 3 days Modified: head/sys/dev/hyperv/pcib/vmbus_pcib.c Modified: head/sys/dev/hyperv/pcib/vmbus_pcib.c ============================================================================== --- head/sys/dev/hyperv/pcib/vmbus_pcib.c Sat Aug 4 14:52:32 2018 (r337321) +++ head/sys/dev/hyperv/pcib/vmbus_pcib.c Sat Aug 4 14:57:23 2018 (r337322) @@ -564,7 +564,7 @@ new_pcichild_device(struct hv_pcibus *hbus, struct pci ret = vmbus_chan_send(hbus->sc->chan, VMBUS_CHANPKT_TYPE_INBAND, VMBUS_CHANPKT_FLAG_RC, - res_req, sizeof(*res_req), (uint64_t)&ctxt.pkt); + res_req, sizeof(*res_req), (uint64_t)(uintptr_t)&ctxt.pkt); if (ret) goto err; @@ -937,7 +937,8 @@ vmbus_pcib_on_channel_callback(struct vmbus_channel *c switch (pkt->cph_type) { case VMBUS_CHANPKT_TYPE_COMP: - comp_packet = (struct pci_packet *)pkt->cph_xactid; + comp_packet = + (struct pci_packet *)(uintptr_t)pkt->cph_xactid; response = (struct pci_response *)pkt; comp_packet->completion_func(comp_packet->compl_ctxt, response, bytes_rxed); @@ -1009,7 +1010,7 @@ hv_pci_protocol_negotiation(struct hv_pcibus *hbus) ret = vmbus_chan_send(hbus->sc->chan, VMBUS_CHANPKT_TYPE_INBAND, VMBUS_CHANPKT_FLAG_RC, version_req, sizeof(*version_req), - (uint64_t)&ctxt.pkt); + (uint64_t)(uintptr_t)&ctxt.pkt); if (ret) goto out; @@ -1070,7 +1071,7 @@ hv_pci_enter_d0(struct hv_pcibus *hbus) ret = vmbus_chan_send(hbus->sc->chan, VMBUS_CHANPKT_TYPE_INBAND, VMBUS_CHANPKT_FLAG_RC, d0_entry, sizeof(*d0_entry), - (uint64_t)&ctxt.pkt); + (uint64_t)(uintptr_t)&ctxt.pkt); if (ret) goto out; @@ -1122,7 +1123,8 @@ hv_send_resources_allocated(struct hv_pcibus *hbus) ret = vmbus_chan_send(hbus->sc->chan, VMBUS_CHANPKT_TYPE_INBAND, VMBUS_CHANPKT_FLAG_RC, - &pkt->message, sizeof(*res_assigned), (uint64_t)pkt); + &pkt->message, sizeof(*res_assigned), + (uint64_t)(uintptr_t)pkt); if (ret) { free_completion(&comp_pkt.host_event); break; @@ -1727,7 +1729,7 @@ vmbus_pcib_map_msi(device_t pcib, device_t child, int ret = vmbus_chan_send(sc->chan, VMBUS_CHANPKT_TYPE_INBAND, VMBUS_CHANPKT_FLAG_RC, int_pkt, sizeof(*int_pkt), - (uint64_t)&ctxt.pkt); + (uint64_t)(uintptr_t)&ctxt.pkt); if (ret) { free_completion(&comp.comp_pkt.host_event); return (ret); From owner-svn-src-all@freebsd.org Sat Aug 4 15:11:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77B43106E3EA for ; Sat, 4 Aug 2018 15:11:19 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 019BF7B9E7 for ; Sat, 4 Aug 2018 15:11:18 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-RoutePath: aGlwcGll X-MHO-User: a1eda9cb-97f8-11e8-904b-1d2e466b3c59 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.ore.mailhop.org (Halon) with ESMTPSA id a1eda9cb-97f8-11e8-904b-1d2e466b3c59; Sat, 04 Aug 2018 15:11:16 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w74FBFoD046128; Sat, 4 Aug 2018 09:11:15 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1533395475.9860.9.camel@freebsd.org> Subject: Re: svn commit: r337273 - head/sys/dev/nvme From: Ian Lepore To: Justin Hibbits , Konstantin Belousov Cc: John Baldwin , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Date: Sat, 04 Aug 2018 09:11:15 -0600 In-Reply-To: References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> <20180804130315.GK6049@kib.kiev.ua> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 15:11:19 -0000 On Sat, 2018-08-04 at 08:29 -0500, Justin Hibbits wrote: > On Sat, Aug 4, 2018, 08:03 Konstantin Belousov wrote: > > > > > On Sat, Aug 04, 2018 at 05:14:31AM -0700, John Baldwin wrote: > > > > > > On 8/4/18 1:08 AM, Konstantin Belousov wrote: > > > > > > > > On Fri, Aug 03, 2018 at 08:04:06PM +0000, Justin Hibbits wrote: > > > > > > > > > > Author: jhibbits > > > > > Date: Fri Aug  3 20:04:06 2018 > > > > > New Revision: 337273 > > > > > URL: https://svnweb.freebsd.org/changeset/base/337273 > > > > > > > > > > Log: > > > > >   nvme(4): Add bus_dmamap_sync() at the end of the request path > > > > > > > > > >   Summary: > > > > >   Some architectures, in this case powerpc64, need explicit > > synchronization > > > > > > > > > > > > > > > > >   barriers vs device accesses. > > > > > > > > > >   Prior to this change, when running 'make buildworld -j72' on a > > 18-core > > > > > > > > > > > > > > > > >   (72-thread) POWER9, I would see controller resets often.  With this > > change, I > > > > > > > > > > > > > > > > >   don't see these resets messages, though another tester still does, > > for yet to be > > > > > > > > > > > > > > > > >   determined reasons, so this may not be a complete fix. > > Additionally, I see a > > > > > > > > > > > > > > > > >   ~5-10% speed up in buildworld times, likely due to not needing to > > reset the > > > > > > > > > > > > > > > > >   controller. > > > > > > > > > >   Reviewed By: jimharris > > > > >   Differential Revision: https://reviews.freebsd.org/D16570 > > > > > > > > > > Modified: > > > > >   head/sys/dev/nvme/nvme_qpair.c > > > > > > > > > > Modified: head/sys/dev/nvme/nvme_qpair.c > > > > > > > ============================================================================== > > > > > > > > > > > > > > > > > --- head/sys/dev/nvme/nvme_qpair.c Fri Aug  3 19:24:04 2018 > > (r337272) > > > > > > > > > > > > > > > > > +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug  3 20:04:06 2018 > > (r337273) > > > > > > > > > > > > > > > > > @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct nvme_qpair > > *qpair, > > > > > > > > > > > > > > > > >            req->retries++; > > > > >            nvme_qpair_submit_tracker(qpair, tr); > > > > >    } else { > > > > > -          if (req->type != NVME_REQUEST_NULL) > > > > > +          if (req->type != NVME_REQUEST_NULL) { > > > > > +                  bus_dmamap_sync(qpair->dma_tag_payload, > > > > > +                      tr->payload_dma_map, > > > > > +                      BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > > > > >                    bus_dmamap_unload(qpair->dma_tag_payload, > > > > >                        tr->payload_dma_map); > > > > > +          } > > > > > > > > > >            nvme_free_request(req); > > > > >            tr->req = NULL; > > > > > @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct nvme_qpair > > *qpai > > > > > > > > > > > > > > > > >             */ > > > > >            return (false); > > > > > > > > > > +  bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > > > > +      BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > > > > >    while (1) { > > > > >            cpl = qpair->cpl[qpair->cq_head]; > > > > > > > > > > @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair > > *qpair, st > > > > > > > > > > > > > > > > >    if (++qpair->sq_tail == qpair->num_entries) > > > > >            qpair->sq_tail = 0; > > > > > > > > > > +  bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > > > > +      BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); > > > > > +#ifndef __powerpc__ > > > > > +  /* > > > > > +   * powerpc's bus_dmamap_sync() already includes a heavyweight > > sync, but > > > > > > > > > > > > > > > > > +   * no other archs do. > > > > > +   */ > > > > >    wmb(); > > > > > +#endif > > > > What is the purpose of this call ?  It is useless without paired read > > > > barrier.  So where is the reciprocal rmb() ? > > > For DMA, the rmb is in the device controller.  However, architectures > > > that need this kind of ordering should do it in their bus_dmmap_sync op, > > > and this explicit one needs to be removed.  (Alpha had a wmb in its > > > bus_dmamap_sync op for this reason.) > > Yes, if something special is needed, it should happen in platform-specific > > busdma code. > > > > Also, if wmb() is needed, then it is not a supposed semantic or > > wmb(), but a specific side-effects of one of the instruction in the > > implementation of wmb(). > > > > As I noted, on x86 it is not needed and detrimental to the performance. > > > According to Jim Harris it is needed for x86. I tried to remove it thinking > it was unnecessary with the sync in place now.  The wmb() was added way > back in r240616 by him, when only x86 was supported by the driver. > bus_dmamap_sync() for all archs except powerpc lack a barrier, from what I > can see.  See the review for more discussion that took place. If it isn't > needed I will gladly remove it. > The arm busdma sync routines definitely invoke the necessary barriers, which on arm are spelled 'dsb' and 'isb' and are in the low-level routines called from the busdma code, such as dcache_wb_pou(), dcache_inv_poc(), etc. -- Ian From owner-svn-src-all@freebsd.org Sat Aug 4 15:21:09 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CC62106E88F; Sat, 4 Aug 2018 15:21:09 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E78047C015; Sat, 4 Aug 2018 15:21:08 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-lj1-f174.google.com with SMTP id w16-v6so7226996ljh.12; Sat, 04 Aug 2018 08:21:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=GW4lXYu0VS+VseO3KBrLpOYA65/d62U5pa3g6NnDvHg=; b=pgeIJPBGT0SB5cvGkv/hJ9o9qTEvG48L0onGxK6FsYEwBxfwlkuvBu4XC6PT3XdcHl vXw9jQhP4/oAn7Jnx+JU6IAGshw+A7zwmernPHgrWQlLVG79qcT7Xz8qazapibrlPE23 GCfC9gPwDWVg+KnYG2sxU2roR6n2s5KLdJDVUJOUGzLd1GGCNkzsv9oC7adoh/12YOxW i3T5pUbybYT42Vmow8jBs8BCts0zrU3JMjpjHG/OiNeqq2krd1DrwbDXuCDPuO1IWBpq hE2rr39i/WDBWolBJ7TGtV2fqu840oXSRThr7ohc8Amxbb0lO1ckY8U/zk36XaJTgpQw XFgA== X-Gm-Message-State: AOUpUlECetaouf4i/EULcK6DDVLotzs3nqE+66lmCyahWGL55HCLB9o+ gA4Olk57pWxnBty2XqA/riu9P0u/u8I2XOTl+N+B0LD1 X-Google-Smtp-Source: AAOMgpdix+OHZx9lQDw7u26JuEO5BIJjyefdBQ3LqjCRnmSk3fTVB/Ujjm5qOJT1kD/3aqrxz370Qdo1xmnH47ojXCs= X-Received: by 2002:a2e:91d6:: with SMTP id u22-v6mr8053219ljg.64.1533396060724; Sat, 04 Aug 2018 08:21:00 -0700 (PDT) MIME-Version: 1.0 References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> <20180804130315.GK6049@kib.kiev.ua> <1533395475.9860.9.camel@freebsd.org> In-Reply-To: <1533395475.9860.9.camel@freebsd.org> From: Justin Hibbits Date: Sat, 4 Aug 2018 10:20:50 -0500 Message-ID: Subject: Re: svn commit: r337273 - head/sys/dev/nvme To: Ian Lepore Cc: Konstantin Belousov , John Baldwin , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 15:21:09 -0000 On Sat, Aug 4, 2018, 10:11 Ian Lepore wrote: > On Sat, 2018-08-04 at 08:29 -0500, Justin Hibbits wrote: > > On Sat, Aug 4, 2018, 08:03 Konstantin Belousov > wrote: > > > > > > > > On Sat, Aug 04, 2018 at 05:14:31AM -0700, John Baldwin wrote: > > > > > > > > On 8/4/18 1:08 AM, Konstantin Belousov wrote: > > > > > > > > > > On Fri, Aug 03, 2018 at 08:04:06PM +0000, Justin Hibbits wrote: > > > > > > > > > > > > Author: jhibbits > > > > > > Date: Fri Aug 3 20:04:06 2018 > > > > > > New Revision: 337273 > > > > > > URL: https://svnweb.freebsd.org/changeset/base/337273 > > > > > > > > > > > > Log: > > > > > > nvme(4): Add bus_dmamap_sync() at the end of the request path > > > > > > > > > > > > Summary: > > > > > > Some architectures, in this case powerpc64, need explicit > > > synchronization > > > > > > > > > > > > > > > > > > > > > barriers vs device accesses. > > > > > > > > > > > > Prior to this change, when running 'make buildworld -j72' on a > > > 18-core > > > > > > > > > > > > > > > > > > > > > (72-thread) POWER9, I would see controller resets often. With > this > > > change, I > > > > > > > > > > > > > > > > > > > > > don't see these resets messages, though another tester still > does, > > > for yet to be > > > > > > > > > > > > > > > > > > > > > determined reasons, so this may not be a complete fix. > > > Additionally, I see a > > > > > > > > > > > > > > > > > > > > > ~5-10% speed up in buildworld times, likely due to not needing > to > > > reset the > > > > > > > > > > > > > > > > > > > > > controller. > > > > > > > > > > > > Reviewed By: jimharris > > > > > > Differential Revision: https://reviews.freebsd.org/D16570 > > > > > > > > > > > > Modified: > > > > > > head/sys/dev/nvme/nvme_qpair.c > > > > > > > > > > > > Modified: head/sys/dev/nvme/nvme_qpair.c > > > > > > > > > > ============================================================================== > > > > > > > > > > > > > > > > > > > > > --- head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 19:24:04 2018 > > > (r337272) > > > > > > > > > > > > > > > > > > > > > +++ head/sys/dev/nvme/nvme_qpair.c Fri Aug 3 20:04:06 2018 > > > (r337273) > > > > > > > > > > > > > > > > > > > > > @@ -401,9 +401,13 @@ nvme_qpair_complete_tracker(struct > nvme_qpair > > > *qpair, > > > > > > > > > > > > > > > > > > > > > req->retries++; > > > > > > nvme_qpair_submit_tracker(qpair, tr); > > > > > > } else { > > > > > > - if (req->type != NVME_REQUEST_NULL) > > > > > > + if (req->type != NVME_REQUEST_NULL) { > > > > > > + bus_dmamap_sync(qpair->dma_tag_payload, > > > > > > + tr->payload_dma_map, > > > > > > + BUS_DMASYNC_POSTREAD | > BUS_DMASYNC_POSTWRITE); > > > > > > bus_dmamap_unload(qpair->dma_tag_payload, > > > > > > tr->payload_dma_map); > > > > > > + } > > > > > > > > > > > > nvme_free_request(req); > > > > > > tr->req = NULL; > > > > > > @@ -487,6 +491,8 @@ nvme_qpair_process_completions(struct > nvme_qpair > > > *qpai > > > > > > > > > > > > > > > > > > > > > */ > > > > > > return (false); > > > > > > > > > > > > + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > > > > > + BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE); > > > > > > while (1) { > > > > > > cpl = qpair->cpl[qpair->cq_head]; > > > > > > > > > > > > @@ -828,7 +834,16 @@ nvme_qpair_submit_tracker(struct nvme_qpair > > > *qpair, st > > > > > > > > > > > > > > > > > > > > > if (++qpair->sq_tail == qpair->num_entries) > > > > > > qpair->sq_tail = 0; > > > > > > > > > > > > + bus_dmamap_sync(qpair->dma_tag, qpair->queuemem_map, > > > > > > + BUS_DMASYNC_PREREAD | BUS_DMASYNC_PREWRITE); > > > > > > +#ifndef __powerpc__ > > > > > > + /* > > > > > > + * powerpc's bus_dmamap_sync() already includes a heavyweight > > > sync, but > > > > > > > > > > > > > > > > > > > > > + * no other archs do. > > > > > > + */ > > > > > > wmb(); > > > > > > +#endif > > > > > What is the purpose of this call ? It is useless without paired > read > > > > > barrier. So where is the reciprocal rmb() ? > > > > For DMA, the rmb is in the device controller. However, architectures > > > > that need this kind of ordering should do it in their bus_dmmap_sync > op, > > > > and this explicit one needs to be removed. (Alpha had a wmb in its > > > > bus_dmamap_sync op for this reason.) > > > Yes, if something special is needed, it should happen in > platform-specific > > > busdma code. > > > > > > Also, if wmb() is needed, then it is not a supposed semantic or > > > wmb(), but a specific side-effects of one of the instruction in the > > > implementation of wmb(). > > > > > > As I noted, on x86 it is not needed and detrimental to the performance. > > > > > According to Jim Harris it is needed for x86. I tried to remove it > thinking > > it was unnecessary with the sync in place now. The wmb() was added way > > back in r240616 by him, when only x86 was supported by the driver. > > bus_dmamap_sync() for all archs except powerpc lack a barrier, from what > I > > can see. See the review for more discussion that took place. If it isn't > > needed I will gladly remove it. > > > > The arm busdma sync routines definitely invoke the necessary barriers, > which on arm are spelled 'dsb' and 'isb' and are in the low-level > routines called from the busdma code, such as dcache_wb_pou(), > dcache_inv_poc(), etc. > > -- Ian > You're right, I missed that when I searched. X86 does not, it appears, even in the case of needing bounce buffers. The wmb() had been in since the beginning. Someone with x86 hardware should test before it's removed. I have none myself. - Justin > From owner-svn-src-all@freebsd.org Sat Aug 4 15:30:57 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03B1C106EBC9; Sat, 4 Aug 2018 15:30:57 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD63A7C70B; Sat, 4 Aug 2018 15:30:56 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8E772114F6; Sat, 4 Aug 2018 15:30:56 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74FUuDq052363; Sat, 4 Aug 2018 15:30:56 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74FUuvr052351; Sat, 4 Aug 2018 15:30:56 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808041530.w74FUuvr052351@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Sat, 4 Aug 2018 15:30:56 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337323 - head/usr.sbin/newsyslog X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/usr.sbin/newsyslog X-SVN-Commit-Revision: 337323 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 15:30:57 -0000 Author: markj Date: Sat Aug 4 15:30:56 2018 New Revision: 337323 URL: https://svnweb.freebsd.org/changeset/base/337323 Log: Fix a flag collision introduced in r327451. PR: 230350 MFC after: 3 days Modified: head/usr.sbin/newsyslog/newsyslog.c Modified: head/usr.sbin/newsyslog/newsyslog.c ============================================================================== --- head/usr.sbin/newsyslog/newsyslog.c Sat Aug 4 14:57:23 2018 (r337322) +++ head/usr.sbin/newsyslog/newsyslog.c Sat Aug 4 15:30:56 2018 (r337323) @@ -133,8 +133,7 @@ __FBSDID("$FreeBSD$"); #define CE_NODUMP 0x0200 /* Set 'nodump' on newly created log file. */ #define CE_PID2CMD 0x0400 /* Replace PID file with a shell command.*/ #define CE_PLAIN0 0x0800 /* Do not compress zero'th history file */ - -#define CE_RFC5424 0x0800 /* Use RFC5424 format rotation message */ +#define CE_RFC5424 0x1000 /* Use RFC5424 format rotation message */ #define MIN_PID 5 /* Don't touch pids lower than this */ #define MAX_PID 99999 /* was lower, see /usr/include/sys/proc.h */ From owner-svn-src-all@freebsd.org Sat Aug 4 16:00:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F01C106F9F7; Sat, 4 Aug 2018 16:00:30 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03CC97DABF; Sat, 4 Aug 2018 16:00:30 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: by mail-it0-x235.google.com with SMTP id 139-v6so5800465itf.0; Sat, 04 Aug 2018 09:00:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ClC+/gYGwRGTK/aDcdxBv6De/2B6HBhKgPZl8a7pW4s=; b=Eu/Nx4JLCyl4OTA0tIDavNz1Jwyz/h2V8BfdBsh5HiX7CIgjy9j7WwdB6Jtslk7KGf kVycqggWI5FyYOxJ0x8ogVNIVzoSKQrkrohafYMQmFRSz/sbbH3zcF2Qaa03BhxmvOPS 38LKZC2FeOthoDOxAKlfB/s21HX5eUB8fkBpYEjRLdRFXBGvpmSRDfz9evHUUj/h3rO2 6Jn0rkjOaOwftLpyrmk3jXszSA67fn3TbDHLekLOSqzIjVeKO3gGtCSXrsm0aoMi3ni1 lbneUURcDIK1YwDR4+6LhiSbMm8bbVbNxAQBQn5/cTE7pw6vSxvowS3qLfKFpZXihR6R oYrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ClC+/gYGwRGTK/aDcdxBv6De/2B6HBhKgPZl8a7pW4s=; b=I0FHR/U4U2F4zPtxuZ3xh3qcPU5fU6LMkgX5HFVWe02VHRmmprFv8qQNeIp1fjdLvt NvwkeRArIXTQzpXdUzx1YI/9Uy2zj43wtqadsSu0OyZdoJEVqvr3xMQ5Uc18hjDAZJG9 i4xnRXYvUDSVpn5DS3QopqP2EUSFH78FvbJ4FHG7T9d95kmSxkbhX68i0jskNMb3uzm/ LkxbdncsrvC1izzFP5hk/K6skhKAFy4pfTtfctT4ge5Rtyg2QqkVZZauNWfMy1SZewSJ d/MZi6mq2FKucphdFdsbBbxl7wqecTcAwSi0LOEelcNGoOaLaJKqTgJEE0W/D+XTriLl Ny8Q== X-Gm-Message-State: AOUpUlEsEvwGMFpjR923U0KYipNnvOB5+Rf5rJw36CBJ5PdbemV9EkWO hcJo1zVfbCOfL+HNOaG90mYDjUYXw5vaKKsFfkQ= X-Google-Smtp-Source: AAOMgpeXxX2lkaPkW1r5kJDbY26N1Wyh188EVm3t34eOm9cchiZwZUAUfg3vOO5HlnZpsZPGEFRTbMdgzpqEm/+lrYk= X-Received: by 2002:a24:d442:: with SMTP id x63-v6mr10606516itg.152.1533398429478; Sat, 04 Aug 2018 09:00:29 -0700 (PDT) MIME-Version: 1.0 References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> <20180804130315.GK6049@kib.kiev.ua> <20180804144733.GO6049@kib.kiev.ua> In-Reply-To: <20180804144733.GO6049@kib.kiev.ua> From: Jim Harris Date: Sat, 4 Aug 2018 09:00:18 -0700 Message-ID: Subject: Re: svn commit: r337273 - head/sys/dev/nvme To: Konstantin Belousov Cc: jhibbits@freebsd.org, John Baldwin , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 16:00:30 -0000 On Sat, Aug 4, 2018 at 7:47 AM Konstantin Belousov wrote: > > Yes, if something special is needed, it should happen in > platform-specific > > > busdma code. > > > > > > Also, if wmb() is needed, then it is not a supposed semantic or > > > wmb(), but a specific side-effects of one of the instruction in the > > > implementation of wmb(). > > > > > > As I noted, on x86 it is not needed and detrimental to the performance. > > > > > > > According to Jim Harris it is needed for x86. I tried to remove it > thinking > > it was unnecessary with the sync in place now. The wmb() was added way > > back in r240616 by him, when only x86 was supported by the driver. > > bus_dmamap_sync() for all archs except powerpc lack a barrier, from what > I > > can see. See the review for more discussion that took place. If it isn't > > needed I will gladly remove it. > > I am very curious why x86 would need MFENCE there. I do not see an > explanation in the review. > On x86, store buffers are not coherent. Parts of the 64-byte submission queue entry could be in store buffers and not globally visible at this point. The wmb() ensures that all of the data is globally visible before the doorbell write. This ensures the NVMe device will DMA the correct data from the submission queue. (Note: the Linux nvme driver also does an explicit wmb() before writing the submission queue doorbell.) From owner-svn-src-all@freebsd.org Sat Aug 4 16:06:58 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B023106FC47; Sat, 4 Aug 2018 16:06:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C6E7B7DF58; Sat, 4 Aug 2018 16:06:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w74G6l4m073052 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 4 Aug 2018 19:06:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w74G6l4m073052 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w74G6lRk073051; Sat, 4 Aug 2018 19:06:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 4 Aug 2018 19:06:47 +0300 From: Konstantin Belousov To: Jim Harris Cc: jhibbits@freebsd.org, John Baldwin , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r337273 - head/sys/dev/nvme Message-ID: <20180804160647.GR6049@kib.kiev.ua> References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> <20180804130315.GK6049@kib.kiev.ua> <20180804144733.GO6049@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 16:06:58 -0000 On Sat, Aug 04, 2018 at 09:00:18AM -0700, Jim Harris wrote: > On Sat, Aug 4, 2018 at 7:47 AM Konstantin Belousov > wrote: > > > > > > Yes, if something special is needed, it should happen in > > platform-specific > > > > busdma code. > > > > > > > > Also, if wmb() is needed, then it is not a supposed semantic or > > > > wmb(), but a specific side-effects of one of the instruction in the > > > > implementation of wmb(). > > > > > > > > As I noted, on x86 it is not needed and detrimental to the performance. > > > > > > > > > > According to Jim Harris it is needed for x86. I tried to remove it > > thinking > > > it was unnecessary with the sync in place now. The wmb() was added way > > > back in r240616 by him, when only x86 was supported by the driver. > > > bus_dmamap_sync() for all archs except powerpc lack a barrier, from what > > I > > > can see. See the review for more discussion that took place. If it isn't > > > needed I will gladly remove it. > > > > I am very curious why x86 would need MFENCE there. I do not see an > > explanation in the review. > > > > On x86, store buffers are not coherent. Parts of the 64-byte submission > queue entry could be in store buffers and not globally visible at this > point. The wmb() ensures that all of the data is globally visible before > the doorbell write. This ensures the NVMe device will DMA the correct data > from the submission queue. Store buffers are not coherent but are in order. If later store is visible, then all previous stores must be visible as well. And wmb() semantic only guarantees ordering, not visibility after the execution. It could be implemented as 'LOCK;ADDL' instead of MFENCE. > > (Note: the Linux nvme driver also does an explicit wmb() before writing the > submission queue doorbell.) From owner-svn-src-all@freebsd.org Sat Aug 4 17:08:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E574C104C0FC for ; Sat, 4 Aug 2018 17:08:37 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yw1-xc29.google.com (mail-yw1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C7C1804DC for ; Sat, 4 Aug 2018 17:08:37 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yw1-xc29.google.com with SMTP id e23-v6so2157689ywe.13 for ; Sat, 04 Aug 2018 10:08:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=9utvUGvG0piGdGCmSgDxxjSf0zPtgly01/BHajUhhPs=; b=dNf0jWLLLtKHFxtU9mydMbsQWcMP6Zfc099w/PIzUW2Eba/WepS5jd+xOe3VLbTZAS UbvVq7Qp7qIc43SG2rZdmvSYyhYnEpsKOQ36bzDXhXCg8514cdVOMv7t8ZoziO/uGNdW 2oXhoD6hxeeMivX4D0hbs7At2KEAKLPUBJvVM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9utvUGvG0piGdGCmSgDxxjSf0zPtgly01/BHajUhhPs=; b=PbgAoHZ3lJ+Q6HQK8iffVBgQ2jOg+rYfH81af/Auvs+EddorWmejQQ7rw98nNSidZ/ MyFrFj3Y7w4N/W6ohada++lnJWtDtSm5ZjK/3SuZOlPOX9l8PqI2NmZjx2rH5/H0I7lV +4kB4QbjLFHnyOe6bq9O5RzA4DM1G2f1mWYly3s0uBcFtlPYMakM+FHt5e0OrouE0WNd Qhyh/ESx9fKKCikLAYymbYftOJHSyShrR8PmwAoBTHS/51guBtMPy8K5ViqEJZ9JX1Ou DrKFUiT+W2tDa1TyrYvBQw8KCc2rhOHpa6a04/8WwPiLmn1jTO1CU8Tp4qRKLHrgov8N EbZA== X-Gm-Message-State: AOUpUlF2CHUExA9PViJUh/lvOYPngqETLM9u6bfqmVC00PmlNCmJjliJ 9rYDna0Ik3tAw8b2kDbRgTXPmKKwh/u5H9M6W3GKVYxh X-Google-Smtp-Source: AAOMgpc+Q0MQdWNWoPi+ed650yauMBPqk1RiL3GjmhVREoGbzmZfv9safkGpRODvALc6cMESuUSr3T/JhheqesvrLHs= X-Received: by 2002:a0d:de02:: with SMTP id h2-v6mr4447563ywe.162.1533402516664; Sat, 04 Aug 2018 10:08:36 -0700 (PDT) MIME-Version: 1.0 References: <201808021828.w72IS2ts055984@repo.freebsd.org> In-Reply-To: <201808021828.w72IS2ts055984@repo.freebsd.org> From: Eitan Adler Date: Sat, 4 Aug 2018 10:08:10 -0700 Message-ID: Subject: Re: svn commit: r337157 - in head: . tools/tools/git To: bwidawsk@freebsd.org Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 17:08:38 -0000 On Thu, 2 Aug 2018 at 11:28, Ben Widawsky wrote: > +git_checkout() > +{ > + # Delete master > + ${GIT} branch -D master > + > + # Make master really be trunk > + ${GIT} checkout -b master trunk Can't this be git branch --force master trunk ? -- Eitan Adler From owner-svn-src-all@freebsd.org Sat Aug 4 17:16:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 829D6104C461 for ; Sat, 4 Aug 2018 17:16:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x234.google.com (mail-it0-x234.google.com [IPv6:2607:f8b0:4001:c0b::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1141980913 for ; Sat, 4 Aug 2018 17:16:38 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x234.google.com with SMTP id v71-v6so12792363itb.3 for ; Sat, 04 Aug 2018 10:16:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lzbHZWNkwGt49UPylYxtzRz9fRyYCQtGM1AW6WV6X4E=; b=pClwFW3YRRLZmV3tIdo8MAFfjhGwxyOT5yGjnPrD0kwHQ1605bGOpLEqXsXblh+31S x3cq4e8h7gF5yEpSQRxF0FkO8yTfAYqMUoxn31Fs4Vv3KidOVj6YgMN5AP3c+GUT4l3C IjdVXurH4dvqP0wCVCgzWELUiUzIcyjjd/fyf98Tg/x75GNUqVaz4ZzgEEhbLwfGfhxS cvxH7avJPtJFsL/3NcpN2zz9BO1ROrR8GwQLlACqH6hc/YyaGOth5g9Kmt//NcYfFG13 dvBmk7q7oehHMJCglN+1HHLnsxpkQbczUazRZuSA6dkGFFEiK5LmJCatWjoiwcJQcWtd VSCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lzbHZWNkwGt49UPylYxtzRz9fRyYCQtGM1AW6WV6X4E=; b=g4Trw6k9VRge+MB3ZrWY7O7Ziozs3Z5wyr1oR4Ngor08PWKWbrEm2dPx2QJcyS2YMy WAauubjOhhUqxn5h77fANHcn+djQgM3kxJUZXosrBtnh7DyEQXf4mXXTGlQJ2uOwL2mN rnwrFXhDt9j/07Foggb3glVHs1I1GsMFo7h5wwJk400MyR5AXU+JkfKD9Z7AnURhzg2l 5Ef23lSEr1W9SwOVVyYhlOL2Ge/T+d0QTtrcpPVq6gv7dEcuxkUve9HWuQp9NhFVvP/y CNUYei65FcAaKMf8ISNKW555vrmY0r/BFBgG5JtaGjD/AdTWnadFr2eJcQBR4yIAYqMY XB/A== X-Gm-Message-State: AOUpUlHUzOOQ56N+rWOA+Y4or2ZiH/aYUUhMG8ChSGh6KFA4yId0g11U 0KMavOwADotYGKMnT8yN3u5XhMNBWwBLNP/kwdSCyQ== X-Google-Smtp-Source: AAOMgpdkHo5B97Xz9M8aINbspVJ3Fy09F3pNlu2zAeUaiOarcltHG2SKJ3XuL2Vz7IIWbvINksnqbme0q5ApZaLZ9pY= X-Received: by 2002:a24:b211:: with SMTP id u17-v6mr10009809ite.1.1533402997336; Sat, 04 Aug 2018 10:16:37 -0700 (PDT) MIME-Version: 1.0 References: <201808021828.w72IS2ts055984@repo.freebsd.org> In-Reply-To: From: Warner Losh Date: Sat, 4 Aug 2018 18:16:24 +0100 Message-ID: Subject: Re: svn commit: r337157 - in head: . tools/tools/git To: Eitan Adler Cc: bwidawsk@freebsd.org, src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 17:16:38 -0000 On Sat, Aug 4, 2018, 6:08 PM Eitan Adler wrote: > On Thu, 2 Aug 2018 at 11:28, Ben Widawsky wrote: > > +git_checkout() > > +{ > > + # Delete master > > + ${GIT} branch -D master > > + > > + # Make master really be trunk > > + ${GIT} checkout -b master trunk > > Can't this be git branch --force master trunk ? > No. That won't work. We do not want to change the location of master. This will reset it to trunk, which is nearly always wrong. Warner > From owner-svn-src-all@freebsd.org Sat Aug 4 18:11:08 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F32B8104E936 for ; Sat, 4 Aug 2018 18:11:07 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yb0-x233.google.com (mail-yb0-x233.google.com [IPv6:2607:f8b0:4002:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 94276821D4 for ; Sat, 4 Aug 2018 18:11:07 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yb0-x233.google.com with SMTP id q5-v6so691524ybk.8 for ; Sat, 04 Aug 2018 11:11:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jpHp8ao4+Wkj/CwsaiMQaDs9bNjMxY/z3alE1UdgOi0=; b=BqjeQoifpNo/2fHGfA/XWptPik2m5kzWd2DS1hlQtBOX+phqy2di8A+UNvNDSEYCru NC3iESPDsOz4k2pKTx+Rr+o3DFVPx/3udcL/j1rYQEZEnv4c6PSwkKZVQfWvI9+jwolb 9TJg/qOntpKXr3bSCzuLiRlTEzyrWt2+ZnJGU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jpHp8ao4+Wkj/CwsaiMQaDs9bNjMxY/z3alE1UdgOi0=; b=TylbUfhO3s8S0zXjGMfypt+m9AlkRJ6uYA4W3P2V/fGzwSutic+yuJFjJZhDQnTNTp il7mjMdbt47eU5DRX82au5bB0ny6D+jFNHLZn+n1aSUAYZOVkiEmesFnhHlLdyksLM3T /q6LvrzIXyuHv/xzyxliDKlPDxe1rqRQujifqEaHTo2OpjP2iY2HR5GBb5BjJEGnnbZd Z0vt96sTGlpVCdyDOvVyAXJh1xTNCmhdNqbkZoNNXMet869mEQ1LLsXJQqXACXX2DCJM QjTW4RHK2I27BqYw9e/lBWOBDCi/FxfC9Jj4SCg4ae1ubpLRtczkjUD2VlTQZcbr0OmR tpKQ== X-Gm-Message-State: AOUpUlFKBYvoDQmcTwezoMtqqYUDzBvmXJn/YejroHAQOGMJEfERfYYX zBdYyyfMR+FP0Y845APTjwywsMetoKzxzVGFctTsFQ== X-Google-Smtp-Source: AAOMgpcJYVteO+FAoBtjQ/9f/rnrIkj7mFwvFf5Jbq+97YzoNxhoHHoe0HYWI/uxLGM0nKQC4SLLi0DAdHREJ8mfPIY= X-Received: by 2002:a25:ba8b:: with SMTP id s11-v6mr4655408ybg.87.1533406266678; Sat, 04 Aug 2018 11:11:06 -0700 (PDT) MIME-Version: 1.0 References: <201808021828.w72IS2ts055984@repo.freebsd.org> In-Reply-To: From: Eitan Adler Date: Sat, 4 Aug 2018 11:10:39 -0700 Message-ID: Subject: Re: svn commit: r337157 - in head: . tools/tools/git To: Warner Losh Cc: bwidawsk@freebsd.org, src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 18:11:08 -0000 On Sat, 4 Aug 2018 at 10:16, Warner Losh wrote: > > > > On Sat, Aug 4, 2018, 6:08 PM Eitan Adler wrote: >> >> On Thu, 2 Aug 2018 at 11:28, Ben Widawsky wrote: >> > +git_checkout() >> > +{ >> > + # Delete master >> > + ${GIT} branch -D master >> > + >> > + # Make master really be trunk >> > + ${GIT} checkout -b master trunk >> >> Can't this be git branch --force master trunk ? > > > No. That won't work. We do not want to change the location of master. This will reset it to trunk, which is nearly always wrong. See this: ${GIT} branch -D master ${GIT} checkout -b master trunk For clarity I'd expect something closer to: git branch --force master trunk git checkout master The end result is the same, but IMHO this is clearer in intent. -- Eitan Adler From owner-svn-src-all@freebsd.org Sat Aug 4 18:36:24 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7DC7104F3FD for ; Sat, 4 Aug 2018 18:36:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C53B82C8C for ; Sat, 4 Aug 2018 18:36:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22b.google.com with SMTP id d9-v6so12531069itf.2 for ; Sat, 04 Aug 2018 11:36:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=m+RFNUQY/lcbZDoWFownm6ei+B6jrmRnRRoWvMeXYbc=; b=iCFX3d0Llx35dKpt9FVyoXJ83LkLRqRadu8nevkED2beo9Z2Ls6xir3d1uj/62yQmt vLTy5whoaXZKLDBLJXvuHpzoFxZP5qQ1eoi11LT9mM5GlQk5MKYg0mve+PEXR7cB7QTV 30RXrJl69dr5FarEjD9qmEkGq2UdwFT1Zqn/Hll5PIctoT3bLHoyt4Lzrz1iKFPc3csC YvcKeH0V1/liXqncXg4ZzxUfAHIOBu2Pu4+z1srHFy1U2gQxOLT7jqwfq21y3UlRrsvg 0jo7yhyQxv+NYratrWLFrYHxqfoReSoUiYUvgqu2t+Klev0lncXxkJDM386O9c24/wqz y/BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=m+RFNUQY/lcbZDoWFownm6ei+B6jrmRnRRoWvMeXYbc=; b=tOUmtI81rrR5zVqRiK1XosdlgxHLi6u1DvYV4XUua2n74WlhZGOWwzFhCbtNofixxJ dHU49lXiX1n7JPHWNVR5MbhXk8Y//GcCGiBUgcfitp3FoC4Ixhg3LpubHUWKCmPiz1kr Wt8DH4Ifkt/tWryUprVMd28u+oKUrnTU4HTwh8UfVz+ol1afgOHG2t1sDt7VneFkH+ZM o06c+h4TMEDtuNvR9//aj+doQSZa3IZW+HATKjX7gZ+UWHdVd0AU/MEs865nzphPld2Q J1ltRLsxAPVsVNwxm7mw8QQ5xmmXYNb9jmDiA/wACUmqkAWhTkziy8W0m/+YPwSrW8FW +qsw== X-Gm-Message-State: AOUpUlEUrTcY8fX4+cVvQS2eGhZd1WViKiXvz0HYQ4lB9jO1aTO3+LYI zdzgLGZT2sAXIpGCBAQlQSXcmTb//Jed1+uqYPiPYJoT4M4= X-Google-Smtp-Source: AAOMgpdBX8F3ZV88V1nNTimYptpdALmelSMtDpQz++0AXvOwQ0oq9HcBw3JQNSG5qD/90kAcLpLjofGW87f1tykbBGM= X-Received: by 2002:a02:bb04:: with SMTP id y4-v6mr7860577jan.5.1533407783541; Sat, 04 Aug 2018 11:36:23 -0700 (PDT) MIME-Version: 1.0 References: <201808021828.w72IS2ts055984@repo.freebsd.org> In-Reply-To: From: Warner Losh Date: Sat, 4 Aug 2018 19:36:11 +0100 Message-ID: Subject: Re: svn commit: r337157 - in head: . tools/tools/git To: Eitan Adler Cc: bwidawsk@freebsd.org, src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 18:36:25 -0000 On Sat, Aug 4, 2018, 7:11 PM Eitan Adler wrote: > On Sat, 4 Aug 2018 at 10:16, Warner Losh wrote: > > > > > > > > On Sat, Aug 4, 2018, 6:08 PM Eitan Adler wrote: > >> > >> On Thu, 2 Aug 2018 at 11:28, Ben Widawsky wrote: > >> > +git_checkout() > >> > +{ > >> > + # Delete master > >> > + ${GIT} branch -D master > >> > + > >> > + # Make master really be trunk > >> > + ${GIT} checkout -b master trunk > >> > >> Can't this be git branch --force master trunk ? > > > > > > No. That won't work. We do not want to change the location of master. > This will reset it to trunk, which is nearly always wrong. > > See this: > > ${GIT} branch -D master > ${GIT} checkout -b master trunk > > For clarity I'd expect something closer to: > > git branch --force master trunk > git checkout master > > The end result is the same, but IMHO this is clearer in intent. > Oh, that... yes, you may be right. There is another script that it would be wrong in, but not this one... Warner > From owner-svn-src-all@freebsd.org Sat Aug 4 19:57:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD8D71051417; Sat, 4 Aug 2018 19:57:12 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 70E8885BF3; Sat, 4 Aug 2018 19:57:12 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 222B41AB40; Sat, 4 Aug 2018 19:57:12 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8rgnodwefsncims6j5o.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240b:b802:c58a:76da:2e2d:7a7c]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id AF4043F9C4; Sat, 4 Aug 2018 21:57:09 +0200 (CEST) From: "Kristof Provost" To: "Ian Lepore" Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r336252 - in head: share/mk stand stand/common stand/efi/loader stand/i386/gptboot stand/i386/gptzfsboot stand/i386/isoboot stand/i386/libi386 stand/i386/loader stand/i386/zfsboot stand... Date: Sat, 04 Aug 2018 21:57:07 +0200 X-Mailer: MailMate (2.0BETAr6116) Message-ID: <9CB310A7-0612-472C-91F0-4B0EB7D75912@FreeBSD.org> In-Reply-To: <201807131750.w6DHoPD9024230@repo.freebsd.org> References: <201807131750.w6DHoPD9024230@repo.freebsd.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_MailMate_7414594C-7AEF-40D0-94A0-BF29CF616A67_=" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 19:57:13 -0000 --=_MailMate_7414594C-7AEF-40D0-94A0-BF29CF616A67_= Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 13 Jul 2018, at 19:50, Ian Lepore wrote: > Author: ian > Date: Fri Jul 13 17:50:25 2018 > New Revision: 336252 > URL: https://svnweb.freebsd.org/changeset/base/336252 > > Log: > Extend loader(8) geli support to all architectures and all disk-like > devices. > > This moves the bulk of the geli support from lib386/biosdisk.c into > a new > geli/gelidev.c which implements a devsw-type device whose > dv_strategy() > function handles geli decryption. Support for all arches comes from > moving > the taste-and-attach code to the devopen() function in libsa. > > After opening any DEVT_DISK device, devopen() calls the new function > geli_probe_and_attach(), which will "attach" the geli code to the > open_file > struct by creating a geli_devdesc instance to replace the > disk_devdesc > instance in the open_file. That routes all IO for the device through > the > geli code. > > A new public geli_add_key() function is added, to allow > arch/vendor-specific > code to add keys obtained from custom hardware or other sources. > > With these changes, geli support will be compiled into all > variations of > loader(8) on all arches because the default is WITH_LOADER_GELI. > > Relnotes: yes > Sponsored by: Microchip Technology Inc > Differential Revision: https://reviews.freebsd.org/D15743 > I ran into a crash during startup with a geli encrypted raid-z1 root pool. I believe this change broke it (although it could have been broken before too). When we iterate over the list of disks and allocate the zfsdsk structures we don’t zero out the `gdev` pointer. In my case that resulted in a geli_read() (called on the bogus pointer) dividing by zero. The attached patch simply changes malloc() to calloc(), so the pointer is always set to NULL. As a side benefit it gets rid of one `#ifdef LOADER_GELI_SUPPORT`. Regards, Kristof --=_MailMate_7414594C-7AEF-40D0-94A0-BF29CF616A67_= Content-Disposition: attachment; filename=gptzfsboot.patch Content-Transfer-Encoding: quoted-printable diff --git a/stand/i386/zfsboot/zfsboot.c b/stand/i386/zfsboot/zfsboot.c index fbdd9ac44e1..e8852775715 100644 --- a/stand/i386/zfsboot/zfsboot.c +++ b/stand/i386/zfsboot/zfsboot.c @@ -707,10 +707,7 @@ main(void) } setheap(heap_next, heap_end); = - zdsk =3D malloc(sizeof(struct zfsdsk)); -#ifdef LOADER_GELI_SUPPORT - zdsk->gdev =3D NULL; -#endif + zdsk =3D calloc(1, sizeof(struct zfsdsk)); zdsk->dsk.drive =3D *(uint8_t *)PTOV(ARGS); zdsk->dsk.type =3D zdsk->dsk.drive & DRV_HARD ? TYPE_AD : TYPE_FD; zdsk->dsk.unit =3D zdsk->dsk.drive & DRV_MASK; @@ -758,7 +755,7 @@ main(void) if (!int13probe(i | DRV_HARD)) break; = - zdsk =3D malloc(sizeof(struct zfsdsk)); + zdsk =3D calloc(1, sizeof(struct zfsdsk)); zdsk->dsk.drive =3D i | DRV_HARD; zdsk->dsk.type =3D zdsk->dsk.drive & TYPE_AD; zdsk->dsk.unit =3D i; --=_MailMate_7414594C-7AEF-40D0-94A0-BF29CF616A67_=-- From owner-svn-src-all@freebsd.org Sat Aug 4 20:04:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BC06F105179B; Sat, 4 Aug 2018 20:04:41 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 666C4860BF; Sat, 4 Aug 2018 20:04:41 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3A557141F9; Sat, 4 Aug 2018 20:04:41 +0000 (UTC) (envelope-from tsoome@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74K4fFM096271; Sat, 4 Aug 2018 20:04:41 GMT (envelope-from tsoome@FreeBSD.org) Received: (from tsoome@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74K4fW4096270; Sat, 4 Aug 2018 20:04:41 GMT (envelope-from tsoome@FreeBSD.org) Message-Id: <201808042004.w74K4fW4096270@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: tsoome set sender to tsoome@FreeBSD.org using -f From: Toomas Soome Date: Sat, 4 Aug 2018 20:04:41 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337326 - head/stand/i386/libi386 X-SVN-Group: head X-SVN-Commit-Author: tsoome X-SVN-Commit-Paths: head/stand/i386/libi386 X-SVN-Commit-Revision: 337326 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 20:04:41 -0000 Author: tsoome Date: Sat Aug 4 20:04:40 2018 New Revision: 337326 URL: https://svnweb.freebsd.org/changeset/base/337326 Log: loader: biosdisk.c has leftover geli header. A small cleanup, remove unneeded #include. Modified: head/stand/i386/libi386/biosdisk.c Modified: head/stand/i386/libi386/biosdisk.c ============================================================================== --- head/stand/i386/libi386/biosdisk.c Sat Aug 4 16:56:39 2018 (r337325) +++ head/stand/i386/libi386/biosdisk.c Sat Aug 4 20:04:40 2018 (r337326) @@ -50,10 +50,6 @@ __FBSDID("$FreeBSD$"); #include "disk.h" #include "libi386.h" -#ifdef LOADER_GELI_SUPPORT -#include "geliboot.h" -#endif /* LOADER_GELI_SUPPORT */ - #define BIOS_NUMDRIVES 0x475 #define BIOSDISK_SECSIZE 512 #define BUFSIZE (1 * BIOSDISK_SECSIZE) From owner-svn-src-all@freebsd.org Sat Aug 4 20:16:37 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3AEA1051B3E; Sat, 4 Aug 2018 20:16:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A278865C7; Sat, 4 Aug 2018 20:16:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 34F3A143AD; Sat, 4 Aug 2018 20:16:37 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74KGaht001581; Sat, 4 Aug 2018 20:16:36 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74KGa6Q001580; Sat, 4 Aug 2018 20:16:36 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808042016.w74KGa6Q001580@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Sat, 4 Aug 2018 20:16:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337327 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 337327 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 20:16:37 -0000 Author: markj Date: Sat Aug 4 20:16:36 2018 New Revision: 337327 URL: https://svnweb.freebsd.org/changeset/base/337327 Log: Style. Modified: head/sys/kern/uipc_usrreq.c Modified: head/sys/kern/uipc_usrreq.c ============================================================================== --- head/sys/kern/uipc_usrreq.c Sat Aug 4 20:04:40 2018 (r337326) +++ head/sys/kern/uipc_usrreq.c Sat Aug 4 20:16:36 2018 (r337327) @@ -357,33 +357,32 @@ unp_pcb_lock2(struct unpcb *unp, struct unpcb *unp2) } static __noinline void -unp_pcb_owned_lock2_slowpath(struct unpcb *unp, struct unpcb **unp2p, int *freed) - +unp_pcb_owned_lock2_slowpath(struct unpcb *unp, struct unpcb **unp2p, + int *freed) { struct unpcb *unp2; unp2 = *unp2p; - unp_pcb_hold((unp2)); - UNP_PCB_UNLOCK((unp)); - UNP_PCB_LOCK((unp2)); - UNP_PCB_LOCK((unp)); - *freed = unp_pcb_rele((unp2)); + unp_pcb_hold(unp2); + UNP_PCB_UNLOCK(unp); + UNP_PCB_LOCK(unp2); + UNP_PCB_LOCK(unp); + *freed = unp_pcb_rele(unp2); if (*freed) *unp2p = NULL; } -#define unp_pcb_owned_lock2(unp, unp2, freed) do { \ - freed = 0; \ - UNP_PCB_LOCK_ASSERT((unp)); \ - UNP_PCB_UNLOCK_ASSERT((unp2)); \ - MPASS(unp != unp2); \ - if (__predict_true(UNP_PCB_TRYLOCK((unp2)))) \ - break; \ - else if ((uintptr_t)(unp2) > (uintptr_t)(unp)) \ - UNP_PCB_LOCK((unp2)); \ - else { \ - unp_pcb_owned_lock2_slowpath((unp), &(unp2), &freed); \ - } \ +#define unp_pcb_owned_lock2(unp, unp2, freed) do { \ + freed = 0; \ + UNP_PCB_LOCK_ASSERT(unp); \ + UNP_PCB_UNLOCK_ASSERT(unp2); \ + MPASS((unp) != (unp2)); \ + if (__predict_true(UNP_PCB_TRYLOCK(unp2))) \ + break; \ + else if ((uintptr_t)(unp2) > (uintptr_t)(unp)) \ + UNP_PCB_LOCK(unp2); \ + else \ + unp_pcb_owned_lock2_slowpath((unp), &(unp2), &freed); \ } while (0) From owner-svn-src-all@freebsd.org Sat Aug 4 20:26:56 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22BAC1051EEE; Sat, 4 Aug 2018 20:26:56 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE27586ABC; Sat, 4 Aug 2018 20:26:55 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 99AF91454D; Sat, 4 Aug 2018 20:26:55 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74KQtRk006470; Sat, 4 Aug 2018 20:26:55 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74KQt6V006467; Sat, 4 Aug 2018 20:26:55 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808042026.w74KQt6V006467@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Sat, 4 Aug 2018 20:26:55 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337328 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 337328 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 20:26:56 -0000 Author: markj Date: Sat Aug 4 20:26:54 2018 New Revision: 337328 URL: https://svnweb.freebsd.org/changeset/base/337328 Log: Don't check rcv sockbuf limits when sending on a unix stream socket. sosend_generic() performs an initial comparison of the amount of data (including control messages) to be transmitted with the send buffer size. When transmitting on a unix socket, we then compare the amount of data being sent with the amount of space in the receive buffer size; if insufficient space is available, sbappendcontrol() returns an error and the data is lost. This is easily triggered by sending control messages together with an amount of data roughly equal to the send buffer size, since the control message size may change in uipc_send() as file descriptors are internalized. Fix the problem by removing the space check in sbappendcontrol(), whose only consumer is the unix sockets code. The stream sockets code uses the SB_STOP mechanism to ensure that senders will block if the receive buffer fills up. PR: 181741 MFC after: 1 month Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D16515 Modified: head/sys/kern/uipc_sockbuf.c head/sys/kern/uipc_usrreq.c head/sys/sys/sockbuf.h Modified: head/sys/kern/uipc_sockbuf.c ============================================================================== --- head/sys/kern/uipc_sockbuf.c Sat Aug 4 20:16:36 2018 (r337327) +++ head/sys/kern/uipc_sockbuf.c Sat Aug 4 20:26:54 2018 (r337328) @@ -955,21 +955,14 @@ sbappendaddr(struct sockbuf *sb, const struct sockaddr return (retval); } -int +void sbappendcontrol_locked(struct sockbuf *sb, struct mbuf *m0, struct mbuf *control) { - struct mbuf *m, *n, *mlast; - int space; + struct mbuf *m, *mlast; - SOCKBUF_LOCK_ASSERT(sb); - - space = m_length(control, &n) + m_length(m0, NULL); - - if (space > sbspace(sb)) - return (0); m_clrprotoflags(m0); - n->m_next = m0; /* concatenate data to control */ + m_last(control)->m_next = m0; SBLASTRECORDCHK(sb); @@ -983,18 +976,15 @@ sbappendcontrol_locked(struct sockbuf *sb, struct mbuf SBLASTMBUFCHK(sb); SBLASTRECORDCHK(sb); - return (1); } -int +void sbappendcontrol(struct sockbuf *sb, struct mbuf *m0, struct mbuf *control) { - int retval; SOCKBUF_LOCK(sb); - retval = sbappendcontrol_locked(sb, m0, control); + sbappendcontrol_locked(sb, m0, control); SOCKBUF_UNLOCK(sb); - return (retval); } /* Modified: head/sys/kern/uipc_usrreq.c ============================================================================== --- head/sys/kern/uipc_usrreq.c Sat Aug 4 20:16:36 2018 (r337327) +++ head/sys/kern/uipc_usrreq.c Sat Aug 4 20:26:54 2018 (r337328) @@ -1174,16 +1174,22 @@ uipc_send(struct socket *so, int flags, struct mbuf *m unp2->unp_flags &= ~UNP_WANTCRED; control = unp_addsockcred(td, control); } + /* - * Send to paired receive port, and then reduce send buffer - * hiwater marks to maintain backpressure. Wake up readers. + * Send to paired receive port and wake up readers. Don't + * check for space available in the receive buffer if we're + * attaching ancillary data; Unix domain sockets only check + * for space in the sending sockbuf, and that check is + * performed one level up the stack. At that level we cannot + * precisely account for the amount of buffer space used + * (e.g., because control messages are not yet internalized). */ switch (so->so_type) { case SOCK_STREAM: if (control != NULL) { - if (sbappendcontrol_locked(&so2->so_rcv, m, - control)) - control = NULL; + sbappendcontrol_locked(&so2->so_rcv, m, + control); + control = NULL; } else sbappend_locked(&so2->so_rcv, m, flags); break; @@ -1192,14 +1198,8 @@ uipc_send(struct socket *so, int flags, struct mbuf *m const struct sockaddr *from; from = &sun_noname; - /* - * Don't check for space available in so2->so_rcv. - * Unix domain sockets only check for space in the - * sending sockbuf, and that check is performed one - * level up the stack. - */ if (sbappendaddr_nospacecheck_locked(&so2->so_rcv, - from, m, control)) + from, m, control)) control = NULL; break; } Modified: head/sys/sys/sockbuf.h ============================================================================== --- head/sys/sys/sockbuf.h Sat Aug 4 20:16:36 2018 (r337327) +++ head/sys/sys/sockbuf.h Sat Aug 4 20:26:54 2018 (r337328) @@ -139,9 +139,9 @@ int sbappendaddr_locked(struct sockbuf *sb, const stru struct mbuf *m0, struct mbuf *control); int sbappendaddr_nospacecheck_locked(struct sockbuf *sb, const struct sockaddr *asa, struct mbuf *m0, struct mbuf *control); -int sbappendcontrol(struct sockbuf *sb, struct mbuf *m0, +void sbappendcontrol(struct sockbuf *sb, struct mbuf *m0, struct mbuf *control); -int sbappendcontrol_locked(struct sockbuf *sb, struct mbuf *m0, +void sbappendcontrol_locked(struct sockbuf *sb, struct mbuf *m0, struct mbuf *control); void sbappendrecord(struct sockbuf *sb, struct mbuf *m0); void sbappendrecord_locked(struct sockbuf *sb, struct mbuf *m0); From owner-svn-src-all@freebsd.org Sat Aug 4 20:29:59 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F9DE1052105; Sat, 4 Aug 2018 20:29:59 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 17E9386DA9; Sat, 4 Aug 2018 20:29:59 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ED46914576; Sat, 4 Aug 2018 20:29:58 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74KTwxZ006643; Sat, 4 Aug 2018 20:29:58 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74KTwUf006642; Sat, 4 Aug 2018 20:29:58 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201808042029.w74KTwUf006642@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Sat, 4 Aug 2018 20:29:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337329 - head/tests/sys/kern X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: head/tests/sys/kern X-SVN-Commit-Revision: 337329 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 20:29:59 -0000 Author: markj Date: Sat Aug 4 20:29:58 2018 New Revision: 337329 URL: https://svnweb.freebsd.org/changeset/base/337329 Log: Fix the regression test for PR 181741. With r337328, the test hangs becase the sendmsg() call will block until the receive buffer is at least partially drained. Fix the problem by using a non-blocking socket and allowing short writes. Also assert that a SCM_CREDS message was received if one was expected. PR: 181741 MFC after: 1 month Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D16516 Modified: head/tests/sys/kern/unix_passfd_test.c Modified: head/tests/sys/kern/unix_passfd_test.c ============================================================================== --- head/tests/sys/kern/unix_passfd_test.c Sat Aug 4 20:26:54 2018 (r337328) +++ head/tests/sys/kern/unix_passfd_test.c Sat Aug 4 20:29:58 2018 (r337329) @@ -126,7 +126,7 @@ samefile(struct stat *sb1, struct stat *sb2) ATF_REQUIRE_MSG(sb1->st_ino == sb2->st_ino, "different inode"); } -static void +static size_t sendfd_payload(int sockfd, int send_fd, void *payload, size_t paylen) { struct iovec iovec; @@ -153,21 +153,37 @@ sendfd_payload(int sockfd, int send_fd, void *payload, cmsghdr->cmsg_type = SCM_RIGHTS; memcpy(CMSG_DATA(cmsghdr), &send_fd, sizeof(int)); - len = sendmsg(sockfd, &msghdr, 0); + len = sendmsg(sockfd, &msghdr, MSG_DONTWAIT); ATF_REQUIRE_MSG(len != -1, "sendmsg failed: %s", strerror(errno)); - ATF_REQUIRE_MSG((size_t)len == paylen, - "sendmsg: %zd messages sent; expected: %zu; %s", len, paylen, - strerror(errno)); + return ((size_t)len); } static void sendfd(int sockfd, int send_fd) { - char ch = 0; + size_t len; + char ch; - sendfd_payload(sockfd, send_fd, &ch, sizeof(ch)); + ch = 0; + len = sendfd_payload(sockfd, send_fd, &ch, sizeof(ch)); + ATF_REQUIRE_MSG(len == sizeof(ch), + "sendmsg: %zu bytes sent; expected %zu; %s", len, sizeof(ch), + strerror(errno)); } +static bool +localcreds(int sockfd) +{ + socklen_t sz; + int rc, val; + + sz = sizeof(val); + rc = getsockopt(sockfd, 0, LOCAL_CREDS, &val, &sz); + ATF_REQUIRE_MSG(rc != -1, "getsockopt(LOCAL_CREDS) failed: %s", + strerror(errno)); + return (val != 0); +} + static void recvfd_payload(int sockfd, int *recv_fd, void *buf, size_t buflen) { @@ -177,6 +193,7 @@ recvfd_payload(int sockfd, int *recv_fd, void *buf, si struct msghdr msghdr; struct iovec iovec; ssize_t len; + bool foundcreds; bzero(&msghdr, sizeof(msghdr)); @@ -197,6 +214,7 @@ recvfd_payload(int sockfd, int *recv_fd, void *buf, si cmsghdr = CMSG_FIRSTHDR(&msghdr); ATF_REQUIRE_MSG(cmsghdr != NULL, "recvmsg: did not receive control message"); + foundcreds = false; *recv_fd = -1; for (; cmsghdr != NULL; cmsghdr = CMSG_NXTHDR(&msghdr, cmsghdr)) { if (cmsghdr->cmsg_level == SOL_SOCKET && @@ -204,10 +222,14 @@ recvfd_payload(int sockfd, int *recv_fd, void *buf, si cmsghdr->cmsg_len == CMSG_LEN(sizeof(int))) { memcpy(recv_fd, CMSG_DATA(cmsghdr), sizeof(int)); ATF_REQUIRE(*recv_fd != -1); - } + } else if (cmsghdr->cmsg_level == SOL_SOCKET && + cmsghdr->cmsg_type == SCM_CREDS) + foundcreds = true; } ATF_REQUIRE_MSG(*recv_fd != -1, "recvmsg: did not receive single-fd message"); + ATF_REQUIRE_MSG(!localcreds(sockfd) || foundcreds, + "recvmsg: expected credentials were not received"); } static void @@ -362,9 +384,9 @@ ATF_TC_BODY(devfs_orphan, tc) /* * Test for PR 181741. Receiver sets LOCAL_CREDS, and kernel prepends a - * control message to the data. Sender sends large payload. - * Payload + SCM_RIGHTS + LOCAL_CREDS hit socket buffer limit, and receiver - * receives truncated data. + * control message to the data. Sender sends large payload using a non-blocking + * socket. Payload + SCM_RIGHTS + LOCAL_CREDS hit socket buffer limit, and + * receiver receives truncated data. */ ATF_TC_WITHOUT_HEAD(rights_creds_payload); ATF_TC_BODY(rights_creds_payload, tc) @@ -375,9 +397,6 @@ ATF_TC_BODY(rights_creds_payload, tc) void *buf; int fd[2], getfd, putfd, rc; - atf_tc_expect_fail("PR 181741: Packet loss when 'control' messages " - "are present with large data"); - len = sizeof(sendspace); rc = sysctlbyname(LOCAL_SENDSPACE_SYSCTL, &sendspace, &len, NULL, 0); @@ -388,12 +407,19 @@ ATF_TC_BODY(rights_creds_payload, tc) ATF_REQUIRE(buf != NULL); domainsocketpair(fd); + tempfile(&putfd); + + rc = fcntl(fd[0], F_SETFL, O_NONBLOCK); + ATF_REQUIRE_MSG(rc != -1, "fcntl(O_NONBLOCK) failed: %s", + strerror(errno)); rc = setsockopt(fd[1], 0, LOCAL_CREDS, &on, sizeof(on)); ATF_REQUIRE_MSG(rc != -1, "setsockopt(LOCAL_CREDS) failed: %s", strerror(errno)); - tempfile(&putfd); - sendfd_payload(fd[0], putfd, buf, sendspace); - recvfd_payload(fd[1], &getfd, buf, sendspace); + + len = sendfd_payload(fd[0], putfd, buf, sendspace); + ATF_REQUIRE_MSG(len < sendspace, "sendmsg: %zu bytes sent", len); + recvfd_payload(fd[1], &getfd, buf, len); + close(putfd); close(getfd); closesocketpair(fd); @@ -421,7 +447,10 @@ ATF_TC_BODY(truncated_rights, tc) devnull(&putfd); nfds = getnfds(); - sendfd_payload(fd[0], putfd, buf, sizeof(buf)); + len = sendfd_payload(fd[0], putfd, buf, sizeof(buf)); + ATF_REQUIRE_MSG(len == sizeof(buf), + "sendmsg: %zd bytes sent; expected %zu; %s", len, sizeof(buf), + strerror(errno)); bzero(&msghdr, sizeof(msghdr)); bzero(message, sizeof(message)); From owner-svn-src-all@freebsd.org Sat Aug 4 20:30:47 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0F621052331 for ; Sat, 4 Aug 2018 20:30:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x242.google.com (mail-it0-x242.google.com [IPv6:2607:f8b0:4001:c0b::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C59886FA4 for ; Sat, 4 Aug 2018 20:30:47 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x242.google.com with SMTP id p81-v6so13191015itp.1 for ; Sat, 04 Aug 2018 13:30:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=bn4/RECcwGF0eoPSpINyOenGMvABfJkM47UM1vOb39g=; b=tryjB1Npr935tU3fppI3HLw2x0EA/6J/lbpr/jF2Vg1NO75aoXaXTRuZ55y6E2JPy2 i0HKencKL/4V1uqsw85gSlFtN0O2sk0mveDefYoGMnj0cdt3EWYLrCPD7Fdj7VqHy9cL L8jw+MdrnAlC+P4SNFCm7WPxW3t22F2/Hv1tLkN/daJ31cQ85+MX7hqP5RT2EN5MSn4/ MExRb+HyqHBE1j44jMzQ8paxXBJivenUfI18KUe1Sbo8EqO0HRBOYJ0F3/lm3GT+DFZi Br/2K0IoHKqJpWrQeI5ZpAqbTn7AbcdCVQiKPDYzv6Sh7gwGhVzTgjAXX2gp+LoThQuu sStQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=bn4/RECcwGF0eoPSpINyOenGMvABfJkM47UM1vOb39g=; b=WAx0QhmTK8a/OyDOsQHQqLCg5zMNhvikoM76+IH4LcQVjEs6N3QFlQ2h3xnj9HYStb FE7BAxP2DCXEkcvOt5B6yDgOlMygHJjIFyRe5ZHJrambnsa0xfMML4jBLhTvcnNcinAF HkMkbQlOSQ4785g2NB3Tv6WlAiCHfrgiJLAcVY8bkUrUWnmuqAuWiSoUW7tQrUpketIU fpns1VbD/T0QToX4a9HUfCTFNElxCyoE6iZQneWsn+xpfa538XztrZJ/K6sSV5ubbWyM aEgjTa+/ppnhx0AelMJkYERVgrVDE5IHSmd+5dcWezWeIQHgOh4YNVWm7qsN41+KCD2s JHQw== X-Gm-Message-State: AOUpUlHSlG19miMc+9e1U9X2UZCw4jCjpCFQ+8ZbKCk5nk7OjAfUnm+U YM7vJkY4woAQ0RS0P5go2BtCkuSBHwSubcaHL+gOTg== X-Google-Smtp-Source: AAOMgpcDeGmDpPoZJvQuTJfFF5x9io3gj6mxXUaLiygVPK4LgMR61rvPvXpDAc3NLW+6g0GLt7aDjYi9f14Z5OGQRHc= X-Received: by 2002:a24:4f52:: with SMTP id c79-v6mr10595895itb.36.1533414646380; Sat, 04 Aug 2018 13:30:46 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:4485:0:0:0:0:0 with HTTP; Sat, 4 Aug 2018 13:30:45 -0700 (PDT) X-Originating-IP: [86.153.210.77] In-Reply-To: <9CB310A7-0612-472C-91F0-4B0EB7D75912@FreeBSD.org> References: <201807131750.w6DHoPD9024230@repo.freebsd.org> <9CB310A7-0612-472C-91F0-4B0EB7D75912@FreeBSD.org> From: Warner Losh Date: Sat, 4 Aug 2018 14:30:45 -0600 X-Google-Sender-Auth: 5fnMcZ-hBvtfV2l-WmjOs4SEwBI Message-ID: Subject: Re: svn commit: r336252 - in head: share/mk stand stand/common stand/efi/loader stand/i386/gptboot stand/i386/gptzfsboot stand/i386/isoboot stand/i386/libi386 stand/i386/loader stand/i386/zfsboot stand... To: Kristof Provost Cc: Ian Lepore , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 20:30:47 -0000 On Sat, Aug 4, 2018 at 1:57 PM, Kristof Provost wrote: > On 13 Jul 2018, at 19:50, Ian Lepore wrote: > > Author: ian > Date: Fri Jul 13 17:50:25 2018 > New Revision: 336252 > URL: https://svnweb.freebsd.org/changeset/base/336252 > > Log: > Extend loader(8) geli support to all architectures and all disk-like > devices. > > This moves the bulk of the geli support from lib386/biosdisk.c into a new > geli/gelidev.c which implements a devsw-type device whose dv_strategy() > function handles geli decryption. Support for all arches comes from movin= g > the taste-and-attach code to the devopen() function in libsa. > > After opening any DEVT_DISK device, devopen() calls the new function > geli_probe_and_attach(), which will "attach" the geli code to the open_fi= le > struct by creating a geli_devdesc instance to replace the disk_devdesc > instance in the open_file. That routes all IO for the device through the > geli code. > > A new public geli_add_key() function is added, to allow > arch/vendor-specific > code to add keys obtained from custom hardware or other sources. > > With these changes, geli support will be compiled into all variations of > loader(8) on all arches because the default is WITH_LOADER_GELI. > > Relnotes: yes > Sponsored by: Microchip Technology Inc > Differential Revision: https://reviews.freebsd.org/D15743 > > I ran into a crash during startup with a geli encrypted raid-z1 root pool= . > > I believe this change broke it (although it could have been broken before > too). > When we iterate over the list of disks and allocate the zfsdsk structures > we don=E2=80=99t zero out the gdev pointer. In my case that resulted in a > geli_read() (called on the bogus pointer) dividing by zero. > > The attached patch simply changes malloc() to calloc(), so the pointer is > always set to NULL. As a side benefit it gets rid of one #ifdef > LOADER_GELI_SUPPORT. > This patch looks fine to me. Warner From owner-svn-src-all@freebsd.org Sat Aug 4 20:45:45 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54C541052AC0; Sat, 4 Aug 2018 20:45:45 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EDFA388232; Sat, 4 Aug 2018 20:45:44 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CA0C9148BC; Sat, 4 Aug 2018 20:45:44 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74KjidY016985; Sat, 4 Aug 2018 20:45:44 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74KjiP7016982; Sat, 4 Aug 2018 20:45:44 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201808042045.w74KjiP7016982@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Sat, 4 Aug 2018 20:45:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337330 - in head/sys: kern sys vm X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys: kern sys vm X-SVN-Commit-Revision: 337330 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 20:45:45 -0000 Author: kib Date: Sat Aug 4 20:45:43 2018 New Revision: 337330 URL: https://svnweb.freebsd.org/changeset/base/337330 Log: Swap in WKILLED processes. Swapped-out process that is WKILLED must be swapped in as soon as possible. The reason is that such process can be killed by OOM and its pages can be only freed if the process exits. To exit, the kernel stack of the process must be mapped. When allocating pages for the stack of the WKILLED process on swap in, use VM_ALLOC_SYSTEM requests to increase the chance of the allocation to succeed. Add counter of the swapped out processes to avoid unneeded iteration over the allprocs list when there is no work to do, reducing the allproc_lock ownership. Reviewed by: alc, markj (previous version) Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential revision: https://reviews.freebsd.org/D16489 Modified: head/sys/kern/kern_sig.c head/sys/kern/sys_process.c head/sys/sys/proc.h head/sys/vm/vm_swapout.c Modified: head/sys/kern/kern_sig.c ============================================================================== --- head/sys/kern/kern_sig.c Sat Aug 4 20:29:58 2018 (r337329) +++ head/sys/kern/kern_sig.c Sat Aug 4 20:45:43 2018 (r337330) @@ -3064,6 +3064,23 @@ postsig(int sig) return (1); } +void +proc_wkilled(struct proc *p) +{ + + PROC_LOCK_ASSERT(p, MA_OWNED); + if ((p->p_flag & P_WKILLED) == 0) { + p->p_flag |= P_WKILLED; + /* + * Notify swapper that there is a process to swap in. + * The notification is racy, at worst it would take 10 + * seconds for the swapper process to notice. + */ + if ((p->p_flag & (P_INMEM | P_SWAPPINGIN)) == 0) + wakeup(&proc0); + } +} + /* * Kill the current process for stated reason. */ @@ -3076,7 +3093,7 @@ killproc(struct proc *p, char *why) p->p_comm); log(LOG_ERR, "pid %d (%s), uid %d, was killed: %s\n", p->p_pid, p->p_comm, p->p_ucred ? p->p_ucred->cr_uid : -1, why); - p->p_flag |= P_WKILLED; + proc_wkilled(p); kern_psignal(p, SIGKILL); } Modified: head/sys/kern/sys_process.c ============================================================================== --- head/sys/kern/sys_process.c Sat Aug 4 20:29:58 2018 (r337329) +++ head/sys/kern/sys_process.c Sat Aug 4 20:45:43 2018 (r337330) @@ -1170,7 +1170,7 @@ kern_ptrace(struct thread *td, int req, pid_t pid, voi * queue cannot accommodate any new signals. */ if (data == SIGKILL) - p->p_flag |= P_WKILLED; + proc_wkilled(p); /* * Unsuspend all threads. To leave a thread Modified: head/sys/sys/proc.h ============================================================================== --- head/sys/sys/proc.h Sat Aug 4 20:29:58 2018 (r337329) +++ head/sys/sys/proc.h Sat Aug 4 20:45:43 2018 (r337330) @@ -1050,6 +1050,7 @@ struct proc *proc_realparent(struct proc *child); void proc_reap(struct thread *td, struct proc *p, int *status, int options); void proc_reparent(struct proc *child, struct proc *newparent); void proc_set_traced(struct proc *p, bool stop); +void proc_wkilled(struct proc *p); struct pstats *pstats_alloc(void); void pstats_fork(struct pstats *src, struct pstats *dst); void pstats_free(struct pstats *ps); Modified: head/sys/vm/vm_swapout.c ============================================================================== --- head/sys/vm/vm_swapout.c Sat Aug 4 20:29:58 2018 (r337329) +++ head/sys/vm/vm_swapout.c Sat Aug 4 20:45:43 2018 (r337330) @@ -158,13 +158,14 @@ static struct mtx vm_daemon_mtx; /* Allow for use by vm_pageout before vm_daemon is initialized. */ MTX_SYSINIT(vm_daemon, &vm_daemon_mtx, "vm daemon", MTX_DEF); +static int swapped_cnt; + static void swapclear(struct proc *); static int swapout(struct proc *); static void vm_swapout_map_deactivate_pages(vm_map_t, long); static void vm_swapout_object_deactivate_pages(pmap_t, vm_object_t, long); static void swapout_procs(int action); static void vm_req_vmdaemon(int req); -static void vm_thread_swapin(struct thread *td); static void vm_thread_swapout(struct thread *td); /* @@ -563,7 +564,7 @@ vm_thread_swapout(struct thread *td) * Bring the kernel stack for a specified thread back in. */ static void -vm_thread_swapin(struct thread *td) +vm_thread_swapin(struct thread *td, int oom_alloc) { vm_object_t ksobj; vm_page_t ma[KSTACK_MAX_PAGES]; @@ -572,7 +573,7 @@ vm_thread_swapin(struct thread *td) pages = td->td_kstack_pages; ksobj = td->td_kstack_obj; VM_OBJECT_WLOCK(ksobj); - (void)vm_page_grab_pages(ksobj, 0, VM_ALLOC_NORMAL | VM_ALLOC_WIRED, ma, + (void)vm_page_grab_pages(ksobj, 0, oom_alloc | VM_ALLOC_WIRED, ma, pages); for (i = 0; i < pages;) { vm_page_assert_xbusied(ma[i]); @@ -605,8 +606,10 @@ void faultin(struct proc *p) { struct thread *td; + int oom_alloc; PROC_LOCK_ASSERT(p, MA_OWNED); + /* * If another process is swapping in this process, * just wait until it finishes. @@ -616,7 +619,11 @@ faultin(struct proc *p) msleep(&p->p_flag, &p->p_mtx, PVM, "faultin", 0); return; } + if ((p->p_flag & P_INMEM) == 0) { + oom_alloc = (p->p_flag & P_WKILLED) != 0 ? VM_ALLOC_SYSTEM : + VM_ALLOC_NORMAL; + /* * Don't let another thread swap process p out while we are * busy swapping it in. @@ -624,6 +631,10 @@ faultin(struct proc *p) ++p->p_lock; p->p_flag |= P_SWAPPINGIN; PROC_UNLOCK(p); + sx_xlock(&allproc_lock); + MPASS(swapped_cnt > 0); + swapped_cnt--; + sx_xunlock(&allproc_lock); /* * We hold no lock here because the list of threads @@ -631,14 +642,14 @@ faultin(struct proc *p) * swapped out. */ FOREACH_THREAD_IN_PROC(p, td) - vm_thread_swapin(td); + vm_thread_swapin(td, oom_alloc); + PROC_LOCK(p); swapclear(p); p->p_swtick = ticks; - wakeup(&p->p_flag); - /* Allow other threads to swap p out now. */ + wakeup(&p->p_flag); --p->p_lock; } } @@ -648,29 +659,41 @@ faultin(struct proc *p) * is enough space for them. Of course, if a process waits for a long * time, it will be swapped in anyway. */ -void -swapper(void) + +static struct proc * +swapper_selector(void) { - struct proc *p, *pp; + struct proc *p, *res; struct thread *td; - int ppri, pri, slptime, swtime; + int min_flag, ppri, pri, slptime, swtime; -loop: - if (vm_page_count_min()) { - vm_wait_min(); - goto loop; - } - - pp = NULL; + sx_assert(&allproc_lock, SA_SLOCKED); + if (swapped_cnt == 0) + return (NULL); + res = NULL; ppri = INT_MIN; - sx_slock(&allproc_lock); + min_flag = vm_page_count_min(); FOREACH_PROC_IN_SYSTEM(p) { PROC_LOCK(p); - if (p->p_state == PRS_NEW || - p->p_flag & (P_SWAPPINGOUT | P_SWAPPINGIN | P_INMEM)) { + if (p->p_state == PRS_NEW || (p->p_flag & (P_SWAPPINGOUT | + P_SWAPPINGIN | P_INMEM)) != 0) { PROC_UNLOCK(p); continue; } + if (p->p_state == PRS_NORMAL && (p->p_flag & P_WKILLED) != 0) { + /* + * A swapped-out process might have mapped a + * large portion of the system's pages as + * anonymous memory. There is no other way to + * release the memory other than to kill the + * process, for which we need to swap it in. + */ + return (p); + } + if (min_flag) { + PROC_UNLOCK(p); + continue; + } swtime = (ticks - p->p_swtick) / hz; FOREACH_THREAD_IN_PROC(p, td) { /* @@ -690,7 +713,7 @@ loop: * selection. */ if (pri > ppri) { - pp = p; + res = p; ppri = pri; } } @@ -698,33 +721,40 @@ loop: } PROC_UNLOCK(p); } - sx_sunlock(&allproc_lock); + if (res != NULL) + PROC_LOCK(res); + return (res); +} - /* - * Nothing to do, back to sleep. - */ - if ((p = pp) == NULL) { - tsleep(&proc0, PVM, "swapin", MAXSLP * hz / 2); - goto loop; - } - PROC_LOCK(p); +void +swapper(void) +{ + struct proc *p; - /* - * Another process may be bringing or may have already - * brought this process in while we traverse all threads. - * Or, this process may even be being swapped out again. - */ - if (p->p_flag & (P_INMEM | P_SWAPPINGOUT | P_SWAPPINGIN)) { - PROC_UNLOCK(p); - goto loop; - } + for (;;) { + sx_slock(&allproc_lock); + p = swapper_selector(); + sx_sunlock(&allproc_lock); - /* - * We would like to bring someone in. - */ - faultin(p); - PROC_UNLOCK(p); - goto loop; + if (p == NULL) { + tsleep(&proc0, PVM, "swapin", MAXSLP * hz / 2); + } else { + PROC_LOCK_ASSERT(p, MA_OWNED); + + /* + * Another process may be bringing or may have + * already brought this process in while we + * traverse all threads. Or, this process may + * have exited or even being swapped out + * again. + */ + if (p->p_state == PRS_NORMAL && (p->p_flag & (P_INMEM | + P_SWAPPINGOUT | P_SWAPPINGIN)) == 0) { + faultin(p); + } + PROC_UNLOCK(p); + } + } } /* @@ -803,7 +833,12 @@ swapout_procs(int action) didswap = true; PROC_UNLOCK(p); - sx_slock(&allproc_lock); + if (didswap) { + sx_xlock(&allproc_lock); + swapped_cnt++; + sx_downgrade(&allproc_lock); + } else + sx_slock(&allproc_lock); PRELE(p); } sx_sunlock(&allproc_lock); From owner-svn-src-all@freebsd.org Sat Aug 4 21:41:12 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17BEA1053B3E; Sat, 4 Aug 2018 21:41:12 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E42BD897B7; Sat, 4 Aug 2018 21:41:11 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C59EC150F8; Sat, 4 Aug 2018 21:41:11 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74LfB1J044398; Sat, 4 Aug 2018 21:41:11 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74LfAb1044394; Sat, 4 Aug 2018 21:41:10 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201808042141.w74LfAb1044394@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Sat, 4 Aug 2018 21:41:10 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337331 - in head/sys: amd64/amd64 arm64/arm64 dev/efidev sys X-SVN-Group: head X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: in head/sys: amd64/amd64 arm64/arm64 dev/efidev sys X-SVN-Commit-Revision: 337331 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 21:41:12 -0000 Author: kevans Date: Sat Aug 4 21:41:10 2018 New Revision: 337331 URL: https://svnweb.freebsd.org/changeset/base/337331 Log: efirt: Don't enter EFI context early, convert addrs to KVA instead efi_enter here was needed because efi_runtime dereference causes a fault outside of EFI context, due to runtime table living in runtime service space. This may cause problems early in boot, though, so instead access it by converting paddr to KVA for access. While here, remove the other direct PHYS_TO_DMAP calls and the explicit DMAP requirement from efidev. Reviewed by: kib MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D16591 Modified: head/sys/amd64/amd64/efirt_machdep.c head/sys/arm64/arm64/efirt_machdep.c head/sys/dev/efidev/efirt.c head/sys/sys/efi.h Modified: head/sys/amd64/amd64/efirt_machdep.c ============================================================================== --- head/sys/amd64/amd64/efirt_machdep.c Sat Aug 4 20:45:43 2018 (r337330) +++ head/sys/amd64/amd64/efirt_machdep.c Sat Aug 4 21:41:10 2018 (r337331) @@ -84,6 +84,19 @@ efi_destroy_1t1_map(void) efi_pml4_page = NULL; } +/* + * Map a physical address from EFI runtime space into KVA space. Returns 0 to + * indicate a failed mapping so that the caller may handle error. + */ +vm_offset_t +efi_phys_to_kva(vm_paddr_t paddr) +{ + + if (paddr >= dmaplimit) + return (0); + return (PHYS_TO_DMAP(paddr)); +} + static vm_page_t efi_1t1_page(void) { Modified: head/sys/arm64/arm64/efirt_machdep.c ============================================================================== --- head/sys/arm64/arm64/efirt_machdep.c Sat Aug 4 20:45:43 2018 (r337330) +++ head/sys/arm64/arm64/efirt_machdep.c Sat Aug 4 21:41:10 2018 (r337331) @@ -143,6 +143,19 @@ efi_1t1_l3(vm_offset_t va) } /* + * Map a physical address from EFI runtime space into KVA space. Returns 0 to + * indicate a failed mapping so that the caller may handle error. + */ +vm_offset_t +efi_phys_to_kva(vm_paddr_t paddr) +{ + + if (!PHYS_IN_DMAP(paddr)) + return (0); + return (PHYS_TO_DMAP(paddr)); +} + +/* * Create the 1:1 virtual to physical map for EFI */ bool Modified: head/sys/dev/efidev/efirt.c ============================================================================== --- head/sys/dev/efidev/efirt.c Sat Aug 4 20:45:43 2018 (r337330) +++ head/sys/dev/efidev/efirt.c Sat Aug 4 21:41:10 2018 (r337331) @@ -131,9 +131,10 @@ efi_init(void) { struct efi_map_header *efihdr; struct efi_md *map; + struct efi_rt *rtdm; caddr_t kmdp; size_t efisz; - int rt_disabled; + int ndesc, rt_disabled; rt_disabled = 0; TUNABLE_INT_FETCH("efi.rt.disabled", &rt_disabled); @@ -146,13 +147,9 @@ efi_init(void) printf("EFI systbl not available\n"); return (0); } - if (!PMAP_HAS_DMAP) { - if (bootverbose) - printf("EFI systbl requires direct map\n"); - return (0); - } - efi_systbl = (struct efi_systbl *)PHYS_TO_DMAP(efi_systbl_phys); - if (efi_systbl->st_hdr.th_sig != EFI_SYSTBL_SIG) { + + efi_systbl = (struct efi_systbl *)efi_phys_to_kva(efi_systbl_phys); + if (efi_systbl == NULL || efi_systbl->st_hdr.th_sig != EFI_SYSTBL_SIG) { efi_systbl = NULL; if (bootverbose) printf("EFI systbl signature invalid\n"); @@ -180,8 +177,8 @@ efi_init(void) if (efihdr->descriptor_size == 0) return (ENOMEM); - if (!efi_create_1t1_map(map, efihdr->memory_size / - efihdr->descriptor_size, efihdr->descriptor_size)) { + ndesc = efihdr->memory_size / efihdr->descriptor_size; + if (!efi_create_1t1_map(map, ndesc, efihdr->descriptor_size)) { if (bootverbose) printf("EFI cannot create runtime map\n"); return (ENOMEM); @@ -196,6 +193,7 @@ efi_init(void) return (ENXIO); } +#if defined(__aarch64__) || defined(__amd64__) /* * Some UEFI implementations have multiple implementations of the * RS->GetTime function. They switch from one we can only use early @@ -203,14 +201,10 @@ efi_init(void) * call RS->SetVirtualAddressMap. As this is not always the case, e.g. * with an old loader.efi, check if the RS->GetTime function is within * the EFI map, and fail to attach if not. - * - * We need to enter into the EFI environment as efi_runtime may point - * to an EFI address. */ - efi_enter(); - if (!efi_is_in_map(map, efihdr->memory_size / efihdr->descriptor_size, - efihdr->descriptor_size, (vm_offset_t)efi_runtime->rt_gettime)) { - efi_leave(); + rtdm = (struct efi_rt *)efi_phys_to_kva((uintptr_t)efi_runtime); + if (rtdm == NULL || !efi_is_in_map(map, ndesc, efihdr->descriptor_size, + (vm_offset_t)rtdm->rt_gettime)) { if (bootverbose) printf( "EFI runtime services table has an invalid pointer\n"); @@ -218,7 +212,7 @@ efi_init(void) efi_destroy_1t1_map(); return (ENXIO); } - efi_leave(); +#endif return (0); } @@ -291,7 +285,7 @@ efi_get_table(struct uuid *uuid, void **ptr) ct = efi_cfgtbl; while (count--) { if (!bcmp(&ct->ct_uuid, uuid, sizeof(*uuid))) { - *ptr = (void *)PHYS_TO_DMAP(ct->ct_data); + *ptr = (void *)efi_phys_to_kva(ct->ct_data); return (0); } ct++; Modified: head/sys/sys/efi.h ============================================================================== --- head/sys/sys/efi.h Sat Aug 4 20:45:43 2018 (r337330) +++ head/sys/sys/efi.h Sat Aug 4 21:41:10 2018 (r337331) @@ -172,6 +172,7 @@ extern vm_paddr_t efi_systbl_phys; /* Internal MD EFI functions */ int efi_arch_enter(void); void efi_arch_leave(void); +vm_offset_t efi_phys_to_kva(vm_paddr_t); bool efi_create_1t1_map(struct efi_md *, int, int); void efi_destroy_1t1_map(void); From owner-svn-src-all@freebsd.org Sat Aug 4 21:54:32 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7D8B105434A; Sat, 4 Aug 2018 21:54:31 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E1368A086; Sat, 4 Aug 2018 21:54:31 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 68A9215400; Sat, 4 Aug 2018 21:54:31 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74LsVir052488; Sat, 4 Aug 2018 21:54:31 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74LsUte052485; Sat, 4 Aug 2018 21:54:30 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201808042154.w74LsUte052485@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: Conrad Meyer Date: Sat, 4 Aug 2018 21:54:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337332 - in head/bin/date: . tests X-SVN-Group: head X-SVN-Commit-Author: cem X-SVN-Commit-Paths: in head/bin/date: . tests X-SVN-Commit-Revision: 337332 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 21:54:32 -0000 Author: cem Date: Sat Aug 4 21:54:30 2018 New Revision: 337332 URL: https://svnweb.freebsd.org/changeset/base/337332 Log: date(1): Add ISO 8601 formatting option The new flag is named '-I'. It is documented in the manual page and covered by basic unit tests. Modified: head/bin/date/date.1 head/bin/date/date.c head/bin/date/tests/format_string_test.sh Modified: head/bin/date/date.1 ============================================================================== --- head/bin/date/date.1 Sat Aug 4 21:41:10 2018 (r337331) +++ head/bin/date/date.1 Sat Aug 4 21:54:30 2018 (r337332) @@ -32,7 +32,7 @@ .\" @(#)date.1 8.3 (Berkeley) 4/28/95 .\" $FreeBSD$ .\" -.Dd June 1, 2018 +.Dd August 4, 2018 .Dt DATE 1 .Os .Sh NAME @@ -64,6 +64,13 @@ .Nm .Op Fl d Ar dst .Op Fl t Ar minutes_west +.Nm +.Op Fl jnu +.Op Fl I Ns Op Ar FMT +.Op Fl f Ar input_fmt +.Op Fl r Ar ... +.Op Fl v Ar ... +.Op Ar new_date .Sh DESCRIPTION When invoked without arguments, the .Nm @@ -113,6 +120,33 @@ provided rather than using the default format. Parsing is done using .Xr strptime 3 . +.It Fl I Ns Op Ar FMT +Use +.St -iso8601 +output format. +.Ar FMT +may be omitted, in which case the default is +.Sq date . +Valid +.Ar FMT +values are +.Sq date , +.Sq hours , +.Sq minutes , +and +.Sq seconds . +The date and time is formatted to the specified precision. +When +.Ar FMT +is +.Sq hours +(or the more precise +.Sq minutes +or +.Sq seconds ) , +the +.St -iso8601 +format includes the timezone. .It Fl j Do not try to set the date. This allows you to use the @@ -401,6 +435,14 @@ sets the time to .Li "2:32 PM" , without modifying the date. .Pp +The command +.Pp +.Dl "TZ=America/Los_Angeles date -Iseconds -r 1533415339" +.Pp +will display +.Pp +.Dl "2018-08-04T13:42:19-07:00" +.Pp Finally the command: .Pp .Dl "date -j -f ""%a %b %d %T %Z %Y"" ""`date`"" ""+%s""" @@ -425,6 +467,19 @@ between and .Xr timed 8 fails. +.Pp +It is invalid to combine the +.Fl I +flag with either +.Fl R +or an output format +.Dq ( + Ns ... ) +operand. +If this occurs, +.Nm +prints: +.Ql multiple output formats specified +and exits with an error status. .Sh SEE ALSO .Xr locale 1 , .Xr gettimeofday 2 , @@ -443,12 +498,22 @@ The utility is expected to be compatible with .St -p1003.2 . The -.Fl d , f , j , n , r , t , +.Fl d , f , I , j , n , r , t , and .Fl v options are all extensions to the standard. +.Pp +The format selected by the +.Fl I +flag is compatible with +.St -iso8601 . .Sh HISTORY A .Nm command appeared in .At v1 . +.Pp +The +.Fl I +flag was added in +.Fx 12.0 . Modified: head/bin/date/date.c ============================================================================== --- head/bin/date/date.c Sat Aug 4 21:41:10 2018 (r337331) +++ head/bin/date/date.c Sat Aug 4 21:54:30 2018 (r337332) @@ -51,6 +51,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -68,10 +69,25 @@ __FBSDID("$FreeBSD$"); static time_t tval; int retval; -static void setthetime(const char *, const char *, int, int); static void badformat(void); +static void iso8601_usage(const char *); +static void multipleformats(void); +static void printdate(const char *); +static void printisodate(struct tm *); +static void setthetime(const char *, const char *, int, int); static void usage(void); +static const struct iso8601_fmt { + const char *refname; + const char *format_string; +} iso8601_fmts[] = { + { "date", "%Y-%m-%d" }, + { "hours", "T%H" }, + { "minutes", ":%M" }, + { "seconds", ":%S" }, +}; +static const struct iso8601_fmt *iso8601_selected; + static const char *rfc2822_format = "%a, %d %b %Y %T %z"; int @@ -79,7 +95,7 @@ main(int argc, char *argv[]) { struct timezone tz; int ch, rflag; - int jflag, nflag, Rflag; + bool Iflag, jflag, nflag, Rflag; const char *format; char buf[1024]; char *endptr, *fmt; @@ -89,15 +105,16 @@ main(int argc, char *argv[]) const struct vary *badv; struct tm *lt; struct stat sb; + size_t i; v = NULL; fmt = NULL; (void) setlocale(LC_TIME, ""); tz.tz_dsttime = tz.tz_minuteswest = 0; rflag = 0; - jflag = nflag = Rflag = 0; + Iflag = jflag = nflag = Rflag = 0; set_timezone = 0; - while ((ch = getopt(argc, argv, "d:f:jnRr:t:uv:")) != -1) + while ((ch = getopt(argc, argv, "d:f:I::jnRr:t:uv:")) != -1) switch((char)ch) { case 'd': /* daylight savings time */ tz.tz_dsttime = strtol(optarg, &endptr, 10) ? 1 : 0; @@ -108,6 +125,22 @@ main(int argc, char *argv[]) case 'f': fmt = optarg; break; + case 'I': + if (Rflag) + multipleformats(); + Iflag = 1; + if (optarg == NULL) { + iso8601_selected = iso8601_fmts; + break; + } + for (i = 0; i < nitems(iso8601_fmts); i++) + if (strcmp(optarg, iso8601_fmts[i].refname) == 0) + break; + if (i == nitems(iso8601_fmts)) + iso8601_usage(optarg); + + iso8601_selected = &iso8601_fmts[i]; + break; case 'j': jflag = 1; /* don't set time */ break; @@ -115,6 +148,8 @@ main(int argc, char *argv[]) nflag = 1; break; case 'R': /* RFC 2822 datetime format */ + if (Iflag) + multipleformats(); Rflag = 1; break; case 'r': /* user specified seconds */ @@ -163,6 +198,8 @@ main(int argc, char *argv[]) /* allow the operands in any order */ if (*argv && **argv == '+') { + if (Iflag) + multipleformats(); format = *argv + 1; ++argv; } @@ -173,8 +210,11 @@ main(int argc, char *argv[]) } else if (fmt != NULL) usage(); - if (*argv && **argv == '+') + if (*argv && **argv == '+') { + if (Iflag) + multipleformats(); format = *argv + 1; + } lt = localtime(&tval); if (lt == NULL) @@ -188,6 +228,9 @@ main(int argc, char *argv[]) } vary_destroy(v); + if (Iflag) + printisodate(lt); + if (format == rfc2822_format) /* * When using RFC 2822 datetime format, don't honor the @@ -196,12 +239,40 @@ main(int argc, char *argv[]) setlocale(LC_TIME, "C"); (void)strftime(buf, sizeof(buf), format, lt); + printdate(buf); +} + +static void +printdate(const char *buf) +{ (void)printf("%s\n", buf); if (fflush(stdout)) err(1, "stdout"); exit(retval); } +static void +printisodate(struct tm *lt) +{ + const struct iso8601_fmt *it; + char fmtbuf[32], buf[32], tzbuf[8]; + + fmtbuf[0] = 0; + for (it = iso8601_fmts; it <= iso8601_selected; it++) + strlcat(fmtbuf, it->format_string, sizeof(fmtbuf)); + + (void)strftime(buf, sizeof(buf), fmtbuf, lt); + + if (iso8601_selected > iso8601_fmts) { + (void)strftime(tzbuf, sizeof(tzbuf), "%z", lt); + memmove(&tzbuf[4], &tzbuf[3], 3); + tzbuf[3] = ':'; + strlcat(buf, tzbuf, sizeof(buf)); + } + + printdate(buf); +} + #define ATOI2(s) ((s) += 2, ((s)[-2] - '0') * 10 + ((s)[-1] - '0')) static void @@ -327,12 +398,27 @@ badformat(void) } static void +iso8601_usage(const char *badarg) +{ + errx(1, "invalid argument '%s' for -I", badarg); +} + +static void +multipleformats(void) +{ + errx(1, "multiple output formats specified"); +} + +static void usage(void) { - (void)fprintf(stderr, "%s\n%s\n", - "usage: date [-jnRu] [-d dst] [-r seconds] [-t west] " - "[-v[+|-]val[ymwdHMS]] ... ", + (void)fprintf(stderr, "%s\n%s\n%s\n", + "usage: date [-jnRu] [-d dst] [-r seconds|file] [-t west] " + "[-v[+|-]val[ymwdHMS]]", " " - "[-f fmt date | [[[[[cc]yy]mm]dd]HH]MM[.ss]] [+format]"); + "[-I[date | hours | minutes | seconds]]", + " " + "[-f fmt date | [[[[[cc]yy]mm]dd]HH]MM[.ss]] [+format]" + ); exit(1); } Modified: head/bin/date/tests/format_string_test.sh ============================================================================== --- head/bin/date/tests/format_string_test.sh Sat Aug 4 21:41:10 2018 (r337331) +++ head/bin/date/tests/format_string_test.sh Sat Aug 4 21:54:30 2018 (r337332) @@ -48,6 +48,55 @@ ${desc}_test_body() { atf_add_test_case ${desc}_test } +iso8601_check() +{ + local arg flags exp_output_1 exp_output_2 + + arg="${1}" + flags="${2}" + exp_output_1="${3}" + exp_output_2="${4}" + + atf_check -o "inline:${exp_output_1}\n" \ + date $flags -r ${TEST1} "-I${arg}" + atf_check -o "inline:${exp_output_2}\n" \ + date $flags -r ${TEST2} "-I${arg}" +} + +iso8601_string_test() +{ + local desc arg exp_output_1 exp_output_2 flags + + desc="${1}" + arg="${2}" + flags="${3}" + exp_output_1="${4}" + exp_output_2="${5}" + + atf_test_case iso8601_${desc}_test + eval " +iso8601_${desc}_test_body() { + iso8601_check '${arg}' '${flags}' '${exp_output_1}' '${exp_output_2}' +}" + atf_add_test_case iso8601_${desc}_test + + if [ -z "$flags" ]; then + atf_test_case iso8601_${desc}_parity + eval " +iso8601_${desc}_parity_body() { + local exp1 exp2 + + atf_require_prog gdate + + exp1=\"\$(gdate --date '@${TEST1}' '-I${arg}')\" + exp2=\"\$(gdate --date '@${TEST2}' '-I${arg}')\" + + iso8601_check '${arg}' '' \"\${exp1}\" \"\${exp2}\" +}" + atf_add_test_case iso8601_${desc}_parity + fi +} + atf_init_test_cases() { format_string_test A A Saturday Monday @@ -89,4 +138,12 @@ atf_init_test_cases() format_string_test z z +0000 +0000 format_string_test percent % % % format_string_test plus + "Sat Feb 7 07:04:03 UTC 1970" "Mon Nov 12 21:20:00 UTC 2001" + + iso8601_string_test default "" "" "1970-02-07" "2001-11-12" + iso8601_string_test date date "" "1970-02-07" "2001-11-12" + iso8601_string_test hours hours "" "1970-02-07T07+00:00" "2001-11-12T21+00:00" + iso8601_string_test minutes minutes "" "1970-02-07T07:04+00:00" "2001-11-12T21:20+00:00" + iso8601_string_test seconds seconds "" "1970-02-07T07:04:03+00:00" "2001-11-12T21:20:00+00:00" + # BSD date(1) does not support fractional seconds at this time. + #iso8601_string_test ns ns "" "1970-02-07T07:04:03,000000000+00:00" "2001-11-12T21:20:00,000000000+00:00" } From owner-svn-src-all@freebsd.org Sat Aug 4 21:57:20 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B3031054465; Sat, 4 Aug 2018 21:57:20 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B3F8E8A239; Sat, 4 Aug 2018 21:57:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8E99D15405; Sat, 4 Aug 2018 21:57:19 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74LvJTi052682; Sat, 4 Aug 2018 21:57:19 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74LvHN8052670; Sat, 4 Aug 2018 21:57:17 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201808042157.w74LvHN8052670@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Sat, 4 Aug 2018 21:57:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337333 - in stable/11: . sys/kern sys/sys tools/build usr.sbin/config X-SVN-Group: stable-11 X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: in stable/11: . sys/kern sys/sys tools/build usr.sbin/config X-SVN-Commit-Revision: 337333 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 21:57:20 -0000 Author: kevans Date: Sat Aug 4 21:57:17 2018 New Revision: 337333 URL: https://svnweb.freebsd.org/changeset/base/337333 Log: kenv MFC: r335998, r336019, r336026, r336036, r336217, r336335, r336337, r336415-r336416, r336419 As an aside- this has been slightly altered from the version in head to keep the MD and config-static environments mutually exclusive by default. This difference is a one-line change in init_static_kenv to setup the MD environment if the config-static environment is empty or if loader_env.disabled is explicitly set to 0. r335998: kern_environment: use any provided environments, evict hintmode/envmode At the moment, hintmode and envmode are used to indicate whether static hints or static env have been provided in the kernel config(5) and the static versions are mutually exclusive with loader(8)-provided environment. hintmode *can* be reconfigured later to pull from the dynamic environment, thus taking advantage of the loader(8) or post-kmem environment setting. This changeset fixes both problems at once to move us from a semi-confusing state to a consistent state: if an environment file, hints file, or loader(8) environment are provided, we use them in a well-known order of precedence: - loader(8) environment - static environment - static hints file Once the dynamic environment is setup this becomes a moot point. The loader(8) and static environments are merged (respecting the above order of precedence), and the static hints are merged in on an as-needed basis after the dynamic environment has been setup. Hints lookup are changed to respect all of the above. Before the dynamic environment is setup, lookups use the above-mentioned order and fallback to the next environment if a matching hint is not found. Once the dynamic environment is setup, that is used on its own since it captures all of the above information plus any dynamic kenv settings that came up later in boot. The following tangentially related changes were made to res_find: - A hintp cookie is now passed in so that related searches continue using the chain of environments (or dynamic environment) without relying on global state - All three environments will be searched if they actually have valid hints to use, rather than just choosing the first environment that actually had a hint and rolling with that only The hintmode sysctl has been ripped out. static_{env,hints}.disabled are still honored and will disable their respective environments from being used for hint lookups and from being merged into the dynamic environment, as expected. r336019: config(8): De-dupe hint/env vars within a single file r335653 flipped the order in which hints/env files are concatenated to match the order in which vars are processed by the kernel. This is the other hammer to drop. Use nv(9) to de-dupe entries within a single `hint` or `env` file, using the latest value specified for a key. This leaves some duplicates if a variable is specified in multiple hint/env files or via `envvar` in a kernel config, but the reversed order of concatenation (from r335653) makes this a non-issue as the latest-specified version will be seen first. This change also silently rewrote hint bits to use the same sanitization process that ian@ wrote for r335642. To the kernel, hints and env vars are basically the same thing through early boot, then get merged into the dynamic environment once kmem becomes available and the dynamic environment is created. They should be subjected to the same restrictions. libnv has been added to -legacy for the time being to support the build of config(8) with the new cnvlist API. r336026: config(8): Fix broken ABI r336019 introduced ${SRCTOP}/sys to the include paths in order to pull in a new sys/{c,}nv.h. This is wrong, because the build tree's ABI isn't guaranteed to match what's running on the host system. Fix instead by removing -I${SRCTOP}/sys and installing the libnv headers with `make -C lib/libnv includes`... this may or may not get re-worked in the future so that a userland lib isn't installing includes from sys/. r336036: kern_environment: Fix SYSINIT ordering The dynamic environment was being initialized at SI_SUB_KMEM, SI_ORDER_ANY. I added the hint-merging at SI_SUB_KMEM, SI_ORDER_ANY as well in r335998 - this can only work by coincidence. Re-do both to operate at SI_SUB_KMEM + 1, SI_ORDER_FIRST and SI_ORDER_SECOND respectively to be safe. It's sufficiently obfuscated away as to when in SU_SUB_KMEM malloc will be available, and the dynamic environment cannot be relied upon there anyways since it's initialized at SI_ORDER_ANY. r336217: kern_environment: Give the static environment a chance to disable MD env This variable has been given the name "loader_env.disabled" as it's the primary way most people will have an MD environment. This restores the previously-default behavior of ignoring the loader(8) environment, which may be useful for vendor distributions or other scenarios where inheriting the loader environment may be considered a security issue or potentially breaking of a more locked-down environment. As the change to config(5) indicates, disabling the loader environment should not be a choice made lightly since it may provide ACPI hints and other useful things that the system can rely on to boot. An UPDATING entry has been added to mention an upgrade path for those that may have relied on the previous behavior. r336335 by arichardson: No longer install sys/nv.h and sys/cnv.h in lib/libnv/Makefile Use tools/build/Makefile to install the headers into ${WORLDTMP}/legacy instead. Compared to r336026 this has the minor advantage that it avoids unncessary header installation when building the non-bootstrap libnv. r336337: Unconditionally build libnv in legacy Rather than using a config(8) built from new tree linking libnv built on host. r336415: config(8): Add compatibility shims for r335998 Plumb the %VERSREQ from Makefile. through to the rest of config(8). We've recorded the config(8) version that we're calling "the end of envmode and hintmode," and we'll write them out for earlier versions. Later kernel version bumps will remove envmode/hintmode from the kernel as needed, which is OK since the current kernel does not use them at all. These compatibility shims really need to go away when the major version rolls over... r336416: Fix GCC 4.2 build after r336415, proper declaration and prototype r336419: config(8): Invert checks; envmode/hintmode reflect "env provided" Relnotes: yes (maybe) [The loader environment may now be used with the config-static environment by specifying loader_env.disabled=0 in the config-static environment] Modified: stable/11/Makefile.inc1 stable/11/sys/kern/kern_environment.c stable/11/sys/kern/subr_hints.c stable/11/sys/sys/systm.h stable/11/tools/build/Makefile stable/11/usr.sbin/config/Makefile stable/11/usr.sbin/config/config.5 stable/11/usr.sbin/config/config.h stable/11/usr.sbin/config/config.y stable/11/usr.sbin/config/configvers.h stable/11/usr.sbin/config/main.c stable/11/usr.sbin/config/mkmakefile.c Directory Properties: stable/11/ (props changed) Modified: stable/11/Makefile.inc1 ============================================================================== --- stable/11/Makefile.inc1 Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/Makefile.inc1 Sat Aug 4 21:57:17 2018 (r337333) @@ -1629,7 +1629,7 @@ legacy: .PHONY @echo "ERROR: Source upgrades from versions prior to ${MINIMUM_SUPPORTED_REL} are not supported."; \ false .endif -.for _tool in tools/build ${_elftoolchain_libs} +.for _tool in tools/build ${_elftoolchain_libs} lib/libnv ${_+_}@${ECHODIR} "===> ${_tool} (obj,includes,all,install)"; \ cd ${.CURDIR}/${_tool}; \ ${MAKE} DIRPRFX=${_tool}/ obj; \ Modified: stable/11/sys/kern/kern_environment.c ============================================================================== --- stable/11/sys/kern/kern_environment.c Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/sys/kern/kern_environment.c Sat Aug 4 21:57:17 2018 (r337333) @@ -55,14 +55,21 @@ __FBSDID("$FreeBSD$"); #include +static char *_getenv_dynamic_locked(const char *name, int *idx); +static char *_getenv_dynamic(const char *name, int *idx); + static MALLOC_DEFINE(M_KENV, "kenv", "kernel environment"); #define KENV_SIZE 512 /* Maximum number of environment strings */ -/* pointer to the static environment */ +/* pointer to the config-generated static environment */ char *kern_envp; -static int env_len; -static int env_pos; + +/* pointer to the md-static environment */ +char *md_envp; +static int md_env_len; +static int md_env_pos; + static char *kernenv_next(char *); /* dynamic environment variables */ @@ -218,16 +225,9 @@ done: * environment obtained from a boot loader, or to provide an empty buffer into * which MD code can store an initial environment using kern_setenv() calls. * - * When a copy of an initial environment is passed in, we start by scanning that - * env for overrides to the compiled-in envmode and hintmode variables. + * kern_envp is set to the static_env generated by config(8). This implements + * the env keyword described in config(5). * - * If the global envmode is 1, the environment is initialized from the global - * static_env[], regardless of the arguments passed. This implements the env - * keyword described in config(5). In this case env_pos is set to env_len, - * causing kern_setenv() to return -1 (if len > 0) or panic (if len == 0) until - * the dynamic environment is available. The envmode and static_env variables - * are defined in env.c which is generated by config(8). - * * If len is non-zero, the caller is providing an empty buffer. The caller will * subsequently use kern_setenv() to add up to len bytes of initial environment * before the dynamic environment is available. @@ -235,72 +235,126 @@ done: * If len is zero, the caller is providing a pre-loaded buffer containing * environment strings. Additional strings cannot be added until the dynamic * environment is available. The memory pointed to must remain stable at least - * until sysinit runs init_dynamic_kenv(). If no initial environment is - * available from the boot loader, passing a NULL pointer allows the static_env - * to be installed if it is configured. + * until sysinit runs init_dynamic_kenv() and preferably until after SI_SUB_KMEM + * is finished so that subr_hints routines may continue to use it until the + * environments have been fully merged at the end of the pass. If no initial + * environment is available from the boot loader, passing a NULL pointer allows + * the static_env to be installed if it is configured. In this case, any call + * to kern_setenv() prior to the setup of the dynamic environment will result in + * a panic. */ void init_static_kenv(char *buf, size_t len) { - char *cp; - - for (cp = buf; cp != NULL && cp[0] != '\0'; cp += strlen(cp) + 1) { - if (strcmp(cp, "static_env.disabled=1") == 0) - envmode = 0; - if (strcmp(cp, "static_hints.disabled=1") == 0) - hintmode = 0; - } + char *eval; - if (envmode == 1) { - kern_envp = static_env; - env_len = len; - env_pos = len; - } else { - kern_envp = buf; - env_len = len; - env_pos = 0; + /* + * Give the static environment a chance to disable the loader(8) + * environment first. This is done with loader_env.disabled=1. + * + * static_env and static_hints may both be disabled, but in slightly + * different ways. For static_env, we just don't setup kern_envp and + * it's as if a static env wasn't even provided. For static_hints, + * we effectively zero out the buffer to stop the rest of the kernel + * from being able to use it. + * + * We're intentionally setting this up so that static_hints.disabled may + * be specified in either the MD env or the static env. This keeps us + * consistent in our new world view. + * + * As a warning, the static environment may not be disabled in any way + * if the static environment has disabled the loader environment. + */ + kern_envp = static_env; + eval = kern_getenv("loader_env.disabled"); + if (*kern_envp == '\0' || (eval != NULL && strcmp(eval, "0") == 0)) { + md_envp = buf; + md_env_len = len; + md_env_pos = 0; + + eval = kern_getenv("static_env.disabled"); + if (eval != NULL && strcmp(eval, "1") == 0) + *kern_envp = '\0'; } + eval = kern_getenv("static_hints.disabled"); + if (eval != NULL && strcmp(eval, "1") == 0) + *static_hints = '\0'; } -/* - * Setup the dynamic kernel environment. - */ static void -init_dynamic_kenv(void *data __unused) +init_dynamic_kenv_from(char *init_env, int *curpos) { - char *cp, *cpnext; + char *cp, *cpnext, *eqpos, *found; size_t len; int i; - kenvp = malloc((KENV_SIZE + 1) * sizeof(char *), M_KENV, - M_WAITOK | M_ZERO); - i = 0; - if (kern_envp && *kern_envp != '\0') { - for (cp = kern_envp; cp != NULL; cp = cpnext) { + if (init_env && *init_env != '\0') { + found = NULL; + i = *curpos; + for (cp = init_env; cp != NULL; cp = cpnext) { cpnext = kernenv_next(cp); len = strlen(cp) + 1; if (len > KENV_MNAMELEN + 1 + KENV_MVALLEN + 1) { printf( "WARNING: too long kenv string, ignoring %s\n", cp); - continue; + goto sanitize; } - if (i < KENV_SIZE) { - kenvp[i] = malloc(len, M_KENV, M_WAITOK); - strcpy(kenvp[i++], cp); - explicit_bzero(cp, strlen(cp)); - } else + eqpos = strchr(cp, '='); + if (eqpos == NULL) { printf( + "WARNING: malformed static env value, ignoring %s\n", + cp); + goto sanitize; + } + *eqpos = 0; + /* + * De-dupe the environment as we go. We don't add the + * duplicated assignments because config(8) will flip + * the order of the static environment around to make + * kernel processing match the order of specification + * in the kernel config. + */ + found = _getenv_dynamic_locked(cp, NULL); + *eqpos = '='; + if (found != NULL) + goto sanitize; + if (i > KENV_SIZE) { + printf( "WARNING: too many kenv strings, ignoring %s\n", cp); + goto sanitize; + } + + kenvp[i] = malloc(len, M_KENV, M_WAITOK); + strcpy(kenvp[i++], cp); +sanitize: + explicit_bzero(cp, len - 1); } + *curpos = i; } - kenvp[i] = NULL; +} +/* + * Setup the dynamic kernel environment. + */ +static void +init_dynamic_kenv(void *data __unused) +{ + int dynamic_envpos; + + kenvp = malloc((KENV_SIZE + 1) * sizeof(char *), M_KENV, + M_WAITOK | M_ZERO); + + dynamic_envpos = 0; + init_dynamic_kenv_from(md_envp, &dynamic_envpos); + init_dynamic_kenv_from(kern_envp, &dynamic_envpos); + kenvp[dynamic_envpos] = NULL; + mtx_init(&kenv_lock, "kernel environment", NULL, MTX_DEF); dynamic_kenv = 1; } -SYSINIT(kenv, SI_SUB_KMEM, SI_ORDER_ANY, init_dynamic_kenv, NULL); +SYSINIT(kenv, SI_SUB_KMEM + 1, SI_ORDER_FIRST, init_dynamic_kenv, NULL); void freeenv(char *env) @@ -316,12 +370,11 @@ freeenv(char *env) * Internal functions for string lookup. */ static char * -_getenv_dynamic(const char *name, int *idx) +_getenv_dynamic_locked(const char *name, int *idx) { char *cp; int len, i; - mtx_assert(&kenv_lock, MA_OWNED); len = strlen(name); for (cp = kenvp[0], i = 0; cp != NULL; cp = kenvp[++i]) { if ((strncmp(cp, name, len) == 0) && @@ -335,12 +388,20 @@ _getenv_dynamic(const char *name, int *idx) } static char * -_getenv_static(const char *name) +_getenv_dynamic(const char *name, int *idx) { + + mtx_assert(&kenv_lock, MA_OWNED); + return (_getenv_dynamic_locked(name, idx)); +} + +static char * +_getenv_static_from(char *chkenv, const char *name) +{ char *cp, *ep; int len; - for (cp = kern_envp; cp != NULL; cp = kernenv_next(cp)) { + for (cp = chkenv; cp != NULL; cp = kernenv_next(cp)) { for (ep = cp; (*ep != '=') && (*ep != 0); ep++) ; if (*ep != '=') @@ -353,6 +414,20 @@ _getenv_static(const char *name) return (NULL); } +static char * +_getenv_static(const char *name) +{ + char *val; + + val = _getenv_static_from(md_envp, name); + if (val != NULL) + return (val); + val = _getenv_static_from(kern_envp, name); + if (val != NULL) + return (val); + return (NULL); +} + /* * Look up an environment variable by name. * Return a pointer to the string if found. @@ -397,20 +472,25 @@ testenv(const char *name) return (0); } +/* + * Set an environment variable in the MD-static environment. This cannot + * feasibly be done on config(8)-generated static environments as they don't + * generally include space for extra variables. + */ static int setenv_static(const char *name, const char *value) { int len; - if (env_pos >= env_len) + if (md_env_pos >= md_env_len) return (-1); /* Check space for x=y and two nuls */ len = strlen(name) + strlen(value); - if (len + 3 < env_len - env_pos) { - len = sprintf(&kern_envp[env_pos], "%s=%s", name, value); - env_pos += len+1; - kern_envp[env_pos] = '\0'; + if (len + 3 < md_env_len - md_env_pos) { + len = sprintf(&md_envp[md_env_pos], "%s=%s", name, value); + md_env_pos += len+1; + md_envp[md_env_pos] = '\0'; return (0); } else return (-1); @@ -426,7 +506,7 @@ kern_setenv(const char *name, const char *value) char *buf, *cp, *oldenv; int namelen, vallen, i; - if (dynamic_kenv == 0 && env_len > 0) + if (dynamic_kenv == 0 && md_env_len > 0) return (setenv_static(name, value)); KENV_CHECK; Modified: stable/11/sys/kern/subr_hints.c ============================================================================== --- stable/11/sys/kern/subr_hints.c Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/sys/kern/subr_hints.c Sat Aug 4 21:57:17 2018 (r337333) @@ -29,61 +29,36 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include #include #include -#define HINTMODE_KENV 0 -#define HINTMODE_STATIC 1 -#define HINTMODE_FALLBACK 2 +#define FBACK_MDENV 0 /* MD env (e.g. loader.conf) */ +#define FBACK_STENV 1 /* Static env */ +#define FBACK_STATIC 2 /* static_hints */ /* - * Access functions for device resources. + * We'll use hintenv_merged to indicate that the dynamic environment has been + * properly prepared for hint usage. This implies that the dynamic environment + * has already been setup (dynamic_kenv) and that we have added any supplied + * static_hints to the dynamic environment. */ +static int hintenv_merged; -static int checkmethod = 1; -static char *hintp; - /* - * Define kern.hintmode sysctl, which only accept value 2, that cause to - * switch from Static KENV mode to Dynamic KENV. So systems that have hints - * compiled into kernel will be able to see/modify KENV (and hints too). + * Access functions for device resources. */ -static int -sysctl_hintmode(SYSCTL_HANDLER_ARGS) +static void +static_hints_to_env(void *data __unused) { const char *cp; char *line, *eq; - int eqidx, error, i, value; + int eqidx, i; - value = hintmode; - - /* Fetch candidate for new hintmode value */ - error = sysctl_handle_int(oidp, &value, 0, req); - if (error || req->newptr == NULL) - return (error); - - if (value != HINTMODE_FALLBACK) - /* Only accept swithing to hintmode 2 */ - return (EINVAL); - - /* - * The rest of the sysctl handler is just making sure that our - * environment is consistent with the world we've already seen. - * If we came from kenv at all, then we have nothing to do: static - * kenv will get merged into dynamic kenv as soon as kmem becomes - * available, dynamic kenv is the environment we'd be setting these - * things in anyways. Therefore, we have nothing left to do unless - * we came from a static hints configuration. - */ - if (hintmode != HINTMODE_STATIC) { - hintmode = value; - return (0); - } - cp = static_hints; while (cp && *cp != '\0') { eq = strchr(cp, '='); @@ -93,103 +68,152 @@ sysctl_hintmode(SYSCTL_HANDLER_ARGS) eqidx = eq - cp; i = strlen(cp); - line = malloc(i+1, M_TEMP, M_WAITOK); + line = malloc(i + 1, M_TEMP, M_WAITOK); strcpy(line, cp); - line[eqidx] = '\0'; - kern_setenv(line, line + eqidx + 1); + line[eqidx] = line[i] = '\0'; + /* + * Before adding a hint to the dynamic environment, check if + * another value for said hint has already been added. This is + * needed because static environment overrides static hints and + * dynamic environment overrides all. + */ + if (testenv(line) == 0) + kern_setenv(line, line + eqidx + 1); free(line, M_TEMP); cp += i + 1; } - - hintmode = value; - return (0); + hintenv_merged = 1; } -SYSCTL_PROC(_kern, OID_AUTO, hintmode, CTLTYPE_INT|CTLFLAG_RW, - &hintmode, 0, sysctl_hintmode, "I", "Get/set current hintmode"); +/* Any time after dynamic env is setup */ +SYSINIT(hintenv, SI_SUB_KMEM + 1, SI_ORDER_SECOND, static_hints_to_env, NULL); /* + * Checks the environment to see if we even have any hints. If it has no hints, + * then res_find can take the hint that there's no point in searching it and + * either move on to the next environment or fail early. + */ +static bool +_res_checkenv(char *envp) +{ + char *cp; + + cp = envp; + while (cp) { + if (strncmp(cp, "hint.", 5) == 0) + return (true); + while (*cp != '\0') + cp++; + cp++; + if (*cp == '\0') + break; + } + return (false); +} + +/* * Evil wildcarding resource string lookup. * This walks the supplied env string table and returns a match. * The start point can be remembered for incremental searches. */ static int -res_find(int *line, int *startln, +res_find(char **hintp_cookie, int *line, int *startln, const char *name, int *unit, const char *resname, const char *value, const char **ret_name, int *ret_namelen, int *ret_unit, const char **ret_resname, int *ret_resnamelen, const char **ret_value) { - int n = 0, hit, i = 0; + int dyn_used = 0, fbacklvl = FBACK_MDENV, hit, i = 0, n = 0; char r_name[32]; - int r_unit, use_kenv = (hintmode != HINTMODE_STATIC && dynamic_kenv); + int r_unit; char r_resname[32]; char r_value[128]; const char *s, *cp; - char *p; + char *hintp, *p; - if (checkmethod) { - hintp = NULL; - switch (hintmode) { - case HINTMODE_KENV: /* loader hints in environment only */ - break; - case HINTMODE_STATIC: /* static hints only */ - hintp = static_hints; - checkmethod = 0; - break; - case HINTMODE_FALLBACK: /* fallback mode */ - if (dynamic_kenv) { - mtx_lock(&kenv_lock); - cp = kenvp[0]; - for (i = 0; cp != NULL; cp = kenvp[++i]) { - if (!strncmp(cp, "hint.", 5)) { - use_kenv = 1; - checkmethod = 0; - break; - } + /* + * We are expecting that the caller will pass us a hintp_cookie that + * they are tracking. Upon entry, if *hintp_cookie is *not* set, this + * indicates to us that we should be figuring out based on the current + * environment where to search. This keeps us sane throughout the + * entirety of a single search. + */ + if (*hintp_cookie == NULL) { + hintp = NULL; + if (hintenv_merged) { + /* + * static_hints, if it was previously used, has + * already been folded in to the environment + * by this point. + */ + mtx_lock(&kenv_lock); + cp = kenvp[0]; + for (i = 0; cp != NULL; cp = kenvp[++i]) { + if (!strncmp(cp, "hint.", 5)) { + hintp = kenvp[0]; + break; } - mtx_unlock(&kenv_lock); - } else { - cp = kern_envp; - while (cp) { - if (strncmp(cp, "hint.", 5) == 0) { - cp = NULL; - hintp = kern_envp; - break; - } - while (*cp != '\0') - cp++; - cp++; - if (*cp == '\0') { - cp = NULL; - hintp = static_hints; - break; - } - } } - break; - default: - break; - } - if (hintp == NULL) { - if (dynamic_kenv) { - use_kenv = 1; - checkmethod = 0; - } else + mtx_unlock(&kenv_lock); + dyn_used = 1; + } else { + /* + * We'll have a chance to keep coming back here until + * we've actually exhausted all of our possibilities. + * We might have chosen the MD/Static env because it + * had some kind of hints, but perhaps it didn't have + * the hint we are looking for. We don't provide any + * fallback when searching the dynamic environment. + */ +fallback: + if (dyn_used || fbacklvl >= FBACK_STATIC) + return (ENOENT); + + if (fbacklvl <= FBACK_MDENV && + _res_checkenv(md_envp)) { + hintp = md_envp; + goto found; + } + fbacklvl++; + + if (fbacklvl <= FBACK_STENV && + _res_checkenv(kern_envp)) { hintp = kern_envp; + goto found; + } + fbacklvl++; + + /* We'll fallback to static_hints if needed/can */ + if (fbacklvl <= FBACK_STATIC && + _res_checkenv(static_hints)) + hintp = static_hints; +found: + fbacklvl++; } + + if (hintp == NULL) + return (ENOENT); + *hintp_cookie = hintp; + } else { + hintp = *hintp_cookie; + if (hintenv_merged && hintp == kenvp[0]) + dyn_used = 1; + else + /* + * If we aren't using the dynamic environment, we need + * to run through the proper fallback procedure again. + * This is so that we do continuations right if we're + * working with *line and *startln. + */ + goto fallback; } - if (use_kenv) { + if (dyn_used) { mtx_lock(&kenv_lock); i = 0; - cp = kenvp[0]; - if (cp == NULL) { - mtx_unlock(&kenv_lock); - return (ENOENT); - } - } else - cp = hintp; + } + + cp = hintp; while (cp) { hit = 1; (*line)++; @@ -198,25 +222,28 @@ res_find(int *line, int *startln, else n = sscanf(cp, "hint.%32[^.].%d.%32[^=]=%127s", r_name, &r_unit, r_resname, r_value); - if (hit && n != 4) { - printf("CONFIG: invalid hint '%s'\n", cp); - p = strchr(cp, 'h'); - *p = 'H'; - hit = 0; + /* We'll circumvent all of the checks if we already know */ + if (hit) { + if (n != 4) { + printf("CONFIG: invalid hint '%s'\n", cp); + p = strchr(cp, 'h'); + *p = 'H'; + hit = 0; + } + if (hit && startln && *startln >= 0 && *line < *startln) + hit = 0; + if (hit && name && strcmp(name, r_name) != 0) + hit = 0; + if (hit && unit && *unit != r_unit) + hit = 0; + if (hit && resname && strcmp(resname, r_resname) != 0) + hit = 0; + if (hit && value && strcmp(value, r_value) != 0) + hit = 0; + if (hit) + break; } - if (hit && startln && *startln >= 0 && *line < *startln) - hit = 0; - if (hit && name && strcmp(name, r_name) != 0) - hit = 0; - if (hit && unit && *unit != r_unit) - hit = 0; - if (hit && resname && strcmp(resname, r_resname) != 0) - hit = 0; - if (hit && value && strcmp(value, r_value) != 0) - hit = 0; - if (hit) - break; - if (use_kenv) { + if (dyn_used) { cp = kenvp[++i]; if (cp == NULL) break; @@ -230,10 +257,10 @@ res_find(int *line, int *startln, } } } - if (use_kenv) + if (dyn_used) mtx_unlock(&kenv_lock); if (cp == NULL) - return ENOENT; + goto fallback; s = cp; /* This is a bit of a hack, but at least is reentrant */ @@ -271,11 +298,13 @@ resource_find(int *line, int *startln, { int i; int un; + char *hintp; *line = 0; + hintp = NULL; /* Search for exact unit matches first */ - i = res_find(line, startln, name, unit, resname, value, + i = res_find(&hintp, line, startln, name, unit, resname, value, ret_name, ret_namelen, ret_unit, ret_resname, ret_resnamelen, ret_value); if (i == 0) @@ -284,7 +313,7 @@ resource_find(int *line, int *startln, return ENOENT; /* If we are still here, search for wildcard matches */ un = -1; - i = res_find(line, startln, name, &un, resname, value, + i = res_find(&hintp, line, startln, name, &un, resname, value, ret_name, ret_namelen, ret_unit, ret_resname, ret_resnamelen, ret_value); if (i == 0) Modified: stable/11/sys/sys/systm.h ============================================================================== --- stable/11/sys/sys/systm.h Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/sys/sys/systm.h Sat Aug 4 21:57:17 2018 (r337333) @@ -148,11 +148,10 @@ void kassert_panic(const char *fmt, ...) __printflike * XXX most of these variables should be const. */ extern int osreldate; -extern int envmode; -extern int hintmode; /* 0 = off. 1 = config, 2 = fallback */ extern int dynamic_kenv; extern struct mtx kenv_lock; extern char *kern_envp; +extern char *md_envp; extern char static_env[]; extern char static_hints[]; /* by config for now */ Modified: stable/11/tools/build/Makefile ============================================================================== --- stable/11/tools/build/Makefile Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/tools/build/Makefile Sat Aug 4 21:57:17 2018 (r337333) @@ -49,4 +49,7 @@ SRCS= dummy.c SUBDIR= cross-build .endif +# Needed to build config (since it uses libnv) +SYSINCS+= ${SRCTOP}/sys/sys/nv.h ${SRCTOP}/sys/sys/cnv.h + .include Modified: stable/11/usr.sbin/config/Makefile ============================================================================== --- stable/11/usr.sbin/config/Makefile Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/usr.sbin/config/Makefile Sat Aug 4 21:57:17 2018 (r337333) @@ -18,7 +18,7 @@ CFLAGS+= -I. -I${SRCDIR} NO_WMISSING_VARIABLE_DECLARATIONS= -LIBADD= l sbuf +LIBADD= l nv sbuf CLEANFILES+= kernconf.c Modified: stable/11/usr.sbin/config/config.5 ============================================================================== --- stable/11/usr.sbin/config/config.5 Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/usr.sbin/config/config.5 Sat Aug 4 21:57:17 2018 (r337333) @@ -23,7 +23,7 @@ .\" .\" $FreeBSD$ .\" -.Dd June 26, 2018 +.Dd July 11, 2018 .Dt CONFIG 5 .Os .Sh NAME @@ -114,13 +114,36 @@ are defined in the file .Pp .It Ic env Ar filename Specifies a filename containing a kernel environment definition. -The kernel normally uses an environment prepared for it at boot time -by +.Pp +The kernel will augment this compiled-in environment with the environment +prepared for it at boot time by .Xr loader 8 . -This directive makes the kernel ignore the boot environment and use -the compiled-in environment instead, unless the boot environment contains -.Va static_env.disabled=1 . +Environment variables specified in the +.Xr loader 8 +environment will take precedence over environment variables specified in +.Ar filename , +and environment variables specified in the dynamic environment take precedence +over both of these. .Pp +.Va loader_env.disabled=1 +may be specified in the static environment to disable the +.Xr loader 8 +environment. +Disabling the +.Xr loader 8 +should be done with caution and due consideration for whether or not it supplies +environment variables needed for properly booting the system. +.Pp +.Va static_env.disabled=1 +may be specified in the +.Xr loader 8 +environment to disable use of the static environment. +This option has no effect if specified in any environment after the +.Xr loader 8 +environment is processed. +This option is not usable in conjunction with +.Va loader_env.disabled . +.Pp This directive is useful for setting kernel tunables in embedded environments that do not start from .Xr loader 8 . @@ -172,9 +195,28 @@ time (see .Xr device.hints 5 ) . This directive configures the kernel to use the static device configuration listed in -.Ar filename , -unless the boot environment contains -.Va static_hints.disabled=1 . +.Ar filename . +.Pp +Hints provided in this static device configuration will be overwritten in the +order in which they're encountered. +Hints in the compiled-in environment takes precedence over compiled-in hints, +and hints in the environment prepared for the kernel by +.Xr loader 8 +takes precedence over hints in the compiled-in environment. +.Pp +Once the dynamic environment becomes available, all compiled-in hints will be +added to the dynamic environment if they do not already have an override in +the dynamic environment. +The dynamic environment will then be used for all searches of hints. +.Pp +.Va static_hints.disabled=1 +may be specified in either a compiled-in environment or the +.Xr loader 8 +environment to disable use of these hints files. +This option has no effect if specified in any environment after the +.Xr loader 8 +environment is processed. +.Pp The file .Ar filename must conform to the syntax specified by Modified: stable/11/usr.sbin/config/config.h ============================================================================== --- stable/11/usr.sbin/config/config.h Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/usr.sbin/config/config.h Sat Aug 4 21:57:17 2018 (r337333) @@ -177,8 +177,6 @@ SLIST_HEAD(, includepath) includepath; extern char *ident; extern char kernconfstr[]; extern int do_trace; -extern int envmode; -extern int hintmode; extern int incignore; char *get_word(FILE *); @@ -212,6 +210,7 @@ extern int debugging; extern int found_defaults; extern int maxusers; +extern int versreq; extern char *PREFIX; /* Config file name - for error messages */ extern char srcdir[]; /* root of the kernel source tree */ Modified: stable/11/usr.sbin/config/config.y ============================================================================== --- stable/11/usr.sbin/config/config.y Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/usr.sbin/config/config.y Sat Aug 4 21:57:17 2018 (r337333) @@ -80,8 +80,7 @@ struct device_head dtab; char *ident; -int envmode; -int hintmode; +char *env; int yyline; const char *yyfile; struct file_list_head ftab; @@ -199,7 +198,6 @@ Config_spec: err(EXIT_FAILURE, "calloc"); hint->hint_name = $2; STAILQ_INSERT_HEAD(&hints, hint, hint_next); - hintmode = 1; } System_spec: @@ -359,7 +357,6 @@ newenvvar(char *name, bool is_file) envvar->env_str = name; envvar->env_is_file = is_file; STAILQ_INSERT_HEAD(&envvars, envvar, envvar_next); - envmode = 1; } /* Modified: stable/11/usr.sbin/config/configvers.h ============================================================================== --- stable/11/usr.sbin/config/configvers.h Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/usr.sbin/config/configvers.h Sat Aug 4 21:57:17 2018 (r337333) @@ -49,5 +49,8 @@ * * $FreeBSD$ */ -#define CONFIGVERS 600015 +#define CONFIGVERS 600016 #define MAJOR_VERS(x) ((x) / 100000) + +/* Last config(8) version to require envmode/hintmode */ +#define CONFIGVERS_ENVMODE_REQ 600015 Modified: stable/11/usr.sbin/config/main.c ============================================================================== --- stable/11/usr.sbin/config/main.c Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/usr.sbin/config/main.c Sat Aug 4 21:57:17 2018 (r337333) @@ -84,12 +84,14 @@ int incignore; * literally). */ int filebased = 0; +int versreq; static void configfile(void); static void get_srcdir(void); static void usage(void); static void cleanheaders(char *); static void kernconfdump(const char *); +static void badversion(void); static void checkversion(void); extern int yyparse(void); @@ -740,8 +742,8 @@ kernconfdump(const char *file) fclose(fp); } -static void -badversion(int versreq) +static void +badversion(void) { fprintf(stderr, "ERROR: version of config(8) does not match kernel!\n"); fprintf(stderr, "config version = %d, ", CONFIGVERS); @@ -761,7 +763,6 @@ checkversion(void) { FILE *ifp; char line[BUFSIZ]; - int versreq; ifp = open_makefile_template(); while (fgets(line, BUFSIZ, ifp) != 0) { @@ -773,7 +774,7 @@ checkversion(void) if (MAJOR_VERS(versreq) == MAJOR_VERS(CONFIGVERS) && versreq <= CONFIGVERS) continue; - badversion(versreq); + badversion(); } fclose(ifp); } Modified: stable/11/usr.sbin/config/mkmakefile.c ============================================================================== --- stable/11/usr.sbin/config/mkmakefile.c Sat Aug 4 21:54:30 2018 (r337332) +++ stable/11/usr.sbin/config/mkmakefile.c Sat Aug 4 21:57:17 2018 (r337333) @@ -47,6 +47,8 @@ static const char rcsid[] = #include #include #include +#include +#include #include #include "y.tab.h" #include "config.h" @@ -60,6 +62,10 @@ static void do_objs(FILE *); static void do_before_depend(FILE *); static int opteq(const char *, const char *); static void read_files(void); +static void sanitize_envline(char *result, const char *src); +static void process_into_file(char *line, FILE *ofp); +static void process_into_nvlist(char *line, nvlist_t *nvl); +static void dump_nvlist(nvlist_t *nvl, FILE *ofp); static void errout(const char *fmt, ...) { @@ -178,65 +184,6 @@ makefile(void) moveifchanged(path("Makefile.new"), path("Makefile")); } -/* - * Build hints.c from the skeleton - */ -void -makehints(void) -{ - FILE *ifp, *ofp; - char line[BUFSIZ]; - char *s; - struct hint *hint; - - ofp = fopen(path("hints.c.new"), "w"); - if (ofp == NULL) - err(1, "%s", path("hints.c.new")); - fprintf(ofp, "#include \n"); - fprintf(ofp, "#include \n"); - fprintf(ofp, "\n"); - fprintf(ofp, "int hintmode = %d;\n", hintmode); - fprintf(ofp, "char static_hints[] = {\n"); - STAILQ_FOREACH(hint, &hints, hint_next) { - ifp = fopen(hint->hint_name, "r"); - if (ifp == NULL) - err(1, "%s", hint->hint_name); - while (fgets(line, BUFSIZ, ifp) != NULL) { - /* zap trailing CR and/or LF */ - while ((s = strrchr(line, '\n')) != NULL) - *s = '\0'; - while ((s = strrchr(line, '\r')) != NULL) - *s = '\0'; - /* remove # comments */ - s = strchr(line, '#'); - if (s) - *s = '\0'; - /* remove any whitespace and " characters */ - s = line; - while (*s) { - if (*s == ' ' || *s == '\t' || *s == '"') { - while (*s) { - s[0] = s[1]; - s++; - } - /* start over */ - s = line; - continue; - } - s++; - } - /* anything left? */ - if (*line == '\0') - continue; *** DIFF OUTPUT TRUNCATED AT 1000 LINES *** From owner-svn-src-all@freebsd.org Sat Aug 4 22:08:25 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 775E7105485A; Sat, 4 Aug 2018 22:08:25 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C8998A78F; Sat, 4 Aug 2018 22:08:25 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0D9C3155B2; Sat, 4 Aug 2018 22:08:25 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74M8Ofi057604; Sat, 4 Aug 2018 22:08:24 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74M8OmD057603; Sat, 4 Aug 2018 22:08:24 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201808042208.w74M8OmD057603@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: Conrad Meyer Date: Sat, 4 Aug 2018 22:08:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337334 - head/lib/libc/sys X-SVN-Group: head X-SVN-Commit-Author: cem X-SVN-Commit-Paths: head/lib/libc/sys X-SVN-Commit-Revision: 337334 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:08:25 -0000 Author: cem Date: Sat Aug 4 22:08:24 2018 New Revision: 337334 URL: https://svnweb.freebsd.org/changeset/base/337334 Log: settimeofday(2): Remove stale note about timezone Contrary to the removed comment, the kernel does appear to use the timezone argument of settimeofday. The comment dates to the BSD4.4 import; I assume it is just stale. Modified: head/lib/libc/sys/gettimeofday.2 Modified: head/lib/libc/sys/gettimeofday.2 ============================================================================== --- head/lib/libc/sys/gettimeofday.2 Sat Aug 4 21:57:17 2018 (r337333) +++ head/lib/libc/sys/gettimeofday.2 Sat Aug 4 22:08:24 2018 (r337334) @@ -28,7 +28,7 @@ .\" @(#)gettimeofday.2 8.2 (Berkeley) 5/26/95 .\" $FreeBSD$ .\" -.Dd December 27, 2015 +.Dd August 4, 2018 .Dt GETTIMEOFDAY 2 .Os .Sh NAME @@ -44,11 +44,6 @@ .Ft int .Fn settimeofday "const struct timeval *tp" "const struct timezone *tzp" .Sh DESCRIPTION -.Bf -symbolic -Note: timezone is no longer used; this information is kept outside -the kernel. -.Ef -.Pp The system's notion of the current Greenwich time and the current time zone is obtained with the .Fn gettimeofday From owner-svn-src-all@freebsd.org Sat Aug 4 22:12:13 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C1581054EC6; Sat, 4 Aug 2018 22:12:13 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C6A4A8AEC9; Sat, 4 Aug 2018 22:12:12 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A7D4A1572E; Sat, 4 Aug 2018 22:12:12 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74MCCe7061697; Sat, 4 Aug 2018 22:12:12 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74MCC5W061696; Sat, 4 Aug 2018 22:12:12 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201808042212.w74MCC5W061696@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Sat, 4 Aug 2018 22:12:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337335 - stable/11/usr.sbin/config X-SVN-Group: stable-11 X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: stable/11/usr.sbin/config X-SVN-Commit-Revision: 337335 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:12:13 -0000 Author: kevans Date: Sat Aug 4 22:12:12 2018 New Revision: 337335 URL: https://svnweb.freebsd.org/changeset/base/337335 Log: MFC r336973-r336975 r336973: config(8): Strip comments from env lines Consolidates the small bits of logic required for preprocessing a line before inclusion into a file or nvlist. r336974: Re-insert variable disappeared during mis-application of r336973 r336975: Remove variable re-inserted during mis-application of r336973 Modified: stable/11/usr.sbin/config/mkmakefile.c Directory Properties: stable/11/ (props changed) Modified: stable/11/usr.sbin/config/mkmakefile.c ============================================================================== --- stable/11/usr.sbin/config/mkmakefile.c Sat Aug 4 22:08:24 2018 (r337334) +++ stable/11/usr.sbin/config/mkmakefile.c Sat Aug 4 22:12:12 2018 (r337335) @@ -63,6 +63,7 @@ static void do_before_depend(FILE *); static int opteq(const char *, const char *); static void read_files(void); static void sanitize_envline(char *result, const char *src); +static bool preprocess(char *line, char *result); static void process_into_file(char *line, FILE *ofp); static void process_into_nvlist(char *line, nvlist_t *nvl); static void dump_nvlist(nvlist_t *nvl, FILE *ofp); @@ -241,16 +242,29 @@ sanitize_envline(char *result, const char *src) *dst = 0; } +/* + * Returns true if the caller may use the string. + */ +static bool +preprocess(char *line, char *result) +{ + char *s; + + /* Strip any comments */ + if ((s = strchr(line, '#')) != NULL) + *s = '\0'; + sanitize_envline(result, line); + /* Return true if it's non-empty */ + return (*result != '\0'); +} + static void process_into_file(char *line, FILE *ofp) { char result[BUFSIZ]; - sanitize_envline(result, line); - /* anything left? */ - if (*result == '\0') - return; - fprintf(ofp, "\"%s\\0\"\n", result); + if (preprocess(line, result)) + fprintf(ofp, "\"%s\\0\"\n", result); } static void @@ -258,15 +272,13 @@ process_into_nvlist(char *line, nvlist_t *nvl) { char result[BUFSIZ], *s; - sanitize_envline(result, line); - /* anything left? */ - if (*result == '\0') - return; - s = strchr(result, '='); - *s = 0; - if (nvlist_exists(nvl, result)) - nvlist_free(nvl, result); - nvlist_add_string(nvl, result, s + 1); + if (preprocess(line, result)) { + s = strchr(result, '='); + *s = '\0'; + if (nvlist_exists(nvl, result)) + nvlist_free(nvl, result); + nvlist_add_string(nvl, result, s + 1); + } } static void From owner-svn-src-all@freebsd.org Sat Aug 4 22:15:06 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 790F7105523D; Sat, 4 Aug 2018 22:15:06 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DFC78B2AA; Sat, 4 Aug 2018 22:15:06 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0F1AC1574C; Sat, 4 Aug 2018 22:15:06 +0000 (UTC) (envelope-from kevans@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74MF5F5062766; Sat, 4 Aug 2018 22:15:05 GMT (envelope-from kevans@FreeBSD.org) Received: (from kevans@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74MF5uj062762; Sat, 4 Aug 2018 22:15:05 GMT (envelope-from kevans@FreeBSD.org) Message-Id: <201808042215.w74MF5uj062762@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kevans set sender to kevans@FreeBSD.org using -f From: Kyle Evans Date: Sat, 4 Aug 2018 22:15:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r337336 - in stable/11/sys: kern sys X-SVN-Group: stable-11 X-SVN-Commit-Author: kevans X-SVN-Commit-Paths: in stable/11/sys: kern sys X-SVN-Commit-Revision: 337336 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:15:06 -0000 Author: kevans Date: Sat Aug 4 22:15:05 2018 New Revision: 337336 URL: https://svnweb.freebsd.org/changeset/base/337336 Log: MFC r336152-r336154, r336157 r336152: subr_hints: Use goto/label instead of series of conditionals r336153: subr_hints: Convert some bool-like ints to bools r336154: subr_hints: Skip static_env and static_hints if they don't contain hints This is possible because, well, they're static. Both the dynamic environment and the MD-environment (generally loader(8) environment) can potentially have room for new variables to be set, and thus do not receive this treatment. r336157: kern_environment: bool'itize dynamic_kenv; fix small style(9) nit Modified: stable/11/sys/kern/kern_environment.c stable/11/sys/kern/subr_hints.c stable/11/sys/sys/systm.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/kern/kern_environment.c ============================================================================== --- stable/11/sys/kern/kern_environment.c Sat Aug 4 22:12:12 2018 (r337335) +++ stable/11/sys/kern/kern_environment.c Sat Aug 4 22:15:05 2018 (r337336) @@ -37,7 +37,6 @@ #include __FBSDID("$FreeBSD$"); -#include #include #include #include @@ -79,7 +78,7 @@ struct mtx kenv_lock; /* * No need to protect this with a mutex since SYSINITS are single threaded. */ -int dynamic_kenv = 0; +bool dynamic_kenv; #define KENV_CHECK if (!dynamic_kenv) \ panic("%s: called before SI_SUB_KMEM", __func__) @@ -98,7 +97,7 @@ sys_kenv(td, uap) size_t len, done, needed, buflen; int error, i; - KASSERT(dynamic_kenv, ("kenv: dynamic_kenv = 0")); + KASSERT(dynamic_kenv, ("kenv: dynamic_kenv = false")); error = 0; if (uap->what == KENV_DUMP) { @@ -352,7 +351,7 @@ init_dynamic_kenv(void *data __unused) kenvp[dynamic_envpos] = NULL; mtx_init(&kenv_lock, "kernel environment", NULL, MTX_DEF); - dynamic_kenv = 1; + dynamic_kenv = true; } SYSINIT(kenv, SI_SUB_KMEM + 1, SI_ORDER_FIRST, init_dynamic_kenv, NULL); @@ -506,7 +505,7 @@ kern_setenv(const char *name, const char *value) char *buf, *cp, *oldenv; int namelen, vallen, i; - if (dynamic_kenv == 0 && md_env_len > 0) + if (!dynamic_kenv && md_env_len > 0) return (setenv_static(name, value)); KENV_CHECK; Modified: stable/11/sys/kern/subr_hints.c ============================================================================== --- stable/11/sys/kern/subr_hints.c Sat Aug 4 22:12:12 2018 (r337335) +++ stable/11/sys/kern/subr_hints.c Sat Aug 4 22:15:05 2018 (r337336) @@ -46,8 +46,10 @@ __FBSDID("$FreeBSD$"); * has already been setup (dynamic_kenv) and that we have added any supplied * static_hints to the dynamic environment. */ -static int hintenv_merged; - +static bool hintenv_merged; +/* Static environment and static hints cannot change, so we'll skip known bad */ +static bool stenv_skip; +static bool sthints_skip; /* * Access functions for device resources. */ @@ -82,7 +84,7 @@ static_hints_to_env(void *data __unused) free(line, M_TEMP); cp += i + 1; } - hintenv_merged = 1; + hintenv_merged = true; } /* Any time after dynamic env is setup */ @@ -122,13 +124,14 @@ res_find(char **hintp_cookie, int *line, int *startln, const char **ret_name, int *ret_namelen, int *ret_unit, const char **ret_resname, int *ret_resnamelen, const char **ret_value) { - int dyn_used = 0, fbacklvl = FBACK_MDENV, hit, i = 0, n = 0; + int fbacklvl = FBACK_MDENV, i = 0, n = 0; char r_name[32]; int r_unit; char r_resname[32]; char r_value[128]; const char *s, *cp; char *hintp, *p; + bool dyn_used = false; /* @@ -155,7 +158,7 @@ res_find(char **hintp_cookie, int *line, int *startln, } } mtx_unlock(&kenv_lock); - dyn_used = 1; + dyn_used = true; } else { /* * We'll have a chance to keep coming back here until @@ -176,17 +179,21 @@ fallback: } fbacklvl++; - if (fbacklvl <= FBACK_STENV && + if (!stenv_skip && fbacklvl <= FBACK_STENV && _res_checkenv(kern_envp)) { hintp = kern_envp; goto found; - } + } else + stenv_skip = true; + fbacklvl++; /* We'll fallback to static_hints if needed/can */ - if (fbacklvl <= FBACK_STATIC && + if (!sthints_skip && fbacklvl <= FBACK_STATIC && _res_checkenv(static_hints)) hintp = static_hints; + else + sthints_skip = true; found: fbacklvl++; } @@ -197,7 +204,7 @@ found: } else { hintp = *hintp_cookie; if (hintenv_merged && hintp == kenvp[0]) - dyn_used = 1; + dyn_used = true; else /* * If we aren't using the dynamic environment, we need @@ -215,34 +222,30 @@ found: cp = hintp; while (cp) { - hit = 1; (*line)++; if (strncmp(cp, "hint.", 5) != 0) - hit = 0; - else - n = sscanf(cp, "hint.%32[^.].%d.%32[^=]=%127s", - r_name, &r_unit, r_resname, r_value); - /* We'll circumvent all of the checks if we already know */ - if (hit) { - if (n != 4) { - printf("CONFIG: invalid hint '%s'\n", cp); - p = strchr(cp, 'h'); - *p = 'H'; - hit = 0; - } - if (hit && startln && *startln >= 0 && *line < *startln) - hit = 0; - if (hit && name && strcmp(name, r_name) != 0) - hit = 0; - if (hit && unit && *unit != r_unit) - hit = 0; - if (hit && resname && strcmp(resname, r_resname) != 0) - hit = 0; - if (hit && value && strcmp(value, r_value) != 0) - hit = 0; - if (hit) - break; + goto nexthint; + n = sscanf(cp, "hint.%32[^.].%d.%32[^=]=%127s", r_name, &r_unit, + r_resname, r_value); + if (n != 4) { + printf("CONFIG: invalid hint '%s'\n", cp); + p = strchr(cp, 'h'); + *p = 'H'; + goto nexthint; } + if (startln && *startln >= 0 && *line < *startln) + goto nexthint; + if (name && strcmp(name, r_name) != 0) + goto nexthint; + if (unit && *unit != r_unit) + goto nexthint; + if (resname && strcmp(resname, r_resname) != 0) + goto nexthint; + if (value && strcmp(value, r_value) != 0) + goto nexthint; + /* Successfully found a hint matching all criteria */ + break; +nexthint: if (dyn_used) { cp = kenvp[++i]; if (cp == NULL) Modified: stable/11/sys/sys/systm.h ============================================================================== --- stable/11/sys/sys/systm.h Sat Aug 4 22:12:12 2018 (r337335) +++ stable/11/sys/sys/systm.h Sat Aug 4 22:15:05 2018 (r337336) @@ -148,7 +148,7 @@ void kassert_panic(const char *fmt, ...) __printflike * XXX most of these variables should be const. */ extern int osreldate; -extern int dynamic_kenv; +extern bool dynamic_kenv; extern struct mtx kenv_lock; extern char *kern_envp; extern char *md_envp; From owner-svn-src-all@freebsd.org Sat Aug 4 22:15:27 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0553110552CB for ; Sat, 4 Aug 2018 22:15:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FA7D8B3D7 for ; Sat, 4 Aug 2018 22:15:26 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22f.google.com with SMTP id d9-v6so12890269itf.2 for ; Sat, 04 Aug 2018 15:15:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Gxn9DkRUSUxjHGee8Lmy02DVUthWt7clCtAJ1NjxI2c=; b=MO85EY4owT6gcoW81tZqzh7rl5Jr5DrklFSE52c9sNq1tj3WwWRZoXUj1ATpG2fnv9 P8rGtn9Ovec2123Zr2z8I/iHv4el0JoG4DZqVZ+qxPLcYYhqNyP4Nma5FHw36mEVZgYE wMlkMW3CcbUxsqSOh2d/PIl8Z/4pJSeZ2oqzAMTWrBf/RuLgJHVWEKpDHea92S/bY/Ei Bn1Pbb+bDDUxF8Wmi97LKZZbQcsBKv2sTg84Zrexn3EG7K6pIJIbcrgJjUBvNn8BAF5S 4I9arVBD3g8YxaQ+nywV/jQYALH3f0dK0L4jPX+VAZBPhGTmRbq7ZMqbSEK4qF1LsCNM MlWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Gxn9DkRUSUxjHGee8Lmy02DVUthWt7clCtAJ1NjxI2c=; b=uf5asmI2YjppJXFJNLq+Q0PfNgXs7keJseYLlTYveHm7Zzp8ahmi4noK5Cgs8L/+bv 8FYhbw/4AszfJz4hNWSwkUJdNFjKsxrAbwmC5upngf+VCzFP28b70ROSUbgxS6dxc0wA YTKI1KlyoRm5xBHeNXXtB4K8OmmWHXVyUcGOr6trCQ4Ssf6TW4pJYL1p997BOz/klW/l a8JtRJQJ9XnopLb/PCijB4d6YHBMd3UsYMM6dhDPa53aqGLnRAKvhyPI0lOXGz3ZuObS KW8taeD1/FtdBXzsAxaWIq7oTpCKPQvtA41zhm26s/MiE7xXZyzNoQLQpqNrtkeQwwrN wd5Q== X-Gm-Message-State: AOUpUlF68P0XpV67uWR+zzCEhp/3o2GilrtN14gKcskodEn8A7pnP1JU eJ/QUt43ifVbazmkOYEltVeqcmcsuYln6dnl43720A== X-Google-Smtp-Source: AAOMgpesmK282f8CFOjnF223hZpJRz0xj5ha4LSADX4/YB6lSVY8LjWDz5c6RYndWNYTyJh3YRraYBnmv7pn26gry+s= X-Received: by 2002:a24:b211:: with SMTP id u17-v6mr10519685ite.1.1533420925941; Sat, 04 Aug 2018 15:15:25 -0700 (PDT) MIME-Version: 1.0 References: <201808042208.w74M8OmD057603@repo.freebsd.org> In-Reply-To: <201808042208.w74M8OmD057603@repo.freebsd.org> From: Warner Losh Date: Sat, 4 Aug 2018 23:15:13 +0100 Message-ID: Subject: Re: svn commit: r337334 - head/lib/libc/sys To: "Conrad E. Meyer" Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:15:27 -0000 How is this stale? It said it unused, and now it doesn't say this are is completely ignored. Warner On Sat, Aug 4, 2018, 11:08 PM Conrad Meyer wrote: > Author: cem > Date: Sat Aug 4 22:08:24 2018 > New Revision: 337334 > URL: https://svnweb.freebsd.org/changeset/base/337334 > > Log: > settimeofday(2): Remove stale note about timezone > > Contrary to the removed comment, the kernel does appear to use the > timezone > argument of settimeofday. The comment dates to the BSD4.4 import; I > assume it > is just stale. > > Modified: > head/lib/libc/sys/gettimeofday.2 > > Modified: head/lib/libc/sys/gettimeofday.2 > > ============================================================================== > --- head/lib/libc/sys/gettimeofday.2 Sat Aug 4 21:57:17 2018 > (r337333) > +++ head/lib/libc/sys/gettimeofday.2 Sat Aug 4 22:08:24 2018 > (r337334) > @@ -28,7 +28,7 @@ > .\" @(#)gettimeofday.2 8.2 (Berkeley) 5/26/95 > .\" $FreeBSD$ > .\" > -.Dd December 27, 2015 > +.Dd August 4, 2018 > .Dt GETTIMEOFDAY 2 > .Os > .Sh NAME > @@ -44,11 +44,6 @@ > .Ft int > .Fn settimeofday "const struct timeval *tp" "const struct timezone *tzp" > .Sh DESCRIPTION > -.Bf -symbolic > -Note: timezone is no longer used; this information is kept outside > -the kernel. > -.Ef > -.Pp > The system's notion of the current Greenwich time and the current time > zone is obtained with the > .Fn gettimeofday > > From owner-svn-src-all@freebsd.org Sat Aug 4 22:16:00 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9F76105539A; Sat, 4 Aug 2018 22:16:00 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 605DE8B53D; Sat, 4 Aug 2018 22:16:00 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 42C811574F; Sat, 4 Aug 2018 22:16:00 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74MG0Xb062848; Sat, 4 Aug 2018 22:16:00 GMT (envelope-from brd@FreeBSD.org) Received: (from brd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74MFxVx062844; Sat, 4 Aug 2018 22:15:59 GMT (envelope-from brd@FreeBSD.org) Message-Id: <201808042215.w74MFxVx062844@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: brd set sender to brd@FreeBSD.org using -f From: Brad Davis Date: Sat, 4 Aug 2018 22:15:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337337 - in head: etc sbin/savecore X-SVN-Group: head X-SVN-Commit-Author: brd X-SVN-Commit-Paths: in head: etc sbin/savecore X-SVN-Commit-Revision: 337337 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:16:00 -0000 Author: brd Date: Sat Aug 4 22:15:59 2018 New Revision: 337337 URL: https://svnweb.freebsd.org/changeset/base/337337 Log: Move etc/minfree to sbin/savecore/. This is prep for pkgbase to have config files tagged as such. Approved by: will (mentor) Differential Revision: https://reviews.freebsd.org/D16558 Added: head/sbin/savecore/minfree - copied unchanged from r337336, head/etc/minfree Deleted: head/etc/minfree Modified: head/etc/Makefile head/sbin/savecore/Makefile Modified: head/etc/Makefile ============================================================================== --- head/etc/Makefile Sat Aug 4 22:15:05 2018 (r337336) +++ head/etc/Makefile Sat Aug 4 22:15:59 2018 (r337337) @@ -271,8 +271,6 @@ distribution: ${INSTALL} -o nobody -g ${BINGRP} -m 644 /dev/null \ ${DESTDIR}/var/db/locate.database .endif - ${INSTALL} -o ${BINOWN} -g ${BINGRP} -m 644 ${.CURDIR}/minfree \ - ${DESTDIR}/var/crash cd ${.CURDIR}/..; ${INSTALL} -o ${BINOWN} -g ${BINGRP} -m 444 \ ${FREEBSD} ${DESTDIR}/ .if ${MK_BOOT} != "no" Modified: head/sbin/savecore/Makefile ============================================================================== --- head/sbin/savecore/Makefile Sat Aug 4 22:15:05 2018 (r337336) +++ head/sbin/savecore/Makefile Sat Aug 4 22:15:59 2018 (r337337) @@ -1,6 +1,8 @@ # $FreeBSD$ PACKAGE=runtime +CONFS= minfree +CONFSDIR= /var/crash PROG= savecore LIBADD= z xo MAN= savecore.8 Copied: head/sbin/savecore/minfree (from r337336, head/etc/minfree) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/sbin/savecore/minfree Sat Aug 4 22:15:59 2018 (r337337, copy of r337336, head/etc/minfree) @@ -0,0 +1 @@ +2048 From owner-svn-src-all@freebsd.org Sat Aug 4 22:24:51 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 829E510558E2; Sat, 4 Aug 2018 22:24:51 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0953C8BB40; Sat, 4 Aug 2018 22:24:50 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w74MOgPx095275; Sat, 4 Aug 2018 15:24:42 -0700 (PDT) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w74MOgLi095274; Sat, 4 Aug 2018 15:24:42 -0700 (PDT) (envelope-from freebsd) From: "Rodney W. Grimes" Message-Id: <201808042224.w74MOgLi095274@pdx.rh.CN85.dnsmgr.net> Subject: Re: svn commit: r337334 - head/lib/libc/sys In-Reply-To: <201808042208.w74M8OmD057603@repo.freebsd.org> To: Conrad Meyer Date: Sat, 4 Aug 2018 15:24:42 -0700 (PDT) CC: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Reply-To: rgrimes@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:24:51 -0000 > Author: cem > Date: Sat Aug 4 22:08:24 2018 > New Revision: 337334 > URL: https://svnweb.freebsd.org/changeset/base/337334 > > Log: > settimeofday(2): Remove stale note about timezone > > Contrary to the removed comment, the kernel does appear to use the timezone > argument of settimeofday. The comment dates to the BSD4.4 import; I assume it > is just stale. In what way is the kernel using TZ? It shouldn't be. Or is it using the TZ to adjust the passed in time to UTC/GMT, which I would of thought is done before calling settimeofday. > Modified: > head/lib/libc/sys/gettimeofday.2 > > Modified: head/lib/libc/sys/gettimeofday.2 > ============================================================================== > --- head/lib/libc/sys/gettimeofday.2 Sat Aug 4 21:57:17 2018 (r337333) > +++ head/lib/libc/sys/gettimeofday.2 Sat Aug 4 22:08:24 2018 (r337334) > @@ -28,7 +28,7 @@ > .\" @(#)gettimeofday.2 8.2 (Berkeley) 5/26/95 > .\" $FreeBSD$ > .\" > -.Dd December 27, 2015 > +.Dd August 4, 2018 > .Dt GETTIMEOFDAY 2 > .Os > .Sh NAME > @@ -44,11 +44,6 @@ > .Ft int > .Fn settimeofday "const struct timeval *tp" "const struct timezone *tzp" > .Sh DESCRIPTION > -.Bf -symbolic > -Note: timezone is no longer used; this information is kept outside > -the kernel. > -.Ef > -.Pp > The system's notion of the current Greenwich time and the current time > zone is obtained with the > .Fn gettimeofday > > -- Rod Grimes rgrimes@freebsd.org From owner-svn-src-all@freebsd.org Sat Aug 4 22:25:43 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11A921055977; Sat, 4 Aug 2018 22:25:43 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BBE878BCB8; Sat, 4 Aug 2018 22:25:42 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9D2AC158ED; Sat, 4 Aug 2018 22:25:42 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74MPgBu067941; Sat, 4 Aug 2018 22:25:42 GMT (envelope-from brd@FreeBSD.org) Received: (from brd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74MPguF067938; Sat, 4 Aug 2018 22:25:42 GMT (envelope-from brd@FreeBSD.org) Message-Id: <201808042225.w74MPguF067938@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: brd set sender to brd@FreeBSD.org using -f From: Brad Davis Date: Sat, 4 Aug 2018 22:25:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337338 - in head: etc usr.sbin/freebsd-update X-SVN-Group: head X-SVN-Commit-Author: brd X-SVN-Commit-Paths: in head: etc usr.sbin/freebsd-update X-SVN-Commit-Revision: 337338 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:25:43 -0000 Author: brd Date: Sat Aug 4 22:25:41 2018 New Revision: 337338 URL: https://svnweb.freebsd.org/changeset/base/337338 Log: Move freebsd-update.conf to usr.sbin/freebsd-update. This is prep for pkgbase so that config files are tagged as such. Approved by: will (mentor) Differential Revision: https://reviews.freebsd.org/D16556 Added: head/usr.sbin/freebsd-update/freebsd-update.conf - copied unchanged from r337337, head/etc/freebsd-update.conf Deleted: head/etc/freebsd-update.conf Modified: head/etc/Makefile head/usr.sbin/freebsd-update/Makefile Modified: head/etc/Makefile ============================================================================== --- head/etc/Makefile Sat Aug 4 22:15:59 2018 (r337337) +++ head/etc/Makefile Sat Aug 4 22:25:41 2018 (r337338) @@ -74,10 +74,6 @@ BIN1+= amd.map BIN1+= auto_master .endif -.if ${MK_FREEBSD_UPDATE} != "no" -BIN1+= freebsd-update.conf -.endif - .if ${MK_FTP} != "no" BIN1+= ftpusers .endif Modified: head/usr.sbin/freebsd-update/Makefile ============================================================================== --- head/usr.sbin/freebsd-update/Makefile Sat Aug 4 22:15:59 2018 (r337337) +++ head/usr.sbin/freebsd-update/Makefile Sat Aug 4 22:25:41 2018 (r337338) @@ -1,5 +1,6 @@ # $FreeBSD$ +CONFS= freebsd-update.conf SCRIPTS=freebsd-update.sh MAN= freebsd-update.8 Copied: head/usr.sbin/freebsd-update/freebsd-update.conf (from r337337, head/etc/freebsd-update.conf) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/freebsd-update/freebsd-update.conf Sat Aug 4 22:25:41 2018 (r337338, copy of r337337, head/etc/freebsd-update.conf) @@ -0,0 +1,76 @@ +# $FreeBSD$ + +# Trusted keyprint. Changing this is a Bad Idea unless you've received +# a PGP-signed email from telling you to +# change it and explaining why. +KeyPrint 800651ef4b4c71c27e60786d7b487188970f4b4169cc055784e21eb71d410cc5 + +# Server or server pool from which to fetch updates. You can change +# this to point at a specific server if you want, but in most cases +# using a "nearby" server won't provide a measurable improvement in +# performance. +ServerName update.FreeBSD.org + +# Components of the base system which should be kept updated. +Components src world kernel + +# Example for updating the userland and the kernel source code only: +# Components src/base src/sys world + +# Paths which start with anything matching an entry in an IgnorePaths +# statement will be ignored. +IgnorePaths + +# Paths which start with anything matching an entry in an IDSIgnorePaths +# statement will be ignored by "freebsd-update IDS". +IDSIgnorePaths /usr/share/man/cat +IDSIgnorePaths /usr/share/man/whatis +IDSIgnorePaths /var/db/locate.database +IDSIgnorePaths /var/log + +# Paths which start with anything matching an entry in an UpdateIfUnmodified +# statement will only be updated if the contents of the file have not been +# modified by the user (unless changes are merged; see below). +UpdateIfUnmodified /etc/ /var/ /root/ /.cshrc /.profile + +# When upgrading to a new FreeBSD release, files which match MergeChanges +# will have any local changes merged into the version from the new release. +MergeChanges /etc/ /boot/device.hints + +### Default configuration options: + +# Directory in which to store downloaded updates and temporary +# files used by FreeBSD Update. +# WorkDir /var/db/freebsd-update + +# Destination to send output of "freebsd-update cron" if an error +# occurs or updates have been downloaded. +# MailTo root + +# Is FreeBSD Update allowed to create new files? +# AllowAdd yes + +# Is FreeBSD Update allowed to delete files? +# AllowDelete yes + +# If the user has modified file ownership, permissions, or flags, should +# FreeBSD Update retain this modified metadata when installing a new version +# of that file? +# KeepModifiedMetadata yes + +# When upgrading between releases, should the list of Components be +# read strictly (StrictComponents yes) or merely as a list of components +# which *might* be installed of which FreeBSD Update should figure out +# which actually are installed and upgrade those (StrictComponents no)? +# StrictComponents no + +# When installing a new kernel perform a backup of the old one first +# so it is possible to boot the old kernel in case of problems. +# BackupKernel yes + +# If BackupKernel is enabled, the backup kernel is saved to this +# directory. +# BackupKernelDir /boot/kernel.old + +# When backing up a kernel also back up debug symbol files? +# BackupKernelSymbolFiles no From owner-svn-src-all@freebsd.org Sat Aug 4 22:31:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E0CF1055C2F; Sat, 4 Aug 2018 22:31:30 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 143CB8C110; Sat, 4 Aug 2018 22:31:30 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E995D15A39; Sat, 4 Aug 2018 22:31:29 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74MVTLF072134; Sat, 4 Aug 2018 22:31:29 GMT (envelope-from brd@FreeBSD.org) Received: (from brd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74MVTsi072132; Sat, 4 Aug 2018 22:31:29 GMT (envelope-from brd@FreeBSD.org) Message-Id: <201808042231.w74MVTsi072132@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: brd set sender to brd@FreeBSD.org using -f From: Brad Davis Date: Sat, 4 Aug 2018 22:31:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337339 - in head: etc usr.sbin/portsnap/portsnap X-SVN-Group: head X-SVN-Commit-Author: brd X-SVN-Commit-Paths: in head: etc usr.sbin/portsnap/portsnap X-SVN-Commit-Revision: 337339 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:31:30 -0000 Author: brd Date: Sat Aug 4 22:31:29 2018 New Revision: 337339 URL: https://svnweb.freebsd.org/changeset/base/337339 Log: Move portsnap.conf to head/usr.sbin/portsnap/portsnap/ This is prep for pkgbase to have config files tagged as such. Approved by: will (mentor) Differential Revision: https://reviews.freebsd.org/D16554 Added: head/usr.sbin/portsnap/portsnap/portsnap.conf - copied unchanged from r337338, head/etc/portsnap.conf Deleted: head/etc/portsnap.conf Modified: head/etc/Makefile head/usr.sbin/portsnap/portsnap/Makefile Modified: head/etc/Makefile ============================================================================== --- head/etc/Makefile Sat Aug 4 22:25:41 2018 (r337338) +++ head/etc/Makefile Sat Aug 4 22:31:29 2018 (r337339) @@ -107,10 +107,6 @@ SSH= ${SRCTOP}/crypto/openssh/ssh_config \ SSL= ${SRCTOP}/crypto/openssl/apps/openssl.cnf .endif -.if ${MK_PORTSNAP} != "no" -BIN1+= portsnap.conf -.endif - .if ${MK_PF} != "no" BIN1+= pf.os .endif Modified: head/usr.sbin/portsnap/portsnap/Makefile ============================================================================== --- head/usr.sbin/portsnap/portsnap/Makefile Sat Aug 4 22:25:41 2018 (r337338) +++ head/usr.sbin/portsnap/portsnap/Makefile Sat Aug 4 22:31:29 2018 (r337339) @@ -1,5 +1,6 @@ # $FreeBSD$ +CONFS= portsnap.conf SCRIPTS=portsnap.sh MAN= portsnap.8 Copied: head/usr.sbin/portsnap/portsnap/portsnap.conf (from r337338, head/etc/portsnap.conf) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/portsnap/portsnap/portsnap.conf Sat Aug 4 22:31:29 2018 (r337339, copy of r337338, head/etc/portsnap.conf) @@ -0,0 +1,35 @@ +# $FreeBSD$ + +# Default directory where compressed snapshots are stored. +# WORKDIR=/var/db/portsnap + +# Default location of the ports tree (target for "update" and "extract"). +# PORTSDIR=/usr/ports + +# Server or server pool from which to fetch updates. You can change +# this to point at a specific server if you want, but in most cases +# using a "nearby" server won't provide a measurable improvement in +# performance. +SERVERNAME=portsnap.FreeBSD.org + +# Trusted keyprint. Changing this is a Bad Idea unless you've received +# a PGP-signed email from telling you to +# change it and explaining why. +KEYPRINT=9b5feee6d69f170e3dd0a2c8e469ddbd64f13f978f2f3aede40c98633216c330 + +# Example of ignoring parts of the ports tree. If you know that you +# absolutely will not need certain parts of the tree, this will save +# some bandwidth and disk space. See the manual page for more details. +# +# WARNING: Working with an incomplete ports tree is not supported and +# can cause problems due to missing dependencies. If you have REFUSE +# directives and experience problems, remove them and update your tree +# before asking for help on the mailing lists. +# +# REFUSE arabic chinese french german hebrew hungarian japanese +# REFUSE korean polish portuguese russian ukrainian vietnamese + +# List of INDEX files to build and the DESCRIBE file to use for each +#INDEX INDEX-10 DESCRIBE.10 +#INDEX INDEX-11 DESCRIBE.11 +INDEX INDEX-12 DESCRIBE.12 From owner-svn-src-all@freebsd.org Sat Aug 4 22:39:53 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 007A51056058 for ; Sat, 4 Aug 2018 22:39:53 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x241.google.com (mail-it0-x241.google.com [IPv6:2607:f8b0:4001:c0b::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F5F28C456 for ; Sat, 4 Aug 2018 22:39:52 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x241.google.com with SMTP id h23-v6so13343049ita.5 for ; Sat, 04 Aug 2018 15:39:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=vyQTfHtxcgMFvYWyj0W4hkX/UHlcTMpRmYMXOSs9kTg=; b=0bKMq2qbcsII+2mkELgOkEGKRN/9k6HSm4RjyNIPjDdcmXnJGmiU58f/usG4iS3wqR vEsIpk4wezIks7F6NmFKMFpKWlHaoCv40M5PJVUdv9LchzF2jwxKMr0Jgji3p0OTLCIN lj9sO890I7oWhnBZdR3QMTcUNbe/EVzYGWZkUZhqTq7AzmgJPaRfs2MPWh4QdEeK/kZ4 EjPMS3royaRSFvorNowGHGgaIsVeteXQuuWtWFnCK0ApnIUxOe52aQ4xh4ajgvVwGthR 4BPczDfQ9CqyIGBtVM1TBBSnthNZZiWGllYoIzYCoOsEt8GdsIxvd99yDejPTiLEtSpd rgfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=vyQTfHtxcgMFvYWyj0W4hkX/UHlcTMpRmYMXOSs9kTg=; b=GPTtdatVlm/3yU3TBIcdlOKoGrB+ijNPbZwvTA9XjRi9oyVr4TdCrfo72MyZKMfYkT 6IXxcaVTp+O7Gs1EIQs8+iBGRcO54d3SivrQGAbSln2CIJhdcW49i5tLFHd0nrqM55tZ /L+Uw3oaTZm150h6E0rVv/83VlgLDxXqYl9knmYzBpBWC2JYabM6R01cmTxK5l8Dt+v+ 8RkkNfpTe27FkXaxd8YoerOKohqKDF9wOlH0fOCXBe0shyKQmS8Rp/iUY0l5msk9z+oc jRcaQqUrLAs2OQ7X0tUyp5cdOvaHIfUgePzDcmVj73dSv+fyy0L/J0DWshnxjGs46nje AbpQ== X-Gm-Message-State: AOUpUlFSZgKoqQnsBC0XWQYLrdu0CAG3+9MglxBh9JCIiCLJ0gDA2ZoY QXeOZDUcYXmh7xefZHyRfPcoWuhmW6mXKzCaAUjoWA== X-Google-Smtp-Source: AAOMgpdp/AnBO3cbQttrFE2LTYxPXrEryDYIGRy2ss/BjIPBkUqD5dbe4Hjohnfydwz8x1EGqBJnd4btjruLsBSdR0w= X-Received: by 2002:a02:3344:: with SMTP id k4-v6mr8361606jak.45.1533422391840; Sat, 04 Aug 2018 15:39:51 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:4485:0:0:0:0:0 with HTTP; Sat, 4 Aug 2018 15:39:51 -0700 (PDT) X-Originating-IP: [86.153.210.77] In-Reply-To: <201808042224.w74MOgLi095274@pdx.rh.CN85.dnsmgr.net> References: <201808042208.w74M8OmD057603@repo.freebsd.org> <201808042224.w74MOgLi095274@pdx.rh.CN85.dnsmgr.net> From: Warner Losh Date: Sat, 4 Aug 2018 16:39:51 -0600 X-Google-Sender-Auth: Xd57GX5iAbLVF8YL3xhDitRRW64 Message-ID: Subject: Re: svn commit: r337334 - head/lib/libc/sys To: "Rodney W. Grimes" Cc: Conrad Meyer , src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:39:53 -0000 On Sat, Aug 4, 2018 at 4:24 PM, Rodney W. Grimes < freebsd@pdx.rh.cn85.dnsmgr.net> wrote: > > Author: cem > > Date: Sat Aug 4 22:08:24 2018 > > New Revision: 337334 > > URL: https://svnweb.freebsd.org/changeset/base/337334 > > > > Log: > > settimeofday(2): Remove stale note about timezone > > > > Contrary to the removed comment, the kernel does appear to use the > timezone > > argument of settimeofday. The comment dates to the BSD4.4 import; I > assume it > > is just stale. > > In what way is the kernel using TZ? It shouldn't be. > Or is it using the TZ to adjust the passed in time to UTC/GMT, > which I would of thought is done before calling settimeofday. Specifically, for compatibility, we store minutes west of UTC on settimeofday and we retrieve it for settimeofday. Otherwise it's 100% unused by anything else at all in the system. Well, technically, we use it for utc_offset, we don't really use that elsewhere (one can find references in utc_offset, but usually we set this via adjkerntz, which is actively used in the system). tz_minuteswest likely should just be removed, and the argument to settimzeofday should just be completely ignored. One could test it with date -t, nothing does that today. That should be removed also. Likewise date -d. So while hyper technically, one could use this, nobody does, nor has since between 4.3 and 4.4 when it was realized that storing the timezone in the kernel was a really stupid idea. That's what the language used in the man page that you removed was trying to say. Warner > > Modified: > > head/lib/libc/sys/gettimeofday.2 > > > > Modified: head/lib/libc/sys/gettimeofday.2 > > ============================================================ > ================== > > --- head/lib/libc/sys/gettimeofday.2 Sat Aug 4 21:57:17 2018 > (r337333) > > +++ head/lib/libc/sys/gettimeofday.2 Sat Aug 4 22:08:24 2018 > (r337334) > > @@ -28,7 +28,7 @@ > > .\" @(#)gettimeofday.2 8.2 (Berkeley) 5/26/95 > > .\" $FreeBSD$ > > .\" > > -.Dd December 27, 2015 > > +.Dd August 4, 2018 > > .Dt GETTIMEOFDAY 2 > > .Os > > .Sh NAME > > @@ -44,11 +44,6 @@ > > .Ft int > > .Fn settimeofday "const struct timeval *tp" "const struct timezone *tzp" > > .Sh DESCRIPTION > > -.Bf -symbolic > > -Note: timezone is no longer used; this information is kept outside > > -the kernel. > > -.Ef > > -.Pp > > The system's notion of the current Greenwich time and the current time > > zone is obtained with the > > .Fn gettimeofday > > > > > > -- > Rod Grimes > rgrimes@freebsd.org > > From owner-svn-src-all@freebsd.org Sat Aug 4 22:41:19 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 373731056120; Sat, 4 Aug 2018 22:41:19 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E06658C653; Sat, 4 Aug 2018 22:41:18 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A779015BE6; Sat, 4 Aug 2018 22:41:18 +0000 (UTC) (envelope-from brd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w74MfI6h074792; Sat, 4 Aug 2018 22:41:18 GMT (envelope-from brd@FreeBSD.org) Received: (from brd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w74MfHpI074788; Sat, 4 Aug 2018 22:41:17 GMT (envelope-from brd@FreeBSD.org) Message-Id: <201808042241.w74MfHpI074788@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: brd set sender to brd@FreeBSD.org using -f From: Brad Davis Date: Sat, 4 Aug 2018 22:41:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r337340 - in head: etc etc/autofs usr.sbin/autofs usr.sbin/autofs/autofs X-SVN-Group: head X-SVN-Commit-Author: brd X-SVN-Commit-Paths: in head: etc etc/autofs usr.sbin/autofs usr.sbin/autofs/autofs X-SVN-Commit-Revision: 337340 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:41:19 -0000 Author: brd Date: Sat Aug 4 22:41:17 2018 New Revision: 337340 URL: https://svnweb.freebsd.org/changeset/base/337340 Log: Move autofs related configs to usr.sbin/autofs/ This is prep for pkgbase to have config files tagged as such. Approved by: will (mentor) Differential Revision: https://reviews.freebsd.org/D16492 Added: head/usr.sbin/autofs/auto_master - copied unchanged from r337339, head/etc/auto_master head/usr.sbin/autofs/autofs/ - copied from r337339, head/etc/autofs/ Deleted: head/etc/auto_master head/etc/autofs/ Modified: head/etc/Makefile head/usr.sbin/autofs/Makefile head/usr.sbin/autofs/autofs/Makefile Modified: head/etc/Makefile ============================================================================== --- head/etc/Makefile Sat Aug 4 22:31:29 2018 (r337339) +++ head/etc/Makefile Sat Aug 4 22:41:17 2018 (r337340) @@ -190,9 +190,6 @@ distribution: echo "./var/db/services.db type=file mode=0644 uname=root gname=wheel"; \ ) | ${METALOG.add} .endif -.if ${MK_AUTOFS} != "no" - ${_+_}cd ${.CURDIR}/autofs; ${MAKE} install -.endif .if ${MK_BLUETOOTH} != "no" ${_+_}cd ${.CURDIR}/bluetooth; ${MAKE} install .endif Modified: head/usr.sbin/autofs/Makefile ============================================================================== --- head/usr.sbin/autofs/Makefile Sat Aug 4 22:31:29 2018 (r337339) +++ head/usr.sbin/autofs/Makefile Sat Aug 4 22:41:17 2018 (r337340) @@ -1,5 +1,6 @@ # $FreeBSD$ +CONFS= auto_master PACKAGE=autofs PROG= automountd SRCS= automount.c @@ -29,5 +30,7 @@ LINKS= ${BINDIR}/automountd ${BINDIR}/automount LINKS+= ${BINDIR}/automountd ${BINDIR}/autounmountd .PATH: ${MOUNT} + +SUBDIR= autofs .include Copied: head/usr.sbin/autofs/auto_master (from r337339, head/etc/auto_master) ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ head/usr.sbin/autofs/auto_master Sat Aug 4 22:41:17 2018 (r337340, copy of r337339, head/etc/auto_master) @@ -0,0 +1,9 @@ +# $FreeBSD$ +# +# Automounter master map, see auto_master(5) for details. +# +/net -hosts -nobrowse,nosuid,intr +# When using the -media special map, make sure to edit devd.conf(5) +# to move the call to "automount -c" out of the comments section. +#/media -media -nosuid,noatime +#/- -noauto Modified: head/usr.sbin/autofs/autofs/Makefile ============================================================================== --- head/etc/autofs/Makefile Sat Aug 4 22:31:29 2018 (r337339) +++ head/usr.sbin/autofs/autofs/Makefile Sat Aug 4 22:41:17 2018 (r337340) @@ -1,6 +1,6 @@ # $FreeBSD$ -FILES= include_ldap \ +CONFS= include_ldap \ include_nis \ include_nis_nullfs \ special_hosts \ @@ -9,7 +9,7 @@ FILES= include_ldap \ special_null NO_OBJ= -FILESDIR= /etc/autofs -FILESMODE= 755 +CONFDIR= /etc/autofs +CONFMODE= 755 .include From owner-svn-src-all@freebsd.org Sat Aug 4 22:46:05 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D4021056375 for ; Sat, 4 Aug 2018 22:46:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 335B78C9E9 for ; Sat, 4 Aug 2018 22:46:05 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x236.google.com with SMTP id g141-v6so13360964ita.4 for ; Sat, 04 Aug 2018 15:46:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=TEBxRak4Dzg/R8aijnb8Fnsk1Mh7ERDSJ8uYtdiyI34=; b=NusJwOZCj9hqkPo/lfpilfPhV+sAOnRuWq9eMwfb2asxwiNki6bHQ2uHiAw0cb6s3F 8KVgTNmgWlYitvkNzPdmVfln2tUgdLb1zMP7yfBs47UNXmS2us6jMZdXhFu1wQeeGw8Y j+NZM6LPu30qHAdUuAvr+UXJU7BxALcE0+/mBFz8jDXP3zAbGI9S6SWkzGgi4aoEfHRV EtqqsecSXth4G/EH5l1qB25JeqoQGi1XCrwTJkM9OWkXT3LP1WEXg02A3BwMlc2fMcvL D0pChz+5jZn4PHYXQuOqmnNMK5aCsJ8YhW6IB2QH8HaURry+09gtQqoCnfw39UOumx2n hBIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=TEBxRak4Dzg/R8aijnb8Fnsk1Mh7ERDSJ8uYtdiyI34=; b=KdS1F4FRPSY4ksEhgzDtwXSPjJde3D3p51hvsYAwvFOIaR+U8ytXBhffzmRxpEcnGh VqXAbsTcT8cNu+joJR8f/X350i+K1Iq8RkhLPVZRfQpDbYG8TByE48GgJxn4rZt/N/Xt JiqelftU+SdwbCo+BVfYDceEzGDMCWpx3ItILtLdhwMhyIoZMxRJ9bilq4M+IHFpN2mC UeXCTpTCYLcTvSpGBnY7v/Gx93tk+ofJBWjWwaJpjvIxKMuj7SVwAwtVqjLb0V/ajVmV xAhPwH9kPzvH1zLDyLVRWoxOm5cw451o5Dq1G6Ni0WCpq/UlheHmd4yCDeByQJ6brnwH b2lg== X-Gm-Message-State: AOUpUlEJ3KWuKw963T2dEqcLYTv5PAl7uBoPFDd9eQmGWTx3iRTbFzPY j8VAxUG1OykHHHq1sZq3kklwFQ+XAKqtLRtAtfLR+g== X-Google-Smtp-Source: AAOMgpd7lbCYKHP66AoDsV9CmRUXlIoOPM1tRLatMRpll8M5ZgVxL893HiNVU8jqNpdTI3+p+ZaYHqZIGJDxAZpetqo= X-Received: by 2002:a24:4f52:: with SMTP id c79-v6mr10813378itb.36.1533422764529; Sat, 04 Aug 2018 15:46:04 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 2002:a4f:4485:0:0:0:0:0 with HTTP; Sat, 4 Aug 2018 15:46:04 -0700 (PDT) X-Originating-IP: [86.153.210.77] In-Reply-To: <201808021925.w72JPhIu086582@repo.freebsd.org> References: <201808021925.w72JPhIu086582@repo.freebsd.org> From: Warner Losh Date: Sat, 4 Aug 2018 16:46:04 -0600 X-Google-Sender-Auth: 1M4XzTqz1Mk1B7eB0jL6Fbc88Sg Message-ID: Subject: Re: svn commit: r337165 - head/sys/fs/fuse To: Conrad Meyer Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:46:05 -0000 On Thu, Aug 2, 2018 at 1:25 PM, Conrad Meyer wrote: > Author: cem > Date: Thu Aug 2 19:25:43 2018 > New Revision: 337165 > URL: https://svnweb.freebsd.org/changeset/base/337165 > > Log: > FUSE: Bump maximum IO size to enable more performant operation > > Various components restrict size of IO passed up to the userspace > filesystem > based on the mount's f_iosize value. The previous default of PAGE_SIZE > is anemic, even for normal filesystems, but especially considering every > FUSE operation involves a kernel <-> userspace IPC upcall. > > Bump to DFLTPHYS (currently 64kB) to match other FUSE implementations. > > Anecdotally, Jakub reports IO read performance increased from 600 MB/s -> > 2700 MB/s with a basic RAM-backed FUSE filesystem. > > PR: 230260 > Reported by: Peter (MooseFS) > Tested by: Jakub Kruszona-Zawadzki > MFC after: 3 days > > Modified: > head/sys/fs/fuse/fuse_vfsops.c > > Modified: head/sys/fs/fuse/fuse_vfsops.c > ============================================================ > ================== > --- head/sys/fs/fuse/fuse_vfsops.c Thu Aug 2 19:19:53 2018 > (r337164) > +++ head/sys/fs/fuse/fuse_vfsops.c Thu Aug 2 19:25:43 2018 > (r337165) > @@ -341,7 +341,7 @@ fuse_vfsop_mount(struct mount *mp) > mp->mnt_kern_flag |= MNTK_USES_BCACHE; > MNT_IUNLOCK(mp); > /* We need this here as this slot is used by getnewvnode() */ > - mp->mnt_stat.f_iosize = PAGE_SIZE; > + mp->mnt_stat.f_iosize = DFLTPHYS; > Why DFLTPHYS instead of MAXPHYS? DFLTPHYS defaults to 64k, while MAXPHYS is 128k. Or do you expect people that want different behavior to raise both? Warner From owner-svn-src-all@freebsd.org Sat Aug 4 22:49:02 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6A2010564AD; Sat, 4 Aug 2018 22:49:02 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it0-f43.google.com (mail-it0-f43.google.com [209.85.214.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 66AAA8CB72; Sat, 4 Aug 2018 22:49:02 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it0-f43.google.com with SMTP id h20-v6so13386707itf.2; Sat, 04 Aug 2018 15:49:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=9F2AzxyBj14X0ymAfTLXJoCeSDYQtZsV9nz48ogeS3M=; b=cXMaZlUjyAEt4uPni4rf24yR0iEnUyz8thoNJ5jzuc6xz3ubk+ns7zlfFttfY/QRhR NOkV1psI5F+MmnRY6YPykmb+NcFONsrLWrNNTUX1Wgut8eYiUS2qymkTgGZaIRNgldVm torYILfmbuoLqRnTp/uAoIjb4BGcgqJc21YCeD4ZMNFWG7rxyeIdvnY363KyKA8Wv5sh MoyBfVr4xCSHG/58gnwlPy2mc2NpeVOWD0t2N2E0VgaDPTNi8s0Y7+XucaIvaYPWkZCQ qDa6BVZWFCzt4Kl9q9tOQmFutJEC6HigJm32iIzjXNaDM8GN6RZfyBg/PxYDZCDbRkP3 Z3wQ== X-Gm-Message-State: AOUpUlHtznigwNlB1GUVwwODDOY81URPHX5xYu8cozBe+YLKv31XBmQZ n6UqDQuECoQCfWx6dTMFY5bILjiG X-Google-Smtp-Source: AAOMgpcxbGVrymV1RTrpGl7JSXGmXKKWiqyv/Wf4iVKZGhXtlsBlHfZpX/WdTFZztVMGVjrwXNJiBA== X-Received: by 2002:a02:45cd:: with SMTP id o74-v6mr7760552jad.43.1533422936187; Sat, 04 Aug 2018 15:48:56 -0700 (PDT) Received: from mail-io0-f175.google.com (mail-io0-f175.google.com. [209.85.223.175]) by smtp.gmail.com with ESMTPSA id x73-v6sm1678653ita.30.2018.08.04.15.48.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 04 Aug 2018 15:48:55 -0700 (PDT) Received: by mail-io0-f175.google.com with SMTP id z20-v6so8041427iol.0; Sat, 04 Aug 2018 15:48:55 -0700 (PDT) X-Received: by 2002:a6b:be46:: with SMTP id o67-v6mr10320298iof.143.1533422935800; Sat, 04 Aug 2018 15:48:55 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 2002:a02:7e0a:0:0:0:0:0 with HTTP; Sat, 4 Aug 2018 15:48:55 -0700 (PDT) In-Reply-To: References: <201808042208.w74M8OmD057603@repo.freebsd.org> From: Conrad Meyer Date: Sat, 4 Aug 2018 15:48:55 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r337334 - head/lib/libc/sys To: Warner Losh Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:49:03 -0000 If not stale, it was inaccurate as-is. It seems to be used by a bunch of different clock-related drivers and routines, e.g., $ rg '(tz_minuteswest|tz_dst|utc_offset)' | wc -l 36 To me there is a disconnect between that use and "timezone is no longer used; this information is kept outside the kernel." If you or someone else wants to reword the "unused" note in a more nuanced fashion, I'm definitely supportive of that. (Or remove the kernel uses aside from pass-thru for gettimeofday and restore the original note, of course.) I was actually going to deorbit the date(1) -t/-d options on the basis of the settimeofday manual page note until I did the due diligence to verify it was actually unused, and discovered that not to be true. All the best, Conrad On Sat, Aug 4, 2018 at 3:15 PM, Warner Losh wrote: > How is this stale? It said it unused, and now it doesn't say this are is > completely ignored. > > Warner > > On Sat, Aug 4, 2018, 11:08 PM Conrad Meyer wrote: >> >> Author: cem >> Date: Sat Aug 4 22:08:24 2018 >> New Revision: 337334 >> URL: https://svnweb.freebsd.org/changeset/base/337334 >> >> Log: >> settimeofday(2): Remove stale note about timezone >> >> Contrary to the removed comment, the kernel does appear to use the >> timezone >> argument of settimeofday. The comment dates to the BSD4.4 import; I >> assume it >> is just stale. >> >> Modified: >> head/lib/libc/sys/gettimeofday.2 >> >> Modified: head/lib/libc/sys/gettimeofday.2 >> >> ============================================================================== >> --- head/lib/libc/sys/gettimeofday.2 Sat Aug 4 21:57:17 2018 >> (r337333) >> +++ head/lib/libc/sys/gettimeofday.2 Sat Aug 4 22:08:24 2018 >> (r337334) >> @@ -28,7 +28,7 @@ >> .\" @(#)gettimeofday.2 8.2 (Berkeley) 5/26/95 >> .\" $FreeBSD$ >> .\" >> -.Dd December 27, 2015 >> +.Dd August 4, 2018 >> .Dt GETTIMEOFDAY 2 >> .Os >> .Sh NAME >> @@ -44,11 +44,6 @@ >> .Ft int >> .Fn settimeofday "const struct timeval *tp" "const struct timezone *tzp" >> .Sh DESCRIPTION >> -.Bf -symbolic >> -Note: timezone is no longer used; this information is kept outside >> -the kernel. >> -.Ef >> -.Pp >> The system's notion of the current Greenwich time and the current time >> zone is obtained with the >> .Fn gettimeofday >> > From owner-svn-src-all@freebsd.org Sat Aug 4 22:54:38 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85442105698F; Sat, 4 Aug 2018 22:54:38 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-it0-f42.google.com (mail-it0-f42.google.com [209.85.214.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23ABC8D0D9; Sat, 4 Aug 2018 22:54:38 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-it0-f42.google.com with SMTP id v71-v6so13395945itb.3; Sat, 04 Aug 2018 15:54:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=rEV+DNzPux6EREf66aQOluLH9UnAgmIeMRKAgM5JXOc=; b=ZfEYDnmUQRFO7VuB/5MrpSYnwUges0HgktL2FG8hMaQZ3FNdBbicyfb4AnZJOLx4P3 UijOuSof01JA0GmqeJpWPTiuh8Hz60CbW93vOLbEmLu8gxUU2Bq4tgxKH8JbsQsFF53r Zlz92du3jnBXqfQJmlRXKGZTOi+Td84etowEMTOEXyWhVMRrFyySsoHmMINBe8V4IE/W /XROg3Diezw5USBHqx9Ooh4LSJMloukDO/1ZSs1p3Lggh/hCtvmExBUn8P/M9I9nWDwR iul8kva2mlyHKqTBiPytE1d7iywG9PnK5+N8+4VZIB2qi4+PRoKhfPzSlWxucR1TCdEO m41g== X-Gm-Message-State: AOUpUlEmZXpEyfv5HxkcvNUIMsFgMdI5oYQ8RLSK3DpueQP+5OMiSoQT Kknd/NvJJOu5vChBM3yZZJ5w7Mm9 X-Google-Smtp-Source: AAOMgpcmantsrvoObeFU5b2cKxeoYIkKiUAC0gN0oLkaZSj9OPR5/YD8VXHUa8J5J2JTgWbNPjc2Ug== X-Received: by 2002:a24:dd88:: with SMTP id t130-v6mr4023490itf.129.1533423271056; Sat, 04 Aug 2018 15:54:31 -0700 (PDT) Received: from mail-it0-f54.google.com (mail-it0-f54.google.com. [209.85.214.54]) by smtp.gmail.com with ESMTPSA id j17-v6sm1553883ita.41.2018.08.04.15.54.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 04 Aug 2018 15:54:30 -0700 (PDT) Received: by mail-it0-f54.google.com with SMTP id e14-v6so12959421itf.1; Sat, 04 Aug 2018 15:54:30 -0700 (PDT) X-Received: by 2002:a24:a308:: with SMTP id p8-v6mr10893249ite.53.1533423270630; Sat, 04 Aug 2018 15:54:30 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 2002:a02:7e0a:0:0:0:0:0 with HTTP; Sat, 4 Aug 2018 15:54:30 -0700 (PDT) In-Reply-To: References: <201808021925.w72JPhIu086582@repo.freebsd.org> From: Conrad Meyer Date: Sat, 4 Aug 2018 15:54:30 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r337165 - head/sys/fs/fuse To: Warner Losh Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 22:54:38 -0000 On Sat, Aug 4, 2018 at 3:46 PM, Warner Losh wrote: > On Thu, Aug 2, 2018 at 1:25 PM, Conrad Meyer wrote: >> >> Author: cem >> Date: Thu Aug 2 19:25:43 2018 >> New Revision: 337165 >> URL: https://svnweb.freebsd.org/changeset/base/337165 >> >> Log: >> FUSE: Bump maximum IO size to enable more performant operation >>... >> - mp->mnt_stat.f_iosize = PAGE_SIZE; >> + mp->mnt_stat.f_iosize = DFLTPHYS; > > Why DFLTPHYS instead of MAXPHYS? > > DFLTPHYS defaults to 64k, while MAXPHYS is 128k. It was chosen arbitrarily. Either is a big improvement over 4k. According to the bug reporter, 64k is a common choice in other implementations. But I have no objection to MAXPHYS. Best, Conrad From owner-svn-src-all@freebsd.org Sat Aug 4 23:01:29 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5307F1056C09 for ; Sat, 4 Aug 2018 23:01:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA62E8D348 for ; Sat, 4 Aug 2018 23:01:28 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22f.google.com with SMTP id q9-v6so8016129ioj.8 for ; Sat, 04 Aug 2018 16:01:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sIRVDCI304lqLYuccWityKgBCfuN2ZGWwGDtSkaWUr8=; b=iq14CLNr0q3kRSo+MuHNj9jkFxfY8b3GUZH7m+apMtRK9JMdddgpGxAX9zG+y7nzf0 OIuooKcY/HwjSB2OvZdXSiLqrzCoyP29mPykJwWpQTR6l4Ts5esMrw4djobHCxdvGNkJ /8Oco+kg+dYs/dc36LF5odg1JijxESvcVa/ovp6zUUDLpQ6XclDfRyXo58gjyveus4zB cx9xerwKlWbvERukVkgJUyhAwiP0XmJVnstw4MLt5zC1SCUVPhKW27rCjlVf495ITfFm MJ4h4Pm4tuk26EvPOsVRuvx05kR5K86zeKSbeePVVVyx5kAghyRMy4U6A9EpbGQiasxM KwUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sIRVDCI304lqLYuccWityKgBCfuN2ZGWwGDtSkaWUr8=; b=Dc8o4xFv1cujIXph1dVYvLj/45ecJBCcZ+z3C67VGl0GVTqoHj88Pxzun0z6smArXC xVoJSiVP02tVJTzI6AAO+jjqxjUcX4nhaZErXOtlC/IHRRvW7gv9E/rhK1AbcFEqlepC CRO/XyZmH7aJk6XUKgY/x72Oqs7B1SieD1e/cmy3p3fNvwRgpiPb3jc4kQz02bwdXoXN RrsTBZjyv2E++NJChTa+ZM/guZ5vkG8mj14ZWJb2SjzxiMndWGGzejR3bNpCPBCFKm8I Ld34ovBhyNmqWuU5X93C2vVr46zn6wuWzjscf35f0X6UWkIe2Rd28JBYeMdarRWzyNEv kONg== X-Gm-Message-State: AOUpUlHfrQKT+udjNkr3sAf2xvc2qi5/6744b/fxLoY5pIhiNRy2K0wQ V9djmMT+5T+lRAsrWnFJgeCYJ7Jw2QIlU2/894hyGA== X-Google-Smtp-Source: AA+uWPx7tvzIxAQbfixm/sTeOX6eopD5J5narh5uEiqDGKYNCL3U2lqCN9TTpmvBvEql9LQLy1O6bmFauLaxxLUg/Sg= X-Received: by 2002:a6b:d004:: with SMTP id x4-v6mr10310673ioa.299.1533423688239; Sat, 04 Aug 2018 16:01:28 -0700 (PDT) MIME-Version: 1.0 References: <201808042208.w74M8OmD057603@repo.freebsd.org> In-Reply-To: From: Warner Losh Date: Sun, 5 Aug 2018 00:01:15 +0100 Message-ID: Subject: Re: svn commit: r337334 - head/lib/libc/sys To: "Conrad E. Meyer" Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 23:01:29 -0000 On Sat, Aug 4, 2018, 11:48 PM Conrad Meyer wrote: > If not stale, it was inaccurate as-is. It seems to be used by a bunch > of different clock-related drivers and routines, e.g., > > $ rg '(tz_minuteswest|tz_dst|utc_offset)' | wc -l > 36 > Right. Nothings sets it any more though and hasn't in 30 years. Adjkerntz is what is used today. The other code was left for still lingering legacy uses in the early 1990s... To me there is a disconnect between that use and "timezone is no > longer used; this information is kept outside the kernel." > Right. It's not used nor set. It was the 4.2 BSD interface... If you or someone else wants to reword the "unused" note in a more > nuanced fashion, I'm definitely supportive of that. (Or remove the > kernel uses aside from pass-thru for gettimeofday and restore the > original note, of course.) > I'll do it. I was actually going to deorbit the date(1) -t/-d options on the basis > of the settimeofday manual page note until I did the due diligence to > verify it was actually unused, and discovered that not to be true. > We should do that. This code has not been relevant in decades. Warner All the best, > Conrad > > > > On Sat, Aug 4, 2018 at 3:15 PM, Warner Losh wrote: > > How is this stale? It said it unused, and now it doesn't say this are is > > completely ignored. > > > > Warner > > > > On Sat, Aug 4, 2018, 11:08 PM Conrad Meyer wrote: > >> > >> Author: cem > >> Date: Sat Aug 4 22:08:24 2018 > >> New Revision: 337334 > >> URL: https://svnweb.freebsd.org/changeset/base/337334 > >> > >> Log: > >> settimeofday(2): Remove stale note about timezone > >> > >> Contrary to the removed comment, the kernel does appear to use the > >> timezone > >> argument of settimeofday. The comment dates to the BSD4.4 import; I > >> assume it > >> is just stale. > >> > >> Modified: > >> head/lib/libc/sys/gettimeofday.2 > >> > >> Modified: head/lib/libc/sys/gettimeofday.2 > >> > >> > ============================================================================== > >> --- head/lib/libc/sys/gettimeofday.2 Sat Aug 4 21:57:17 2018 > >> (r337333) > >> +++ head/lib/libc/sys/gettimeofday.2 Sat Aug 4 22:08:24 2018 > >> (r337334) > >> @@ -28,7 +28,7 @@ > >> .\" @(#)gettimeofday.2 8.2 (Berkeley) 5/26/95 > >> .\" $FreeBSD$ > >> .\" > >> -.Dd December 27, 2015 > >> +.Dd August 4, 2018 > >> .Dt GETTIMEOFDAY 2 > >> .Os > >> .Sh NAME > >> @@ -44,11 +44,6 @@ > >> .Ft int > >> .Fn settimeofday "const struct timeval *tp" "const struct timezone > *tzp" > >> .Sh DESCRIPTION > >> -.Bf -symbolic > >> -Note: timezone is no longer used; this information is kept outside > >> -the kernel. > >> -.Ef > >> -.Pp > >> The system's notion of the current Greenwich time and the current time > >> zone is obtained with the > >> .Fn gettimeofday > >> > > > From owner-svn-src-all@freebsd.org Sat Aug 4 23:02:50 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9085A1056CA9 for ; Sat, 4 Aug 2018 23:02:50 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1CA698D662 for ; Sat, 4 Aug 2018 23:02:50 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x22a.google.com with SMTP id z19-v6so8029409ioh.4 for ; Sat, 04 Aug 2018 16:02:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6ZLLTbJKjPDcWSs72gWsUymWUOHh3sLhB+lY9IvA9CI=; b=kk7syF5eY5Khfnd1fHCqTM4pD+XRvgnLYfG4aDrFlhV53qtRG8O+ht++3KTD35ao5l geFoSzenijaCx2Ql3eVd0YBE6/VRUHwwyoLI1oNMqQ9agxnnmXcKxzqKfvYWVQbWX/fA N7/LrRoWlXT14AM5OWLYjhfSK45IY/BB1469V/tOaHVt4wh/xqigekCNPGGwoCDlr6jw gDLr5Jrm7ITp2TxFtu7w9ibfepNhypMQbr1jhpxVYghE+r4d1ILvPf3ThjQeJ8xo2C3S An1dGU6enx2FJjAgJ/g2ssS4ZD3oyBFFNxo3z153yPOlfknO+bwrjYix4YU8XaJ5soBA ZbSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6ZLLTbJKjPDcWSs72gWsUymWUOHh3sLhB+lY9IvA9CI=; b=Gp2d1ZJszt4v/roFwbHF5P3Fe1jZHNUbS81EVAtJS4kyhlAtjSMSy8hKcH3tQ4cuWe gwavwVuYtRC2NyDrrxc3NYZr/Y/E61Zeuh8boUXC0GMRuVoyCw3pMuWr8k3nOD2uTFbE JkePJPuI78R8tcevhPn/CsXp+OhJqQFtT8NscQCwZZpT0M/V2R8WdtKJ67YcBRNcumRe TcHtu1wojSEqLTM7FEXoegGkqby4V44GT9OFX/dMD8DK12VDJRIvgggZdjEZbTDaCpA7 SAlJFJuRv6RUBNFw/PcSqMEICrKoyRQd/kAwt13FySEf//7rcOryBnOVGEXERyvVxhsF zruA== X-Gm-Message-State: AOUpUlEke30CrtbXB2knoPB1pBakx1ZQxw8mO66ql8VquC+QLCiRVjN/ Pnp1aR0pC0p7MOgbTO1C12u5vaFE03p9Z6octLx8qA== X-Google-Smtp-Source: AA+uWPzQxMii3opXM7rpRlNbwRzymMowt/8JV8rWRx/hWmCr34e8iZ6WOajikVlYtqTUrVBeG5zNrHWR0SyhFjif2Xg= X-Received: by 2002:a6b:f719:: with SMTP id k25-v6mr4469207iog.37.1533423769506; Sat, 04 Aug 2018 16:02:49 -0700 (PDT) MIME-Version: 1.0 References: <201808021925.w72JPhIu086582@repo.freebsd.org> In-Reply-To: From: Warner Losh Date: Sun, 5 Aug 2018 00:02:36 +0100 Message-ID: Subject: Re: svn commit: r337165 - head/sys/fs/fuse To: "Conrad E. Meyer" Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 23:02:50 -0000 On Sat, Aug 4, 2018, 11:56 PM Conrad Meyer wrote: > On Sat, Aug 4, 2018 at 3:54 PM, Conrad Meyer wrote: > > It was chosen arbitrarily. Either is a big improvement over 4k. > > > > According to the bug reporter, 64k is a common choice in other > > implementations. But I have no objection to MAXPHYS. > > (It should probably be globally tunable/sysctlable, at least, or > perhaps a mount option. This was just the minimum diff change to > remove a large bottleneck.) > It's a good change. I've always been a bit unclear on when one should use one or the other.. both will work.. Warner > From owner-svn-src-all@freebsd.org Sat Aug 4 23:04:18 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 042471056D54; Sat, 4 Aug 2018 23:04:18 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-io0-f181.google.com (mail-io0-f181.google.com [209.85.223.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 931DE8D7DC; Sat, 4 Aug 2018 23:04:17 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-io0-f181.google.com with SMTP id w11-v6so8038261iob.2; Sat, 04 Aug 2018 16:04:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=Q8EegBdmTJGwC6+Ber4WBYsdJ5d70x55ZROXjCwGhP8=; b=En4bjDPTWBwWtAMGB1YjM7eWj/PdEaFp1R+dsrqyHJsHfSGBz2Roo7frwvvgXzLTBb 4QNDTzAy0K58Osd8iVCLUhHx/iJF6ajjx4CYPxBeCJrUuZUOhsoH0IWkUZ8pD6Az3ebf 0QtUy3RtZs47FOSoo/APF0XCDavwHJKYfweB8L6To2eRvZrnV2VQODfrEBST23d1Lc5t Z2BekLmPKXzfLTFlUIYmTeHOeSa4zzlgpt1oiBrQvqtYOsYvb6+/HVdTCr6eYigJozY9 oLsFzR+Ff6H2C7fnhtwNTMsHiMZz7KOC1dyNxmJjY6iULxVp/DKa2Czu174jGUpn0AfP 0nLw== X-Gm-Message-State: AOUpUlFaL+nFUsXzb9d/XEEjLiB4anLL3q3mbAm3MOT7XeICxtaxrbui cxa6DYw8FSeglkQ/wt9zUAkEjZe7 X-Google-Smtp-Source: AA+uWPwZV70EUkJDd9HKJdVn7hE86steZo9zDIfEf/21Tk8yNedIHMkvEx/pBFln0/OHMFheDFz1HQ== X-Received: by 2002:a6b:b0c5:: with SMTP id z188-v6mr10379292ioe.220.1533423376937; Sat, 04 Aug 2018 15:56:16 -0700 (PDT) Received: from mail-io0-f178.google.com (mail-io0-f178.google.com. [209.85.223.178]) by smtp.gmail.com with ESMTPSA id n127-v6sm1712591itn.7.2018.08.04.15.56.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 04 Aug 2018 15:56:16 -0700 (PDT) Received: by mail-io0-f178.google.com with SMTP id k16-v6so7977903iom.12; Sat, 04 Aug 2018 15:56:16 -0700 (PDT) X-Received: by 2002:a6b:b0c5:: with SMTP id z188-v6mr10379281ioe.220.1533423376698; Sat, 04 Aug 2018 15:56:16 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 2002:a02:7e0a:0:0:0:0:0 with HTTP; Sat, 4 Aug 2018 15:56:16 -0700 (PDT) In-Reply-To: References: <201808021925.w72JPhIu086582@repo.freebsd.org> From: Conrad Meyer Date: Sat, 4 Aug 2018 15:56:16 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r337165 - head/sys/fs/fuse To: Warner Losh Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 23:04:18 -0000 On Sat, Aug 4, 2018 at 3:54 PM, Conrad Meyer wrote: > It was chosen arbitrarily. Either is a big improvement over 4k. > > According to the bug reporter, 64k is a common choice in other > implementations. But I have no objection to MAXPHYS. (It should probably be globally tunable/sysctlable, at least, or perhaps a mount option. This was just the minimum diff change to remove a large bottleneck.)