From nobody Thu Oct 31 00:55:01 2024 X-Original-To: dev-commits-ports-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xf5BF2ppwz5ZfLt; Thu, 31 Oct 2024 00:55:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xf5BF1QSXz4dmj; Thu, 31 Oct 2024 00:55:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730336101; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=98pnBiJykbPfiyWN0gVeOvknEEHjQlEP+G61kinoAZM=; b=Qx3ngmoz4AEJs+iLQJ8UjdMUHFMWREeWQWbrvUi1z+CE+SKq5J4+gq/g48SMQ7diVzIvTJ 81MX5RkFcV2fFNtjrQ9/qs3CSh97/gtXs2ZwG0PCV+AaJ53oztCLaX5nzLeBCm7+FsGLol 0mcfCz5Y9QTKoMdqt4HBO6yl5/7V0XkspuFbW0PqB0ST/pHXiRUOQvLS+54tLXKX6XHcQC jWsDOrL+rjPw8TqwN7VUqzJvUqLwVjtBlY/S9HooPYjYAjiP3EB5rLBLpPTZbPpO3mOJO+ AYXCwfO9+brqXqtrtjr4BGv1RQeXzvFGpu8vtcdQtVIpzGMBlI+QwSYf6Xhpbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730336101; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=98pnBiJykbPfiyWN0gVeOvknEEHjQlEP+G61kinoAZM=; b=IoTG3xQhXSmEhf8Eoo3+OKwAAS7E7Lj7hURwBQunuAIGoQRaxGWKyhduHsNj0IH8r+HY/M Sd4kC7wZrmzrhbwUvzq06YkAxZlZw/Etywd7YYLIe7X6oVzfeCE+SNbs26IgbCFgXkT5Au NXudy2dcHcYsXg/xRuTJGvfEGAFUqb4Z+LOrg35QZ82L0rVIaLutotGFSf3rYKP4APxoOU 2xUqPUDFmK1+hSSY3fj5AfmAdvzMYyfCTNySFnpxL3bUMUTBkSqUsz7lBEBzhOBP6TtUzf mjneThlztuzAzYRSJ83VEXXN7wfgsjBeBFHjzVcgYNt1PNpO/q/lWBXwGrucOA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1730336101; a=rsa-sha256; cv=none; b=GH+c8dVQXTPG8VYg2QVBrLwcQ4+9BFRM4cOd1gl14DFo6e0rSP5hmF/xLQEX0uJSZ9sWsf ++zcp2dJDj4sFunWD98A412ObcCU1uEzwypE/yxsxgIxfWmHBvvqZv3CPNVzyhXAc7Uggt dnWJ09UUFcaXTdOstkmfjnNm2gZPcCZgO2TI6UfNbGYGfDCtbhJVew/x8DQ9ugVjQv6K+5 CIrN+xo9inMwi5jKlX8SHgSWLeg88Qh02nuAroaoe1ThEAwRNrl6bhcBcuA2TztToNGBH9 K6iEbxyBAXfiBr3tL0KTPqbczPlRFNUDoL5r3W4AqlOL395BfKlxSXOmq3a5uQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Xf5BF1277zFTl; Thu, 31 Oct 2024 00:55:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 49V0t1od077762; Thu, 31 Oct 2024 00:55:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 49V0t12K077759; Thu, 31 Oct 2024 00:55:01 GMT (envelope-from git) Date: Thu, 31 Oct 2024 00:55:01 GMT Message-Id: <202410310055.49V0t12K077759@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-branches@FreeBSD.org From: Jan Beich Subject: git: 2e2d801cf977 - 2024Q4 - graphics/mesa-devel: update to 24.2.b.4926 List-Id: Commits to the quarterly branches of the FreeBSD ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-ports-branches@freebsd.org Sender: owner-dev-commits-ports-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jbeich X-Git-Repository: ports X-Git-Refname: refs/heads/2024Q4 X-Git-Reftype: branch X-Git-Commit: 2e2d801cf977c386e0b4c350b6b1b38f70011ffb Auto-Submitted: auto-generated The branch 2024Q4 has been updated by jbeich: URL: https://cgit.FreeBSD.org/ports/commit/?id=2e2d801cf977c386e0b4c350b6b1b38f70011ffb commit 2e2d801cf977c386e0b4c350b6b1b38f70011ffb Author: Jan Beich AuthorDate: 2024-10-30 16:21:16 +0000 Commit: Jan Beich CommitDate: 2024-10-31 00:54:52 +0000 graphics/mesa-devel: update to 24.2.b.4926 Changes: https://gitlab.freedesktop.org/mesa/mesa/-/compare/c0bceaf0576...e474d4ebeef (cherry picked from commit 35e33b5b91868165936e4063f394acd76e60a530) --- graphics/mesa-devel/Makefile | 4 +- graphics/mesa-devel/distinfo | 6 +- graphics/mesa-devel/files/patch-revert | 1075 ++++++++++++++++++++++++++++++++ 3 files changed, 1080 insertions(+), 5 deletions(-) diff --git a/graphics/mesa-devel/Makefile b/graphics/mesa-devel/Makefile index 9475bfe6cea8..21b6744f6db6 100644 --- a/graphics/mesa-devel/Makefile +++ b/graphics/mesa-devel/Makefile @@ -1,6 +1,6 @@ PORTNAME= mesa -DISTVERSION= 24.2-branchpoint-4566 -DISTVERSIONSUFFIX= -gc0bceaf0576 +DISTVERSION= 24.2-branchpoint-4926 +DISTVERSIONSUFFIX= -ge474d4ebeef CATEGORIES= graphics PKGNAMESUFFIX= -devel diff --git a/graphics/mesa-devel/distinfo b/graphics/mesa-devel/distinfo index a607cfb66920..38d25a771c9f 100644 --- a/graphics/mesa-devel/distinfo +++ b/graphics/mesa-devel/distinfo @@ -1,6 +1,6 @@ -TIMESTAMP = 1729714788 -SHA256 (mesa-24.2-branchpoint-4566-gc0bceaf0576.tar.bz2) = 3e35051d08e71231990004b16870dcf9863108621c60d31c2c65eb6a95c41cbf -SIZE (mesa-24.2-branchpoint-4566-gc0bceaf0576.tar.bz2) = 38952383 +TIMESTAMP = 1730322260 +SHA256 (mesa-24.2-branchpoint-4926-ge474d4ebeef.tar.bz2) = 69a2e52d5bc3ca11c10b24042818d95725b97f20ac741ddfa021402ca1fd36d5 +SIZE (mesa-24.2-branchpoint-4926-ge474d4ebeef.tar.bz2) = 38960769 SHA256 (700efacda59c.patch) = 0d567fe737ad1404e1f12d7cd018826d9095c23835f1ed5aaa1c81cb58d3d008 SIZE (700efacda59c.patch) = 983 SHA256 (2930dcbb3329.patch) = f90ab77950ba1a56d165f0bc8a3cbd9c5f624bb5c67d1c7f337316027e8295e8 diff --git a/graphics/mesa-devel/files/patch-revert b/graphics/mesa-devel/files/patch-revert new file mode 100644 index 000000000000..cba4cbffdf09 --- /dev/null +++ b/graphics/mesa-devel/files/patch-revert @@ -0,0 +1,1075 @@ +Revert https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30988 +until https://gitlab.freedesktop.org/mesa/mesa/-/issues/12083 + +--- src/gallium/drivers/zink/ci/zink-anv-adl-fails.txt.orig 2024-10-30 21:04:20 UTC ++++ src/gallium/drivers/zink/ci/zink-anv-adl-fails.txt +@@ -77,6 +77,7 @@ glx@glx-multi-window-single-context,Fail + KHR-GL46.sparse_texture2_tests.SparseTexture2Commitment_texture_2d_multisample_rgb10_a2,Fail + + glx@glx-multi-window-single-context,Fail ++glx@glx-multithread-clearbuffer,Crash + glx@glx_arb_create_context_es2_profile@invalid opengl es version,Fail + glx@glx_arb_create_context_no_error@no error,Fail + glx@glx_arb_create_context_robustness@invalid reset notification strategy,Fail +--- src/gallium/drivers/zink/ci/zink-anv-adl-flakes.txt.orig 2024-10-30 21:04:20 UTC ++++ src/gallium/drivers/zink/ci/zink-anv-adl-flakes.txt +@@ -834,4 +834,3 @@ glx@glx-visuals-stencil -pixmap + wayland-dEQP-EGL.functional.wide_color.window_fp16_default_colorspace + + glx@glx-visuals-stencil -pixmap +-glx@glx-multithread-clearbuffer +--- src/util/mesa_cache_db.c.orig 2024-10-30 21:04:20 UTC ++++ src/util/mesa_cache_db.c +@@ -16,7 +16,6 @@ + #include + #include + #include +-#include + #include + + #include "crc32.h" +@@ -51,6 +50,14 @@ struct PACKED mesa_index_db_file_entry { + uint64_t cache_db_file_offset; + }; + ++struct mesa_index_db_hash_entry { ++ uint64_t cache_db_file_offset; ++ uint64_t index_db_file_offset; ++ uint64_t last_access_time; ++ uint32_t size; ++ bool evicted; ++}; ++ + static inline bool mesa_db_seek_end(FILE *file) + { + return !fseek(file, 0, SEEK_END); +@@ -84,46 +91,21 @@ static bool + } + + static bool +-mesa_db_reopen_file(struct mesa_cache_db_file *db_file); +- +-static void +-mesa_db_close_file(struct mesa_cache_db_file *db_file); +- +-static int +-mesa_db_flock(FILE *file, int op) +-{ +- int ret; +- +- do { +- ret = flock(fileno(file), op); +- } while (ret < 0 && errno == EINTR); +- +- return ret; +-} +- +-static bool + mesa_db_lock(struct mesa_cache_db *db) + { + simple_mtx_lock(&db->flock_mtx); + +- if (!mesa_db_reopen_file(&db->index) || +- !mesa_db_reopen_file(&db->cache)) +- goto close_files; ++ if (flock(fileno(db->cache.file), LOCK_EX) == -1) ++ goto unlock_mtx; + +- if (mesa_db_flock(db->cache.file, LOCK_EX) < 0) +- goto close_files; +- +- if (mesa_db_flock(db->index.file, LOCK_EX) < 0) ++ if (flock(fileno(db->index.file), LOCK_EX) == -1) + goto unlock_cache; + + return true; + + unlock_cache: +- mesa_db_flock(db->cache.file, LOCK_UN); +-close_files: +- mesa_db_close_file(&db->index); +- mesa_db_close_file(&db->cache); +- ++ flock(fileno(db->cache.file), LOCK_UN); ++unlock_mtx: + simple_mtx_unlock(&db->flock_mtx); + + return false; +@@ -132,12 +114,8 @@ mesa_db_unlock(struct mesa_cache_db *db) + static void + mesa_db_unlock(struct mesa_cache_db *db) + { +- mesa_db_flock(db->index.file, LOCK_UN); +- mesa_db_flock(db->cache.file, LOCK_UN); +- +- mesa_db_close_file(&db->index); +- mesa_db_close_file(&db->cache); +- ++ flock(fileno(db->index.file), LOCK_UN); ++ flock(fileno(db->cache.file), LOCK_UN); + simple_mtx_unlock(&db->flock_mtx); + } + +@@ -255,34 +233,6 @@ mesa_db_zap(struct mesa_cache_db *db) + return true; + } + +-static struct mesa_index_db_file_entry * +-mesa_db_index_entry_get(struct mesa_cache_db *db, size_t offset) +-{ +- return (struct mesa_index_db_file_entry *) +- ((char*)db->index_entries + offset); +-} +- +-static void +-mesa_db_index_entry_insert(struct mesa_cache_db *db, +- struct mesa_index_db_file_entry *index_entry) +-{ +- size_t offset = (char*)index_entry - (char*)db->index_entries; +- +- offset += sizeof(struct mesa_db_file_header); +- _mesa_hash_table_u64_insert(db->index_db, index_entry->hash, (char*)(intptr_t)offset); +-} +- +-static struct mesa_index_db_file_entry * +-mesa_db_index_entry_search(struct mesa_cache_db *db, uint64_t key) +-{ +- size_t index_offset = (intptr_t)_mesa_hash_table_u64_search(db->index_db, key); +- +- if (!index_offset) +- return NULL; +- +- return mesa_db_index_entry_get(db, index_offset - sizeof(struct mesa_db_file_header)); +-} +- + static bool + mesa_db_index_entry_valid(struct mesa_index_db_file_entry *entry) + { +@@ -297,104 +247,46 @@ static bool + } + + static bool +-mesa_db_resize_index_entries(struct mesa_cache_db *db, off_t size) +-{ +- int page_size = getpagesize(); +- size_t page_mask = page_size - 1; +- off_t old_num_pages, new_num_pages; +- +- if (db->index_entries_size == size) +- return true; +- +- new_num_pages = (size + page_mask) / page_size; +- +- if (size) { +- if (db->index_entries_size) { +- old_num_pages = (db->index_entries_size + page_mask) / page_size; +- +- if (new_num_pages != old_num_pages) { +- db->index_entries = mremap(db->index_entries, old_num_pages * page_size, +- new_num_pages * page_size, MREMAP_MAYMOVE); +- if (db->index_entries == MAP_FAILED) { +- fprintf(stderr, "%s: mremap failed with error %d (%s)\n", +- __func__, errno, strerror(errno)); +- goto error; +- } +- } +- } else { +- db->index_entries = mmap(NULL, new_num_pages * page_size, PROT_READ | PROT_WRITE, +- MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE, -1, 0); +- if (db->index_entries == MAP_FAILED) { +- fprintf(stderr, "%s: mmap failed with error %d (%s)\n", +- __func__, errno, strerror(errno)); +- goto error; +- } +- } +- } else { +- if (db->index_entries_size) { +- old_num_pages = (db->index_entries_size + page_mask) / page_size; +- +- munmap(db->index_entries, old_num_pages * page_size); +- } +- +- db->index_entries = NULL; +- } +- +- db->index_entries_size = size; +- return true; +- +-error: +- _mesa_hash_table_u64_clear(db->index_db); +- db->index_entries = NULL; +- db->index_entries_size = 0; +- return false; +-} +- +-static bool + mesa_db_update_index(struct mesa_cache_db *db) + { +- struct mesa_index_db_file_entry *index_entry; ++ struct mesa_index_db_hash_entry *hash_entry; ++ struct mesa_index_db_file_entry index_entry; + size_t file_length; +- size_t old_entries, new_entries; +- int i; + + if (!mesa_db_seek_end(db->index.file)) + return false; + + file_length = ftell(db->index.file); +- if (file_length < db->index.offset) +- return false; + + if (!mesa_db_seek(db->index.file, db->index.offset)) + return false; + +- new_entries = (file_length - db->index.offset) / sizeof(*index_entry); +- if (!new_entries) +- return true; ++ while (db->index.offset < file_length) { ++ if (!mesa_db_read(db->index.file, &index_entry)) ++ break; + +- old_entries = db->index_entries_size / sizeof(*index_entry); +- +- if (!mesa_db_resize_index_entries(db, (old_entries + new_entries) * sizeof(*index_entry))) +- return false; +- +- _mesa_hash_table_reserve(db->index_db->table, old_entries + new_entries); +- +- index_entry = mesa_db_index_entry_get(db, old_entries * sizeof(*index_entry)); +- if (!mesa_db_read_data(db->index.file, index_entry, new_entries * sizeof(*index_entry))) +- return false; +- +- for (i = 0; i < new_entries; i++, index_entry++) { + /* Check whether the index entry looks valid or we have a corrupted DB */ +- if (!mesa_db_index_entry_valid(index_entry)) ++ if (!mesa_db_index_entry_valid(&index_entry)) + break; + +- mesa_db_index_entry_insert(db, index_entry); ++ hash_entry = ralloc(db->mem_ctx, struct mesa_index_db_hash_entry); ++ if (!hash_entry) ++ break; + +- db->index.offset += sizeof(*index_entry); ++ hash_entry->cache_db_file_offset = index_entry.cache_db_file_offset; ++ hash_entry->index_db_file_offset = db->index.offset; ++ hash_entry->last_access_time = index_entry.last_access_time; ++ hash_entry->size = index_entry.size; ++ ++ _mesa_hash_table_u64_insert(db->index_db, index_entry.hash, hash_entry); ++ ++ db->index.offset += sizeof(index_entry); + } + +- return mesa_db_seek(db->index.file, db->index.offset) && +- db->index.offset == file_length; ++ if (!mesa_db_seek(db->index.file, db->index.offset)) ++ return false; ++ ++ return db->index.offset == file_length; + } + + static void +@@ -403,8 +295,6 @@ mesa_db_hash_table_reset(struct mesa_cache_db *db) + _mesa_hash_table_u64_clear(db->index_db); + ralloc_free(db->mem_ctx); + db->mem_ctx = ralloc_context(NULL); +- +- mesa_db_resize_index_entries(db, 0); + } + + static bool +@@ -433,6 +323,10 @@ mesa_db_load(struct mesa_cache_db *db, bool reload) + !mesa_db_load_header(&db->index) || + db->cache.uuid != db->index.uuid) { + ++ /* This is unexpected to happen on reload, bail out */ ++ if (reload) ++ goto fail; ++ + if (!mesa_db_recreate_files(db)) + goto fail; + } else { +@@ -444,17 +338,9 @@ mesa_db_load(struct mesa_cache_db *db, bool reload) + if (reload) + mesa_db_hash_table_reset(db); + +- /* The update failed so we assume the files are corrupt and +- * recreate them. +- */ +- if (!mesa_db_update_index(db)) { +- mesa_db_recreate_files(db); +- db->index.offset = ftell(db->index.file); ++ if (!mesa_db_update_index(db)) ++ goto fail; + +- if (!mesa_db_update_index(db)) +- goto fail; +- } +- + if (!reload) + mesa_db_unlock(db); + +@@ -478,21 +364,10 @@ mesa_db_reload(struct mesa_cache_db *db) + return mesa_db_load(db, true); + } + +-static FILE * +-mesa_db_fopen(const char *path) ++static void ++touch_file(const char* path) + { +- /* The fopen("r+b") mode doesn't auto-create new file, hence we need to +- * explicitly create the file first. +- */ +- int fd = open(path, O_CREAT | O_CLOEXEC | O_RDWR, 0644); +- if (fd < 0) +- return NULL; +- +- FILE *f = fdopen(fd, "r+b"); +- if (!f) +- close(fd); +- +- return f; ++ close(open(path, O_CREAT | O_CLOEXEC, 0644)); + } + + static bool +@@ -503,7 +378,12 @@ mesa_db_open_file(struct mesa_cache_db_file *db_file, + if (asprintf(&db_file->path, "%s/%s", cache_path, filename) == -1) + return false; + +- db_file->file = mesa_db_fopen(db_file->path); ++ /* The fopen("r+b") mode doesn't auto-create new file, hence we need to ++ * explicitly create the file first. ++ */ ++ touch_file(db_file->path); ++ ++ db_file->file = fopen(db_file->path, "r+b"); + if (!db_file->file) { + free(db_file->path); + return false; +@@ -512,34 +392,10 @@ mesa_db_open_file(struct mesa_cache_db_file *db_file, + return true; + } + +-static bool +-mesa_db_reopen_file(struct mesa_cache_db_file *db_file) +-{ +- if (db_file->file) +- return true; +- +- db_file->file = mesa_db_fopen(db_file->path); +- if (!db_file->file) +- return false; +- +- return true; +-} +- + static void + mesa_db_close_file(struct mesa_cache_db_file *db_file) + { +- if (db_file->file) { +- fclose(db_file->file); +- db_file->file = NULL; +- } +-} +- +-static void +-mesa_db_free_file(struct mesa_cache_db_file *db_file) +-{ +- if (db_file->file) +- fclose(db_file->file); +- ++ fclose(db_file->file); + free(db_file->path); + } + +@@ -556,16 +412,11 @@ mesa_db_remove_file(struct mesa_cache_db_file *db_file + return true; + } + +-struct sort_entry { +- struct mesa_index_db_file_entry *index_entry; +- bool evicted; +-}; +- + static int + entry_sort_lru(const void *_a, const void *_b, void *arg) + { +- const struct mesa_index_db_file_entry *a = ((const struct sort_entry *)_a)->index_entry; +- const struct mesa_index_db_file_entry *b = ((const struct sort_entry *)_b)->index_entry; ++ const struct mesa_index_db_hash_entry *a = *((const struct mesa_index_db_hash_entry **)_a); ++ const struct mesa_index_db_hash_entry *b = *((const struct mesa_index_db_hash_entry **)_b); + + /* In practice it's unlikely that we will get two entries with the + * same timestamp, but technically it's possible to happen if OS +@@ -579,8 +430,8 @@ entry_sort_offset(const void *_a, const void *_b, void + static int + entry_sort_offset(const void *_a, const void *_b, void *arg) + { +- const struct mesa_index_db_file_entry *a = ((const struct sort_entry *)_a)->index_entry; +- const struct mesa_index_db_file_entry *b = ((const struct sort_entry *)_b)->index_entry; ++ const struct mesa_index_db_hash_entry *a = *((const struct mesa_index_db_hash_entry **)_a); ++ const struct mesa_index_db_hash_entry *b = *((const struct mesa_index_db_hash_entry **)_b); + struct mesa_cache_db *db = arg; + + /* Two entries will never have the identical offset, otherwise DB is +@@ -598,14 +449,14 @@ mesa_db_compact(struct mesa_cache_db *db, int64_t blob + + static bool + mesa_db_compact(struct mesa_cache_db *db, int64_t blob_size, +- struct mesa_index_db_file_entry *remove_entry) ++ struct mesa_index_db_hash_entry *remove_entry) + { + uint32_t num_entries, buffer_size = sizeof(struct mesa_index_db_file_entry); + struct mesa_db_file_header cache_header, index_header; + FILE *compacted_cache = NULL, *compacted_index = NULL; +- struct mesa_index_db_file_entry *index_entry; +- struct sort_entry *entries; +- bool success = false; ++ struct mesa_index_db_file_entry index_entry; ++ struct mesa_index_db_hash_entry **entries; ++ bool success = false, compact = false; + void *buffer = NULL; + unsigned int i = 0; + +@@ -614,15 +465,12 @@ mesa_db_compact(struct mesa_cache_db *db, int64_t blob + return false; + + num_entries = _mesa_hash_table_num_entries(db->index_db->table); +- if (!num_entries) +- return true; +- + entries = calloc(num_entries, sizeof(*entries)); + if (!entries) + return false; + +- compacted_cache = mesa_db_fopen(db->cache.path); +- compacted_index = mesa_db_fopen(db->index.path); ++ compacted_cache = fopen(db->cache.path, "r+b"); ++ compacted_index = fopen(db->index.path, "r+b"); + if (!compacted_cache || !compacted_index) + goto cleanup; + +@@ -634,18 +482,19 @@ mesa_db_compact(struct mesa_cache_db *db, int64_t blob + index_header.uuid != db->uuid) + goto cleanup; + +- for (i = 0, index_entry = db->index_entries; i < num_entries; i++, index_entry++) { +- entries[i].index_entry = index_entry; +- entries[i].evicted = index_entry == remove_entry; +- buffer_size = MAX2(buffer_size, blob_file_size(index_entry->size)); ++ hash_table_foreach(db->index_db->table, entry) { ++ entries[i] = entry->data; ++ entries[i]->evicted = (entries[i] == remove_entry); ++ buffer_size = MAX2(buffer_size, blob_file_size(entries[i]->size)); ++ i++; + } + + util_qsort_r(entries, num_entries, sizeof(*entries), + entry_sort_lru, db); + + for (i = 0; blob_size > 0 && i < num_entries; i++) { +- blob_size -= blob_file_size(entries[i].index_entry->size); +- entries[i].evicted = true; ++ blob_size -= blob_file_size(entries[i]->size); ++ entries[i]->evicted = true; + } + + util_qsort_r(entries, num_entries, sizeof(*entries), +@@ -665,38 +514,59 @@ mesa_db_compact(struct mesa_cache_db *db, int64_t blob + !mesa_db_write_header(&db->index, 0, false)) + goto cleanup; + +- /* Skip non-evicted entries at the start of the files */ +- for (i = 0; i < num_entries; i++) { +- if (entries[i].evicted) +- break; +- } +- + /* Sync the file pointers */ +- if (!mesa_db_seek(compacted_cache, entries[i].index_entry->cache_db_file_offset) || +- !mesa_db_seek(compacted_index, ftell(db->index.file) + +- i * sizeof(struct mesa_index_db_file_entry))) ++ if (!mesa_db_seek(compacted_cache, ftell(db->cache.file)) || ++ !mesa_db_seek(compacted_index, ftell(db->index.file))) + goto cleanup; + + /* Do the compaction */ +- for (; i < num_entries; i++) { +- struct mesa_index_db_file_entry *index_entry = entries[i].index_entry; ++ for (i = 0; i < num_entries; i++) { ++ blob_size = blob_file_size(entries[i]->size); + +- if (entries[i].evicted) ++ /* Sanity-check the cache-read offset */ ++ if (ftell(db->cache.file) != entries[i]->cache_db_file_offset) ++ goto cleanup; ++ ++ if (entries[i]->evicted) { ++ /* Jump over the evicted entry */ ++ if (!mesa_db_seek_cur(db->cache.file, blob_size) || ++ !mesa_db_seek_cur(db->index.file, sizeof(index_entry))) ++ goto cleanup; ++ ++ compact = true; + continue; ++ } + +- blob_size = blob_file_size(index_entry->size); ++ if (compact) { ++ /* Compact the cache file */ ++ if (!mesa_db_read_data(db->cache.file, buffer, blob_size) || ++ !mesa_db_cache_entry_valid(buffer) || ++ !mesa_db_write_data(compacted_cache, buffer, blob_size)) ++ goto cleanup; + +- /* Compact the cache file */ +- if (!mesa_db_seek(db->cache.file, index_entry->cache_db_file_offset) || +- !mesa_db_read_data(db->cache.file, buffer, blob_size) || +- !mesa_db_cache_entry_valid(buffer) || +- !mesa_db_write_data(compacted_cache, buffer, blob_size)) +- goto cleanup; ++ /* Compact the index file */ ++ if (!mesa_db_read(db->index.file, &index_entry) || ++ !mesa_db_index_entry_valid(&index_entry) || ++ index_entry.cache_db_file_offset != entries[i]->cache_db_file_offset || ++ index_entry.size != entries[i]->size) ++ goto cleanup; + +- index_entry->cache_db_file_offset = ftell(compacted_cache) - blob_size; ++ index_entry.cache_db_file_offset = ftell(compacted_cache) - blob_size; + +- if (!mesa_db_write(compacted_index, index_entry)) +- goto cleanup; ++ if (!mesa_db_write(compacted_index, &index_entry)) ++ goto cleanup; ++ } else { ++ /* Sanity-check the cache-write offset */ ++ if (ftell(compacted_cache) != entries[i]->cache_db_file_offset) ++ goto cleanup; ++ ++ /* Jump over the unchanged entry */ ++ if (!mesa_db_seek_cur(db->index.file, sizeof(index_entry)) || ++ !mesa_db_seek_cur(compacted_index, sizeof(index_entry)) || ++ !mesa_db_seek_cur(db->cache.file, blob_size) || ++ !mesa_db_seek_cur(compacted_cache, blob_size)) ++ goto cleanup; ++ } + } + + fflush(compacted_cache); +@@ -762,9 +632,9 @@ close_index: + + ralloc_free(db->mem_ctx); + close_index: +- mesa_db_free_file(&db->index); ++ mesa_db_close_file(&db->index); + close_cache: +- mesa_db_free_file(&db->cache); ++ mesa_db_close_file(&db->cache); + + return false; + } +@@ -792,14 +662,8 @@ mesa_cache_db_close(struct mesa_cache_db *db) + simple_mtx_destroy(&db->flock_mtx); + ralloc_free(db->mem_ctx); + +- mesa_db_resize_index_entries(db, 0); +- if (db->index_entries) { +- munmap(db->index_entries, 0); +- db->index_entries = NULL; +- } +- +- mesa_db_free_file(&db->index); +- mesa_db_free_file(&db->cache); ++ mesa_db_close_file(&db->index); ++ mesa_db_close_file(&db->cache); + } + + void +@@ -822,8 +686,8 @@ mesa_cache_db_read_entry(struct mesa_cache_db *db, + { + uint64_t hash = to_mesa_cache_db_hash(cache_key_160bit); + struct mesa_cache_db_file_entry cache_entry; +- struct mesa_index_db_file_entry *index_entry; +- long seek_pos; ++ struct mesa_index_db_file_entry index_entry; ++ struct mesa_index_db_hash_entry *hash_entry; + void *data = NULL; + + if (!mesa_db_lock(db)) +@@ -838,11 +702,11 @@ mesa_cache_db_read_entry(struct mesa_cache_db *db, + if (!mesa_db_update_index(db)) + goto fail_fatal; + +- index_entry = mesa_db_index_entry_search(db, hash); +- if (!index_entry) ++ hash_entry = _mesa_hash_table_u64_search(db->index_db, hash); ++ if (!hash_entry) + goto fail; + +- if (!mesa_db_seek(db->cache.file, index_entry->cache_db_file_offset) || ++ if (!mesa_db_seek(db->cache.file, hash_entry->cache_db_file_offset) || + !mesa_db_read(db->cache.file, &cache_entry) || + !mesa_db_cache_entry_valid(&cache_entry)) + goto fail_fatal; +@@ -858,13 +722,18 @@ mesa_cache_db_read_entry(struct mesa_cache_db *db, + util_hash_crc32(data, cache_entry.size) != cache_entry.crc) + goto fail_fatal; + +- index_entry->last_access_time = os_time_get_nano(); ++ if (!mesa_db_seek(db->index.file, hash_entry->index_db_file_offset) || ++ !mesa_db_read(db->index.file, &index_entry) || ++ !mesa_db_index_entry_valid(&index_entry) || ++ index_entry.cache_db_file_offset != hash_entry->cache_db_file_offset || ++ index_entry.size != hash_entry->size) ++ goto fail_fatal; + +- seek_pos = ((char*)index_entry - (char*)db->index_entries) + +- sizeof(struct mesa_db_file_header); ++ index_entry.last_access_time = os_time_get_nano(); ++ hash_entry->last_access_time = index_entry.last_access_time; + +- if (!mesa_db_seek(db->index.file, seek_pos) || +- !mesa_db_write(db->index.file, index_entry)) ++ if (!mesa_db_seek(db->index.file, hash_entry->index_db_file_offset) || ++ !mesa_db_write(db->index.file, &index_entry)) + goto fail_fatal; + + fflush(db->index.file); +@@ -904,9 +773,9 @@ mesa_cache_db_entry_write(struct mesa_cache_db *db, + const void *blob, size_t blob_size) + { + uint64_t hash = to_mesa_cache_db_hash(cache_key_160bit); ++ struct mesa_index_db_hash_entry *hash_entry = NULL; + struct mesa_cache_db_file_entry cache_entry; +- struct mesa_index_db_file_entry *index_entry; +- off_t index_offset; ++ struct mesa_index_db_file_entry index_entry; + + if (!mesa_db_lock(db)) + return false; +@@ -929,41 +798,46 @@ mesa_cache_db_entry_write(struct mesa_cache_db *db, + goto fail_fatal; + } + +- index_entry = mesa_db_index_entry_search(db, hash); +- if (index_entry) ++ hash_entry = _mesa_hash_table_u64_search(db->index_db, hash); ++ if (hash_entry) { ++ hash_entry = NULL; + goto fail; ++ } + + if (!mesa_db_seek_end(db->cache.file) || + !mesa_db_seek_end(db->index.file)) + goto fail_fatal; + +- index_offset = db->index_entries_size; +- if (!mesa_db_resize_index_entries(db, index_offset + sizeof(*index_entry))) +- goto fail; +- +- index_entry = mesa_db_index_entry_get(db, index_offset); +- + memcpy(cache_entry.key, cache_key_160bit, sizeof(cache_entry.key)); + cache_entry.crc = util_hash_crc32(blob, blob_size); + cache_entry.size = blob_size; + +- index_entry->hash = hash; +- index_entry->size = blob_size; +- index_entry->last_access_time = os_time_get_nano(); +- index_entry->cache_db_file_offset = ftell(db->cache.file); ++ index_entry.hash = hash; ++ index_entry.size = blob_size; ++ index_entry.last_access_time = os_time_get_nano(); ++ index_entry.cache_db_file_offset = ftell(db->cache.file); + ++ hash_entry = ralloc(db->mem_ctx, struct mesa_index_db_hash_entry); ++ if (!hash_entry) ++ goto fail; ++ ++ hash_entry->cache_db_file_offset = index_entry.cache_db_file_offset; ++ hash_entry->index_db_file_offset = ftell(db->index.file); ++ hash_entry->last_access_time = index_entry.last_access_time; ++ hash_entry->size = index_entry.size; ++ + if (!mesa_db_write(db->cache.file, &cache_entry) || + !mesa_db_write_data(db->cache.file, blob, blob_size) || +- !mesa_db_write(db->index.file, index_entry)) ++ !mesa_db_write(db->index.file, &index_entry)) + goto fail_fatal; + + fflush(db->cache.file); + fflush(db->index.file); + +- mesa_db_index_entry_insert(db, index_entry); +- + db->index.offset = ftell(db->index.file); + ++ _mesa_hash_table_u64_insert(db->index_db, hash, hash_entry); ++ + mesa_db_unlock(db); + + return true; +@@ -973,6 +847,9 @@ fail: + fail: + mesa_db_unlock(db); + ++ if (hash_entry) ++ ralloc_free(hash_entry); ++ + return false; + } + +@@ -982,7 +859,7 @@ mesa_cache_db_entry_remove(struct mesa_cache_db *db, + { + uint64_t hash = to_mesa_cache_db_hash(cache_key_160bit); + struct mesa_cache_db_file_entry cache_entry; +- struct mesa_index_db_file_entry *index_entry; ++ struct mesa_index_db_hash_entry *hash_entry; + + if (!mesa_db_lock(db)) + return NULL; +@@ -996,11 +873,11 @@ mesa_cache_db_entry_remove(struct mesa_cache_db *db, + if (!mesa_db_update_index(db)) + goto fail_fatal; + +- index_entry = mesa_db_index_entry_search(db, hash); +- if (!index_entry) ++ hash_entry = _mesa_hash_table_u64_search(db->index_db, hash); ++ if (!hash_entry) + goto fail; + +- if (!mesa_db_seek(db->cache.file, index_entry->cache_db_file_offset) || ++ if (!mesa_db_seek(db->cache.file, hash_entry->cache_db_file_offset) || + !mesa_db_read(db->cache.file, &cache_entry) || + !mesa_db_cache_entry_valid(&cache_entry)) + goto fail_fatal; +@@ -1008,7 +885,7 @@ mesa_cache_db_entry_remove(struct mesa_cache_db *db, + if (memcmp(cache_entry.key, cache_key_160bit, sizeof(cache_entry.key))) + goto fail; + +- if (!mesa_db_compact(db, 0, index_entry)) ++ if (!mesa_db_compact(db, 0, hash_entry)) + goto fail_fatal; + + mesa_db_unlock(db); +@@ -1066,8 +943,7 @@ mesa_cache_db_eviction_score(struct mesa_cache_db *db) + mesa_cache_db_eviction_score(struct mesa_cache_db *db) + { + int64_t eviction_size = mesa_cache_db_eviction_size(db); +- struct mesa_index_db_file_entry *index_entry; +- struct sort_entry *entries; ++ struct mesa_index_db_hash_entry **entries; + unsigned num_entries, i = 0; + double eviction_score = 0; + +@@ -1085,16 +961,15 @@ mesa_cache_db_eviction_score(struct mesa_cache_db *db) + if (!entries) + goto fail; + +- for (i = 0, index_entry = db->index_entries; i < num_entries; i++) +- entries[i].index_entry = index_entry++; ++ hash_table_foreach(db->index_db->table, entry) ++ entries[i++] = entry->data; + + util_qsort_r(entries, num_entries, sizeof(*entries), + entry_sort_lru, db); + + for (i = 0; eviction_size > 0 && i < num_entries; i++) { +- index_entry = entries[i].index_entry; +- uint64_t entry_age = os_time_get_nano() - index_entry->last_access_time; +- unsigned entry_size = blob_file_size(index_entry->size); ++ uint64_t entry_age = os_time_get_nano() - entries[i]->last_access_time; ++ unsigned entry_size = blob_file_size(entries[i]->size); + + /* Eviction score is a sum of weighted cache entry sizes, + * where weight doubles for each month of entry's age. +--- src/util/mesa_cache_db.h.orig 2024-10-30 21:04:20 UTC ++++ src/util/mesa_cache_db.h +@@ -32,8 +32,6 @@ struct mesa_cache_db { + struct hash_table_u64 *index_db; + struct mesa_cache_db_file cache; + struct mesa_cache_db_file index; +- void *index_entries; +- size_t index_entries_size; + uint64_t max_cache_size; + simple_mtx_t flock_mtx; + void *mem_ctx; +--- src/util/mesa_cache_db_multipart.c.orig 2024-10-30 21:04:20 UTC ++++ src/util/mesa_cache_db_multipart.c +@@ -18,109 +18,67 @@ mesa_cache_db_multipart_open(struct mesa_cache_db_mult + #if DETECT_OS_WINDOWS + return false; + #else ++ char *part_path = NULL; ++ unsigned int i; ++ + db->num_parts = debug_get_num_option("MESA_DISK_CACHE_DATABASE_NUM_PARTS", 50); +- db->cache_path = cache_path; ++ + db->parts = calloc(db->num_parts, sizeof(*db->parts)); + if (!db->parts) + return false; + +- simple_mtx_init(&db->lock, mtx_plain); ++ for (i = 0; i < db->num_parts; i++) { ++ bool db_opened = false; + +- return true; +-#endif +-} ++ if (asprintf(&part_path, "%s/part%u", cache_path, i) == -1) ++ goto close_db; + +-static bool +-mesa_cache_db_multipart_init_part_locked(struct mesa_cache_db_multipart *db, +- unsigned int part) +-{ +-#if DETECT_OS_WINDOWS +- return false; +-#else +- struct mesa_cache_db *db_part; +- bool db_opened = false; +- char *part_path = NULL; ++ if (mkdir(part_path, 0755) == -1 && errno != EEXIST) ++ goto free_path; + +- if (db->parts[part]) +- return true; ++ /* DB opening may fail only in a case of a severe problem, ++ * like IO error. ++ */ ++ db_opened = mesa_cache_db_open(&db->parts[i], part_path); ++ if (!db_opened) ++ goto free_path; + +- if (asprintf(&part_path, "%s/part%u", db->cache_path, part) == -1) +- return false; +- +- if (mkdir(part_path, 0755) == -1 && errno != EEXIST) +- goto free_path; +- +- db_part = calloc(1, sizeof(*db_part)); +- if (!db_part) +- goto free_path; +- +- /* DB opening may fail only in a case of a severe problem, +- * like IO error. +- */ +- db_opened = mesa_cache_db_open(db_part, part_path); +- if (!db_opened) { +- free(db_part); +- goto free_path; ++ free(part_path); + } + +- if (db->max_cache_size) +- mesa_cache_db_set_size_limit(db_part, db->max_cache_size / db->num_parts); +- + /* remove old pre multi-part cache */ +- mesa_db_wipe_path(db->cache_path); ++ mesa_db_wipe_path(cache_path); + +- __sync_synchronize(); ++ return true; + +- db->parts[part] = db_part; +- + free_path: + free(part_path); ++close_db: ++ while (i--) ++ mesa_cache_db_close(&db->parts[i]); + +- return db_opened; ++ free(db->parts); ++ ++ return false; + #endif + } + +-static bool +-mesa_cache_db_multipart_init_part(struct mesa_cache_db_multipart *db, +- unsigned int part) +-{ +- bool ret; +- +- if (db->parts[part]) +- return true; +- +- simple_mtx_lock(&db->lock); +- ret = mesa_cache_db_multipart_init_part_locked(db, part); +- simple_mtx_unlock(&db->lock); +- +- return ret; +-} +- + void + mesa_cache_db_multipart_close(struct mesa_cache_db_multipart *db) + { +- while (db->num_parts--) { +- if (db->parts[db->num_parts]) { +- mesa_cache_db_close(db->parts[db->num_parts]); +- free(db->parts[db->num_parts]); +- } +- } ++ while (db->num_parts--) ++ mesa_cache_db_close(&db->parts[db->num_parts]); + + free(db->parts); +- simple_mtx_destroy(&db->lock); + } + + void + mesa_cache_db_multipart_set_size_limit(struct mesa_cache_db_multipart *db, + uint64_t max_cache_size) + { +- for (unsigned int part = 0; part < db->num_parts; part++) { +- if (db->parts[part]) +- mesa_cache_db_set_size_limit(db->parts[part], *** 142 LINES SKIPPED ***