Date: Mon, 24 Aug 2020 20:43:00 +0000 From: bugzilla-noreply@freebsd.org To: elastic@FreeBSD.org Subject: [Bug 248889] textproc/elasticsearch7: Update to 7.9.0 Message-ID: <bug-248889-37421@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248889 Bug ID: 248889 Summary: textproc/elasticsearch7: Update to 7.9.0 Product: Ports & Packages Version: Latest Hardware: Any URL: https://www.elastic.co/guide/en/elasticsearch/referenc e/7.9/release-notes-7.9.0.html OS: Any Status: New Severity: Affects Only Me Priority: --- Component: Individual Port(s) Assignee: elastic@FreeBSD.org Reporter: juraj@lutter.sk Assignee: elastic@FreeBSD.org Flags: maintainer-feedback?(elastic@FreeBSD.org) Attachment #217505 maintainer-approval+ Flags: Created attachment 217505 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D217505&action= =3Dedit textproc/elasticsearch7: Update to 7.9.0 Hi, please find the file attached. Changelog: Security updates * A field disclosure flaw was found in Elasticsearch when running a scrolling search with field level security. If a user runs the same query another more privileged user recently ran, the scrolling search can leak fields that should be hidden. This could result in an attacker gaining additional permissions against a restricted index. All versions of Elasticsearch before 7.9.0 and 6.8.12 are affected by this flaw. You must upgrade to Elasticsearch version 7.9.0 or 6.8.12 to obtain the fix. Known issues * Upgrading to 7.9.0 from an earlier version will result in incorrect mappings on the machine learning annotations index, and possibly also on the machine learning config index. This will lead to some pages in the machine learning UI not displaying correctly, and may prevent machine learning jobs being created or updated. The best way to avoid this problem if you read about this known issue before upgrading is to manually update the mappings on these indices in your old Elasticsearch version before upgrading to 7.9.0. If you find out about the issue after upgrading then reindexing is required to recover. Full details of the mitigations are in Upgrade to 7.9.0 causes incorrect mappings article. Breaking changes Script Cache + Script cache size and rate limiting are per-context Field capabilities API + Constant_keyword fields are now described by their family type keyword instead of constant_keyword Snapshot restore throttling + Restoring from a snapshot (which is a particular form of recovery) is now properly taking recovery throttling into account (i.e. the indices.recovery.max_bytes_per_sec setting). The max_restore_bytes_per_sec setting is also now defaulting to unlimited, whereas previously it was set to 40mb, which is the default that's used for indices.recovery.max_bytes_per_sec. This means that no behavioral change will be observed by clusters where the recovery and restore settings had not been adapted from the defaults. Thread pool write queue size + The WRITE thread pool default queue size (thread_pool.write.size) has been increased from 200 to 10000. A small queue size (200) caused issues when users wanted to send small indexing requests with a high client count. Additional memory-oriented back pressure has been introduced with the indexing_pressure.memory.limit setting. This setting configures a limit to the number of bytes allowed to be consumed by outstanding indexing requests. Dangling indices + Automatically importing dangling indices is now deprecated, disabled by default, and will be removed in Elasticsearch 8.0. See the Breaking Java changes Aggregations + Improve cardinality measure used to build aggs Features/Ingest + Add optional description parameter to ingest processors. New features Aggregations + Add moving percentiles pipeline aggregation + Add normalize pipeline aggregation + Add variable width histogram aggregation + Add pipeline inference aggregation + Speed up time interval arounding around daylight savings time (DST) Geo + Override doc_value parameter in Spatial XPack module Machine Learning + Add update data frame analytics jobs API + Introduce model_plot_config.annotations_enabled setting for anomaly detection jobs + Report significant changes to anomaly detection models in annotations of the results Mapping + Merge mappings for composable index templates + Wildcard field optimised for wildcard queries Search + Allow index filtering in field capabilities API Enhancements Aggregations + Add support for numeric range keys + Added standard deviation / variance sampling to extended stats + Give significance lookups their own home + Increase search.max_buckets to 65,535 + Optimize date_histograms across daylight savings time + Return clear error message if aggregation type is invalid + Save memory on numeric significant terms when not top + Save memory when auto_date_histogram is not on top + Save memory when date_histogram is not on top (issues: + Save memory when histogram agg is not on top + Save memory when numeric terms agg is not top + Save memory when parent and child are not on top + Save memory when rare_terms is not on top + Save memory when significant_text is not on top + Save memory when string terms are not on top + Speed up reducing auto_date_histo with a time zone + Speed up rounding in auto_date_histogram Allocation + Account for remaining recovery in disk allocator Analysis + Add max_token_length setting to the CharGroupTokenizer + Expose discard_compound_token option to kuromoji_tokenizer + Support multiple tokens on LHS in stemmer_override rules ( Authentication + Add http proxy support for OIDC realm + Improve threadpool usage and error handling for API key validation + Support handling LogoutResponse from SAML idP (issues: Authorization + Add cache for application privileges + Add monitor and view_index_metadata privileges to built-in kibana_system role + Improve role cache efficiency for API key roles CCR + Allow follower indices to override leader settings CRUD + Retry failed replication due to transient errors Engine + Don't log on RetentionLeaseSync error handler after an index has been deleted Features/Data streams + Add support for snapshot and restore to data streams + Data stream creation validation allows for prefixed indices + Disallow deletion of composable template if in use by data stream + Validate alias operations don't target data streams Features/ILM+SLM + Add data stream support to searchable snapshot action + Add data stream support to the shrink action + Add support for rolling over data streams + Check the managed index is not a data stream write index Features/Indices APIs + Add default composable templates for new indexing strategy + Add index block api + Add new flag to check whether alias exists on remove + Add prefer_v2_templates parameter to reindex + Add template simulation API for simulating template composition Features/Ingest + Add ignore_empty_value parameter in set ingest processor + Support if_seq_no and if_primary_term for ingest Features/Java High Level REST Client + Add support for data streams + Enable decompression of response within LowLevelRestClient Features/Java Low Level REST Client + Add isRunning method to RestClient + Add RequestConfig support to RequestOptions Infra/Circuit Breakers + Enhance real memory circuit breaker with G1 GC Infra/Core + Introduce node.roles setting Infra/Packaging + Remove DEBUG-level logging from actions in Docker (issues: Infra/Plugins + Improved ExtensiblePlugin Infra/Resiliency + Adds resiliency to read-only filesystems Machine Learning + Accounting for model size when models are not cached. + Adds new for_export flag to GET _ml/inference API + Adds WKT geometry detection in find_file_structure + Calculate cache misses for inference and return in stats + Delete auto-generated annotations when job is deleted. + Delete auto-generated annotations when model snapshot is reverted + Delete expired data by job + Introduce Annotation.event field + Add support for larger forecasts in memory via max_model_memory setting + Don't lose precision when saving model state + Parallelize the feature importance calculation for classification and regression over trees + Add an option to do categorization independently for each partition + Memory usage is reported during job initialization + More realistic memory estimation for classification and regression means that these analyses will require lower memory limits than before + Checkpoint state to allow efficient failover during coarse parameter search for classification and regression + Improve data access patterns to speed up classification and regression + Performance improvements for classification and regression, particularly running multithreaded + Improve runtime and memory usage training deep trees for classification and regression + Improvement in handling large inference model definitions + Add a peak_model_bytes field to model_size_stats Mapping + Add regex query support to wildcard field + Make keyword a family of field types + Store parsed mapping settings in IndexSettings + Wildcard field - add support for custom null values Network + Make the number of transport threads equal to the number of available CPUs Recovery + Implement dangling indices API + Reestablish peer recovery after network errors + Sending operations concurrently in peer recovery Reindex + Throw an illegal_argument_exception when max_docs is less than slices SQL + Implement TIME_PARSE function for parsing strings into TIME values + Implement TOP as an alternative to LIMIT + Implement TRIM function + Improve performances of LTRIM/RTRIM + Make CASTing string to DATETIME more lenient + Redact credentials in connection exceptions + Relax parsing of date/time escaped literals + Add support for scalars within LIKE/RLIKE Search + Add description to submit and get async search, as well as cancel tasks + Add matchBoolPrefix static method in query builders + Add range query support to wildcard field + Group docIds by segment in FetchPhase to better use LRU cache + Improve error handling when decoding async execution ids + Specify reason whenever async search gets cancelled + Use index sort range query when possible. Security + Add machine learning admin permissions to the kibana_system role + Just log 401 stacktraces Snapshot/Restore + Deduplicate Index Metadata in BlobStore + Default to zero replicas for searchable snapshots + Enable fully concurrent snapshot operations + Support cloning of searchable snapshot indices + Track GET/LIST Azure Storage API calls + Track GET/LIST GoogleCloudStorage API calls + Track PUT/PUT_BLOCK operations on AzureBlobStore. + Track multipart/resumable uploads GCS API calls + Track upload requests on S3 repositories Task Management + Add index name to refresh mapping task + Cancel task and descendants on channel disconnects (issues: Transform + Add support for terms agg in transforms + Adds geotile_grid support in group_by Bug fixes Aggregations + Fix auto_date_histogram interval + Fix bug in faster interval rounding + Fix bug in parent and child aggregators when parent field not defined + Fix missing null values for std_deviation_bounds in ext. stats aggs Allocation + Reword INDEX_READ_ONLY_ALLOW_DELETE_BLOCK message (issues: Authentication + Map only specific type of OIDC Claims Authorization + Change privilege of enrich stats API to monitor Engine + Fix local translog recovery not updating safe commit in edge case + Hide AlreadyClosedException on IndexCommit release Features/ILM+SLM + Normalized prefix for rollover API Features/Indices APIs + Don't allow invalid template combinations + Handle cluster.max_shards_per_node in YAML config Features/Ingest + Fix ingest simulate verbose on failure with conditional Geo + Check for degenerated lines when calculating the centroid + Fix bug in circuit-breaker check for geoshape grid aggregations Infra/Scripting + Fix source return bug in scripting Machine Learning + Fix wire serialization for flush acknowledgements + Make waiting for renormalization optional for internally flushing job + Tail the C++ logging pipe before connecting other pipes + Fix numerical issues leading to blow up of the model plot bounds + Fix causes for inverted forecast confidence interval bounds + Restrict growth of max matching string length for categories Mapping + Wildcard field fix for scripts - changed value type from BytesRef to String SQL + Introduce JDBC option for meta pattern escaping Search + Don't omit empty arrays when filtering _source (issues: + Fix casting of scaled_float in sorts Snapshot/Restore + Account for recovery throttling when restoring snapshot + Fix noisy logging during snapshot delete + Fix S3ClientSettings leak Upgrades Search + Update to lucene snapshot e7c625430ed Testport result on 12-STABLE: https://freebsd-stable.builder.wilbury.net/data/12_STABLE_GENERIC_amd64-def= ault/2020-08-23_09h47m41s/logs/elasticsearch7-7.9.0.log Testport result on 13-CURRENT: https://freebsd-current.builder.wilbury.net/data/13_CURRENT_GENERIC_amd64-d= efault/2020-08-23_09h50m26s/logs/elasticsearch7-7.9.0.log --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-248889-37421>