From owner-freebsd-elastic@freebsd.org Mon Aug 24 21:32:35 2020 Return-Path: Delivered-To: freebsd-elastic@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C27AF3CDF54 for ; Mon, 24 Aug 2020 21:32:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Bb51v3ZyTz4Jcp for ; Mon, 24 Aug 2020 21:32:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 7B0333CE13A; Mon, 24 Aug 2020 21:32:35 +0000 (UTC) Delivered-To: elastic@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7ACD33CE03E for ; Mon, 24 Aug 2020 21:32:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bb51v2kGzz4JBs for ; Mon, 24 Aug 2020 21:32:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 40C8B1F158 for ; Mon, 24 Aug 2020 21:32:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 07OLWZZR059419 for ; Mon, 24 Aug 2020 21:32:35 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 07OLWZJm059418 for elastic@FreeBSD.org; Mon, 24 Aug 2020 21:32:35 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: elastic@FreeBSD.org Subject: [Bug 248893] sysutils/logstash7: Update to 7.9.0 Date: Mon, 24 Aug 2020 21:32:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: juraj@lutter.sk X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: elastic@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform bug_file_loc op_sys bug_status bug_severity priority component assigned_to reporter flagtypes.name attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-elastic@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Support of ElasticSearch-related ports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Aug 2020 21:32:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248893 Bug ID: 248893 Summary: sysutils/logstash7: Update to 7.9.0 Product: Ports & Packages Version: Latest Hardware: Any URL: https://www.elastic.co/guide/en/logstash/current/logst ash-7-9-0.html OS: Any Status: New Severity: Affects Only Me Priority: --- Component: Individual Port(s) Assignee: elastic@FreeBSD.org Reporter: juraj@lutter.sk Assignee: elastic@FreeBSD.org Flags: maintainer-feedback?(elastic@FreeBSD.org) Attachment #217509 maintainer-approval+ Flags: Created attachment 217509 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D217509&action= =3Dedit sysutils/logstash7: Update to 7.9.0 Hi, please find the patch attached. Changelog New features and enhancements ECS support in Elasticsearch output plugin This release is the first step toward Elastic Common Schema (ECS) support in Logstash. With 7.9, you can configure the Elasticsearch output plugin to manage index templates that are compatible with the Elastic Common Schema (ECS). The ECS compatibility setting in the Elasticsearch output plugin makes this possible. See Compatibility with the Elastic Common Schema (ECS) in the Elasticsearch output plugin docs for more information. Expanded API key support With this release, we've continued expanding support for Elasticsearch API keys. Support for API keys in the Elasticsearch output plugin arrived in Logstash 7.8.0. Logstash 7.9.0 introduces support for Elasticsearch API keys in the Elasticsearch input plugin, the Elasticsearch filter plugin, and Logstash monitoring and management. Check out Grant access using API keys for more information about using API keys with Logstash and Elasticsearch. Implementation details are in ARM64 support (experimental) Logstash runs on arm machines! We have tested Logstash against arm64, and we are looking to make docker and other images available soon. ARM artifacts are not yet supported for production, and we're offering them as "experimental" to early adopters. Improved support in App Search output We replaced the deprecated Java client library for the Elastic App Search output plugin with the Ruby client library, and expanded integration testing. These changes provide a foundation for expanding App Search integration and quality assurance in future releases. Improvements to persistent queue (PQ) We've enhanced persistent queues to better manage exceptions and error handling which could sometimes result in a LockException when the queue file lock was not properly released. Under some conditions, a complex pipeline that is slower to initialize could be recreated when it was not done initializing, causing a LockException. Implementation details are in These changes result in better stability of persistent queues. Improvements to pipeline workers error handling Worker threads were not correctly monitored for a worker loop exception resulting in a complete logstash crash upon any exception even when multiple pipelines are running. Now only the failed pipeline is terminated. If pipeline reloading is enabled, you can edit the config and have the failed pipeline reloaded. Implementation details are in and Performance improvement on startup and pipeline restarts This release contains several optimizations to pipeline compilation, an essential step of the pipeline initialization process. These changes significantly improve startup and pipeline-restart performance for complex pipelines. (For technical details, check out this PR: ) From our tests in three different pipelines with eight workers each, we have seen times decrease from 9 - 28 minutes to around 1 minute. To aid the development of pipelines, especially the performance impact of compilation, Logstash now reports the time taken to compile each pipeline as a log entry such as: [2020-08-12T14:10:29,[INFO ][logstash.javapipeline ][main] Pipeline Java ex ecution initialization time {"seconds"=3D>0.7} Performance improvements and notable issues fixed * Support white space as a delimiter on list-type params * Support using unix pipe as local config file * Logging improvements + Display Java pipeline initialization time to help with troubleshooting and diagnostics + Logging framework enhancement to allow more finetuned logging + Better logging after definition improvements and script routes in log4j and + Improved Logstash startup logging to ensure that starting logstash entry happens before any other log entries * Fix: Add back pipelines queue.data and queue.capacity subdocuments for _node/stats * Fix: Avoid reloading pipelines that have no changes * Fix: Removed unnecessary calls that, under some circumstances, could cause pipeline startup issues for pipelines that were slow to initialize * Fix: Allow trailing newlines in config fragments to resolve an issue in which split configs were corrupted when merged * Fix: Resolve issue in which pipeline init fails for a slow pipeline when monitoring is enabled * Fix: Ignore default username when no password is set for monitoring and management * Refactor code refactor to launch ruby thread from ruby code instead of java (as a workaround for jruby bug) * Updates to dependencies + Update log4j dependency to 2.13.3 + Update jruby to 9.2.12.0 Plugin releases Rubydebug Codec - 3.1.0 * Replace stale awesome_print library with maintained fork called amazing_print Elasticsearch Filter - 3.9.0 * Add support to define a proxy with the proxy config option * Added api_key support * [DOC] Removed outdated compatibility notice Memcached Filter - 1.1.0 * Added better exception handling Elasticsearch Input - 4.7.0 * Added api_key support File Input - 4.2.1 * Fix: Skip sincedb eviction if read mode completion deletes file during flush * Fix: Watched files performance with huge filesets * Updated logging to include full traces in debug (and trace) levels Imap Input - 3.1.0 * Adds an option to recursively search the message parts for attachment and inline attachment filenames. If the save_attachments option is set to true, the content of attachments is included in the attachments.data field. The attachment data can then be used by the Elasticsearch Ingest Attachment Processor Plugin Kafka Integration - 10.4.0 * Added the input isolation_level to allow fine control of whether to return transactional messages * Added the input and output client_dns_lookup parameter to allow control of how DNS requests are made Rabbitmq Integration - 7.1.0 * Added support in Output plugin for sprintf templates in values provided to message_properties * Added support for extended metadata including raw payload to events generated by the Input Plugin * Fixes an issue with custom port assignment, in which the custom port was not being applied when more than one host was supplied * Fixes bug where attempting to read from undeclared exchange resulted in infinite retry loop * Fixes bug where failing to establish initial connection resulted in a pipeline that refused to shut down Elastic_app_search Output - 1.1.0 * Switched AppSearch client library from Java to Ruby * Covered with integration tests and dockerized local AppSearch server instance. Elasticsearch Output - 10.6.1 * Fixed an issue introduced in 10.6.0 that broke Logstash Core's monitoring feature when this plugin is run in Logstash 7.7-7.8. * Added ecs_compatiblity mode, for managing ECS-compatable templates Testport on 12-STABLE: https://freebsd-stable.builder.wilbury.net/data/12_STABLE_GENERIC_amd64-def= ault/2020-08-23_09h37m26s/logs/logstash7-7.9.0.log Testport on 13-CURRENT: https://freebsd-current.builder.wilbury.net/data/13_CURRENT_GENERIC_amd64-d= efault/2020-08-23_09h41m30s/logs/logstash7-7.9.0.log --=20 You are receiving this mail because: You are the assignee for the bug.=