Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 24 Aug 2020 21:32:35 +0000
From:      bugzilla-noreply@freebsd.org
To:        elastic@FreeBSD.org
Subject:   [Bug 248893] sysutils/logstash7: Update to 7.9.0
Message-ID:  <bug-248893-37421@https.bugs.freebsd.org/bugzilla/>

next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248893

            Bug ID: 248893
           Summary: sysutils/logstash7: Update to 7.9.0
           Product: Ports & Packages
           Version: Latest
          Hardware: Any
               URL: https://www.elastic.co/guide/en/logstash/current/logst
                    ash-7-9-0.html
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: Individual Port(s)
          Assignee: elastic@FreeBSD.org
          Reporter: juraj@lutter.sk
          Assignee: elastic@FreeBSD.org
             Flags: maintainer-feedback?(elastic@FreeBSD.org)
 Attachment #217509 maintainer-approval+
             Flags:

Created attachment 217509
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D217509&action=
=3Dedit
sysutils/logstash7: Update to 7.9.0

Hi,

please find the patch attached.

Changelog
New features and enhancements

ECS support in Elasticsearch output plugin

   This release is the first step toward Elastic Common Schema (ECS)
   support in Logstash. With 7.9, you can configure the Elasticsearch
   output plugin to manage index templates that are compatible with the
   Elastic Common Schema (ECS). The ECS compatibility setting in
   the Elasticsearch output plugin makes this possible.

   See Compatibility with the Elastic Common Schema (ECS) in the
   Elasticsearch output plugin docs for more information.

Expanded API key support

   With this release, we've continued expanding support for Elasticsearch
   API keys. Support for API keys in the Elasticsearch output plugin
   arrived in Logstash 7.8.0. Logstash 7.9.0 introduces support for
   Elasticsearch API keys in the Elasticsearch input plugin, the
   Elasticsearch filter plugin, and Logstash monitoring and
   management.

   Check out Grant access using API keys for more information about
   using API keys with Logstash and Elasticsearch. Implementation details
   are in

ARM64 support (experimental)

   Logstash runs on arm machines! We have tested Logstash against arm64,
   and we are looking to make docker and other images available soon.

   ARM artifacts are not yet supported for production, and we're offering
   them as "experimental" to early adopters.

Improved support in App Search output

   We replaced the deprecated Java client library for the Elastic App
   Search output plugin with the Ruby client library, and expanded
   integration testing. These changes provide a foundation for expanding
   App Search integration and quality assurance in future releases.

Improvements to persistent queue (PQ)

   We've enhanced persistent queues to better manage exceptions and error
   handling which could sometimes result in a LockException when the queue
   file lock was not properly released. Under some conditions, a complex
   pipeline that is slower to initialize could be recreated when it was
   not done initializing, causing a LockException. Implementation details
   are in

   These changes result in better stability of persistent queues.

Improvements to pipeline workers error handling

   Worker threads were not correctly monitored for a worker loop exception
   resulting in a complete logstash crash upon any exception even when
   multiple pipelines are running. Now only the failed pipeline is
   terminated. If pipeline reloading is enabled, you can edit the config
   and have the failed pipeline reloaded. Implementation details are in
    and

Performance improvement on startup and pipeline restarts

   This release contains several optimizations to pipeline compilation, an
   essential step of the pipeline initialization process. These changes
   significantly improve startup and pipeline-restart performance for
   complex pipelines. (For technical details, check out this PR:
   )

   From our tests in three different pipelines with eight workers each, we
   have seen times decrease from 9 - 28 minutes to around 1 minute.

   To aid the development of pipelines, especially the performance impact
   of compilation, Logstash now reports the time taken to compile each
   pipeline as a log entry such as:
[2020-08-12T14:10:29,[INFO ][logstash.javapipeline  ][main] Pipeline Java ex
ecution initialization time {"seconds"=3D>0.7}

Performance improvements and notable issues fixed

     * Support white space as a delimiter on list-type params
     * Support using unix pipe as local config file
     * Logging improvements
          + Display Java pipeline initialization time to help with
            troubleshooting and diagnostics
          + Logging framework enhancement to allow more finetuned logging

          + Better logging after definition improvements and script routes
            in log4j  and
          + Improved Logstash startup logging to ensure that starting
            logstash entry happens before any other log entries
     * Fix: Add back pipelines queue.data and queue.capacity subdocuments
       for _node/stats
     * Fix: Avoid reloading pipelines that have no changes
     * Fix: Removed unnecessary calls that, under some circumstances,
       could cause pipeline startup issues for pipelines that were slow to
      initialize
     * Fix: Allow trailing newlines in config fragments to resolve an
       issue in which split configs were corrupted when merged
     * Fix: Resolve issue in which pipeline init fails for a slow pipeline
       when monitoring is enabled
     * Fix: Ignore default username when no password is set for monitoring
       and management
     * Refactor code refactor to launch ruby thread from ruby code instead
       of java (as a workaround for jruby bug)
     * Updates to dependencies
          + Update log4j dependency to 2.13.3
          + Update jruby to 9.2.12.0

Plugin releases

   Rubydebug Codec - 3.1.0
     * Replace stale awesome_print library with maintained fork called
       amazing_print

   Elasticsearch Filter - 3.9.0
     * Add support to define a proxy with the proxy config option
     * Added api_key support
     * [DOC] Removed outdated compatibility notice

   Memcached Filter - 1.1.0
     * Added better exception handling

   Elasticsearch Input - 4.7.0
     * Added api_key support

   File Input - 4.2.1
     * Fix: Skip sincedb eviction if read mode completion deletes file
       during flush
     * Fix: Watched files performance with huge filesets
     * Updated logging to include full traces in debug (and trace) levels

   Imap Input - 3.1.0
     * Adds an option to recursively search the message parts for
       attachment and inline attachment filenames. If the save_attachments
       option is set to true, the content of attachments is included in
       the attachments.data field. The attachment data can then be used by
       the Elasticsearch Ingest Attachment Processor Plugin

   Kafka Integration - 10.4.0
     * Added the input isolation_level to allow fine control of whether to
       return transactional messages
     * Added the input and output client_dns_lookup parameter to allow
       control of how DNS requests are made

   Rabbitmq Integration - 7.1.0
     * Added support in Output plugin for sprintf templates in values
       provided to message_properties
     * Added support for extended metadata including raw payload to events
       generated by the Input Plugin
     * Fixes an issue with custom port assignment, in which the custom
       port was not being applied when more than one host was supplied

     * Fixes bug where attempting to read from undeclared exchange
       resulted in infinite retry loop
     * Fixes bug where failing to establish initial connection resulted in
       a pipeline that refused to shut down

   Elastic_app_search Output - 1.1.0
     * Switched AppSearch client library from Java to Ruby
     * Covered with integration tests and dockerized local AppSearch
       server instance.

   Elasticsearch Output - 10.6.1
     * Fixed an issue introduced in 10.6.0 that broke Logstash Core's
       monitoring feature when this plugin is run in Logstash 7.7-7.8.

     * Added ecs_compatiblity mode, for managing ECS-compatable templates


Testport on 12-STABLE:
https://freebsd-stable.builder.wilbury.net/data/12_STABLE_GENERIC_amd64-def=
ault/2020-08-23_09h37m26s/logs/logstash7-7.9.0.log

Testport on 13-CURRENT:
https://freebsd-current.builder.wilbury.net/data/13_CURRENT_GENERIC_amd64-d=
efault/2020-08-23_09h41m30s/logs/logstash7-7.9.0.log

--=20
You are receiving this mail because:
You are the assignee for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-248893-37421>