From owner-freebsd-elastic@freebsd.org Tue Nov 24 20:59:51 2020 Return-Path: Delivered-To: freebsd-elastic@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 87E47469CAB for ; Tue, 24 Nov 2020 20:59:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4Cgbxg3Ffyz3Hj0 for ; Tue, 24 Nov 2020 20:59:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 6F888469C5F; Tue, 24 Nov 2020 20:59:51 +0000 (UTC) Delivered-To: elastic@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6F4DA469CAA for ; Tue, 24 Nov 2020 20:59:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cgbxg2fcwz3HWP for ; Tue, 24 Nov 2020 20:59:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4E18E252DF for ; Tue, 24 Nov 2020 20:59:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AOKxpaF028364 for ; Tue, 24 Nov 2020 20:59:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0AOKxpIB028363 for elastic@FreeBSD.org; Tue, 24 Nov 2020 20:59:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" From: bugzilla-noreply@freebsd.org To: elastic@FreeBSD.org Subject: maintainer-feedback requested: [Bug 251354] sysutils/beats7: Update to 7.10.0 Date: Tue, 24 Nov 2020 20:59:51 +0000 X-Bugzilla-Type: request X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: elastic@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? Message-ID: In-Reply-To: References: X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-elastic@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Support of ElasticSearch-related ports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Nov 2020 20:59:51 -0000 Bugzilla Automation has asked freebsd-elastic (Nobod= y) for maintainer-feedback: Bug 251354: sysutils/beats7: Update to 7.10.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D251354 --- Description --- Hi, please find the patch attached. Breaking changes Affecting all Beats * Added certificate TLS verification mode to ignore server name mismatch. * Remove redundant cloudfoundry.*.timestamp fields. This value is set = in @timestamp. * Allow embedding of CAs, Certificate of private keys for anything that supports TLS in outputs and inputs * API address is a required setting in add_cloudfoundry_metadata. Auditbeat * Change network.direction values to ECS recommended values (inbound, outbound). * Docker container needs to be explicitly run as user root for auditin= g. * File integrity dataset no longer includes the leading dot in file.extension values (e.g. it will report "png" instead of ".png") = to comply with ECS. Filebeat * Cisco * CrowdStrike * Fortinet * iptables * Checkpoint * Netflow * Zeek (forwarded tag is not included by default) * Suricata (forwarded tag is not included by default) * CoreDNS (forwarded tag is not included by default) * Envoy Proxy (forwarded tag is not included by default) * Move file metrics to dataset endpoint * Fix PANW field spelling "veredict" to "verdict" on event.action * Tracking session end reason in panw module. * API address and shard ID are required settings in the Cloud Foundry input. Heartbeat Journalbeat Metricbeat * Remove "invalid zero" metrics on Windows and Darwin, don't report linux-only memory and disk I/O metrics when running under agent. * API address and shard ID are required settings in the Cloud Foundry module. Packetbeat Winlogbeat Functionbeat Bugfixes Affecting all Beats * Remove unnecessary restarts of metricsets while using Node autodiscover * [Metricbeat][Kubernetes] Change cluster_ip field from ip to keyword. * [Autodiscover] Handle input-not-finished errors in config reload. * Orderly close processors when processing pipelines are not needed anymore to release their resources. * Fix parsing of expired licences. Auditbeat * auditd: Fix spelling of anomaly in event.category. * auditd: Fix typo in event.action of removed-user-role-from. * auditd: Fix typo in event.action of used-suspicious-link. Filebeat * Fix mapping of fortinet.firewall.mem as integer. * Fix auditd module syscall table for ppc64 and ppc64le. * Fix Filebeat OOMs on very long lines, * Ignore missing in Zeek module when dropping unecessary fields. * Fix event.outcome logic for azure/siginlogs fileset * Improve validation checks for Azure configuration * Fix event.kind for system/syslog pipeline * Fix event.type for zeek/ssl and duplicate event.category for zeek/connection * Remove wrongly mapped tls.client.server_name from fortinet/firewall fileset. * Handle multiple upstreams in ingress-controller. * Provide backwards compatibility for the append processor when Elasticsearch is less than.10.0. * Fix checkpoint module when logs contain time field. * Fix syslog RFC parsing in the CheckPoint module. * Fix incorrect connection state mapping in zeek connection pipeline. * Fix for field [source] not present as part of path [source.ip] error in azure pipelines. * Fix handing missing eventtime and assignip field being set to N/A for fortinet module. Heartbeat * Add support for new service_name option to all monitors.. Journalbeat Metricbeat * Add support for azure light metricset app_stats. * Fix ec2 disk and network metrics to use Sum statistic method. * Fix ec2 disk and network metrics to use Sum statistic method. * Update fields.yml in the azure module, missing metrics field. * Disable Kafka metricsets based on Jolokia by default. They require a different configuration. * Fix timestamp handling in remote_write. * Visualization title fixes in aws, azure and googlecloud compute dashboards. * Fix retrieving resources by ID for the azure module. * Use timestamp from CloudWatch API when creating events. * Report the correct windows events for system/filesystem * Fix regular expression in windows/permfon. * Fix azure storage event format. * Fix panic in kubernetes autodiscover related to keystores * [Kubernetes] Remove redundant dockersock volume mount * Revert change to report process.memory.rss as process.memory.wss on Windows. * Add interval information to monitor metricset in azure. * Remove io.time from windows * Fix instance name in perfmon metricset. Packetbeat * Add "network" to event.category Winlogbeat * Fix invalid IP addresses in DNS query results from Sysmon data. * Fix event.outcome in the security module for non-English languages. * Fields from Winlogbeat modules were not being included in index templates and patterns. * Protect against accessing undefined variables in Sysmon module. Functionbeat * Fix catchall bucket config errors by adding more validation. * Fix Google Cloud Function configuration issue. Added Affecting all Beats * Add minimum cache TTL for successful DNS responses. * Add support for DNS over TLS for the dns processor. * Add leader election for Kubernetes autodiscover. * Add capability of enriching process metadata with container id also for non-privileged containers in add_process_metadata processor. * Add replace_fields config option in add_host_metadata for replacing host fields. * Add ingress controller dashboards. * Added experimental citrix module. * Added experimental cyberark module. * Added experimental proofpoint module. * Added experimental snort module. * Added experimental symantec module. * Added experimental dataset barracuda/spamfirewall. * Added experimental dataset cisco/meraki. * Added experimental dataset f5/bigipafm. * Added experimental dataset fortinet/fortimail. * Added experimental dataset fortinet/fortimanager. * Added experimental dataset juniper/netscreen. * Added experimental dataset sophos/utm. * Add Cloud Foundry tags in related events. * Cloud Foundry metadata is cached to disk. * Add option to select the type of index template to load: legacy, component, index. * Release add_cloudfoundry_metadata as GA. * Added Kafka version.2 to the list of supported versions. Auditbeat * Add enrichment of auditd seccomp events with name of the architectur= e, syscall, and signal. Filebeat * Add support for reading auditd logs that are prefixed with node=3D. * Add event.ingested to all Filebeat modules. * Add event.ingested for Suricata module * Add support for custom header and headersecret for filebeat http_endpoint input * Convert httpjson to v2 input * Add event.ingested to all Filebeat modules. * Return error when log harvester tries to open a named pipe. * Avoid goroutine leaks in Filebeat readers. * Improve Zeek x509 module with x509 ECS mappings * Improve Zeek SSL module with x509 ECS mappings * Added new properties field support for event.outcome in azure module * Improve Zeek Kerberos module with x509 ECS mappings * Improve Fortinet firewall module with x509 ECS mappings * Improve Santa module with x509 ECS mappings * Improve Suricata Eve module with x509 ECS mappings * Added new module for Zoom webhooks * Add type and sub_type to panw panos fileset * Always attempt community_id processor on zeek module * Add related.hosts ecs field to all modules * Keep cursor state between httpjson input restarts * Convert aws s3 to v2 input * Add support for additional fields from V2 ALB logs. * Release Cloud Foundry input as GA. * New Cisco Umbrella dataset * New juniper.srx dataset for Juniper SRX logs. * Adding support for Microsoft Defender (Microsoft Threat Protection) * Adding support for FIPS in s3 input * Update Okta documentation for new stateful restarts. Heartbeat * Add index and pipeline settings to monitor configurations. Journalbeat Metricbeat * Add state_statefulset metricset to Metricbeat recommended configuration for k8s. * Infer types in Prometheus remote_write. * Add cloud.instance.name into aws ec2 metricset. * Add host inventory metrics into aws ec2 metricset. * Add scope setting for Elasticsearch module, allowing it to monitor an Elasticsearch cluster behind a load-balancing proxy. * Add state_daemonset metricset for Kubernetes Metricbeat module * Add host inventory metrics to googlecloud compute metricset. * Add host inventory metrics to azure compute_vm metricset. * Add host inventory metrics to system module. * Add billing data collection from Cost Explorer into aws billing metricset. * Migrate compute_vm metricset to a light one, map cloud.instance.id field. * Request prometheus endpoints to be gzipped by default * Add latency config parameter into aws module. * Add billing metricset into googlecloud module. * Release all kubernetes state metricsets as GA * Move compute_vm_scaleset to light metricset. * Sanitize event.host. * Add support for different Azure Cloud environments in the metricbeat azure module. * Add overview and platform health dashboards to Cloud Foundry module. * Release lambda metricset in aws module as GA. * Add dashboard for pubsub metricset in googlecloud module. * Move Prometheus query & remote_write to GA. * Map cloud data filed cloud.account.id to azure subscription. * Expand unsupported option from namespace to metrics in the azure module. Packetbeat * Add an example to packetbeat.yml of using the forwarded tag to disab= le * Add-continue support * Add initial SIP protocol support Functionbeat Winlogbeat Elastic Log Driver - Add support to change beat name, and support for Kibana Logs. Deprecated * N/A Testport on 11.4, 12.1, 12.2, 13.0 OK