From owner-freebsd-ports@freebsd.org Thu May 24 12:48:01 2018 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9436EEF90E for ; Thu, 24 May 2018 12:48:01 +0000 (UTC) (envelope-from kpnemesis@gmail.com) Received: from mail-wm0-x242.google.com (mail-wm0-x242.google.com [IPv6:2a00:1450:400c:c09::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 332C369C07 for ; Thu, 24 May 2018 12:48:01 +0000 (UTC) (envelope-from kpnemesis@gmail.com) Received: by mail-wm0-x242.google.com with SMTP id f8-v6so4913287wmc.4 for ; Thu, 24 May 2018 05:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=AHCDGWWXYR6cMZdruOQCqLMm1gxYGqeI1SWYVk9hUu0=; b=asQfolje3vXGXkK4cYIxsInaMjZ18OSS6I6s+BPfkSU/8CqSp+ph1UJ4zjGg+fxJSI yZJRPU8eQ0rq6AEfhw0DvMzngao7SI6sEzfpd6SckmOn7LBrjKv2VeY0/GDF38JNeffB IaCwtbLFt5kSkJRFFfu6WMhGt0+OxXNqti8CqJJyEKCV4Nk9Sz+r36jK1SfMO6miZnJz PcRUSJruY9hUDU8WJyUFxNNjXWHyQFri1wZ4L8JcpTCCioC0jyXxj7evhMyaamB76fBt dBZzSepwHk74xqxPyZbRkyk1Kf2ei+zn25AmEpiEqGFAjfA/asS0i8GD24PUjonMawUn JjPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=AHCDGWWXYR6cMZdruOQCqLMm1gxYGqeI1SWYVk9hUu0=; b=uICg4zGaRIyv1f3to0MYepxakZnXQP2RZHr8mLDp9hhooeg6hX3GcDBnwf2Cjl6BpE QXgYqJ1dzyK8XxXXi93BEUfU+YR/OUaka4BZrAqnTFDNIk9mUkWiI28K1hLdd121O7N6 ebWHB+3WTQqkbxfyS0QrbwsnWwhYLUa+h5ATm1rZFoKE7x9RjnOPp0mCiGx8Vf2sbcQR fN81HZIp+ysmL7iMqNrTKG3nEz3KIhhU2oo+HiZI0ZcH+eLB4IOHa4m6lxgwrCL9dZWw iBI3QKq6QK1a1cAlKPBcUGwlOFmOtCz+r/5eAvQyiQKpq3TtizCaXX/wGNJ1jA1lcqxg wmHQ== X-Gm-Message-State: ALKqPwdSVFaxmz81qiNlZsGoTRDBSYmTIvQWbrY7UINCXbicCeykl8Jo nyrnskcELhMfUGrrzSY8+DLF/QJxW81ttbV0F1o= X-Google-Smtp-Source: AB8JxZocSDqNGI4bHMkcTsCzwatimirUhE34rH0+QRY8T+nnoywEHNrv0zQMK5mehSqX3hXVJoxq4rNpGOFFKR9GfLI= X-Received: by 2002:a50:a722:: with SMTP id h31-v6mr11843363edc.288.1527166080214; Thu, 24 May 2018 05:48:00 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a50:f114:0:0:0:0:0 with HTTP; Thu, 24 May 2018 05:47:39 -0700 (PDT) In-Reply-To: <4e0c6da9-1942-8a64-cd26-89c7f3cfe6c0@belgacom.net> References: <4e0c6da9-1942-8a64-cd26-89c7f3cfe6c0@belgacom.net> From: Kernel Panic Date: Thu, 24 May 2018 13:47:39 +0100 Message-ID: Subject: Re: Logstash failing to process messages To: Benny Goemans Cc: freebsd-ports@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2018 12:48:02 -0000 Thanks for getting back to me, yes I suspect it has something to do with my filters though I've no idea which one it could be as I'm filtering on beats and syslog inputs. As a work around I've just added a cron command to restart Logstash every morning at 01:00, though obviously that means I'm losing non-beat events whilst it restarts. Please let me know if upgrading to the latest versions helps you, if it doesn't then perhaps a PR needs to be filed. On 24 May 2018 at 11:25, Benny Goemans wrote: > I have seen the same issue. In my case however, I had about OOM caused by > parsing long grok patterns. I didn't have these in 5.3 either so I suspect > it's a memory leak somewhere. > I have since upgraded everything to 6.x and am waiting to see if the same > issue persists. > > Regards, > Benny Goemans > > On 23-05-2018 17:23, Kernel Panic wrote: > >> Hello, I'll just list the versions before I start: >> >> FreeBSD 11.1 >> >> Logstash 6.23 >> Elasticsearch 5.6.8 >> Kibana 5.6.8 >> >> The issue I'm having is that after a few days Logstash will stop >> processing >> any messages; I'm using the same config file that I used with Logstash >> 5.3.0 which worked without issue and was rock-solid. There's nothing in >> the >> Logstash log file apart from messages about a field in my Cisco logs being >> the wrong type and therefore failing to index, however this has always >> been >> the case. I have tried enabling the 'dead letter' feature in Logstash to >> process these Cisco logs but that just makes Logstash even more unstable. >> >> The Logstash service doesn't actually crash, it just stops processing >> messages and fails to respond to the restart command so I end up having to >> reboot the server. I should say though that Logstash continues to respond >> the the monitor API commands. >> >> I have tried updating all Logstash plugins however that has not fixed the >> issue. >> >> As I said, I never had any problems with Logstash 5.3.0 but the latest >> version (and version 5.6.8) just seem to become unstable after a few days. >> >> Any help is greatly appreciated. >> _______________________________________________ >> freebsd-ports@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-ports >> To unsubscribe, send any mail to "freebsd-ports-unsubscribe@freebsd.org" >> > > >