From owner-freebsd-questions@freebsd.org Mon Jul 12 05:44:29 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8AA00651345 for ; Mon, 12 Jul 2021 05:44:29 +0000 (UTC) (envelope-from serejk@febras.net) Received: from prima.febras.net (prima.febras.net [62.76.193.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "*.febras.net", Issuer "Sectigo RSA Organization Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNXlJ3ctTz3NkF for ; Mon, 12 Jul 2021 05:44:28 +0000 (UTC) (envelope-from serejk@febras.net) Received: from mail.febras.net (localhost [127.0.0.1]) by prima.febras.net ("FEB RAS network Mail Server") with ESMTP id CBAE755B902; Mon, 12 Jul 2021 15:44:16 +1000 (VLAT) MIME-Version: 1.0 Date: Mon, 12 Jul 2021 15:44:16 +1000 From: Korolev Sergey To: KK CHN Cc: freebsd-questions Subject: Re: Analyzing Log files of very large size Organization: =?UTF-8?Q?=D0=92=D0=A6_=D0=94=D0=92=D0=9E_=D0=A0=D0=90?= =?UTF-8?Q?=D0=9D?= Reply-To: Mail-Reply-To: In-Reply-To: References: Message-ID: X-Sender: serejk@febras.net User-Agent: RoundCube Webmail/0.5.4 X-FEBRAS-Info: Contact e-mail: admin@febras.net X-FEBRAS-ID: CBAE755B902.A0F10 X-FEBRAS: clean X-FEBRAS-SpamCheck: not spam, SpamAssassin (not cached, score=-0.5, required 5, ALL_TRUSTED -1.00, BAYES_40 -0.00, HTML_MESSAGE 0.00, URIBL_BLOCKED 0.00, VOWEL_TOCC_5 0.50) X-FEBRAS-From: serejk@febras.net X-FEBRAS-To: freebsd-questions@freebsd.org, kkchn.in@gmail.com X-Spam-Status: No X-Rspamd-Queue-Id: 4GNXlJ3ctTz3NkF X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of serejk@febras.net designates 62.76.193.23 as permitted sender) smtp.mailfrom=serejk@febras.net X-Spamd-Result: default: False [-3.20 / 15.00]; HAS_REPLYTO(0.00)[serejk@febras.net]; R_SPF_ALLOW(-0.20)[+ip4:62.76.193.23]; REPLYTO_ADDR_EQ_FROM(0.00)[]; HAS_ORG_HEADER(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.90)[-0.895]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[62.76.193.23:from]; ASN(0.00)[asn:34017, ipnet:62.76.193.0/24, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; TAGGED_RCPT(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DMARC_NA(0.00)[febras.net]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; SPAMHAUS_ZRD(0.00)[62.76.193.23:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-questions] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2021 05:44:29 -0000 I think, that proper tools usually highly depends on desired result, so my reasoning is quite general. People here advise to use Perl and also split one large file into managable pieces - all that is very good, I vote for that. But I don`t know Perl at all, so I usually get along with standard shell utilities: grep, tr, awk, sed, etc. I used to parse big maillogs with them successfully. On Sun, 11 Jul 2021 19:43:41 +0530, KK CHN wrote: > Yes, it is. > > On Sun, Jul 11, 2021 at 6:02 PM Korolev Sergey wrote: > >> Is it a plain text file? On 11 Jul 2021, at 22:13, KK CHN wrote: List, I am in a requirement to analyze large log files of sonic wall firewall around 50 GB. for a suspect attack. What tools and solutions need to be deployed for handling this much large files and pls enlighten me with your expertise and reference materials if any. All are tcp / ip communications, DNS UDP transports .. Regards, Kris _______________________________________________ freebsd-questions@freebsd.org [2] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-questions [3] To unsubscribe, send any mail to " freebsd-questions-unsubscribe@freebsd.org [4]" > > _______________________________________________ > freebsd-questions@freebsd.org [6] mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions [7] > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org [8]" Links: ------ [1] mailto:kkchn.in@gmail.com [2] mailto:freebsd-questions@freebsd.org [3] https://lists.freebsd.org/mailman/listinfo/freebsd-questions [4] mailto:freebsd-questions-unsubscribe@freebsd.org [5] mailto:serejk@febras.net [6] mailto:freebsd-questions@freebsd.org [7] https://lists.freebsd.org/mailman/listinfo/freebsd-questions [8] mailto:freebsd-questions-unsubscribe@freebsd.org