From owner-freebsd-cloud@freebsd.org Wed Aug 30 23:19:51 2017 Return-Path: Delivered-To: freebsd-cloud@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5162EE0C74F for ; Wed, 30 Aug 2017 23:19:51 +0000 (UTC) (envelope-from raf@rafal.net) Received: from mxout-08.mxes.net (mxout-08.mxes.net [216.86.168.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B29084396 for ; Wed, 30 Aug 2017 23:19:50 +0000 (UTC) (envelope-from raf@rafal.net) Received: from [192.168.40.37] (unknown [86.40.118.125]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTPSA id 215AE509B6; Wed, 30 Aug 2017 19:19:48 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: AWS CloudWatch Logs Agent From: Rafal Lukawiecki In-Reply-To: <0100015e3565144c-18dc3e52-fcba-4b30-bf2f-58b99b55562d-000000@email.amazonses.com> Date: Thu, 31 Aug 2017 00:19:47 +0100 Cc: freebsd-cloud@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <456A3074-08D2-4141-81EF-5CD6B1B53899@rafal.net> References: <4FFC6F3B-E8A1-49EE-A05A-C6B9B0A615FF@rafal.net> <0100015e3565144c-18dc3e52-fcba-4b30-bf2f-58b99b55562d-000000@email.amazonses.com> To: Colin Percival X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-cloud@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "FreeBSD on cloud platforms \(EC2, GCE, Azure, etc.\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2017 23:19:51 -0000 Thank you, Colin, for your suggestion. I have never run the Linux = emulation system in FreeBSD yet, I suppose this is a good time to try = it. However, I did not realise that it would be able to cope with setup = of rc/runlevels/deamons. I thought it was only able to run more mundane = utilities, rather than assist in system/boot init set-up. Regarding your comment about CloudWatch metrics, I am *also* using = CloudWatch to report in-machine memory and CPU utilisation in my = existing Linux machines. I have followed this guide: = http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html It was easy. Essentially, I have installed the script from = http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScrip= ts-1.2.1.zip and in my crontab I have something along the lines of: */5 * * * * /opt/aws-scripts-mon/mon-put-instance-data.pl --mem-util = --mem-used --mem-avail --swap-util --swap-used --disk-path=3D/ = --disk-space-util --disk-space-used --disk-space-avail --from-cron The metrics magically and reliably appear in CloudWatch so that I can = set-up alarms etc. It works really well in Linux, I hope to replicate = the success with FreeBSD. As for the CW Logs, unfortunately the script at curl = https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-set= up.py is a bit too daunting for me to try rewriting using my brand-new = rc.d scripting skills, with only a hundred lines working (!) so far = behind my belt. Thank you for all your help, which is much appreciated. I hope to be = able to give back somehow one day=E2=80=A6if I manage to persist in this = adventure. Fingers crossed. Rafal -- Rafal Lukawiecki Data Scientist and Director=20 Project Botticelli Ltd > On 31 Aug 2017, at 00:06, Colin Percival wrote: >=20 > On 08/30/17 07:07, Rafal Lukawiecki wrote: >> One nice AWS component is its CloudWatch Logs engine that collects = streams >> of syslog-generated data and centralises them, with a simple but = usable UI. >> AWS provides an automated set-up for the awslogs daemon agent, but = from a >> cursory look at the code, although parts of it are plain Python, much = of it >> is very Linux-focused shell script, with much regard for runlevels, >> logrotate etc and not immediately usable as an rc.d script. >>=20 >> I was wondering if anyone has hacked that script yet to run on = FreeBSD or >> if you have any other suggestions how to help me get AWS CloudWatch = Logs >> integrated into FreeBSD running on AWS. >=20 > Huh, somehow I never noticed CloudWatch Logs; the aspect of CloudWatch = I had > been thinking could be useful for FreeBSD was to create an agent which = would > record memory/swap/disk usage to CloudWatch. >=20 > Unfortunately I don't have time to work on either right now; but for a = very > simple proof of principle I'd suggest running the Linux CloudWatch = Logs agent > via the Linux emulation system. >=20 > --=20 > Colin Percival > Security Officer Emeritus, FreeBSD | The power to serve > Founder, Tarsnap | www.tarsnap.com | Online backups for the truly = paranoid