From owner-freebsd-net@FreeBSD.ORG Mon Sep 17 02:28:39 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6FCD1106566C for ; Mon, 17 Sep 2012 02:28:39 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 307BB8FC0A for ; Mon, 17 Sep 2012 02:28:38 +0000 (UTC) Received: by obbun3 with SMTP id un3so10520915obb.13 for ; Sun, 16 Sep 2012 19:28:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=rddSSOkfI9IhOI5AVYO1a3NUnraiL78cFMFx/r9iRyE=; b=jiTTZZLNN0aOeM6YWl6Md8xEUEF3BQ67S8Zyj0O0YLWddpSDqS+65Wdk6Ey00PGcxN tPr55QH7QOvTZ9lwDZpNI+zgSY7h61RknaszBFwi1UP295cVk8IULI41IRA07FFlYWe9 p8g8gTvxLd+nQL4RMWbyI++WWsbcpFcOSoRnw5WHjo8rFDOqWVLAMcTBeEneJPuRTAWy zUwcoLGhkxlbeU848veISg6RziV4NJTKFuMy2w4yZGLjcj8PGDj8L0y6n4Z65wG2OIHh d7hWatfuCiW6u+F+pnqRIlc3VDJ+8+KZ3rohdqDW6qzT9I7reksBxZ7qLIPFzPlc5GAC 3Mzg== MIME-Version: 1.0 Received: by 10.60.12.234 with SMTP id b10mr10484783oec.72.1347848918410; Sun, 16 Sep 2012 19:28:38 -0700 (PDT) Received: by 10.76.143.194 with HTTP; Sun, 16 Sep 2012 19:28:38 -0700 (PDT) In-Reply-To: <50564BE9.6050104@sentex.net> References: <50564BE9.6050104@sentex.net> Date: Sun, 16 Sep 2012 22:28:38 -0400 Message-ID: From: Zaphod Beeblebrox To: Mike Tancsa Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-net@freebsd.org, Ivan Alexandrovich Subject: Re: getting counters for a plenty of vlan ifaces X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2012 02:28:39 -0000 On Sun, Sep 16, 2012 at 6:00 PM, Mike Tancsa wrote: > On 9/16/2012 10:41 AM, Ivan Alexandrovich wrote: >> >> We are running freebsd9.0 on a router with >> more than 1000 of subscriber's vlan interfaces. >> Outgoing packet rate is approximately 40 kpps. >> >> There's a need to collect bytes and packets >> counters for all those vlan interfaces every >> minute (or even twice a minute) and store them > > Hi, > We approach it a little differently and collect all the data via > netflow, or in this case argus. I sample the parent interface and save > all the flow data which argus is smart enough to parse out at the vlan > level. You can then run all sorts of fine grained reports this way. We > use it on a system with about 900 ng interfaces. I know that many people like netflow, but consider you're adding a processing point per packet to solve a once per minute interface sample. Netflow has always struck me as a solution for closed systems --- giving access to all possible information at moderate expense such that you would then never have an excuse to want changes in the operating system of the router. It strikes me that a little kernel module that provided a kernel call that (when called) walked the list of interfaces (in kernel) building a table as described and then shipping that table to userland in one go would be exceedingly cheep to call. It would also not be part of the packet forwarding path and not a potential constant cost during a DDOS. If someone wanted me to write a little .ko for that and an associated userland utility, I'd be happy to do the work.