From owner-freebsd-isp@FreeBSD.ORG Sun Sep 6 20:06:25 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C46421065679; Sun, 6 Sep 2009 20:06:25 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from no.spam.no.ddos.ru (no.spam.no.ddos.ru [77.73.233.132]) by mx1.freebsd.org (Postfix) with ESMTP id 5440C8FC08; Sun, 6 Sep 2009 20:06:23 +0000 (UTC) Received: from ws.ipfw.ru (secured.by.ipfw.ru [81.200.11.182]) by no.spam.no.ddos.ru (Postfix) with ESMTPA id 1833B37A708; Sun, 6 Sep 2009 19:46:27 +0000 (UTC) Message-ID: <4AA41192.6080708@ipfw.ru> Date: Sun, 06 Sep 2009 23:46:26 +0400 From: "Alexander V. Chernikov" User-Agent: Thunderbird 2.0.0.22 (X11/20090818) MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: multipart/mixed; boundary="------------010607050501030003050405" Cc: freebsd-isp@freebsd.org Subject: Call for testers: ng_netflow with v9 and IPv6 support X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Sep 2009 20:06:25 -0000 This is a multi-part message in MIME format. --------------010607050501030003050405 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi! There are patches for 7.2R and . to make ng_netflow export IPv4/IPv6 flows via V9 protocol (RFC3954). Your feedback is welcome. Alternative links: http://static.ipfw.ru/patches/netflow_v9_20090906_72.diff http://static.ipfw.ru/patches/netflow_v9_20090906_curr.diff node runtime control messages: Set export version: setversion { version=X }. X=[59], defaults to 9 Set export interface mtu setmtu { mtu=X }. X has to be <= MHLEN at the moment, defaults to 1500 Set v9 template announce conditions settemplateperiodic { time=X packets=Y }. Send templates every X seconds or Y packets, X=600, Y=500 by default --------------010607050501030003050405 Content-Type: text/plain; name="netflow_v9_20090906_curr.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="netflow_v9_20090906_curr.diff" diff -urN sys/netgraph/_netflow.orig/netflow.c sys/netgraph/netflow/netflow.c --- sys/netgraph/_netflow.orig/netflow.c 2009-09-01 02:37:44.000000000 +0400 +++ sys/netgraph/netflow/netflow.c 2009-09-06 17:04:28.000000000 +0400 @@ -30,6 +30,8 @@ static const char rcs_id[] = "@(#) $FreeBSD: src/sys/netgraph/netflow/netflow.c,v 1.33 2008/12/15 06:10:57 qingli Exp $"; +#include "opt_inet6.h" + #include #include #include @@ -37,14 +39,18 @@ #include #include #include +#include #include +#include #include #include +#include #include #include #include +#include #include #include @@ -94,12 +100,99 @@ ((t) << 6) + /* 64 */ \ ((t) << 5) + /* 32 */ \ ((t) << 3)) /* 8 */ +/* + * Defines for different neflow version dispatchers + * + */ +static int export_add(item_p, struct flow_entry *); +static int export_add_v9(item_p, struct flow_entry *); +static int export_send(priv_p, item_p, int flags); +static int export_send_v9(priv_p, item_p, int flags); + +typedef int (*record_add_ptr)(item_p, struct flow_entry *); +typedef int (*record_send_ptr)(priv_p, item_p, int); +struct _netflow_dispatchers { + record_add_ptr record_add; + record_send_ptr record_send; +}; + +static struct _netflow_dispatchers netflow_dispatcher[] = +{ + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { export_add, export_send }, /* Version 5 */ + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { export_add_v9, export_send_v9 }, /* Version 9 */ + { NULL, NULL } +}; + +record_add_ptr record_add = NULL; +record_send_ptr record_send = NULL; MALLOC_DECLARE(M_NETFLOW_HASH); MALLOC_DEFINE(M_NETFLOW_HASH, "netflow_hash", "NetFlow hash"); -static int export_add(item_p, struct flow_entry *); -static int export_send(priv_p, item_p, int flags); + +static struct netflow_v9_template _netflow_v9_record_ipv4_tcp[] = +{ + { NETFLOW_V9_FIELD_IPV4_SRC_ADDR, 4}, + { NETFLOW_V9_FIELD_IPV4_DST_ADDR, 4}, + { NETFLOW_V9_FIELD_IPV4_NEXT_HOP, 4}, + { NETFLOW_V9_FIELD_INPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_OUTPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_IN_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_IN_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_FIRST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_LAST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_L4_SRC_PORT, 2}, + { NETFLOW_V9_FIELD_L4_DST_PORT, 2}, + { NETFLOW_V9_FIELD_TCP_FLAGS, 1}, + { NETFLOW_V9_FIELD_PROTOCOL, 1}, + { NETFLOW_V9_FIELD_TOS, 1}, + { NETFLOW_V9_FIELD_SRC_AS, 4}, + { NETFLOW_V9_FIELD_DST_AS, 4}, + { NETFLOW_V9_FIELD_SRC_MASK, 1}, + { NETFLOW_V9_FIELD_DST_MASK, 1}, + {0, 0} +}; + +static struct netflow_v9_template _netflow_v9_record_ipv6_tcp[] = +{ + { NETFLOW_V9_FIELD_IPV6_SRC_ADDR, 16}, + { NETFLOW_V9_FIELD_IPV6_DST_ADDR, 16}, + { NETFLOW_V9_FIELD_IPV6_NEXT_HOP, 16}, + { NETFLOW_V9_FIELD_INPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_OUTPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_IN_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_IN_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_FIRST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_LAST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_L4_SRC_PORT, 2}, + { NETFLOW_V9_FIELD_L4_DST_PORT, 2}, + { NETFLOW_V9_FIELD_TCP_FLAGS, 1}, + { NETFLOW_V9_FIELD_PROTOCOL, 1}, + { NETFLOW_V9_FIELD_TOS, 1}, + { NETFLOW_V9_FIELD_SRC_AS, 4}, + { NETFLOW_V9_FIELD_DST_AS, 4}, + { NETFLOW_V9_FIELD_SRC_MASK, 1}, + { NETFLOW_V9_FIELD_DST_MASK, 1}, + {0, 0} +}; + + +static int generate_v9_templates(priv_p priv); + +MALLOC_DECLARE(M_NETFLOW_GENERAL); +MALLOC_DEFINE(M_NETFLOW_GENERAL, "netflog_general", "plog, templates data"); /* Generate hash for a given flow record. */ static __inline uint32_t @@ -108,10 +201,24 @@ switch (r->r_ip_p) { case IPPROTO_TCP: case IPPROTO_UDP: - return FULL_HASH(r->r_src.s_addr, r->r_dst.s_addr, + return FULL_HASH(r->src.r_src.s_addr, r->dst.r_dst.s_addr, r->r_sport, r->r_dport); default: - return ADDR_HASH(r->r_src.s_addr, r->r_dst.s_addr); + return ADDR_HASH(r->src.r_src.s_addr, r->dst.r_dst.s_addr); + } +} + +/* Generate hash for a given flow record. Use lower 4 octets from v6 addresses */ +static __inline uint32_t +ip6_hash(struct flow_rec *r) +{ + switch (r->r_ip_p) { + case IPPROTO_TCP: + case IPPROTO_UDP: + return FULL_HASH(r->src.r_src6.__u6_addr.__u6_addr32[3], r->dst.r_dst6.__u6_addr.__u6_addr32[3], + r->r_sport, r->r_dport); + default: + return ADDR_HASH(r->src.r_src6.__u6_addr.__u6_addr32[3], r->dst.r_dst6.__u6_addr.__u6_addr32[3]); } } @@ -126,6 +233,7 @@ atomic_add_32(&priv->info.nfinfo_used, 1); + return (0); } @@ -141,6 +249,7 @@ /* * Detach export datagram from priv, if there is any. * If there is no, allocate a new one. + * -- V9/IPv6 ready */ static item_p get_export_dgram(priv_p priv) @@ -166,14 +275,163 @@ return (NULL); dgram = mtod(m, struct netflow_v5_export_dgram *); dgram->header.count = 0; - dgram->header.version = htons(NETFLOW_V5); + dgram->header.version = htons(priv->version); + + if (priv->version == NETFLOW_V9) { + atomic_fetchadd_32(&priv->sent_packets, 1); + + /* + * Let's insert mbuf tag to store some info + */ + struct netflow_v9_mbuf_tag *t; + struct m_tag *mt = m_tag_alloc(MTAG_NETFLOW, MTAG_NETFLOW_V9, sizeof(struct netflow_v9_mbuf_tag), M_NOWAIT); + if (mt == NULL) { + m_freem(m); + return (NULL); + } + + m_tag_init(m); + m_tag_prepend(m, mt); + t = (struct netflow_v9_mbuf_tag *)(mt + 1); + t->length = sizeof(struct netflow_v9_header); + t->count = 0; + t->mtu = priv->mtu; + t->flow_header = t->length; + + /* + * Check if we need to insert templates into packet + */ + + struct timespec ts; + struct netflow_v9_flowset_header *fl; + + getnanotime(&ts); + if ((ts.tv_sec >= priv->templ_time + priv->templ_last_ts) || (priv->sent_packets >= priv->templ_packets + priv->templ_last_pkt)) { + //vlog("INSERTING TEMPLATE: ts: %lu last: %u packets: %u last: %u delay: %u", ts.tv_sec, priv->templ_last_ts, priv->sent_packets, priv->templ_last_pkt, priv->templ_time); + atomic_store_rel_32(&priv->templ_last_ts, ts.tv_sec); + atomic_store_rel_32(&priv->templ_last_pkt, priv->sent_packets); + //vlog("ts: %lu last_t: %u last_p: %u delay: %u", ts.tv_sec, priv->templ_last_ts, priv->templ_last_pkt, priv->templ_time); + + fl = priv->v9_flowsets[0]; + bcopy(fl, (char *)dgram + t->length, ntohs(fl->length)); + + t->length += ntohs(fl->length); + t->flow_header = t->length; + t->count += priv->flowset_records[0]; + } + + } } return (item); } /* + * Pre-compiles flow exporter for all possible FlowSets + * so we can add flowset to packet via simple memcpy() + */ +#define __push(x) *p++ = htons((x)) +static int +generate_v9_templates(priv_p priv) +{ + uint16_t *p, *template_fields_cnt; + int cnt; + + int flowset_size = sizeof(struct netflow_v9_flowset_header) + + _NETFLOW_V9_TEMPLATE_SIZE(_netflow_v9_record_ipv4_tcp) + /* netflow_v9_record_ipv4_tcp */ + _NETFLOW_V9_TEMPLATE_SIZE(_netflow_v9_record_ipv6_tcp); /* netflow_v9_record_ipv6_tcp */ + + priv->v9_flowsets[0] = malloc(flowset_size, M_NETFLOW_GENERAL, M_WAITOK | M_ZERO); + if (priv->v9_flowsets[0] == NULL) + return (ENOMEM); + + + if (flowset_size % 4) + flowset_size += 4 - (flowset_size % 4); /* Padding to 4-byte boundary */ + + priv->flowsets_count = 1; + p = (uint16_t *)priv->v9_flowsets[0]; + *p++ = 0; /* Flowset ID, 0 is reserved for Template FlowSets */ + *p++ = htons(flowset_size); /* Total FlowSet length */ + + /* + * Most common TCP/UDP IPv4 template, ID = 256 + */ + *p++ = htons(0x100 + NETFLOW_V9_FLOW_V4_L4); + template_fields_cnt = p++; + for (cnt = 0; _netflow_v9_record_ipv4_tcp[cnt].field_id != 0; cnt++) { + *p++ = htons(_netflow_v9_record_ipv4_tcp[cnt].field_id); + *p++ = htons(_netflow_v9_record_ipv4_tcp[cnt].field_length); + } + *template_fields_cnt = htons(cnt); + + /* + * TCP/UDP IPv6 template, ID = 257 + */ + *p++ = htons(0x100 + NETFLOW_V9_FLOW_V6_L4); + template_fields_cnt = p++; + for (cnt = 0; _netflow_v9_record_ipv6_tcp[cnt].field_id != 0; cnt++) { + *p++ = htons(_netflow_v9_record_ipv6_tcp[cnt].field_id); + *p++ = htons(_netflow_v9_record_ipv6_tcp[cnt].field_length); + } + *template_fields_cnt = htons(cnt); + + + priv->flowset_records[0] = 2; + return (0); +} + +/* + * Switches version used for netflow export + * + */ +void +ng_netflow_switch_version(priv_p priv, int ver, int boot) +{ + item_p item = NULL; + + if ((ver != NETFLOW_V9) && (ver != NETFLOW_V5)) + return; + + if ((ver == priv->version) && (boot == 0)) + return; + + /* + * All new threads acquiring export datagram will wait for lock + * so we can change pointers. + * Existing threads with export datagram already held will call + * wrong export function which will do version check at the beginning + */ + + mtx_lock(&priv->export_mtx); + /* XXX: Need to be machine-independent here */ + record_add = netflow_dispatcher[ver].record_add; + record_send = netflow_dispatcher[ver].record_send; + //atomic_store_rel_ptr((unsigned long *)record_add, (unsigned long)netflow_dispatcher[ver].record_add); + //atomic_store_rel_ptr((unsigned long *)record_send, (unsigned long)netflow_dispatcher[ver].record_send); + item = priv->export_item; + priv->export_item = NULL; + mtx_unlock(&priv->export_mtx); + + if (ver == NETFLOW_V9) { + priv->templ_last_pkt = 0; + priv->templ_last_ts = 0; + } + + //vlog("NEW: add=%p send=%p", record_add, record_send); + + if (item != NULL) + (*netflow_dispatcher[priv->version].record_send)(priv, item, NG_NOFLAGS); + + if (boot) + log(LOG_DEBUG, "ng_netflow: v%d export started\n", ver); + else + log(LOG_DEBUG, "ng_netflow: export switched: v%d -> v%d\n", priv->version, ver); + priv->version = ver; +} + +/* * Re-attach incomplete datagram back to priv. * If there is already another one, then send incomplete. */ static void @@ -190,13 +448,14 @@ mtx_unlock(&priv->export_mtx); } else { mtx_unlock(&priv->export_mtx); - export_send(priv, item, flags); + (*record_send)(priv, item, flags); } } /* * The flow is over. Call export_add() and free it. If datagram is * full, then call export_send(). + * -- v9/IPv6 nearly ready */ static __inline void expire_flow(priv_p priv, item_p *item, struct flow_entry *fle, int flags) @@ -208,8 +467,8 @@ uma_zfree_arg(priv->zone, fle, priv); return; } - if (export_add(*item, fle) > 0) { - export_send(priv, *item, flags); + if ((*record_add)(*item, fle) > 0) { + (*record_send)(priv, *item, flags); *item = NULL; } uma_zfree_arg(priv->zone, fle, priv); @@ -234,11 +493,14 @@ * to be sure. */ static __inline int -hash_insert(priv_p priv, struct flow_hash_entry *hsh, struct flow_rec *r, +hash_insert(priv_p priv, struct flow_hash_entry *hsh, struct flow_rec *r, uint16_t eproto, int plen, uint8_t tcp_flags) { struct flow_entry *fle; struct sockaddr_in sin; +#ifdef INET6 + struct sockaddr_in6 sin6; +#endif struct rtentry *rt; mtx_assert(&hsh->mtx, MA_OWNED); @@ -257,6 +519,7 @@ bcopy(r, &fle->f.r, sizeof(struct flow_rec)); fle->f.bytes = plen; fle->f.packets = 1; + fle->f.proto = eproto; fle->f.tcp_flags = tcp_flags; fle->f.first = fle->f.last = time_uptime; @@ -265,46 +528,94 @@ * First we do route table lookup on destination address. So we can * fill in out_ifx, dst_mask, nexthop, and dst_as in future releases. */ - bzero(&sin, sizeof(sin)); - sin.sin_len = sizeof(struct sockaddr_in); - sin.sin_family = AF_INET; - sin.sin_addr = fle->f.r.r_dst; - /* XXX MRT 0 as a default.. need the m here to get fib */ - rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, 0); - if (rt != NULL) { - fle->f.fle_o_ifx = rt->rt_ifp->if_index; - - if (rt->rt_flags & RTF_GATEWAY && - rt->rt_gateway->sa_family == AF_INET) - fle->f.next_hop = - ((struct sockaddr_in *)(rt->rt_gateway))->sin_addr; - - if (rt_mask(rt)) - fle->f.dst_mask = bitcount32(((struct sockaddr_in *) - rt_mask(rt))->sin_addr.s_addr); - else if (rt->rt_flags & RTF_HOST) - /* Give up. We can't determine mask :( */ - fle->f.dst_mask = 32; - - RTFREE_LOCKED(rt); - } - - /* Do route lookup on source address, to fill in src_mask. */ - bzero(&sin, sizeof(sin)); - sin.sin_len = sizeof(struct sockaddr_in); - sin.sin_family = AF_INET; - sin.sin_addr = fle->f.r.r_src; - /* XXX MRT 0 as a default revisit. need the mbuf for fib*/ - rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, 0); - if (rt != NULL) { - if (rt_mask(rt)) - fle->f.src_mask = bitcount32(((struct sockaddr_in *) - rt_mask(rt))->sin_addr.s_addr); - else if (rt->rt_flags & RTF_HOST) - /* Give up. We can't determine mask :( */ - fle->f.src_mask = 32; + if (eproto == ETHERTYPE_IP) { + bzero(&sin, sizeof(sin)); + sin.sin_len = sizeof(struct sockaddr_in); + sin.sin_family = AF_INET; + sin.sin_addr = fle->f.r.dst.r_dst; + /* XXX MRT 0 as a default.. need the m here to get fib */ + rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, 0); + if (rt != NULL) { + fle->f.fle_o_ifx = rt->rt_ifp->if_index; + + if (rt->rt_flags & RTF_GATEWAY && + rt->rt_gateway->sa_family == AF_INET) + fle->f.n.next_hop = + ((struct sockaddr_in *)(rt->rt_gateway))->sin_addr; + + if (rt_mask(rt)) + fle->f.dst_mask = bitcount32(((struct sockaddr_in *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) + /* Give up. We can't determine mask :( */ + fle->f.dst_mask = 32; + + RTFREE_LOCKED(rt); + } + + /* Do route lookup on source address, to fill in src_mask. */ + bzero(&sin, sizeof(sin)); + sin.sin_len = sizeof(struct sockaddr_in); + sin.sin_family = AF_INET; + sin.sin_addr = fle->f.r.src.r_src; + /* XXX MRT 0 as a default revisit. need the mbuf for fib*/ + rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, 0); + if (rt != NULL) { + if (rt_mask(rt)) + fle->f.src_mask = bitcount32(((struct sockaddr_in *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) + /* Give up. We can't determine mask :( */ + fle->f.src_mask = 32; + + RTFREE_LOCKED(rt); + } + } else { +#ifdef INET6 + bzero(&sin6, sizeof(sin6)); + sin6.sin6_len = sizeof(struct sockaddr_in6); + sin6.sin6_family = AF_INET6; + sin6.sin6_addr = fle->f.r.dst.r_dst6; + /* XXX fib works for AF_INET only */ + rt = rtalloc1_fib((struct sockaddr *)&sin6, 0, 0, 0); + if (rt != NULL) { + fle->f.fle_o_ifx = rt->rt_ifp->if_index; + + if (rt->rt_flags & RTF_GATEWAY && + rt->rt_gateway->sa_family == AF_INET6) + fle->f.n.next_hop6 = + ((struct sockaddr_in6 *)(rt->rt_gateway))->sin6_addr; + if (rt_mask(rt)) +/* + fle->f.dst_mask = bitcount32(((struct sockaddr_in6 *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) */ + /* Give up. We can't determine mask :( */ + fle->f.dst_mask = 128; + + RTFREE_LOCKED(rt); + } + + /* Do route lookup on source address, to fill in src_mask. */ + bzero(&sin6, sizeof(sin6)); + sin6.sin6_len = sizeof(struct sockaddr_in6); + sin6.sin6_family = AF_INET6; + sin6.sin6_addr = fle->f.r.src.r_src6; + /* XXX MRT 0 as a default revisit. need the mbuf for fib*/ + rt = rtalloc1_fib((struct sockaddr *)&sin6, 0, 0, 0); + if (rt != NULL) { +/* + if (rt_mask(rt)) + fle->f.src_mask = bitcount32(((struct sockaddr_in6 *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) */ + /* Give up. We can't determine mask :( */ + fle->f.src_mask = 128; + + RTFREE_LOCKED(rt); + } - RTFREE_LOCKED(rt); +#endif } /* Push new flow at the and of hash. */ @@ -347,6 +658,10 @@ mtx_init(&priv->export_mtx, "export dgram lock", NULL, MTX_DEF); + generate_v9_templates(priv); + + ng_netflow_switch_version(priv, priv->version, 1); + return (0); } @@ -359,6 +674,7 @@ item_p item = NULL; int i; + /* * We are going to free probably billable data. * Expire everything before freeing it. @@ -371,7 +687,7 @@ } if (item != NULL) - export_send(priv, item, NG_QUEUE); + (*record_send)(priv, item, NG_QUEUE); uma_zdestroy(priv->zone); @@ -383,46 +699,34 @@ if (priv->hash) free(priv->hash, M_NETFLOW_HASH); + /* FreeFlow Tables */ + for (i = 0; i < priv->flowsets_count; i++) + free(priv->v9_flowsets[i], M_NETFLOW_GENERAL); + mtx_destroy(&priv->export_mtx); } -/* Insert packet from into flow cache. */ +/* + * Insert packet from into flow cache. Assume size/version check passed + */ int -ng_netflow_flow_add(priv_p priv, struct ip *ip, unsigned int src_if_index) +ng_netflow_flow_add(priv_p priv, caddr_t ip_ptr, caddr_t upper_ptr, uint8_t upper_proto, uint8_t is_frag, unsigned int src_if_index) { register struct flow_entry *fle, *fle1; struct flow_hash_entry *hsh; struct flow_rec r; + struct ip *ip = NULL; +#ifdef INET6 + struct ip6_hdr *ip6 = NULL; +#endif item_p item = NULL; - int hlen, plen; + int plen; int error = 0; uint8_t tcp_flags = 0; - - /* Try to fill flow_rec r */ - bzero(&r, sizeof(r)); - /* check version */ - if (ip->ip_v != IPVERSION) - return (EINVAL); - - /* verify min header length */ - hlen = ip->ip_hl << 2; - - if (hlen < sizeof(struct ip)) - return (EINVAL); - - r.r_src = ip->ip_src; - r.r_dst = ip->ip_dst; - - /* save packet length */ - plen = ntohs(ip->ip_len); - - r.r_ip_p = ip->ip_p; - r.r_tos = ip->ip_tos; - - r.r_i_ifx = src_if_index; + uint16_t eproto; /* - * XXX NOTE: only first fragment of fragmented TCP, UDP and + * XXX Fragmentation NOTE: only first fragment of fragmented TCP, UDP and * ICMP packet will be recorded with proper s_port and d_port. * Following fragments will be recorded simply as IP packet with * ip_proto = ip->ip_p and s_port, d_port set to zero. @@ -430,22 +734,91 @@ * ip packet assebmling here. Anyway, (in)famous trafd works this way - * and nobody complains yet :) */ - if ((ip->ip_off & htons(IP_OFFMASK)) == 0) - switch(r.r_ip_p) { - case IPPROTO_TCP: - { - register struct tcphdr *tcp; - - tcp = (struct tcphdr *)((caddr_t )ip + hlen); - r.r_sport = tcp->th_sport; - r.r_dport = tcp->th_dport; - tcp_flags = tcp->th_flags; - break; + + /* Try to fill flow_rec r */ + bzero(&r, sizeof(r)); + ip = (struct ip *)ip_ptr; + if (ip->ip_v == IPVERSION) { + eproto = ETHERTYPE_IP; + r.src.r_src = ip->ip_src; + r.dst.r_dst = ip->ip_dst; + + /* Assume L$ template by default */ + r.flow_type = NETFLOW_V9_FLOW_V4_L4; + + /* save packet length */ + plen = ntohs(ip->ip_len); + + r.r_tos = ip->ip_tos; + + if (is_frag == 0) { + switch(upper_proto) { + case IPPROTO_TCP: + { + register struct tcphdr *tcp; + + tcp = (struct tcphdr *)upper_ptr; + r.r_ports = *(uint32_t *)upper_ptr; + tcp_flags = tcp->th_flags; + /* r.flow_type = NETFLOW_V9_FLOW_V4_L4; */ + break; + } + case IPPROTO_UDP: + case IPPROTO_SCTP: + { + r.r_ports = *(uint32_t *)upper_ptr; + /* r.flow_type = NETFLOW_V9_FLOW_V4_L4; */ + break; + } + + } } + + } else { +#ifdef INET6 + /* IPv6 traffic */ + ip = NULL; + ip6 = (struct ip6_hdr *)ip_ptr; + eproto = ETHERTYPE_IPV6; + + r.src.r_src6 = ip6->ip6_src; + r.dst.r_dst6 = ip6->ip6_dst; + + /* Assume L4 template by default */ + r.flow_type = NETFLOW_V9_FLOW_V6_L4; + + /* save packet length */ + plen = ntohs(ip6->ip6_plen) + sizeof(struct ip6_hdr); + + //r.r_tos = ip->ip_tos; + + if (is_frag == 0) { + switch(upper_proto) { + case IPPROTO_TCP: + { + register struct tcphdr *tcp; + + tcp = (struct tcphdr *)upper_ptr; + r.r_ports = *(uint32_t *)upper_ptr; + tcp_flags = tcp->th_flags; + /* r.flow_type = NETFLOW_V9_FLOW_V6_L4; */ + break; + } case IPPROTO_UDP: - r.r_ports = *(uint32_t *)((caddr_t )ip + hlen); - break; + case IPPROTO_SCTP: + { + r.r_ports = *(uint32_t *)upper_ptr; + /* r.flow_type = NETFLOW_V9_FLOW_V6_L4; */ + break; + } + + } } +#endif + } + + r.r_ip_p = upper_proto; + r.r_i_ifx = src_if_index; /* Update node statistics. XXX: race... */ priv->info.nfinfo_packets ++; @@ -486,7 +859,7 @@ * - it is going to overflow counter */ if (tcp_flags & TH_FIN || tcp_flags & TH_RST || AGED(fle) || - (fle->f.bytes >= (UINT_MAX - IF_MAXMTU)) ) { + (fle->f.bytes >= (cntr_max - IF_MAXMTU)) ) { TAILQ_REMOVE(&hsh->head, fle, fle_hash); expire_flow(priv, &item, fle, NG_QUEUE); atomic_add_32(&priv->info.nfinfo_act_exp, 1); @@ -502,7 +875,7 @@ } } } else /* A new flow entry. */ - error = hash_insert(priv, hsh, &r, plen, tcp_flags); + error = hash_insert(priv, hsh, &r, eproto, plen, tcp_flags); mtx_unlock(&hsh->mtx); @@ -611,6 +984,65 @@ return (error); } +/* We have full datagram in privdata. Send it to export hook. */ +static int +export_send_v9(priv_p priv, item_p item, int flags) +{ + struct mbuf *m = NGI_M(item); + struct netflow_v9_export_dgram *dgram = mtod(m, + struct netflow_v9_export_dgram *); + struct netflow_v9_header *header = &dgram->header; + struct timespec ts; + int error = 0; + uint16_t len = 0; + + struct netflow_v9_mbuf_tag *t; + struct netflow_v9_flowset_header *fs = NULL; + struct m_tag *mt = m_tag_locate(m, MTAG_NETFLOW, MTAG_NETFLOW_V9, NULL); + + if (mt == NULL) { + /* */ + log(LOG_DEBUG, "ng_netflow: V9 export packet without tag!\n"); + return (0); + } + + t = (struct netflow_v9_mbuf_tag *)(mt + 1); + + /* Close FlowSet if not closed */ + if (t->length != t->flow_header) { + fs = (struct netflow_v9_flowset_header *)(mtod(m, char *) + t->flow_header); + len = (uint16_t)(t->length - t->flow_header); + if (len % 4) { + t->length += 4 - (len % 4); + len += 4 - (len % 4); + } + fs->length = htons(len); + } + + + + /* Fill mbuf header. */ + m->m_len = m->m_pkthdr.len = t->length; + + /* Fill export header. */ + header->count = t->count; + header->sys_uptime = htonl(MILLIUPTIME(time_uptime)); + getnanotime(&ts); + header->unix_secs = htonl(ts.tv_sec); + header->seq_num = htonl(atomic_fetchadd_32(&priv->flow_seq, 1)); + header->count = htons(t->count); + header->source_id = htonl(NG_NODE_ID(priv->node)); + + /* remove tag */ + m_tag_delete_chain(m, NULL); + + if (priv->export != NULL) + NG_FWD_ITEM_HOOK_FLAGS(error, item, priv->export, flags); + else + NG_FREE_ITEM(item); + + return (error); +} /* Add export record to dgram. */ static int @@ -628,9 +1060,9 @@ ("ng_netflow: export too big")); /* Fill in export record. */ - rec->src_addr = fle->f.r.r_src.s_addr; - rec->dst_addr = fle->f.r.r_dst.s_addr; - rec->next_hop = fle->f.next_hop.s_addr; + rec->src_addr = fle->f.r.src.r_src.s_addr; + rec->dst_addr = fle->f.r.dst.r_dst.s_addr; + rec->next_hop = fle->f.n.next_hop.s_addr; rec->i_ifx = htons(fle->f.fle_i_ifx); rec->o_ifx = htons(fle->f.fle_o_ifx); rec->packets = htonl(fle->f.packets); @@ -654,6 +1086,135 @@ return (0); } +/* Add V9 record to dgram. */ +static int +export_add_v9(item_p item, struct flow_entry *fle) +{ + size_t len = 0; + uint16_t new_flow = 0; + void *offset_ptr; + struct netflow_v9_mbuf_tag *t; + struct netflow_v9_flowset_header *fs = NULL; + struct mbuf *m = NGI_M(item); + struct m_tag *mt = m_tag_locate(m, MTAG_NETFLOW, MTAG_NETFLOW_V9, NULL); + + if (mt == NULL) { + /* */ + log(LOG_DEBUG, "ng_netflow: V9 export packet without tag!\n"); + return (0); + } + + t = (struct netflow_v9_mbuf_tag *)(mt + 1); + offset_ptr = (mtod(m, char *) + t->length); + + /* Check if new records has the same template */ + if (fle->f.r.flow_type != t->flow_type) { + new_flow = 1; + + /* 'Close' old FlowSet */ + fs = (struct netflow_v9_flowset_header *)(mtod(m, char *) + t->flow_header); + len = (uint16_t)(t->length - t->flow_header); + if (len % 4) { + t->length += 4 - (len % 4); + len += 4 - (len % 4); + } + fs->length = htons(len); + + /* Prepare 'new', but do not modify any counters here because switch can fail */ + t->flow_type = NETFLOW_V9_FLOW_FAKE; + t->flow_header = t->length; + fs = (struct netflow_v9_flowset_header *)(mtod(m, char *) + t->length); + offset_ptr = (fs + 1); + } + + + switch (fle->f.r.flow_type) { + case NETFLOW_V9_FLOW_V4_L4: + { + /* IPv4 TCP/UDP/[SCTP] */ + struct netflow_v9_record_ipv4_tcp *rec = (struct netflow_v9_record_ipv4_tcp *)offset_ptr; + + + rec->src_addr = fle->f.r.src.r_src.s_addr; + rec->dst_addr = fle->f.r.dst.r_dst.s_addr; + rec->next_hop = fle->f.n.next_hop.s_addr; + rec->i_ifx = htons(fle->f.fle_i_ifx); + rec->o_ifx = htons(fle->f.fle_o_ifx); + rec->i_packets = htonl(fle->f.packets); + rec->i_octets = htonl(fle->f.bytes); + rec->o_packets = htonl(0); + rec->o_octets = htonl(0); + rec->first = htonl(MILLIUPTIME(fle->f.first)); + rec->last = htonl(MILLIUPTIME(fle->f.last)); + rec->s_port = fle->f.r.r_sport; + rec->d_port = fle->f.r.r_dport; + rec->flags = fle->f.tcp_flags; + rec->prot = fle->f.r.r_ip_p; + rec->tos = fle->f.r.r_tos; + rec->dst_mask = fle->f.dst_mask; + rec->src_mask = fle->f.src_mask; + + /* Not supported fields. */ + rec->src_as = rec->dst_as = 0; + + len = sizeof(struct netflow_v9_record_ipv4_tcp); + break; + } +#ifdef INET6 + case NETFLOW_V9_FLOW_V6_L4: + { + /* IPv6 TCP/UDP/[SCTP] */ + struct netflow_v9_record_ipv6_tcp *rec = (struct netflow_v9_record_ipv6_tcp *)offset_ptr; + + /* ACHTUNG! unchecked code! */ + rec->src_addr = fle->f.r.src.r_src6; + rec->dst_addr = fle->f.r.dst.r_dst6; + rec->next_hop = fle->f.n.next_hop6; + rec->i_ifx = htons(fle->f.fle_i_ifx); + rec->o_ifx = htons(fle->f.fle_o_ifx); + rec->i_packets = htonl(fle->f.packets); + rec->i_octets = htonl(fle->f.bytes); + rec->first = htonl(MILLIUPTIME(fle->f.first)); + rec->last = htonl(MILLIUPTIME(fle->f.last)); + rec->s_port = fle->f.r.r_sport; + rec->d_port = fle->f.r.r_dport; + rec->flags = fle->f.tcp_flags; + rec->prot = fle->f.r.r_ip_p; + rec->tos = fle->f.r.r_tos; + rec->dst_mask = fle->f.dst_mask; + rec->src_mask = fle->f.src_mask; + + /* Not supported fields. */ + rec->src_as = rec->dst_as = 0; + + len = sizeof(struct netflow_v9_record_ipv6_tcp); + break; + } +#endif + default: + { + log(LOG_DEBUG, "ng_netflow: Don't know what to do with %d flow type!\n", fle->f.r.flow_type); + return (0); + } + } + + if (new_flow) { + /* Generate data segment ID */ + fs->id = htons(0x100 + fle->f.r.flow_type); + t->flow_type = fle->f.r.flow_type; + t->length += sizeof(struct netflow_v9_flowset_header); + } + + t->length += len; + t->count++; + + if (t->length + NETFLOW_V9_MAX_RECORD_SIZE + 4 >= _NETFLOW_V9_MAX_SIZE(t->mtu)) + return (1); /* end of datagram */ + else + return (0); +} + + /* Periodic flow expiry run. */ void ng_netflow_expire(void *arg) diff -urN sys/netgraph/_netflow.orig/netflow.h sys/netgraph/netflow/netflow.h --- sys/netgraph/_netflow.orig/netflow.h 2009-09-01 02:37:44.000000000 +0400 +++ sys/netgraph/netflow/netflow.h 2009-09-06 12:41:29.000000000 +0400 @@ -46,6 +46,7 @@ #define NETFLOW_V1 1 #define NETFLOW_V5 5 +#define NETFLOW_V9 9 struct netflow_v1_header { @@ -69,6 +70,16 @@ uint16_t pad; /* Pad to word boundary */ } __attribute__((__packed__)); +struct netflow_v9_header +{ + uint16_t version; /* NetFlow version */ + uint16_t count; /* Total number of records in packet */ + uint32_t sys_uptime; /* System uptime */ + uint32_t unix_secs; /* Current seconds since 0000 UTC 1970 */ + uint32_t seq_num; /* Sequence number */ + uint32_t source_id; /* Observation Domain id */ +} __attribute__((__packed__)); + struct netflow_v1_record { uint32_t src_addr; /* Source IP address */ @@ -115,6 +126,142 @@ uint16_t pad2; /* Pad to word boundary */ } __attribute__((__packed__)); + + +/* RFC3954 field definitions */ +#define NETFLOW_V9_FIELD_IN_BYTES 1 /* Input bytes count for a flow. Default 4, can be 8 */ +#define NETFLOW_V9_FIELD_IN_PKTS 2 /* Incoming counter with number of packets associated with an IP Flow. Default 4 */ +#define NETFLOW_V9_FIELD_FLOWS 3 /* Number of Flows that were aggregated. Default 4 */ +#define NETFLOW_V9_FIELD_PROTOCOL 4 /* IP protocol byte. 1 */ +#define NETFLOW_V9_FIELD_TOS 5 /* Type of service byte setting when entering the incoming interface. 1 */ +#define NETFLOW_V9_FIELD_TCP_FLAGS 6 /* TCP flags; cumulative of all the TCP flags seen in this Flow. 1 */ +#define NETFLOW_V9_FIELD_L4_SRC_PORT 7 /* TCP/UDP source port number. 2 */ +#define NETFLOW_V9_FIELD_IPV4_SRC_ADDR 8 /* IPv4 source address. 4 */ +#define NETFLOW_V9_FIELD_SRC_MASK 9 /* The number of contiguous bits in the source subnet mask (i.e., the mask in slash notation). 1 */ +#define NETFLOW_V9_FIELD_INPUT_SNMP 10 /* Input interface index. Default 2 */ +#define NETFLOW_V9_FIELD_L4_DST_PORT 11 /* TCP/UDP destination port number. 2 */ +#define NETFLOW_V9_FIELD_IPV4_DST_ADDR 12 /* IPv4 destination address. 4 */ +#define NETFLOW_V9_FIELD_DST_MASK 13 /* The number of contiguous bits in the destination subnet mask (i.e., the mask in slash notation). 1 */ +#define NETFLOW_V9_FIELD_OUTPUT_SNMP 14 /* Output interface index. Default 2 */ +#define NETFLOW_V9_FIELD_IPV4_NEXT_HOP 15 /* IPv4 address of the next-hop router. 4 */ +#define NETFLOW_V9_FIELD_SRC_AS 16 /* Source BGP autonomous system number. Default 2, can be 4 */ +#define NETFLOW_V9_FIELD_DST_AS 17 /* Destination BGP autonomous system number. Default 2, can be 4 */ +#define NETFLOW_V9_FIELD_BGP_IPV4_NEXT_HOP 18 /* Next-hop router's IP address in the BGP domain. 4 */ +#define NETFLOW_V9_FIELD_MUL_DST_PKTS 19 /* IP multicast outgoing packet counter for packets associated with IP flow. Default 4 */ +#define NETFLOW_V9_FIELD_MUL_DST_BYTES 20 /* IP multicast outgoing Octet (byte) counter for the number of bytes associated with IP flow. Default 4 */ +#define NETFLOW_V9_FIELD_LAST_SWITCHED 21 /* sysUptime in msec at which the last packet of this Flow was switched. 4 */ +#define NETFLOW_V9_FIELD_FIRST_SWITCHED 22 /* sysUptime in msec at which the first packet of this Flow was switched. 4 */ +#define NETFLOW_V9_FIELD_OUT_BYTES 23 /* Outgoing counter for the number of bytes associated with an IP Flow. Default 4 */ +#define NETFLOW_V9_FIELD_OUT_PKTS 24 /* Outgoing counter for the number of packets associated with an IP Flow. Default 4 */ +#define NETFLOW_V9_FIELD_IPV6_SRC_ADDR 27 /* IPv6 source address. 16 */ +#define NETFLOW_V9_FIELD_IPV6_DST_ADDR 28 /* IPv6 destination address. 16 */ +#define NETFLOW_V9_FIELD_IPV6_SRC_MASK 29 /* Length of the IPv6 source mask in contiguous bits. 1 */ +#define NETFLOW_V9_FIELD_IPV6_DST_MASK 30 /* Length of the IPv6 destination mask in contiguous bits. 1 */ +#define NETFLOW_V9_FIELD_IPV6_FLOW_LABEL 31 /* IPv6 flow label as per RFC 2460 definition. 3 */ +#define NETFLOW_V9_FIELD_ICMP_TYPE 32 /* Internet Control Message Protocol (ICMP) packet type; reported as ICMP Type * 256 + ICMP code. 2 */ +#define NETFLOW_V9_FIELD_MUL_IGMP_TYPE 33 /* Internet Group Management Protocol (IGMP) packet type. 1 */ +#define NETFLOW_V9_FIELD_SAMPLING_INTERVAL 34 /* When using sampled NetFlow, the rate at which packets are sampled; for example, a value of 100 indicates that one of every hundred packets is sampled. 4 */ +#define NETFLOW_V9_FIELD_SAMPLING_ALGORITHM 35 /* For sampled NetFlow platform-wide: 0x01 deterministic sampling 0x02 random sampling. 1 */ +#define NETFLOW_V9_FIELD_FLOW_ACTIVE_TIMEOUT 36 /* Timeout value (in seconds) for active flow entries in the NetFlow cache. 2 */ +#define NETFLOW_V9_FIELD_FLOW_INACTIVE_TIMEOUT 37 /* Timeout value (in seconds) for inactive Flow entries in the NetFlow cache. 2 */ +#define NETFLOW_V9_FIELD_ENGINE_TYPE 38 /* Type of Flow switching engine (route processor, linecard, etc...). 1 */ +#define NETFLOW_V9_FIELD_ENGINE_ID 39 /* ID number of the Flow switching engine. 1 */ +#define NETFLOW_V9_FIELD_TOTAL_BYTES_EXP 40 /* Counter with for the number of bytes exported by the Observation Domain. Default 4 */ +#define NETFLOW_V9_FIELD_TOTAL_PKTS_EXP 41 /* Counter with for the number of packets exported by the Observation Domain. Default 4 */ +#define NETFLOW_V9_FIELD_TOTAL_FLOWS_EXP 42 /* Counter with for the number of flows exported by the Observation Domain. Default 4 */ +#define NETFLOW_V9_FIELD_MPLS_TOP_LABEL_TYPE 46 /* MPLS Top Label Type. 1 */ +#define NETFLOW_V9_FIELD_MPLS_TOP_LABEL_IP_ADDR 47 /* Forwarding Equivalent Class corresponding to the MPLS Top Label. 4 */ +#define NETFLOW_V9_FIELD_FLOW_SAMPLER_ID 48 /* Identifier shown in "show flow-sampler". 1 */ +#define NETFLOW_V9_FIELD_FLOW_SAMPLER_MODE 49 /* The type of algorithm used for sampling data. 2 */ +#define NETFLOW_V9_FIELD_FLOW_SAMPLER_RANDOM_INTERVAL 50 /* Packet interval at which to sample. 4. */ +#define NETFLOW_V9_FIELD_DST_TOS 55 /* Type of Service byte setting when exiting outgoing interface. 1. */ +#define NETFLOW_V9_FIELD_SRC_MAC 56 /* Source MAC Address. 6 */ +#define NETFLOW_V9_FIELD_DST_MAC 57 /* Destination MAC Address. 6 */ +#define NETFLOW_V9_FIELD_SRC_VLAN 58 /* Virtual LAN identifier associated with ingress interface. 2 */ +#define NETFLOW_V9_FIELD_DST_VLAN 59 /* irtual LAN identifier associated with egress interface. 2 */ +#define NETFLOW_V9_FIELD_IP_PROTOCOL_VERSION 60 /* Internet Protocol Version. Set to 4 for IPv4, set to 6 for IPv6. If not present in the template, then version 4 is assumed. 1. */ +#define NETFLOW_V9_FIELD_DIRECTION 61 /* Flow direction: 0 - ingress flow 1 - egress flow. 1 */ +#define NETFLOW_V9_FIELD_IPV6_NEXT_HOP 62 /* IPv6 address of the next-hop router. 16 */ +#define NETFLOW_V9_FIELD_BGP_IPV6_NEXT_HOP 63 /* Next-hop router in the BGP domain. 16 */ +#define NETFLOW_V9_FIELD_IPV6_OPTION_HEADERS 64 /* Bit-encoded field identifying IPv6 option headers found in the flow */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_1 70 /* MPLS label at position 1 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_2 71 /* MPLS label at position 2 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_3 72 /* MPLS label at position 3 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_4 73 /* MPLS label at position 4 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_5 74 /* MPLS label at position 5 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_6 75 /* MPLS label at position 6 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_7 76 /* MPLS label at position 7 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_8 77 /* MPLS label at position 8 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_9 78 /* MPLS label at position 9 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_10 79 /* MPLS label at position 10 in the stack. 3 */ + +#ifdef COUNTERS_64 +#define cntr uint64_t +#define cntr_max UINT64_MAX +#else +#define cntr uint32_t +#define cntr_max UINT_MAX +#endif + +struct netflow_v9_template +{ + int field_id; + int field_length; +}; + +/* Template ID for tcp/udp v4 streams ID:257 (0x100 + NETFLOW_V9_FLOW_V4_L4) */ +struct netflow_v9_record_ipv4_tcp +{ + uint32_t src_addr; /* Source IPv4 address (IPV4_SRC_ADDR) */ + uint32_t dst_addr; /* Destination IPv4 address (IPV4_DST_ADDR) */ + uint32_t next_hop; /* Next hop IPv4 address (IPV4_NEXT_HOP) */ + uint16_t i_ifx; /* Source interface index (INPUT_SNMP) */ + uint16_t o_ifx; /* Destination interface index (OUTPUT_SNMP) */ + cntr i_packets; /* Number of incoming packets in a flow (IN_PKTS) */ + cntr i_octets; /* Number of incoming octets in a flow (IN_BYTES) */ + cntr o_packets; /* Number of outgoing packets in a flow (OUT_PKTS) */ + cntr o_octets; /* Number of outgoing octets in a flow (OUT_BYTES) */ + uint32_t first; /* System uptime at start of a flow (FIRST_SWITCHED) */ + uint32_t last; /* System uptime at end of a flow (LAST_SWITCHED) */ + uint16_t s_port; /* Source port (L4_SRC_PORT) */ + uint16_t d_port; /* Destination port (L4_DST_PORT) */ + uint8_t flags; /* Cumulative OR of tcp flags (TCP_FLAGS) */ + uint8_t prot; /* IP protocol */ + uint8_t tos; /* IP type of service IN (or OUT) (TOS) */ + uint32_t src_as; /* Src peer/origin Autonomous System (SRC_AS) */ + uint32_t dst_as; /* Dst peer/origin Autonomous System (DST_AS) */ + uint8_t src_mask; /* Source route's mask bits (SRC_MASK) */ + uint8_t dst_mask; /* Destination route's mask bits (DST_MASK) */ +} __attribute__((__packed__)); + + +/* Template ID for tcp/udp v6 streams ID: 260 (0x100 + NETFLOW_V9_FLOW_V6_L4) */ +struct netflow_v9_record_ipv6_tcp +{ + struct in6_addr src_addr; /* Source IPv6 address (IPV6_SRC_ADDR) */ + struct in6_addr dst_addr; /* Destination IPv6 address (IPV6_DST_ADDR) */ + struct in6_addr next_hop; /* Next hop IPv6 address (IPV6_NEXT_HOP) */ + uint16_t i_ifx; /* Source interface index (INPUT_SNMP) */ + uint16_t o_ifx; /* Destination interface index (OUTPUT_SNMP) */ + cntr i_packets; /* Number of incoming packets in a flow (IN_PKTS) */ + cntr i_octets; /* Number of incoming octets in a flow (IN_BYTES) */ + cntr o_packets; /* Number of outgoing packets in a flow (OUT_PKTS) */ + cntr o_octets; /* Number of outgoing octets in a flow (OUT_BYTES) */ + uint32_t first; /* System uptime at start of a flow (FIRST_SWITCHED) */ + uint32_t last; /* System uptime at end of a flow (LAST_SWITCHED) */ + uint16_t s_port; /* Source port (L4_SRC_PORT) */ + uint16_t d_port; /* Destination port (L4_DST_PORT) */ + uint8_t flags; /* Cumulative OR of tcp flags (TCP_FLAGS) */ + uint8_t prot; /* IP protocol */ + uint8_t tos; /* IP type of service IN (or OUT) (TOS) */ + uint32_t src_as; /* Src peer/origin Autonomous System (SRC_AS) */ + uint32_t dst_as; /* Dst peer/origin Autonomous System (DST_AS) */ + uint8_t src_mask; /* Source route's mask bits (SRC_MASK) */ + uint8_t dst_mask; /* Destination route's mask bits (DST_MASK) */ +} __attribute__((__packed__)); + +#define vlog(x, ...) log(LOG_DEBUG, "%s:%d " x "\n", __FUNCTION__, __LINE__, ##__VA_ARGS__) + #define NETFLOW_V1_MAX_RECORDS 24 #define NETFLOW_V5_MAX_RECORDS 30 @@ -123,7 +270,50 @@ #define NETFLOW_V5_MAX_SIZE (sizeof(netflow_v5_header)+ \ sizeof(netflow_v5_record)*NETFLOW_V5_MAX_RECORDS) +#define BASE_MTU 1500 +#define MIN_MTU sizeof(struct netflow_v5_header) +#define MAX_MTU 16384 +#define NETFLOW_V9_MAX_SIZE _NETFLOW_V9_MAX_SIZE(BASE_MTU) +#define _NETFLOW_V9_MAX_SIZE(x) (x) - sizeof(struct ip6_hdr) - sizeof(struct udphdr) +#define NETFLOW_V9_MAX_FLOWSETS 2 + +#define NETFLOW_V9_MAX_RECORD_SIZE sizeof(struct netflow_v9_record_ipv6_tcp) +#define NETFLOW_V9_MAX_PACKETS_TEMPL 500 /* Send data templates every ... packets */ +#define NETFLOW_V9_MAX_TIME_TEMPL 600 /* Send data templates every ... seconds */ +#define NETFLOW_V9_MAX_TEMPLATES 16 /* Not a real value */ +#define _NETFLOW_V9_TEMPLATE_SIZE(x) (sizeof(x) / sizeof(struct netflow_v9_template)) * 4 +//#define _NETFLOW_V9_TEMPLATE_SIZE(x) ((x) + 1) * 4 + +/* Flow Templates */ +#define NETFLOW_V9_FLOW_V4_L4 1 /* IPv4 TCP/UDP packet */ +#define NETFLOW_V9_FLOW_V4_ICMP 2 /* IPv4 ICMP packet, currently unused */ +#define NETFLOW_V9_FLOW_V4_L3 3 /* IPv4 IP packet */ +#define NETFLOW_V9_FLOW_V6_L4 4 /* IPv6 TCP/UDP packet */ +#define NETFLOW_V9_FLOW_V6_ICMP 5 /* IPv6 ICMP packet, currently unused */ +#define NETFLOW_V9_FLOW_V6_L3 6 /* IPv6 IP packet */ + +#define NETFLOW_V9_FLOW_FAKE 65535 /* Not uset used in real flowsets! */ + struct netflow_v5_export_dgram { struct netflow_v5_header header; struct netflow_v5_record r[NETFLOW_V5_MAX_RECORDS]; } __attribute__((__packed__)); + +struct netflow_v9_export_dgram { + struct netflow_v9_header header; + char data; /* MTU can change, record length is dynamic */ +}; + +struct netflow_v9_flowset_header { + uint16_t id; /* FlowSet id */ + uint16_t length; /* FlowSet length */ +} __attribute__((__packed__)); + +struct netflow_v9_mbuf_tag { + struct m_tag tag; /* pointer to mbuf tag */ + uint16_t length; /* Current packet length */ + uint16_t count; /* current records count */ + uint16_t mtu; /* max MTU shapshot */ + uint16_t flow_type; /* current flowset */ + uint16_t flow_header; /* offset pointing to current flow header */ +}; diff -urN sys/netgraph/_netflow.orig/ng_netflow.c sys/netgraph/netflow/ng_netflow.c --- sys/netgraph/_netflow.orig/ng_netflow.c 2009-09-01 02:37:44.000000000 +0400 +++ sys/netgraph/netflow/ng_netflow.c 2009-09-06 16:36:08.000000000 +0400 @@ -30,6 +30,8 @@ static const char rcs_id[] = "@(#) $FreeBSD: src/sys/netgraph/netflow/ng_netflow.c,v 1.20 2009/05/13 02:26:34 mav Exp $"; +#include "opt_inet6.h" + #include #include #include @@ -48,8 +50,11 @@ #include #include #include +#include +#include #include #include +#include #include #include @@ -114,6 +119,30 @@ &ng_netflow_setconfig_type_fields }; +/* Parse type for ng_netflow_setversion */ +static const struct ng_parse_struct_field ng_netflow_setversion_type_fields[] + = NG_NETFLOW_SETVERSION_TYPE; +static const struct ng_parse_type ng_netflow_setversion_type = { + &ng_parse_struct_type, + &ng_netflow_setversion_type_fields +}; + +/* Parse type for ng_netflow_settemplateperiodic */ +static const struct ng_parse_struct_field ng_netflow_settemplateperiodic_type_fields[] + = NG_NETFLOW_SETTEMPLATEPERIODIC_TYPE; +static const struct ng_parse_type ng_netflow_settemplateperiodic_type = { + &ng_parse_struct_type, + &ng_netflow_settemplateperiodic_type_fields +}; + +/* Parse type for ng_netflow_setmtu */ +static const struct ng_parse_struct_field ng_netflow_setmtu_type_fields[] + = NG_NETFLOW_SETMTU_TYPE; +static const struct ng_parse_type ng_netflow_setmtu_type = { + &ng_parse_struct_type, + &ng_netflow_setmtu_type_fields +}; + /* List of commands and how to convert arguments to/from ASCII */ static const struct ng_cmdlist ng_netflow_cmds[] = { { @@ -158,6 +187,27 @@ &ng_netflow_setconfig_type, NULL }, + { + NGM_NETFLOW_COOKIE, + NGM_NETFLOW_SETVERSION, + "setversion", + &ng_netflow_setversion_type, + NULL + }, + { + NGM_NETFLOW_COOKIE, + NGM_NETFLOW_SETTEMPLATEPERIODIC, + "settemplateperiodic", + &ng_netflow_settemplateperiodic_type, + NULL + }, + { + NGM_NETFLOW_COOKIE, + NGM_NETFLOW_SETMTU, + "setmtu", + &ng_netflow_setmtu_type, + NULL + }, { 0 } }; @@ -202,6 +252,13 @@ for (i = 0; i < NG_NETFLOW_MAXIFACES; i++) priv->ifaces[i].info.conf = NG_NETFLOW_CONF_INGRESS; + /* Set v9 defaults */ + priv->version = NETFLOW_V9; + priv->templ_time = NETFLOW_V9_MAX_TIME_TEMPL; + priv->templ_packets = NETFLOW_V9_MAX_PACKETS_TEMPL; + priv->mtu = BASE_MTU; + priv->flow_seq = 0; + /* Initialize callout handle */ callout_init(&priv->exp_callout, CALLOUT_MPSAFE); @@ -434,6 +491,52 @@ break; } + case NGM_NETFLOW_SETVERSION: + { + struct ng_netflow_setversion *set; + + if (msg->header.arglen != sizeof(struct ng_netflow_setversion)) + ERROUT(EINVAL); + + set = (struct ng_netflow_setversion *)msg->data; + if ((set->version != NETFLOW_V9) && (set->version != NETFLOW_V5)) + ERROUT(EINVAL); + + ng_netflow_switch_version(priv, set->version, 0); + + break; + } + case NGM_NETFLOW_SETTEMPLATEPERIODIC: + { + struct ng_netflow_settemplateperiodic *set; + + if (msg->header.arglen != sizeof(struct ng_netflow_settemplateperiodic)) + ERROUT(EINVAL); + + set = (struct ng_netflow_settemplateperiodic *)msg->data; + + /* XXX: Set atomic here */ + priv->templ_packets = set->packets; + priv->templ_time = set->time; + + break; + } + case NGM_NETFLOW_SETMTU: + { + struct ng_netflow_setmtu *set; + + if (msg->header.arglen != sizeof(struct ng_netflow_setmtu)) + ERROUT(EINVAL); + + set = (struct ng_netflow_setmtu *)msg->data; + if ((set->mtu < MIN_MTU) || (set->mtu > MAX_MTU)) + ERROUT(EINVAL); + + /* XXX: Set atomic here */ + priv->mtu = set->mtu; + + break; + } case NGM_NETFLOW_SHOW: { uint32_t *last; @@ -481,12 +584,15 @@ const priv_p priv = NG_NODE_PRIVATE(node); const iface_p iface = NG_HOOK_PRIVATE(hook); hook_p out; - struct mbuf *m = NULL; - struct ip *ip; + struct mbuf *m, *m_old = NULL; + struct ip *ip = NULL; + struct ip6_hdr *ip6 = NULL; struct m_tag *mtag; - int pullup_len = 0; + int pullup_len = 0, off; + uint8_t upper_proto = 0, is_frag = 0; int error = 0, bypass = 0; unsigned int src_if_index; + caddr_t upper_ptr = NULL; if (hook == priv->export) { /* @@ -541,6 +647,7 @@ } NGI_GET_M(item, m); + m_old = m; /* Increase counters. */ iface->info.ifinfo_packets++; @@ -569,6 +676,22 @@ } \ } while (0) +/* +#define M_HCHECK(length) do { \ + pullup_len += length + sizeof(ip6_ext); \ + if ((m)->m_pkthdr.len < (pullup_len)) { \ + pullup_len -= length + sizeof(ip6_ext); + M_CHECK(length); + ` + } \ + if ((m)->m_len < (pullup_len) && \ + (((m) = m_pullup((m),(pullup_len))) == NULL)) { \ + error = ENOBUFS; \ + goto done; \ + } \ +} while (0) +*/ + switch (iface->info.ifinfo_dlt) { case DLT_EN10MB: /* Ethernet */ { @@ -586,6 +709,33 @@ eh = mtod(m, struct ether_header *); ip = (struct ip *)(eh + 1); break; +#ifdef INET6 + case ETHERTYPE_IPV6: + /* + * This is done not to pullup mbuf twice on every ext + * header + */ + pullup_len += sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + if (m->m_pkthdr.len >= pullup_len) { + if ((m = m_pullup(m, pullup_len)) == NULL) { + error = ENOBUFS; + goto done; + } + pullup_len -= sizeof(struct ip6_ext); + } else { + pullup_len -= sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + M_CHECK(sizeof(struct ip6_hdr)); + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + if (ip6->ip6_nxt != IPPROTO_NONE) { + error = EINVAL; + goto bypass; + } + } + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + break; +#endif case ETHERTYPE_VLAN: { struct ether_vlan_header *evh; @@ -597,6 +747,29 @@ M_CHECK(sizeof(struct ip)); ip = (struct ip *)(evh + 1); break; + } else if (ntohs(evh->evl_proto) == ETHERTYPE_IPV6) { +#ifdef INET6 + pullup_len += sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + if (m->m_pkthdr.len >= pullup_len) { + if ((m = m_pullup(m, pullup_len)) == NULL) { + error = ENOBUFS; + goto done; + } + pullup_len -= sizeof(struct ip6_ext); + } else { + pullup_len -= sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + M_CHECK(sizeof(struct ip6_hdr)); + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + if (ip6->ip6_nxt != IPPROTO_NONE) { + error = EINVAL; + goto bypass; + } + } + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + break; +#endif } } default: @@ -607,18 +780,32 @@ case DLT_RAW: /* IP packets */ M_CHECK(sizeof(struct ip)); ip = mtod(m, struct ip *); +#ifdef INET6 + if (ip->ip_v == 6) { + /* IPv6 packet */ + ip = NULL; + M_CHECK(sizeof(struct ip6_hdr)); + ip6 = mtod(m, struct ip6_hdr *); + } +#endif break; default: goto bypass; break; } - if ((ip->ip_off & htons(IP_OFFMASK)) == 0) { + if ((ip != NULL) && ((ip->ip_off & htons(IP_OFFMASK)) == 0)) { + if ((ip->ip_v != IPVERSION) || + ((ip->ip_hl << 2) < sizeof(struct ip))) + goto bypass; /* - * In case of IP header with options, we haven't pulled + * In case of IPv4 header with options, we haven't pulled * up enough, yet. */ pullup_len += (ip->ip_hl << 2) - sizeof(struct ip); + off = pullup_len; + //vlog("upper_offset = %d", off); + upper_proto = ip->ip_p; switch (ip->ip_p) { case IPPROTO_TCP: @@ -627,38 +814,161 @@ case IPPROTO_UDP: M_CHECK(sizeof(struct udphdr)); break; + case IPPROTO_SCTP: + M_CHECK(sizeof(struct sctphdr)); + break; } - } + } else if (ip != NULL) { + is_frag = 1; + upper_proto = ip->ip_p; + if ((ip->ip_v != IPVERSION) || + ((ip->ip_hl << 2) < sizeof(struct ip))) + goto bypass; + } else if (ip6 != NULL) { +#ifdef INET6 + /* Check if we can export */ + if (priv->version < NETFLOW_V9) + goto bypass; + + /* Loop thru IPv6 extended headers to get upper layer header / frag */ + int cur = ip6->ip6_nxt, hdr_off = 0; + struct ip6_ext *ip6e; + struct ip6_frag *ip6f; + + off = pullup_len; + upper_proto = cur; + + if ((ip6->ip6_vfc & IPV6_VERSION_MASK) != IPV6_VERSION) + goto bypass; + + while (42) { + + /* + * At the moment we're 'standing' with pullup_len and + * off + */ + switch (cur) { + + /* Ensure pointer is contiguous */ + case IPPROTO_TCP: + M_CHECK(sizeof(struct tcphdr)); + goto loopend; + case IPPROTO_UDP: + M_CHECK(sizeof(struct udphdr)); + goto loopend; + case IPPROTO_SCTP: + M_CHECK(sizeof(struct sctphdr)); + goto loopend; + + /* Loop until 'real' upper layer headers */ + case IPPROTO_HOPOPTS: + case IPPROTO_ROUTING: + case IPPROTO_DSTOPTS: + ip6e = (struct ip6_ext *)(mtod(m, caddr_t) + off); + upper_proto = ip6e->ip6e_nxt; + hdr_off = (ip6e->ip6e_len + 1) << 3; + break; - switch (iface->info.ifinfo_dlt) { - case DLT_EN10MB: - { - struct ether_header *eh; + /* RFC4302, can be before DSTOPTS */ + case IPPROTO_AH: + ip6e = (struct ip6_ext *)(mtod(m, caddr_t) + off); + upper_proto = ip6e->ip6e_nxt; + hdr_off = (ip6e->ip6e_len + 2) << 2; + break; - eh = mtod(m, struct ether_header *); - switch (ntohs(eh->ether_type)) { - case ETHERTYPE_IP: - ip = (struct ip *)(eh + 1); - break; - case ETHERTYPE_VLAN: - { - struct ether_vlan_header *evh; + + case IPPROTO_FRAGMENT: + ip6f = (struct ip6_frag *)(mtod(m, caddr_t) + off); + upper_proto = ip6f->ip6f_nxt; + hdr_off = sizeof(struct ip6_frag); + off += hdr_off; + is_frag = 1; + goto loopend; + +/* + case IPPROTO_NONE: + goto loopend; +*/ + default: + goto loopend; + } - evh = mtod(m, struct ether_vlan_header *); - ip = (struct ip *)(evh + 1); + off += hdr_off + sizeof(struct ip6_ext); + cur = upper_proto; + if (m->m_pkthdr.len >= off) { + if ((m = m_pullup(m, off)) == NULL) { + error = ENOBUFS; + goto done; + } + off -= sizeof(struct ip6_ext); + pullup_len += hdr_off; + } else { + /* + * Packet ends somewhere here. + * if its next header is not IPPROTO_NONE it is + * possibly mailformed packet. Let's count as RAW IP + */ + upper_proto = IPPROTO_NONE; + off -= sizeof(struct ip6_ext); + goto loopend; + } + + } +#endif + } + +#ifdef INET6 +loopend: +#endif + if (m != m_old) { + switch (iface->info.ifinfo_dlt) { + case DLT_EN10MB: /* Ethernet */ + { + struct ether_header *eh; + + eh = mtod(m, struct ether_header *); + switch (ntohs(eh->ether_type)) { + case ETHERTYPE_IP: + ip = (struct ip *)(eh + 1); + break; + case ETHERTYPE_IPV6: + ip6 = (struct ip6_hdr *)(eh + 1); + break; + case ETHERTYPE_VLAN: + { + struct ether_vlan_header *evh; + + evh = mtod(m, struct ether_vlan_header *); + if (ntohs(evh->evl_proto) == ETHERTYPE_IP) { + ip = (struct ip *)(evh + 1); + break; + } else if (ntohs(evh->evl_proto) == ETHERTYPE_IPV6) { +#ifdef INET6 + ip6 = (struct ip6_hdr *)(evh + 1); + break; +#endif + } + } + default: + panic("ng_netflow entered deadcode"); + } + break; + } + case DLT_RAW: /* IP packets */ + ip = mtod(m, struct ip *); +#ifdef INET6 + if (ip->ip_v == 6) { + /* IPv6 packet */ + ip = NULL; + ip6 = mtod(m, struct ip6_hdr *); + } +#endif break; - } default: panic("ng_netflow entered deadcode"); } - break; - } - case DLT_RAW: - ip = mtod(m, struct ip *); - break; - default: - panic("ng_netflow entered deadcode"); } + upper_ptr = (caddr_t)(mtod(m, caddr_t) + off); #undef M_CHECK @@ -670,7 +980,7 @@ } else src_if_index = iface->info.ifinfo_index; - error = ng_netflow_flow_add(priv, ip, src_if_index); + error = ng_netflow_flow_add(priv, (ip != NULL) ? (caddr_t)ip : (caddr_t)ip6, upper_ptr, upper_proto, is_frag, src_if_index); bypass: if (out != NULL) { diff -urN sys/netgraph/_netflow.orig/ng_netflow.h sys/netgraph/netflow/ng_netflow.h --- sys/netgraph/_netflow.orig/ng_netflow.h 2009-09-01 02:37:44.000000000 +0400 +++ sys/netgraph/netflow/ng_netflow.h 2009-09-06 16:36:17.000000000 +0400 @@ -51,6 +51,9 @@ NGM_NETFLOW_SETIFINDEX = 5, /* set interface index */ NGM_NETFLOW_SETTIMEOUTS = 6, /* set active/inactive flow timeouts */ NGM_NETFLOW_SETCONFIG = 7, /* set flow generation options */ + NGM_NETFLOW_SETVERSION = 8, /* set flow export version */ + NGM_NETFLOW_SETTEMPLATEPERIODIC = 9, /* set v9 flow template periodic */ + NGM_NETFLOW_SETMTU = 10, /* set outgoing interface MTU */ }; /* This structure is returned by the NGM_NETFLOW_INFO message */ @@ -105,10 +108,34 @@ u_int32_t conf; /* new config */ }; +/* This structure is passed to NGM_NETFLOW_SETVERSION */ +struct ng_netflow_setversion { + u_int8_t version; /* netflow version */ +}; + +/* This structure is passed to NGM_NETFLOW_SETTEMPLATEPERIODIC */ +struct ng_netflow_settemplateperiodic { + u_int16_t time; /* max time between announce */ + u_int16_t packets; /* max packets between announce */ +}; + +/* This structure is passed to NGM_NETFLOW_SETMTU */ +struct ng_netflow_setmtu { + u_int16_t mtu; /* MTU for packet */ +}; + /* This is unique data, which identifies flow */ struct flow_rec { - struct in_addr r_src; - struct in_addr r_dst; + uint16_t flow_type; /* IPv4 L4/L3 Ipv6 L4/L3 flow, see NETFLOW_V9_FLOW* */ + uint16_t ether_proto; /* unused */ + union { + struct in_addr r_src; + struct in6_addr r_src6; + } src; + union { + struct in_addr r_dst; + struct in6_addr r_dst6; + } dst; union { struct { uint16_t s_port; /* source TCP/UDP port */ @@ -137,7 +164,10 @@ /* A flow entry which accumulates statistics */ struct flow_entry_data { struct flow_rec r; - struct in_addr next_hop; + union { + struct in_addr next_hop; + struct in6_addr next_hop6; + } n; uint16_t fle_o_ifx; /* output interface index */ #define fle_i_ifx r.misc.i.i_ifx uint8_t dst_mask; /* destination route mask bits */ @@ -146,6 +176,7 @@ u_long bytes; long first; /* uptime on first packet */ long last; /* uptime on last packet */ + uint16_t proto; /* IP/IPv6 */ u_char tcp_flags; /* cumulative OR */ }; @@ -227,6 +258,26 @@ { NULL } \ } +/* Parse the setversion structure */ +#define NG_NETFLOW_SETVERSION_TYPE { \ + { "version", &ng_parse_uint8_type }, \ + { NULL } \ +} + +/* Parse the settemplateperiodic structure */ +#define NG_NETFLOW_SETTEMPLATEPERIODIC_TYPE { \ + { "time", &ng_parse_uint16_type }, \ + { "packets", &ng_parse_uint16_type }, \ + { NULL } \ +} + +/* Parse the setmtu structure */ +#define NG_NETFLOW_SETMTU_TYPE { \ + { "mtu", &ng_parse_uint16_type }, \ + { NULL } \ +} + + /* Private hook data */ struct ng_netflow_iface { hook_p hook; /* NULL when disconnected */ @@ -270,6 +321,20 @@ item_p export_item; struct mtx export_mtx; uint32_t flow_seq; /* current flow sequence */ + /* + * RFC 3954 clause 7.3 + * "Both options MUST be configurable by the user on the Exporter." + */ + uint16_t templ_time; /* time between sending templates */ + uint16_t templ_packets; /* packets between sending templates */ + uint32_t templ_last_ts; /* unixtime of last template announce */ + uint32_t templ_last_pkt; /* packets count on last template announce */ + uint32_t sent_packets; /* packets sent by exporeter; can be reached another way ? */ + u_char version; /* Netflow export version */ + u_char flowsets_count; /* current flowsets used */ + u_char flowset_records[NETFLOW_V9_MAX_FLOWSETS - 1]; /* Count of records in each flowset */ + uint16_t mtu; /* export interface MTU */ + struct netflow_v9_flowset_header *v9_flowsets[NETFLOW_V9_MAX_FLOWSETS - 1]; /* Pointers to pre-compiled flowsets */ struct ng_netflow_iface ifaces[NG_NETFLOW_MAXIFACES]; }; @@ -286,14 +351,15 @@ #define MTAG_NETFLOW 1221656444 #define MTAG_NETFLOW_CALLED 0 +#define MTAG_NETFLOW_V9 1 /* Prototypes for netflow.c */ int ng_netflow_cache_init(priv_p); void ng_netflow_cache_flush(priv_p); void ng_netflow_copyinfo(priv_p, struct ng_netflow_info *); timeout_t ng_netflow_expire; -int ng_netflow_flow_add(priv_p, struct ip *, unsigned int src_if_index); +int ng_netflow_flow_add(priv_p, caddr_t ip_ptr, caddr_t upper_ptr, uint8_t upper_proto, uint8_t is_frag, unsigned int src_if_index); int ng_netflow_flow_show(priv_p, uint32_t last, struct ng_mesg *); - +void ng_netflow_switch_version(priv_p priv, int ver, int boot); #endif /* _KERNEL */ #endif /* _NG_NETFLOW_H_ */ diff -urN sys/modules/netgraph/netflow/Makefile.orig sys/modules/netgraph/netflow/Makefile --- sys/modules/netgraph/netflow/Makefile.orig 2009-09-06 16:31:18.000000000 +0400 +++ sys/modules/netgraph/netflow/Makefile 2009-09-06 16:33:32.000000000 +0400 @@ -3,9 +3,19 @@ # Author: Gleb Smirnoff # +.include + .PATH: ${.CURDIR}/../../../netgraph/netflow KMOD= ng_netflow -SRCS= ng_netflow.c netflow.c +SRCS= ng_netflow.c netflow.c opt_inet6.h + +.if !defined(KERNBUILDDIR) + +.if ${MK_INET6_SUPPORT} != "no" +opt_inet6.h: + echo "#define INET6 1" > ${.TARGET} +.endif +.endif .include --------------010607050501030003050405 Content-Type: text/plain; name="netflow_v9_20090906_72.diff" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="netflow_v9_20090906_72.diff" diff -urN sys/netgraph/netflow/netflow.c.orig sys/netgraph/netflow/netflow.c --- sys/netgraph/netflow/netflow.c.orig 2009-01-09 23:55:26.000000000 +0300 +++ sys/netgraph/netflow/netflow.c 2009-09-06 18:51:53.000000000 +0400 @@ -30,6 +30,8 @@ static const char rcs_id[] = "@(#) $FreeBSD: src/sys/netgraph/netflow/netflow.c,v 1.25.2.3 2009/01/09 20:55:26 mav Exp $"; +#include "opt_inet6.h" + #include #include #include @@ -37,14 +39,18 @@ #include #include #include +#include #include +#include #include #include +#include #include #include #include +#include #include #include @@ -94,12 +100,99 @@ ((t) << 6) + /* 64 */ \ ((t) << 5) + /* 32 */ \ ((t) << 3)) /* 8 */ +/* + * Defines for different neflow version dispatchers + * + */ +static int export_add(item_p, struct flow_entry *); +static int export_add_v9(item_p, struct flow_entry *); +static int export_send(priv_p, item_p, int flags); +static int export_send_v9(priv_p, item_p, int flags); + +typedef int (*record_add_ptr)(item_p, struct flow_entry *); +typedef int (*record_send_ptr)(priv_p, item_p, int); +struct _netflow_dispatchers { + record_add_ptr record_add; + record_send_ptr record_send; +}; + +static struct _netflow_dispatchers netflow_dispatcher[] = +{ + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { export_add, export_send }, /* Version 5 */ + { NULL, NULL }, + { NULL, NULL }, + { NULL, NULL }, + { export_add_v9, export_send_v9 }, /* Version 9 */ + { NULL, NULL } +}; + +record_add_ptr record_add = NULL; +record_send_ptr record_send = NULL; MALLOC_DECLARE(M_NETFLOW_HASH); MALLOC_DEFINE(M_NETFLOW_HASH, "netflow_hash", "NetFlow hash"); -static int export_add(item_p, struct flow_entry *); -static int export_send(priv_p, item_p, int flags); + +static struct netflow_v9_template _netflow_v9_record_ipv4_tcp[] = +{ + { NETFLOW_V9_FIELD_IPV4_SRC_ADDR, 4}, + { NETFLOW_V9_FIELD_IPV4_DST_ADDR, 4}, + { NETFLOW_V9_FIELD_IPV4_NEXT_HOP, 4}, + { NETFLOW_V9_FIELD_INPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_OUTPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_IN_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_IN_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_FIRST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_LAST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_L4_SRC_PORT, 2}, + { NETFLOW_V9_FIELD_L4_DST_PORT, 2}, + { NETFLOW_V9_FIELD_TCP_FLAGS, 1}, + { NETFLOW_V9_FIELD_PROTOCOL, 1}, + { NETFLOW_V9_FIELD_TOS, 1}, + { NETFLOW_V9_FIELD_SRC_AS, 4}, + { NETFLOW_V9_FIELD_DST_AS, 4}, + { NETFLOW_V9_FIELD_SRC_MASK, 1}, + { NETFLOW_V9_FIELD_DST_MASK, 1}, + {0, 0} +}; + +static struct netflow_v9_template _netflow_v9_record_ipv6_tcp[] = +{ + { NETFLOW_V9_FIELD_IPV6_SRC_ADDR, 16}, + { NETFLOW_V9_FIELD_IPV6_DST_ADDR, 16}, + { NETFLOW_V9_FIELD_IPV6_NEXT_HOP, 16}, + { NETFLOW_V9_FIELD_INPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_OUTPUT_SNMP, 2}, + { NETFLOW_V9_FIELD_IN_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_IN_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_PKTS, sizeof(cntr)}, + { NETFLOW_V9_FIELD_OUT_BYTES, sizeof(cntr)}, + { NETFLOW_V9_FIELD_FIRST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_LAST_SWITCHED, 4}, + { NETFLOW_V9_FIELD_L4_SRC_PORT, 2}, + { NETFLOW_V9_FIELD_L4_DST_PORT, 2}, + { NETFLOW_V9_FIELD_TCP_FLAGS, 1}, + { NETFLOW_V9_FIELD_PROTOCOL, 1}, + { NETFLOW_V9_FIELD_TOS, 1}, + { NETFLOW_V9_FIELD_SRC_AS, 4}, + { NETFLOW_V9_FIELD_DST_AS, 4}, + { NETFLOW_V9_FIELD_SRC_MASK, 1}, + { NETFLOW_V9_FIELD_DST_MASK, 1}, + {0, 0} +}; + + +static int generate_v9_templates(priv_p priv); + +MALLOC_DECLARE(M_NETFLOW_GENERAL); +MALLOC_DEFINE(M_NETFLOW_GENERAL, "netflog_general", "plog, templates data"); /* Generate hash for a given flow record. */ static __inline uint32_t @@ -108,10 +201,24 @@ switch (r->r_ip_p) { case IPPROTO_TCP: case IPPROTO_UDP: - return FULL_HASH(r->r_src.s_addr, r->r_dst.s_addr, + return FULL_HASH(r->src.r_src.s_addr, r->dst.r_dst.s_addr, + r->r_sport, r->r_dport); + default: + return ADDR_HASH(r->src.r_src.s_addr, r->dst.r_dst.s_addr); + } +} + +/* Generate hash for a given flow record. Use lower 4 octets from v6 addresses */ +static __inline uint32_t +ip6_hash(struct flow_rec *r) +{ + switch (r->r_ip_p) { + case IPPROTO_TCP: + case IPPROTO_UDP: + return FULL_HASH(r->src.r_src6.__u6_addr.__u6_addr32[3], r->dst.r_dst6.__u6_addr.__u6_addr32[3], r->r_sport, r->r_dport); default: - return ADDR_HASH(r->r_src.s_addr, r->r_dst.s_addr); + return ADDR_HASH(r->src.r_src6.__u6_addr.__u6_addr32[3], r->dst.r_dst6.__u6_addr.__u6_addr32[3]); } } @@ -126,6 +233,7 @@ atomic_add_32(&priv->info.nfinfo_used, 1); + return (0); } @@ -141,6 +249,7 @@ /* * Detach export datagram from priv, if there is any. * If there is no, allocate a new one. + * -- V9/IPv6 ready */ static item_p get_export_dgram(priv_p priv) @@ -166,14 +275,163 @@ return (NULL); dgram = mtod(m, struct netflow_v5_export_dgram *); dgram->header.count = 0; - dgram->header.version = htons(NETFLOW_V5); + dgram->header.version = htons(priv->version); + + if (priv->version == NETFLOW_V9) { + atomic_fetchadd_32(&priv->sent_packets, 1); + + /* + * Let's insert mbuf tag to store some info + */ + struct netflow_v9_mbuf_tag *t; + struct m_tag *mt = m_tag_alloc(MTAG_NETFLOW, MTAG_NETFLOW_V9, sizeof(struct netflow_v9_mbuf_tag), M_NOWAIT); + if (mt == NULL) { + m_freem(m); + return (NULL); + } + + m_tag_init(m); + m_tag_prepend(m, mt); + t = (struct netflow_v9_mbuf_tag *)(mt + 1); + t->length = sizeof(struct netflow_v9_header); + t->count = 0; + t->mtu = priv->mtu; + t->flow_header = t->length; + + /* + * Check if we need to insert templates into packet + */ + + struct timespec ts; + struct netflow_v9_flowset_header *fl; + + getnanotime(&ts); + if ((ts.tv_sec >= priv->templ_time + priv->templ_last_ts) || (priv->sent_packets >= priv->templ_packets + priv->templ_last_pkt)) { + //vlog("INSERTING TEMPLATE: ts: %lu last: %u packets: %u last: %u delay: %u", ts.tv_sec, priv->templ_last_ts, priv->sent_packets, priv->templ_last_pkt, priv->templ_time); + atomic_store_rel_32(&priv->templ_last_ts, ts.tv_sec); + atomic_store_rel_32(&priv->templ_last_pkt, priv->sent_packets); + //vlog("ts: %lu last_t: %u last_p: %u delay: %u", ts.tv_sec, priv->templ_last_ts, priv->templ_last_pkt, priv->templ_time); + + fl = priv->v9_flowsets[0]; + bcopy(fl, (char *)dgram + t->length, ntohs(fl->length)); + + t->length += ntohs(fl->length); + t->flow_header = t->length; + t->count += priv->flowset_records[0]; + } + + } } return (item); } /* + * Pre-compiles flow exporter for all possible FlowSets + * so we can add flowset to packet via simple memcpy() + */ +#define __push(x) *p++ = htons((x)) +static int +generate_v9_templates(priv_p priv) +{ + uint16_t *p, *template_fields_cnt; + int cnt; + + int flowset_size = sizeof(struct netflow_v9_flowset_header) + + _NETFLOW_V9_TEMPLATE_SIZE(_netflow_v9_record_ipv4_tcp) + /* netflow_v9_record_ipv4_tcp */ + _NETFLOW_V9_TEMPLATE_SIZE(_netflow_v9_record_ipv6_tcp); /* netflow_v9_record_ipv6_tcp */ + + priv->v9_flowsets[0] = malloc(flowset_size, M_NETFLOW_GENERAL, M_WAITOK | M_ZERO); + if (priv->v9_flowsets[0] == NULL) + return (ENOMEM); + + + if (flowset_size % 4) + flowset_size += 4 - (flowset_size % 4); /* Padding to 4-byte boundary */ + + priv->flowsets_count = 1; + p = (uint16_t *)priv->v9_flowsets[0]; + *p++ = 0; /* Flowset ID, 0 is reserved for Template FlowSets */ + *p++ = htons(flowset_size); /* Total FlowSet length */ + + /* + * Most common TCP/UDP IPv4 template, ID = 256 + */ + *p++ = htons(0x100 + NETFLOW_V9_FLOW_V4_L4); + template_fields_cnt = p++; + for (cnt = 0; _netflow_v9_record_ipv4_tcp[cnt].field_id != 0; cnt++) { + *p++ = htons(_netflow_v9_record_ipv4_tcp[cnt].field_id); + *p++ = htons(_netflow_v9_record_ipv4_tcp[cnt].field_length); + } + *template_fields_cnt = htons(cnt); + + /* + * TCP/UDP IPv6 template, ID = 257 + */ + *p++ = htons(0x100 + NETFLOW_V9_FLOW_V6_L4); + template_fields_cnt = p++; + for (cnt = 0; _netflow_v9_record_ipv6_tcp[cnt].field_id != 0; cnt++) { + *p++ = htons(_netflow_v9_record_ipv6_tcp[cnt].field_id); + *p++ = htons(_netflow_v9_record_ipv6_tcp[cnt].field_length); + } + *template_fields_cnt = htons(cnt); + + + priv->flowset_records[0] = 2; + return (0); +} + +/* + * Switches version used for netflow export + * + */ +void +ng_netflow_switch_version(priv_p priv, int ver, int boot) +{ + item_p item = NULL; + + if ((ver != NETFLOW_V9) && (ver != NETFLOW_V5)) + return; + + if ((ver == priv->version) && (boot == 0)) + return; + + /* + * All new threads acquiring export datagram will wait for lock + * so we can change pointers. + * Existing threads with export datagram already held will call + * wrong export function which will do version check at the beginning + */ + + mtx_lock(&priv->export_mtx); + /* XXX: Need to be machine-independent here */ + record_add = netflow_dispatcher[ver].record_add; + record_send = netflow_dispatcher[ver].record_send; + //atomic_store_rel_ptr((unsigned long *)record_add, (unsigned long)netflow_dispatcher[ver].record_add); + //atomic_store_rel_ptr((unsigned long *)record_send, (unsigned long)netflow_dispatcher[ver].record_send); + item = priv->export_item; + priv->export_item = NULL; + mtx_unlock(&priv->export_mtx); + + if (ver == NETFLOW_V9) { + priv->templ_last_pkt = 0; + priv->templ_last_ts = 0; + } + + //vlog("NEW: add=%p send=%p", record_add, record_send); + + if (item != NULL) + (*netflow_dispatcher[priv->version].record_send)(priv, item, NG_NOFLAGS); + + if (boot) + log(LOG_DEBUG, "ng_netflow: v%d export started\n", ver); + else + log(LOG_DEBUG, "ng_netflow: export switched: v%d -> v%d\n", priv->version, ver); + priv->version = ver; +} + +/* * Re-attach incomplete datagram back to priv. * If there is already another one, then send incomplete. */ static void @@ -190,13 +448,14 @@ mtx_unlock(&priv->export_mtx); } else { mtx_unlock(&priv->export_mtx); - export_send(priv, item, flags); + (*record_send)(priv, item, flags); } } /* * The flow is over. Call export_add() and free it. If datagram is * full, then call export_send(). + * -- v9/IPv6 nearly ready */ static __inline void expire_flow(priv_p priv, item_p *item, struct flow_entry *fle, int flags) @@ -208,8 +467,8 @@ uma_zfree_arg(priv->zone, fle, priv); return; } - if (export_add(*item, fle) > 0) { - export_send(priv, *item, flags); + if ((*record_add)(*item, fle) > 0) { + (*record_send)(priv, *item, flags); *item = NULL; } uma_zfree_arg(priv->zone, fle, priv); @@ -234,12 +493,15 @@ * to be sure. */ static __inline int -hash_insert(priv_p priv, struct flow_hash_entry *hsh, struct flow_rec *r, +hash_insert(priv_p priv, struct flow_hash_entry *hsh, struct flow_rec *r, uint16_t eproto, int plen, uint8_t tcp_flags) { struct flow_entry *fle; - struct route ro; - struct sockaddr_in *sin; + struct sockaddr_in sin; +#ifdef INET6 + struct sockaddr_in6 sin6; +#endif + struct rtentry *rt; mtx_assert(&hsh->mtx, MA_OWNED); @@ -257,6 +519,7 @@ bcopy(r, &fle->f.r, sizeof(struct flow_rec)); fle->f.bytes = plen; fle->f.packets = 1; + fle->f.proto = eproto; fle->f.tcp_flags = tcp_flags; fle->f.first = fle->f.last = time_uptime; @@ -265,52 +528,93 @@ * First we do route table lookup on destination address. So we can * fill in out_ifx, dst_mask, nexthop, and dst_as in future releases. */ - bzero((caddr_t)&ro, sizeof(ro)); - sin = (struct sockaddr_in *)&ro.ro_dst; - sin->sin_len = sizeof(*sin); - sin->sin_family = AF_INET; - sin->sin_addr = fle->f.r.r_dst; - /* XXX MRT 0 as a default.. need the m here to get fib */ - rtalloc_ign_fib(&ro, RTF_CLONING, 0); - if (ro.ro_rt != NULL) { - struct rtentry *rt = ro.ro_rt; - - fle->f.fle_o_ifx = rt->rt_ifp->if_index; - - if (rt->rt_flags & RTF_GATEWAY && - rt->rt_gateway->sa_family == AF_INET) - fle->f.next_hop = - ((struct sockaddr_in *)(rt->rt_gateway))->sin_addr; - - if (rt_mask(rt)) - fle->f.dst_mask = bitcount32(((struct sockaddr_in *) - rt_mask(rt))->sin_addr.s_addr); - else if (rt->rt_flags & RTF_HOST) - /* Give up. We can't determine mask :( */ - fle->f.dst_mask = 32; - - RTFREE(ro.ro_rt); - } - - /* Do route lookup on source address, to fill in src_mask. */ - - bzero((caddr_t)&ro, sizeof(ro)); - sin = (struct sockaddr_in *)&ro.ro_dst; - sin->sin_len = sizeof(*sin); - sin->sin_family = AF_INET; - sin->sin_addr = fle->f.r.r_src; - rtalloc_ign_fib(&ro, RTF_CLONING, 0); /* XXX MRT */ - if (ro.ro_rt != NULL) { - struct rtentry *rt = ro.ro_rt; - - if (rt_mask(rt)) - fle->f.src_mask = bitcount32(((struct sockaddr_in *) - rt_mask(rt))->sin_addr.s_addr); - else if (rt->rt_flags & RTF_HOST) - /* Give up. We can't determine mask :( */ - fle->f.src_mask = 32; - - RTFREE(ro.ro_rt); + if (eproto == ETHERTYPE_IP) { + bzero(&sin, sizeof(sin)); + sin.sin_len = sizeof(struct sockaddr_in); + sin.sin_family = AF_INET; + sin.sin_addr = fle->f.r.dst.r_dst; + /* XXX MRT 0 as a default.. need the m here to get fib */ + rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, 0); + if (rt != NULL) { + fle->f.fle_o_ifx = rt->rt_ifp->if_index; + + if (rt->rt_flags & RTF_GATEWAY && + rt->rt_gateway->sa_family == AF_INET) + fle->f.n.next_hop = + ((struct sockaddr_in *)(rt->rt_gateway))->sin_addr; + + if (rt_mask(rt)) + fle->f.dst_mask = bitcount32(((struct sockaddr_in *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) + /* Give up. We can't determine mask :( */ + fle->f.dst_mask = 32; + + RTFREE_LOCKED(rt); + } + + /* Do route lookup on source address, to fill in src_mask. */ + bzero(&sin, sizeof(sin)); + sin.sin_len = sizeof(struct sockaddr_in); + sin.sin_family = AF_INET; + sin.sin_addr = fle->f.r.src.r_src; + /* XXX MRT 0 as a default revisit. need the mbuf for fib*/ + rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, 0); + if (rt != NULL) { + if (rt_mask(rt)) + fle->f.src_mask = bitcount32(((struct sockaddr_in *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) + /* Give up. We can't determine mask :( */ + fle->f.src_mask = 32; + + RTFREE_LOCKED(rt); + } + } else { +#ifdef INET6 + bzero(&sin6, sizeof(sin6)); + sin6.sin6_len = sizeof(struct sockaddr_in6); + sin6.sin6_family = AF_INET6; + sin6.sin6_addr = fle->f.r.dst.r_dst6; + /* XXX fib works for AF_INET only */ + rt = rtalloc1_fib((struct sockaddr *)&sin6, 0, 0, 0); + if (rt != NULL) { + fle->f.fle_o_ifx = rt->rt_ifp->if_index; + + if (rt->rt_flags & RTF_GATEWAY && + rt->rt_gateway->sa_family == AF_INET6) + fle->f.n.next_hop6 = + ((struct sockaddr_in6 *)(rt->rt_gateway))->sin6_addr; + if (rt_mask(rt)) +/* + fle->f.dst_mask = bitcount32(((struct sockaddr_in6 *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) */ + /* Give up. We can't determine mask :( */ + fle->f.dst_mask = 128; + + RTFREE_LOCKED(rt); + } + + /* Do route lookup on source address, to fill in src_mask. */ + bzero(&sin6, sizeof(sin6)); + sin6.sin6_len = sizeof(struct sockaddr_in6); + sin6.sin6_family = AF_INET6; + sin6.sin6_addr = fle->f.r.src.r_src6; + /* XXX MRT 0 as a default revisit. need the mbuf for fib*/ + rt = rtalloc1_fib((struct sockaddr *)&sin6, 0, 0, 0); + if (rt != NULL) { +/* + if (rt_mask(rt)) + fle->f.src_mask = bitcount32(((struct sockaddr_in6 *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) */ + /* Give up. We can't determine mask :( */ + fle->f.src_mask = 128; + + RTFREE_LOCKED(rt); + } +#endif } /* Push new flow at the and of hash. */ @@ -354,6 +658,10 @@ mtx_init(&priv->export_mtx, "export dgram lock", NULL, MTX_DEF); + generate_v9_templates(priv); + + ng_netflow_switch_version(priv, priv->version, 1); + return (0); } @@ -366,6 +674,7 @@ item_p item = NULL; int i; + /* * We are going to free probably billable data. * Expire everything before freeing it. @@ -378,7 +687,7 @@ } if (item != NULL) - export_send(priv, item, NG_QUEUE); + (*record_send)(priv, item, NG_QUEUE); uma_zdestroy(priv->zone); @@ -390,46 +699,34 @@ if (priv->hash) FREE(priv->hash, M_NETFLOW_HASH); + /* FreeFlow Tables */ + for (i = 0; i < priv->flowsets_count; i++) + free(priv->v9_flowsets[i], M_NETFLOW_GENERAL); + mtx_destroy(&priv->export_mtx); } -/* Insert packet from into flow cache. */ +/* + * Insert packet from into flow cache. Assume size/version check passed + */ int -ng_netflow_flow_add(priv_p priv, struct ip *ip, unsigned int src_if_index) +ng_netflow_flow_add(priv_p priv, caddr_t ip_ptr, caddr_t upper_ptr, uint8_t upper_proto, uint8_t is_frag, unsigned int src_if_index) { register struct flow_entry *fle, *fle1; struct flow_hash_entry *hsh; struct flow_rec r; + struct ip *ip = NULL; +#ifdef INET6 + struct ip6_hdr *ip6 = NULL; +#endif item_p item = NULL; - int hlen, plen; + int plen; int error = 0; uint8_t tcp_flags = 0; - - /* Try to fill flow_rec r */ - bzero(&r, sizeof(r)); - /* check version */ - if (ip->ip_v != IPVERSION) - return (EINVAL); - - /* verify min header length */ - hlen = ip->ip_hl << 2; - - if (hlen < sizeof(struct ip)) - return (EINVAL); - - r.r_src = ip->ip_src; - r.r_dst = ip->ip_dst; - - /* save packet length */ - plen = ntohs(ip->ip_len); - - r.r_ip_p = ip->ip_p; - r.r_tos = ip->ip_tos; - - r.r_i_ifx = src_if_index; + uint16_t eproto; /* - * XXX NOTE: only first fragment of fragmented TCP, UDP and + * XXX Fragmentation NOTE: only first fragment of fragmented TCP, UDP and * ICMP packet will be recorded with proper s_port and d_port. * Following fragments will be recorded simply as IP packet with * ip_proto = ip->ip_p and s_port, d_port set to zero. @@ -437,22 +734,91 @@ * ip packet assebmling here. Anyway, (in)famous trafd works this way - * and nobody complains yet :) */ - if ((ip->ip_off & htons(IP_OFFMASK)) == 0) - switch(r.r_ip_p) { - case IPPROTO_TCP: - { - register struct tcphdr *tcp; - - tcp = (struct tcphdr *)((caddr_t )ip + hlen); - r.r_sport = tcp->th_sport; - r.r_dport = tcp->th_dport; - tcp_flags = tcp->th_flags; - break; + + /* Try to fill flow_rec r */ + bzero(&r, sizeof(r)); + ip = (struct ip *)ip_ptr; + if (ip->ip_v == IPVERSION) { + eproto = ETHERTYPE_IP; + r.src.r_src = ip->ip_src; + r.dst.r_dst = ip->ip_dst; + + /* Assume L$ template by default */ + r.flow_type = NETFLOW_V9_FLOW_V4_L4; + + /* save packet length */ + plen = ntohs(ip->ip_len); + + r.r_tos = ip->ip_tos; + + if (is_frag == 0) { + switch(upper_proto) { + case IPPROTO_TCP: + { + register struct tcphdr *tcp; + + tcp = (struct tcphdr *)upper_ptr; + r.r_ports = *(uint32_t *)upper_ptr; + tcp_flags = tcp->th_flags; + /* r.flow_type = NETFLOW_V9_FLOW_V4_L4; */ + break; + } + case IPPROTO_UDP: + case IPPROTO_SCTP: + { + r.r_ports = *(uint32_t *)upper_ptr; + /* r.flow_type = NETFLOW_V9_FLOW_V4_L4; */ + break; + } + + } } + + } else { +#ifdef INET6 + /* IPv6 traffic */ + ip = NULL; + ip6 = (struct ip6_hdr *)ip_ptr; + eproto = ETHERTYPE_IPV6; + + r.src.r_src6 = ip6->ip6_src; + r.dst.r_dst6 = ip6->ip6_dst; + + /* Assume L4 template by default */ + r.flow_type = NETFLOW_V9_FLOW_V6_L4; + + /* save packet length */ + plen = ntohs(ip6->ip6_plen) + sizeof(struct ip6_hdr); + + //r.r_tos = ip->ip_tos; + + if (is_frag == 0) { + switch(upper_proto) { + case IPPROTO_TCP: + { + register struct tcphdr *tcp; + + tcp = (struct tcphdr *)upper_ptr; + r.r_ports = *(uint32_t *)upper_ptr; + tcp_flags = tcp->th_flags; + /* r.flow_type = NETFLOW_V9_FLOW_V6_L4; */ + break; + } case IPPROTO_UDP: - r.r_ports = *(uint32_t *)((caddr_t )ip + hlen); - break; + case IPPROTO_SCTP: + { + r.r_ports = *(uint32_t *)upper_ptr; + /* r.flow_type = NETFLOW_V9_FLOW_V6_L4; */ + break; + } + + } } +#endif + } + + r.r_ip_p = upper_proto; + r.r_i_ifx = src_if_index; /* Update node statistics. XXX: race... */ priv->info.nfinfo_packets ++; @@ -493,7 +859,7 @@ * - it is going to overflow counter */ if (tcp_flags & TH_FIN || tcp_flags & TH_RST || AGED(fle) || - (fle->f.bytes >= (UINT_MAX - IF_MAXMTU)) ) { + (fle->f.bytes >= (cntr_max - IF_MAXMTU)) ) { TAILQ_REMOVE(&hsh->head, fle, fle_hash); expire_flow(priv, &item, fle, NG_QUEUE); atomic_add_32(&priv->info.nfinfo_act_exp, 1); @@ -509,7 +875,7 @@ } } } else /* A new flow entry. */ - error = hash_insert(priv, hsh, &r, plen, tcp_flags); + error = hash_insert(priv, hsh, &r, eproto, plen, tcp_flags); mtx_unlock(&hsh->mtx); @@ -618,6 +984,65 @@ return (error); } +/* We have full datagram in privdata. Send it to export hook. */ +static int +export_send_v9(priv_p priv, item_p item, int flags) +{ + struct mbuf *m = NGI_M(item); + struct netflow_v9_export_dgram *dgram = mtod(m, + struct netflow_v9_export_dgram *); + struct netflow_v9_header *header = &dgram->header; + struct timespec ts; + int error = 0; + uint16_t len = 0; + + struct netflow_v9_mbuf_tag *t; + struct netflow_v9_flowset_header *fs = NULL; + struct m_tag *mt = m_tag_locate(m, MTAG_NETFLOW, MTAG_NETFLOW_V9, NULL); + + if (mt == NULL) { + /* */ + log(LOG_DEBUG, "ng_netflow: V9 export packet without tag!\n"); + return (0); + } + + t = (struct netflow_v9_mbuf_tag *)(mt + 1); + + /* Close FlowSet if not closed */ + if (t->length != t->flow_header) { + fs = (struct netflow_v9_flowset_header *)(mtod(m, char *) + t->flow_header); + len = (uint16_t)(t->length - t->flow_header); + if (len % 4) { + t->length += 4 - (len % 4); + len += 4 - (len % 4); + } + fs->length = htons(len); + } + + + + /* Fill mbuf header. */ + m->m_len = m->m_pkthdr.len = t->length; + + /* Fill export header. */ + header->count = t->count; + header->sys_uptime = htonl(MILLIUPTIME(time_uptime)); + getnanotime(&ts); + header->unix_secs = htonl(ts.tv_sec); + header->seq_num = htonl(atomic_fetchadd_32(&priv->flow_seq, 1)); + header->count = htons(t->count); + header->source_id = htonl(NG_NODE_ID(priv->node)); + + /* remove tag */ + m_tag_delete_chain(m, NULL); + + if (priv->export != NULL) + NG_FWD_ITEM_HOOK_FLAGS(error, item, priv->export, flags); + else + NG_FREE_ITEM(item); + + return (error); +} /* Add export record to dgram. */ static int @@ -635,9 +1060,9 @@ ("ng_netflow: export too big")); /* Fill in export record. */ - rec->src_addr = fle->f.r.r_src.s_addr; - rec->dst_addr = fle->f.r.r_dst.s_addr; - rec->next_hop = fle->f.next_hop.s_addr; + rec->src_addr = fle->f.r.src.r_src.s_addr; + rec->dst_addr = fle->f.r.dst.r_dst.s_addr; + rec->next_hop = fle->f.n.next_hop.s_addr; rec->i_ifx = htons(fle->f.fle_i_ifx); rec->o_ifx = htons(fle->f.fle_o_ifx); rec->packets = htonl(fle->f.packets); @@ -661,6 +1086,135 @@ return (0); } +/* Add V9 record to dgram. */ +static int +export_add_v9(item_p item, struct flow_entry *fle) +{ + size_t len = 0; + uint16_t new_flow = 0; + void *offset_ptr; + struct netflow_v9_mbuf_tag *t; + struct netflow_v9_flowset_header *fs = NULL; + struct mbuf *m = NGI_M(item); + struct m_tag *mt = m_tag_locate(m, MTAG_NETFLOW, MTAG_NETFLOW_V9, NULL); + + if (mt == NULL) { + /* */ + log(LOG_DEBUG, "ng_netflow: V9 export packet without tag!\n"); + return (0); + } + + t = (struct netflow_v9_mbuf_tag *)(mt + 1); + offset_ptr = (mtod(m, char *) + t->length); + + /* Check if new records has the same template */ + if (fle->f.r.flow_type != t->flow_type) { + new_flow = 1; + + /* 'Close' old FlowSet */ + fs = (struct netflow_v9_flowset_header *)(mtod(m, char *) + t->flow_header); + len = (uint16_t)(t->length - t->flow_header); + if (len % 4) { + t->length += 4 - (len % 4); + len += 4 - (len % 4); + } + fs->length = htons(len); + + /* Prepare 'new', but do not modify any counters here because switch can fail */ + t->flow_type = NETFLOW_V9_FLOW_FAKE; + t->flow_header = t->length; + fs = (struct netflow_v9_flowset_header *)(mtod(m, char *) + t->length); + offset_ptr = (fs + 1); + } + + + switch (fle->f.r.flow_type) { + case NETFLOW_V9_FLOW_V4_L4: + { + /* IPv4 TCP/UDP/[SCTP] */ + struct netflow_v9_record_ipv4_tcp *rec = (struct netflow_v9_record_ipv4_tcp *)offset_ptr; + + + rec->src_addr = fle->f.r.src.r_src.s_addr; + rec->dst_addr = fle->f.r.dst.r_dst.s_addr; + rec->next_hop = fle->f.n.next_hop.s_addr; + rec->i_ifx = htons(fle->f.fle_i_ifx); + rec->o_ifx = htons(fle->f.fle_o_ifx); + rec->i_packets = htonl(fle->f.packets); + rec->i_octets = htonl(fle->f.bytes); + rec->o_packets = htonl(0); + rec->o_octets = htonl(0); + rec->first = htonl(MILLIUPTIME(fle->f.first)); + rec->last = htonl(MILLIUPTIME(fle->f.last)); + rec->s_port = fle->f.r.r_sport; + rec->d_port = fle->f.r.r_dport; + rec->flags = fle->f.tcp_flags; + rec->prot = fle->f.r.r_ip_p; + rec->tos = fle->f.r.r_tos; + rec->dst_mask = fle->f.dst_mask; + rec->src_mask = fle->f.src_mask; + + /* Not supported fields. */ + rec->src_as = rec->dst_as = 0; + + len = sizeof(struct netflow_v9_record_ipv4_tcp); + break; + } +#ifdef INET6 + case NETFLOW_V9_FLOW_V6_L4: + { + /* IPv6 TCP/UDP/[SCTP] */ + struct netflow_v9_record_ipv6_tcp *rec = (struct netflow_v9_record_ipv6_tcp *)offset_ptr; + + /* ACHTUNG! unchecked code! */ + rec->src_addr = fle->f.r.src.r_src6; + rec->dst_addr = fle->f.r.dst.r_dst6; + rec->next_hop = fle->f.n.next_hop6; + rec->i_ifx = htons(fle->f.fle_i_ifx); + rec->o_ifx = htons(fle->f.fle_o_ifx); + rec->i_packets = htonl(fle->f.packets); + rec->i_octets = htonl(fle->f.bytes); + rec->first = htonl(MILLIUPTIME(fle->f.first)); + rec->last = htonl(MILLIUPTIME(fle->f.last)); + rec->s_port = fle->f.r.r_sport; + rec->d_port = fle->f.r.r_dport; + rec->flags = fle->f.tcp_flags; + rec->prot = fle->f.r.r_ip_p; + rec->tos = fle->f.r.r_tos; + rec->dst_mask = fle->f.dst_mask; + rec->src_mask = fle->f.src_mask; + + /* Not supported fields. */ + rec->src_as = rec->dst_as = 0; + + len = sizeof(struct netflow_v9_record_ipv6_tcp); + break; + } +#endif + default: + { + log(LOG_DEBUG, "ng_netflow: Don't know what to do with %d flow type!\n", fle->f.r.flow_type); + return (0); + } + } + + if (new_flow) { + /* Generate data segment ID */ + fs->id = htons(0x100 + fle->f.r.flow_type); + t->flow_type = fle->f.r.flow_type; + t->length += sizeof(struct netflow_v9_flowset_header); + } + + t->length += len; + t->count++; + + if (t->length + NETFLOW_V9_MAX_RECORD_SIZE + 4 >= _NETFLOW_V9_MAX_SIZE(t->mtu)) + return (1); /* end of datagram */ + else + return (0); +} + + /* Periodic flow expiry run. */ void ng_netflow_expire(void *arg) diff -urN sys/netgraph/netflow/ng_netflow.c.orig sys/netgraph/netflow/ng_netflow.c --- sys/netgraph/netflow/ng_netflow.c.orig 2009-01-09 23:55:26.000000000 +0300 +++ sys/netgraph/netflow/ng_netflow.c 2009-09-06 18:38:40.000000000 +0400 @@ -30,6 +30,8 @@ static const char rcs_id[] = "@(#) $FreeBSD: src/sys/netgraph/netflow/ng_netflow.c,v 1.14.2.4 2009/01/09 20:55:26 mav Exp $"; +#include "opt_inet6.h" + #include #include #include @@ -48,8 +50,11 @@ #include #include #include +#include +#include #include #include +#include #include #include @@ -114,6 +119,30 @@ &ng_netflow_setconfig_type_fields }; +/* Parse type for ng_netflow_setversion */ +static const struct ng_parse_struct_field ng_netflow_setversion_type_fields[] + = NG_NETFLOW_SETVERSION_TYPE; +static const struct ng_parse_type ng_netflow_setversion_type = { + &ng_parse_struct_type, + &ng_netflow_setversion_type_fields +}; + +/* Parse type for ng_netflow_settemplateperiodic */ +static const struct ng_parse_struct_field ng_netflow_settemplateperiodic_type_fields[] + = NG_NETFLOW_SETTEMPLATEPERIODIC_TYPE; +static const struct ng_parse_type ng_netflow_settemplateperiodic_type = { + &ng_parse_struct_type, + &ng_netflow_settemplateperiodic_type_fields +}; + +/* Parse type for ng_netflow_setmtu */ +static const struct ng_parse_struct_field ng_netflow_setmtu_type_fields[] + = NG_NETFLOW_SETMTU_TYPE; +static const struct ng_parse_type ng_netflow_setmtu_type = { + &ng_parse_struct_type, + &ng_netflow_setmtu_type_fields +}; + /* List of commands and how to convert arguments to/from ASCII */ static const struct ng_cmdlist ng_netflow_cmds[] = { { @@ -158,6 +187,27 @@ &ng_netflow_setconfig_type, NULL }, + { + NGM_NETFLOW_COOKIE, + NGM_NETFLOW_SETVERSION, + "setversion", + &ng_netflow_setversion_type, + NULL + }, + { + NGM_NETFLOW_COOKIE, + NGM_NETFLOW_SETTEMPLATEPERIODIC, + "settemplateperiodic", + &ng_netflow_settemplateperiodic_type, + NULL + }, + { + NGM_NETFLOW_COOKIE, + NGM_NETFLOW_SETMTU, + "setmtu", + &ng_netflow_setmtu_type, + NULL + }, { 0 } }; @@ -202,6 +252,13 @@ for (i = 0; i < NG_NETFLOW_MAXIFACES; i++) priv->ifaces[i].info.conf = NG_NETFLOW_CONF_INGRESS; + /* Set v9 defaults */ + priv->version = NETFLOW_V9; + priv->templ_time = NETFLOW_V9_MAX_TIME_TEMPL; + priv->templ_packets = NETFLOW_V9_MAX_PACKETS_TEMPL; + priv->mtu = BASE_MTU; + priv->flow_seq = 0; + /* Initialize callout handle */ callout_init(&priv->exp_callout, CALLOUT_MPSAFE); @@ -434,6 +491,52 @@ break; } + case NGM_NETFLOW_SETVERSION: + { + struct ng_netflow_setversion *set; + + if (msg->header.arglen != sizeof(struct ng_netflow_setversion)) + ERROUT(EINVAL); + + set = (struct ng_netflow_setversion *)msg->data; + if ((set->version != NETFLOW_V9) && (set->version != NETFLOW_V5)) + ERROUT(EINVAL); + + ng_netflow_switch_version(priv, set->version, 0); + + break; + } + case NGM_NETFLOW_SETTEMPLATEPERIODIC: + { + struct ng_netflow_settemplateperiodic *set; + + if (msg->header.arglen != sizeof(struct ng_netflow_settemplateperiodic)) + ERROUT(EINVAL); + + set = (struct ng_netflow_settemplateperiodic *)msg->data; + + /* XXX: Set atomic here */ + priv->templ_packets = set->packets; + priv->templ_time = set->time; + + break; + } + case NGM_NETFLOW_SETMTU: + { + struct ng_netflow_setmtu *set; + + if (msg->header.arglen != sizeof(struct ng_netflow_setmtu)) + ERROUT(EINVAL); + + set = (struct ng_netflow_setmtu *)msg->data; + if ((set->mtu < MIN_MTU) || (set->mtu > MAX_MTU)) + ERROUT(EINVAL); + + /* XXX: Set atomic here */ + priv->mtu = set->mtu; + + break; + } case NGM_NETFLOW_SHOW: { uint32_t *last; @@ -481,12 +584,15 @@ const priv_p priv = NG_NODE_PRIVATE(node); const iface_p iface = NG_HOOK_PRIVATE(hook); hook_p out; - struct mbuf *m = NULL; - struct ip *ip; + struct mbuf *m, *m_old = NULL; + struct ip *ip = NULL; + struct ip6_hdr *ip6 = NULL; struct m_tag *mtag; - int pullup_len = 0; + int pullup_len = 0, off = 0; + uint8_t upper_proto = 0, is_frag = 0; int error = 0, bypass = 0; unsigned int src_if_index; + caddr_t upper_ptr = NULL; if (hook == priv->export) { /* @@ -541,6 +647,7 @@ } NGI_GET_M(item, m); + m_old = m; /* Increase counters. */ iface->info.ifinfo_packets++; @@ -569,6 +676,22 @@ } \ } while (0) +/* +#define M_HCHECK(length) do { \ + pullup_len += length + sizeof(ip6_ext); \ + if ((m)->m_pkthdr.len < (pullup_len)) { \ + pullup_len -= length + sizeof(ip6_ext); + M_CHECK(length); + ` + } \ + if ((m)->m_len < (pullup_len) && \ + (((m) = m_pullup((m),(pullup_len))) == NULL)) { \ + error = ENOBUFS; \ + goto done; \ + } \ +} while (0) +*/ + switch (iface->info.ifinfo_dlt) { case DLT_EN10MB: /* Ethernet */ { @@ -586,6 +709,33 @@ eh = mtod(m, struct ether_header *); ip = (struct ip *)(eh + 1); break; +#ifdef INET6 + case ETHERTYPE_IPV6: + /* + * This is done not to pullup mbuf twice on every ext + * header + */ + pullup_len += sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + if (m->m_pkthdr.len >= pullup_len) { + if ((m = m_pullup(m, pullup_len)) == NULL) { + error = ENOBUFS; + goto done; + } + pullup_len -= sizeof(struct ip6_ext); + } else { + pullup_len -= sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + M_CHECK(sizeof(struct ip6_hdr)); + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + if (ip6->ip6_nxt != IPPROTO_NONE) { + error = EINVAL; + goto bypass; + } + } + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + break; +#endif case ETHERTYPE_VLAN: { struct ether_vlan_header *evh; @@ -597,6 +747,29 @@ M_CHECK(sizeof(struct ip)); ip = (struct ip *)(evh + 1); break; + } else if (ntohs(evh->evl_proto) == ETHERTYPE_IPV6) { +#ifdef INET6 + pullup_len += sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + if (m->m_pkthdr.len >= pullup_len) { + if ((m = m_pullup(m, pullup_len)) == NULL) { + error = ENOBUFS; + goto done; + } + pullup_len -= sizeof(struct ip6_ext); + } else { + pullup_len -= sizeof(struct ip6_hdr) + sizeof(struct ip6_ext); + M_CHECK(sizeof(struct ip6_hdr)); + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + if (ip6->ip6_nxt != IPPROTO_NONE) { + error = EINVAL; + goto bypass; + } + } + eh = mtod(m, struct ether_header *); + ip6 = (struct ip6_hdr *)(eh + 1); + break; +#endif } } default: @@ -607,18 +780,32 @@ case DLT_RAW: /* IP packets */ M_CHECK(sizeof(struct ip)); ip = mtod(m, struct ip *); +#ifdef INET6 + if (ip->ip_v == 6) { + /* IPv6 packet */ + ip = NULL; + M_CHECK(sizeof(struct ip6_hdr)); + ip6 = mtod(m, struct ip6_hdr *); + } +#endif break; default: goto bypass; break; } - if ((ip->ip_off & htons(IP_OFFMASK)) == 0) { + if ((ip != NULL) && ((ip->ip_off & htons(IP_OFFMASK)) == 0)) { + if ((ip->ip_v != IPVERSION) || + ((ip->ip_hl << 2) < sizeof(struct ip))) + goto bypass; /* - * In case of IP header with options, we haven't pulled + * In case of IPv4 header with options, we haven't pulled * up enough, yet. */ pullup_len += (ip->ip_hl << 2) - sizeof(struct ip); + off = pullup_len; + //vlog("upper_offset = %d", off); + upper_proto = ip->ip_p; switch (ip->ip_p) { case IPPROTO_TCP: @@ -627,38 +814,161 @@ case IPPROTO_UDP: M_CHECK(sizeof(struct udphdr)); break; + case IPPROTO_SCTP: + M_CHECK(sizeof(struct sctphdr)); + break; } - } + } else if (ip != NULL) { + is_frag = 1; + upper_proto = ip->ip_p; + if ((ip->ip_v != IPVERSION) || + ((ip->ip_hl << 2) < sizeof(struct ip))) + goto bypass; + } else if (ip6 != NULL) { +#ifdef INET6 + /* Check if we can export */ + if (priv->version < NETFLOW_V9) + goto bypass; + + /* Loop thru IPv6 extended headers to get upper layer header / frag */ + int cur = ip6->ip6_nxt, hdr_off = 0; + struct ip6_ext *ip6e; + struct ip6_frag *ip6f; + + off = pullup_len; + upper_proto = cur; + + if ((ip6->ip6_vfc & IPV6_VERSION_MASK) != IPV6_VERSION) + goto bypass; + + while (42) { + + /* + * At the moment we're 'standing' with pullup_len and + * off + */ + switch (cur) { + + /* Ensure pointer is contiguous */ + case IPPROTO_TCP: + M_CHECK(sizeof(struct tcphdr)); + goto loopend; + case IPPROTO_UDP: + M_CHECK(sizeof(struct udphdr)); + goto loopend; + case IPPROTO_SCTP: + M_CHECK(sizeof(struct sctphdr)); + goto loopend; + + /* Loop until 'real' upper layer headers */ + case IPPROTO_HOPOPTS: + case IPPROTO_ROUTING: + case IPPROTO_DSTOPTS: + ip6e = (struct ip6_ext *)(mtod(m, caddr_t) + off); + upper_proto = ip6e->ip6e_nxt; + hdr_off = (ip6e->ip6e_len + 1) << 3; + break; - switch (iface->info.ifinfo_dlt) { - case DLT_EN10MB: - { - struct ether_header *eh; + /* RFC4302, can be before DSTOPTS */ + case IPPROTO_AH: + ip6e = (struct ip6_ext *)(mtod(m, caddr_t) + off); + upper_proto = ip6e->ip6e_nxt; + hdr_off = (ip6e->ip6e_len + 2) << 2; + break; - eh = mtod(m, struct ether_header *); - switch (ntohs(eh->ether_type)) { - case ETHERTYPE_IP: - ip = (struct ip *)(eh + 1); - break; - case ETHERTYPE_VLAN: - { - struct ether_vlan_header *evh; + + case IPPROTO_FRAGMENT: + ip6f = (struct ip6_frag *)(mtod(m, caddr_t) + off); + upper_proto = ip6f->ip6f_nxt; + hdr_off = sizeof(struct ip6_frag); + off += hdr_off; + is_frag = 1; + goto loopend; + +/* + case IPPROTO_NONE: + goto loopend; +*/ + default: + goto loopend; + } - evh = mtod(m, struct ether_vlan_header *); - ip = (struct ip *)(evh + 1); + off += hdr_off + sizeof(struct ip6_ext); + cur = upper_proto; + if (m->m_pkthdr.len >= off) { + if ((m = m_pullup(m, off)) == NULL) { + error = ENOBUFS; + goto done; + } + off -= sizeof(struct ip6_ext); + pullup_len += hdr_off; + } else { + /* + * Packet ends somewhere here. + * if its next header is not IPPROTO_NONE it is + * possibly mailformed packet. Let's count as RAW IP + */ + upper_proto = IPPROTO_NONE; + off -= sizeof(struct ip6_ext); + goto loopend; + } + + } +#endif + } + +#ifdef INET6 +loopend: +#endif + if (m != m_old) { + switch (iface->info.ifinfo_dlt) { + case DLT_EN10MB: /* Ethernet */ + { + struct ether_header *eh; + + eh = mtod(m, struct ether_header *); + switch (ntohs(eh->ether_type)) { + case ETHERTYPE_IP: + ip = (struct ip *)(eh + 1); + break; + case ETHERTYPE_IPV6: + ip6 = (struct ip6_hdr *)(eh + 1); + break; + case ETHERTYPE_VLAN: + { + struct ether_vlan_header *evh; + + evh = mtod(m, struct ether_vlan_header *); + if (ntohs(evh->evl_proto) == ETHERTYPE_IP) { + ip = (struct ip *)(evh + 1); + break; + } else if (ntohs(evh->evl_proto) == ETHERTYPE_IPV6) { +#ifdef INET6 + ip6 = (struct ip6_hdr *)(evh + 1); + break; +#endif + } + } + default: + panic("ng_netflow entered deadcode"); + } + break; + } + case DLT_RAW: /* IP packets */ + ip = mtod(m, struct ip *); +#ifdef INET6 + if (ip->ip_v == 6) { + /* IPv6 packet */ + ip = NULL; + ip6 = mtod(m, struct ip6_hdr *); + } +#endif break; - } default: panic("ng_netflow entered deadcode"); } - break; - } - case DLT_RAW: - ip = mtod(m, struct ip *); - break; - default: - panic("ng_netflow entered deadcode"); } + upper_ptr = (caddr_t)(mtod(m, caddr_t) + off); #undef M_CHECK @@ -670,7 +980,7 @@ } else src_if_index = iface->info.ifinfo_index; - error = ng_netflow_flow_add(priv, ip, src_if_index); + error = ng_netflow_flow_add(priv, (ip != NULL) ? (caddr_t)ip : (caddr_t)ip6, upper_ptr, upper_proto, is_frag, src_if_index); bypass: if (out != NULL) { diff -urN sys/netgraph/_netflow.orig/netflow.h sys/netgraph/netflow/netflow.h --- sys/netgraph/_netflow.orig/netflow.h 2009-09-01 02:37:44.000000000 +0400 +++ sys/netgraph/netflow/netflow.h 2009-09-06 12:41:29.000000000 +0400 @@ -46,6 +46,7 @@ #define NETFLOW_V1 1 #define NETFLOW_V5 5 +#define NETFLOW_V9 9 struct netflow_v1_header { @@ -69,6 +70,16 @@ uint16_t pad; /* Pad to word boundary */ } __attribute__((__packed__)); +struct netflow_v9_header +{ + uint16_t version; /* NetFlow version */ + uint16_t count; /* Total number of records in packet */ + uint32_t sys_uptime; /* System uptime */ + uint32_t unix_secs; /* Current seconds since 0000 UTC 1970 */ + uint32_t seq_num; /* Sequence number */ + uint32_t source_id; /* Observation Domain id */ +} __attribute__((__packed__)); + struct netflow_v1_record { uint32_t src_addr; /* Source IP address */ @@ -115,6 +126,142 @@ uint16_t pad2; /* Pad to word boundary */ } __attribute__((__packed__)); + + +/* RFC3954 field definitions */ +#define NETFLOW_V9_FIELD_IN_BYTES 1 /* Input bytes count for a flow. Default 4, can be 8 */ +#define NETFLOW_V9_FIELD_IN_PKTS 2 /* Incoming counter with number of packets associated with an IP Flow. Default 4 */ +#define NETFLOW_V9_FIELD_FLOWS 3 /* Number of Flows that were aggregated. Default 4 */ +#define NETFLOW_V9_FIELD_PROTOCOL 4 /* IP protocol byte. 1 */ +#define NETFLOW_V9_FIELD_TOS 5 /* Type of service byte setting when entering the incoming interface. 1 */ +#define NETFLOW_V9_FIELD_TCP_FLAGS 6 /* TCP flags; cumulative of all the TCP flags seen in this Flow. 1 */ +#define NETFLOW_V9_FIELD_L4_SRC_PORT 7 /* TCP/UDP source port number. 2 */ +#define NETFLOW_V9_FIELD_IPV4_SRC_ADDR 8 /* IPv4 source address. 4 */ +#define NETFLOW_V9_FIELD_SRC_MASK 9 /* The number of contiguous bits in the source subnet mask (i.e., the mask in slash notation). 1 */ +#define NETFLOW_V9_FIELD_INPUT_SNMP 10 /* Input interface index. Default 2 */ +#define NETFLOW_V9_FIELD_L4_DST_PORT 11 /* TCP/UDP destination port number. 2 */ +#define NETFLOW_V9_FIELD_IPV4_DST_ADDR 12 /* IPv4 destination address. 4 */ +#define NETFLOW_V9_FIELD_DST_MASK 13 /* The number of contiguous bits in the destination subnet mask (i.e., the mask in slash notation). 1 */ +#define NETFLOW_V9_FIELD_OUTPUT_SNMP 14 /* Output interface index. Default 2 */ +#define NETFLOW_V9_FIELD_IPV4_NEXT_HOP 15 /* IPv4 address of the next-hop router. 4 */ +#define NETFLOW_V9_FIELD_SRC_AS 16 /* Source BGP autonomous system number. Default 2, can be 4 */ +#define NETFLOW_V9_FIELD_DST_AS 17 /* Destination BGP autonomous system number. Default 2, can be 4 */ +#define NETFLOW_V9_FIELD_BGP_IPV4_NEXT_HOP 18 /* Next-hop router's IP address in the BGP domain. 4 */ +#define NETFLOW_V9_FIELD_MUL_DST_PKTS 19 /* IP multicast outgoing packet counter for packets associated with IP flow. Default 4 */ +#define NETFLOW_V9_FIELD_MUL_DST_BYTES 20 /* IP multicast outgoing Octet (byte) counter for the number of bytes associated with IP flow. Default 4 */ +#define NETFLOW_V9_FIELD_LAST_SWITCHED 21 /* sysUptime in msec at which the last packet of this Flow was switched. 4 */ +#define NETFLOW_V9_FIELD_FIRST_SWITCHED 22 /* sysUptime in msec at which the first packet of this Flow was switched. 4 */ +#define NETFLOW_V9_FIELD_OUT_BYTES 23 /* Outgoing counter for the number of bytes associated with an IP Flow. Default 4 */ +#define NETFLOW_V9_FIELD_OUT_PKTS 24 /* Outgoing counter for the number of packets associated with an IP Flow. Default 4 */ +#define NETFLOW_V9_FIELD_IPV6_SRC_ADDR 27 /* IPv6 source address. 16 */ +#define NETFLOW_V9_FIELD_IPV6_DST_ADDR 28 /* IPv6 destination address. 16 */ +#define NETFLOW_V9_FIELD_IPV6_SRC_MASK 29 /* Length of the IPv6 source mask in contiguous bits. 1 */ +#define NETFLOW_V9_FIELD_IPV6_DST_MASK 30 /* Length of the IPv6 destination mask in contiguous bits. 1 */ +#define NETFLOW_V9_FIELD_IPV6_FLOW_LABEL 31 /* IPv6 flow label as per RFC 2460 definition. 3 */ +#define NETFLOW_V9_FIELD_ICMP_TYPE 32 /* Internet Control Message Protocol (ICMP) packet type; reported as ICMP Type * 256 + ICMP code. 2 */ +#define NETFLOW_V9_FIELD_MUL_IGMP_TYPE 33 /* Internet Group Management Protocol (IGMP) packet type. 1 */ +#define NETFLOW_V9_FIELD_SAMPLING_INTERVAL 34 /* When using sampled NetFlow, the rate at which packets are sampled; for example, a value of 100 indicates that one of every hundred packets is sampled. 4 */ +#define NETFLOW_V9_FIELD_SAMPLING_ALGORITHM 35 /* For sampled NetFlow platform-wide: 0x01 deterministic sampling 0x02 random sampling. 1 */ +#define NETFLOW_V9_FIELD_FLOW_ACTIVE_TIMEOUT 36 /* Timeout value (in seconds) for active flow entries in the NetFlow cache. 2 */ +#define NETFLOW_V9_FIELD_FLOW_INACTIVE_TIMEOUT 37 /* Timeout value (in seconds) for inactive Flow entries in the NetFlow cache. 2 */ +#define NETFLOW_V9_FIELD_ENGINE_TYPE 38 /* Type of Flow switching engine (route processor, linecard, etc...). 1 */ +#define NETFLOW_V9_FIELD_ENGINE_ID 39 /* ID number of the Flow switching engine. 1 */ +#define NETFLOW_V9_FIELD_TOTAL_BYTES_EXP 40 /* Counter with for the number of bytes exported by the Observation Domain. Default 4 */ +#define NETFLOW_V9_FIELD_TOTAL_PKTS_EXP 41 /* Counter with for the number of packets exported by the Observation Domain. Default 4 */ +#define NETFLOW_V9_FIELD_TOTAL_FLOWS_EXP 42 /* Counter with for the number of flows exported by the Observation Domain. Default 4 */ +#define NETFLOW_V9_FIELD_MPLS_TOP_LABEL_TYPE 46 /* MPLS Top Label Type. 1 */ +#define NETFLOW_V9_FIELD_MPLS_TOP_LABEL_IP_ADDR 47 /* Forwarding Equivalent Class corresponding to the MPLS Top Label. 4 */ +#define NETFLOW_V9_FIELD_FLOW_SAMPLER_ID 48 /* Identifier shown in "show flow-sampler". 1 */ +#define NETFLOW_V9_FIELD_FLOW_SAMPLER_MODE 49 /* The type of algorithm used for sampling data. 2 */ +#define NETFLOW_V9_FIELD_FLOW_SAMPLER_RANDOM_INTERVAL 50 /* Packet interval at which to sample. 4. */ +#define NETFLOW_V9_FIELD_DST_TOS 55 /* Type of Service byte setting when exiting outgoing interface. 1. */ +#define NETFLOW_V9_FIELD_SRC_MAC 56 /* Source MAC Address. 6 */ +#define NETFLOW_V9_FIELD_DST_MAC 57 /* Destination MAC Address. 6 */ +#define NETFLOW_V9_FIELD_SRC_VLAN 58 /* Virtual LAN identifier associated with ingress interface. 2 */ +#define NETFLOW_V9_FIELD_DST_VLAN 59 /* irtual LAN identifier associated with egress interface. 2 */ +#define NETFLOW_V9_FIELD_IP_PROTOCOL_VERSION 60 /* Internet Protocol Version. Set to 4 for IPv4, set to 6 for IPv6. If not present in the template, then version 4 is assumed. 1. */ +#define NETFLOW_V9_FIELD_DIRECTION 61 /* Flow direction: 0 - ingress flow 1 - egress flow. 1 */ +#define NETFLOW_V9_FIELD_IPV6_NEXT_HOP 62 /* IPv6 address of the next-hop router. 16 */ +#define NETFLOW_V9_FIELD_BGP_IPV6_NEXT_HOP 63 /* Next-hop router in the BGP domain. 16 */ +#define NETFLOW_V9_FIELD_IPV6_OPTION_HEADERS 64 /* Bit-encoded field identifying IPv6 option headers found in the flow */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_1 70 /* MPLS label at position 1 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_2 71 /* MPLS label at position 2 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_3 72 /* MPLS label at position 3 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_4 73 /* MPLS label at position 4 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_5 74 /* MPLS label at position 5 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_6 75 /* MPLS label at position 6 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_7 76 /* MPLS label at position 7 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_8 77 /* MPLS label at position 8 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_9 78 /* MPLS label at position 9 in the stack. 3 */ +#define NETFLOW_V9_FIELD_MPLS_LABEL_10 79 /* MPLS label at position 10 in the stack. 3 */ + +#ifdef COUNTERS_64 +#define cntr uint64_t +#define cntr_max UINT64_MAX +#else +#define cntr uint32_t +#define cntr_max UINT_MAX +#endif + +struct netflow_v9_template +{ + int field_id; + int field_length; +}; + +/* Template ID for tcp/udp v4 streams ID:257 (0x100 + NETFLOW_V9_FLOW_V4_L4) */ +struct netflow_v9_record_ipv4_tcp +{ + uint32_t src_addr; /* Source IPv4 address (IPV4_SRC_ADDR) */ + uint32_t dst_addr; /* Destination IPv4 address (IPV4_DST_ADDR) */ + uint32_t next_hop; /* Next hop IPv4 address (IPV4_NEXT_HOP) */ + uint16_t i_ifx; /* Source interface index (INPUT_SNMP) */ + uint16_t o_ifx; /* Destination interface index (OUTPUT_SNMP) */ + cntr i_packets; /* Number of incoming packets in a flow (IN_PKTS) */ + cntr i_octets; /* Number of incoming octets in a flow (IN_BYTES) */ + cntr o_packets; /* Number of outgoing packets in a flow (OUT_PKTS) */ + cntr o_octets; /* Number of outgoing octets in a flow (OUT_BYTES) */ + uint32_t first; /* System uptime at start of a flow (FIRST_SWITCHED) */ + uint32_t last; /* System uptime at end of a flow (LAST_SWITCHED) */ + uint16_t s_port; /* Source port (L4_SRC_PORT) */ + uint16_t d_port; /* Destination port (L4_DST_PORT) */ + uint8_t flags; /* Cumulative OR of tcp flags (TCP_FLAGS) */ + uint8_t prot; /* IP protocol */ + uint8_t tos; /* IP type of service IN (or OUT) (TOS) */ + uint32_t src_as; /* Src peer/origin Autonomous System (SRC_AS) */ + uint32_t dst_as; /* Dst peer/origin Autonomous System (DST_AS) */ + uint8_t src_mask; /* Source route's mask bits (SRC_MASK) */ + uint8_t dst_mask; /* Destination route's mask bits (DST_MASK) */ +} __attribute__((__packed__)); + + +/* Template ID for tcp/udp v6 streams ID: 260 (0x100 + NETFLOW_V9_FLOW_V6_L4) */ +struct netflow_v9_record_ipv6_tcp +{ + struct in6_addr src_addr; /* Source IPv6 address (IPV6_SRC_ADDR) */ + struct in6_addr dst_addr; /* Destination IPv6 address (IPV6_DST_ADDR) */ + struct in6_addr next_hop; /* Next hop IPv6 address (IPV6_NEXT_HOP) */ + uint16_t i_ifx; /* Source interface index (INPUT_SNMP) */ + uint16_t o_ifx; /* Destination interface index (OUTPUT_SNMP) */ + cntr i_packets; /* Number of incoming packets in a flow (IN_PKTS) */ + cntr i_octets; /* Number of incoming octets in a flow (IN_BYTES) */ + cntr o_packets; /* Number of outgoing packets in a flow (OUT_PKTS) */ + cntr o_octets; /* Number of outgoing octets in a flow (OUT_BYTES) */ + uint32_t first; /* System uptime at start of a flow (FIRST_SWITCHED) */ + uint32_t last; /* System uptime at end of a flow (LAST_SWITCHED) */ + uint16_t s_port; /* Source port (L4_SRC_PORT) */ + uint16_t d_port; /* Destination port (L4_DST_PORT) */ + uint8_t flags; /* Cumulative OR of tcp flags (TCP_FLAGS) */ + uint8_t prot; /* IP protocol */ + uint8_t tos; /* IP type of service IN (or OUT) (TOS) */ + uint32_t src_as; /* Src peer/origin Autonomous System (SRC_AS) */ + uint32_t dst_as; /* Dst peer/origin Autonomous System (DST_AS) */ + uint8_t src_mask; /* Source route's mask bits (SRC_MASK) */ + uint8_t dst_mask; /* Destination route's mask bits (DST_MASK) */ +} __attribute__((__packed__)); + +#define vlog(x, ...) log(LOG_DEBUG, "%s:%d " x "\n", __FUNCTION__, __LINE__, ##__VA_ARGS__) + #define NETFLOW_V1_MAX_RECORDS 24 #define NETFLOW_V5_MAX_RECORDS 30 @@ -123,7 +270,50 @@ #define NETFLOW_V5_MAX_SIZE (sizeof(netflow_v5_header)+ \ sizeof(netflow_v5_record)*NETFLOW_V5_MAX_RECORDS) +#define BASE_MTU 1500 +#define MIN_MTU sizeof(struct netflow_v5_header) +#define MAX_MTU 16384 +#define NETFLOW_V9_MAX_SIZE _NETFLOW_V9_MAX_SIZE(BASE_MTU) +#define _NETFLOW_V9_MAX_SIZE(x) (x) - sizeof(struct ip6_hdr) - sizeof(struct udphdr) +#define NETFLOW_V9_MAX_FLOWSETS 2 + +#define NETFLOW_V9_MAX_RECORD_SIZE sizeof(struct netflow_v9_record_ipv6_tcp) +#define NETFLOW_V9_MAX_PACKETS_TEMPL 500 /* Send data templates every ... packets */ +#define NETFLOW_V9_MAX_TIME_TEMPL 600 /* Send data templates every ... seconds */ +#define NETFLOW_V9_MAX_TEMPLATES 16 /* Not a real value */ +#define _NETFLOW_V9_TEMPLATE_SIZE(x) (sizeof(x) / sizeof(struct netflow_v9_template)) * 4 +//#define _NETFLOW_V9_TEMPLATE_SIZE(x) ((x) + 1) * 4 + +/* Flow Templates */ +#define NETFLOW_V9_FLOW_V4_L4 1 /* IPv4 TCP/UDP packet */ +#define NETFLOW_V9_FLOW_V4_ICMP 2 /* IPv4 ICMP packet, currently unused */ +#define NETFLOW_V9_FLOW_V4_L3 3 /* IPv4 IP packet */ +#define NETFLOW_V9_FLOW_V6_L4 4 /* IPv6 TCP/UDP packet */ +#define NETFLOW_V9_FLOW_V6_ICMP 5 /* IPv6 ICMP packet, currently unused */ +#define NETFLOW_V9_FLOW_V6_L3 6 /* IPv6 IP packet */ + +#define NETFLOW_V9_FLOW_FAKE 65535 /* Not uset used in real flowsets! */ + struct netflow_v5_export_dgram { struct netflow_v5_header header; struct netflow_v5_record r[NETFLOW_V5_MAX_RECORDS]; } __attribute__((__packed__)); + +struct netflow_v9_export_dgram { + struct netflow_v9_header header; + char data; /* MTU can change, record length is dynamic */ +}; + +struct netflow_v9_flowset_header { + uint16_t id; /* FlowSet id */ + uint16_t length; /* FlowSet length */ +} __attribute__((__packed__)); + +struct netflow_v9_mbuf_tag { + struct m_tag tag; /* pointer to mbuf tag */ + uint16_t length; /* Current packet length */ + uint16_t count; /* current records count */ + uint16_t mtu; /* max MTU shapshot */ + uint16_t flow_type; /* current flowset */ + uint16_t flow_header; /* offset pointing to current flow header */ +}; diff -urN sys/netgraph/_netflow.orig/ng_netflow.h sys/netgraph/netflow/ng_netflow.h --- sys/netgraph/_netflow.orig/ng_netflow.h 2009-09-01 02:37:44.000000000 +0400 +++ sys/netgraph/netflow/ng_netflow.h 2009-09-06 16:36:17.000000000 +0400 @@ -51,6 +51,9 @@ NGM_NETFLOW_SETIFINDEX = 5, /* set interface index */ NGM_NETFLOW_SETTIMEOUTS = 6, /* set active/inactive flow timeouts */ NGM_NETFLOW_SETCONFIG = 7, /* set flow generation options */ + NGM_NETFLOW_SETVERSION = 8, /* set flow export version */ + NGM_NETFLOW_SETTEMPLATEPERIODIC = 9, /* set v9 flow template periodic */ + NGM_NETFLOW_SETMTU = 10, /* set outgoing interface MTU */ }; /* This structure is returned by the NGM_NETFLOW_INFO message */ @@ -105,10 +108,34 @@ u_int32_t conf; /* new config */ }; +/* This structure is passed to NGM_NETFLOW_SETVERSION */ +struct ng_netflow_setversion { + u_int8_t version; /* netflow version */ +}; + +/* This structure is passed to NGM_NETFLOW_SETTEMPLATEPERIODIC */ +struct ng_netflow_settemplateperiodic { + u_int16_t time; /* max time between announce */ + u_int16_t packets; /* max packets between announce */ +}; + +/* This structure is passed to NGM_NETFLOW_SETMTU */ +struct ng_netflow_setmtu { + u_int16_t mtu; /* MTU for packet */ +}; + /* This is unique data, which identifies flow */ struct flow_rec { - struct in_addr r_src; - struct in_addr r_dst; + uint16_t flow_type; /* IPv4 L4/L3 Ipv6 L4/L3 flow, see NETFLOW_V9_FLOW* */ + uint16_t ether_proto; /* unused */ + union { + struct in_addr r_src; + struct in6_addr r_src6; + } src; + union { + struct in_addr r_dst; + struct in6_addr r_dst6; + } dst; union { struct { uint16_t s_port; /* source TCP/UDP port */ @@ -137,7 +164,10 @@ /* A flow entry which accumulates statistics */ struct flow_entry_data { struct flow_rec r; - struct in_addr next_hop; + union { + struct in_addr next_hop; + struct in6_addr next_hop6; + } n; uint16_t fle_o_ifx; /* output interface index */ #define fle_i_ifx r.misc.i.i_ifx uint8_t dst_mask; /* destination route mask bits */ @@ -146,6 +176,7 @@ u_long bytes; long first; /* uptime on first packet */ long last; /* uptime on last packet */ + uint16_t proto; /* IP/IPv6 */ u_char tcp_flags; /* cumulative OR */ }; @@ -227,6 +258,26 @@ { NULL } \ } +/* Parse the setversion structure */ +#define NG_NETFLOW_SETVERSION_TYPE { \ + { "version", &ng_parse_uint8_type }, \ + { NULL } \ +} + +/* Parse the settemplateperiodic structure */ +#define NG_NETFLOW_SETTEMPLATEPERIODIC_TYPE { \ + { "time", &ng_parse_uint16_type }, \ + { "packets", &ng_parse_uint16_type }, \ + { NULL } \ +} + +/* Parse the setmtu structure */ +#define NG_NETFLOW_SETMTU_TYPE { \ + { "mtu", &ng_parse_uint16_type }, \ + { NULL } \ +} + + /* Private hook data */ struct ng_netflow_iface { hook_p hook; /* NULL when disconnected */ @@ -270,6 +321,20 @@ item_p export_item; struct mtx export_mtx; uint32_t flow_seq; /* current flow sequence */ + /* + * RFC 3954 clause 7.3 + * "Both options MUST be configurable by the user on the Exporter." + */ + uint16_t templ_time; /* time between sending templates */ + uint16_t templ_packets; /* packets between sending templates */ + uint32_t templ_last_ts; /* unixtime of last template announce */ + uint32_t templ_last_pkt; /* packets count on last template announce */ + uint32_t sent_packets; /* packets sent by exporeter; can be reached another way ? */ + u_char version; /* Netflow export version */ + u_char flowsets_count; /* current flowsets used */ + u_char flowset_records[NETFLOW_V9_MAX_FLOWSETS - 1]; /* Count of records in each flowset */ + uint16_t mtu; /* export interface MTU */ + struct netflow_v9_flowset_header *v9_flowsets[NETFLOW_V9_MAX_FLOWSETS - 1]; /* Pointers to pre-compiled flowsets */ struct ng_netflow_iface ifaces[NG_NETFLOW_MAXIFACES]; }; @@ -286,14 +351,15 @@ #define MTAG_NETFLOW 1221656444 #define MTAG_NETFLOW_CALLED 0 +#define MTAG_NETFLOW_V9 1 /* Prototypes for netflow.c */ int ng_netflow_cache_init(priv_p); void ng_netflow_cache_flush(priv_p); void ng_netflow_copyinfo(priv_p, struct ng_netflow_info *); timeout_t ng_netflow_expire; -int ng_netflow_flow_add(priv_p, struct ip *, unsigned int src_if_index); +int ng_netflow_flow_add(priv_p, caddr_t ip_ptr, caddr_t upper_ptr, uint8_t upper_proto, uint8_t is_frag, unsigned int src_if_index); int ng_netflow_flow_show(priv_p, uint32_t last, struct ng_mesg *); - +void ng_netflow_switch_version(priv_p priv, int ver, int boot); #endif /* _KERNEL */ #endif /* _NG_NETFLOW_H_ */ diff -urN sys/modules/netgraph/netflow/Makefile.orig sys/modules/netgraph/netflow/Makefile --- sys/modules/netgraph/netflow/Makefile.orig 2009-09-06 16:31:18.000000000 +0400 +++ sys/modules/netgraph/netflow/Makefile 2009-09-06 16:33:32.000000000 +0400 @@ -3,9 +3,19 @@ # Author: Gleb Smirnoff # +.include + .PATH: ${.CURDIR}/../../../netgraph/netflow KMOD= ng_netflow -SRCS= ng_netflow.c netflow.c +SRCS= ng_netflow.c netflow.c opt_inet6.h + +.if !defined(KERNBUILDDIR) + +.if ${MK_INET6_SUPPORT} != "no" +opt_inet6.h: + echo "#define INET6 1" > ${.TARGET} +.endif +.endif .include --------------010607050501030003050405-- From owner-freebsd-isp@FreeBSD.ORG Sun Sep 6 22:31:53 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 453DF106566B; Sun, 6 Sep 2009 22:31:53 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-ew0-f208.google.com (mail-ew0-f208.google.com [209.85.219.208]) by mx1.freebsd.org (Postfix) with ESMTP id 9F5718FC08; Sun, 6 Sep 2009 22:31:52 +0000 (UTC) Received: by ewy4 with SMTP id 4so433236ewy.36 for ; Sun, 06 Sep 2009 15:31:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:cc:content-type; bh=sutaW3IPsfV3eys2+pVWlFqLpUnQfD8zn1Z8gPCXNog=; b=BtjNLnwqoBYhHiB4hkvApoioAS8G7+ZHzdM68wveZp5mx/ik876C9yy+8XSAh0OwPk 0ilg4Rklc9DI2SssxT7F7jQtIDuWshoFtjuf7yEY1dFlHIAvBt2fx3hVJhK3tHuh7Kro Nnmk8CtLz54+OAUogncF4EPcogbRd+9s1sK9A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=r5fKkDA/0on5BmEp88YiMvwY/+/m13sQ5QbHBCCpttZ/xrh+hqRRxeICl6ZWYvJWvs +1tWiF9ZjhpM+LxGb0i4u80Xv/CZhd4BXhuVI8CGrHL22LhFqebzaZmIZiJZOjeJfNvI vqZ82zTwAIUhMz/1QZog0Wj4XtcRK6RLS/QRA= MIME-Version: 1.0 Received: by 10.211.154.17 with SMTP id g17mr864569ebo.32.1252275070144; Sun, 06 Sep 2009 15:11:10 -0700 (PDT) Date: Sun, 6 Sep 2009 18:11:10 -0400 Message-ID: From: grarpamp To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-isp@freebsd.org Subject: Call for testers: ng_netflow with v9 and IPv6 support X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Sep 2009 22:31:53 -0000 Wouldn't it be better to support the obvious formal emergent standards track protocol instead of the legacy informational one? Or to perform both via sysctl or other arguments/defines, with the standard IPFIX being the default mode? Have you reviewed the nProbe code for other various ideas? Thanks for working on netflow :) And any form of v6 is good to have finally. ========================================================================= Network Working Group B. Claise, Ed. Request for Comments: 5101 Cisco Systems, Inc. Category: Standards Track January 2008 Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information This document specifies an Internet standards track protocol for the Internet community Network Working Group J. Quittek Request for Comments: 5102 NEC Category: Standards Track S. Bryant B. Claise P. Aitken Cisco Systems, Inc. J. Meyer PayPal January 2008 Information Model for IP Flow Information Export This document specifies an Internet standards track protocol for the Internet community Network Working Group B. Trammell Request for Comments: 5103 CERT/NetSA Category: Standards Track E. Boschi Hitachi Europe January 2008 Bidirectional Flow Export Using IP Flow Information Export (IPFIX) This document specifies an Internet standards track protocol for the Internet community Network Working Group E. Boschi Request for Comments: 5153 Hitachi Europe Category: Informational L. Mark Fraunhofer FOKUS J. Quittek M. Stiemerling NEC P. Aitken Cisco Systems, Inc. April 2008 IP Flow Information Export (IPFIX) Implementation Guidelines This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Network Working Group S. Leinen Request for Comments: 3955 SWITCH Category: Informational October 2004 Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX) This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Network Working Group J. Quittek Request for Comments: 3917 NEC Europe Ltd. Category: Informational T. Zseby Fraunhofer FOKUS B. Claise Cisco Systems S. Zander Swinburne University October 2004 Requirements for IP Flow Information Export (IPFIX) This memo provides information for the Internet community. It does not specify an Internet standard of any kind. And others... ========================================================================= Network Working Group B. Claise, Ed. Request for Comments: 3954 Cisco Systems Category: Informational October 2004 Cisco Systems NetFlow Services Export Version 9 This RFC documents the NetFlow services export protocol Version 9 as it was when submitted to the IETF as a basis for further work in the IPFIX WG. This RFC itself is not a candidate for any level of Internet Standard. The IETF disclaims any knowledge of the fitness of this RFC for any purpose... And others... ========================================================================= From owner-freebsd-isp@FreeBSD.ORG Mon Sep 7 01:37:00 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8382106566B for ; Mon, 7 Sep 2009 01:37:00 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outL.internet-mail-service.net (outl.internet-mail-service.net [216.240.47.235]) by mx1.freebsd.org (Postfix) with ESMTP id 9D4D18FC12 for ; Mon, 7 Sep 2009 01:37:00 +0000 (UTC) Received: from idiom.com (mx0.idiom.com [216.240.32.160]) by out.internet-mail-service.net (Postfix) with ESMTP id 74522B98AB; Sun, 6 Sep 2009 18:25:25 -0700 (PDT) X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (home.elischer.org [216.240.48.38]) by idiom.com (Postfix) with ESMTP id CD7E42D6006; Sun, 6 Sep 2009 18:25:24 -0700 (PDT) Message-ID: <4AA46103.9000108@elischer.org> Date: Sun, 06 Sep 2009 18:25:23 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: grarpamp References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-isp@freebsd.org, freebsd-net@freebsd.org Subject: Re: Call for testers: ng_netflow with v9 and IPv6 support X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2009 01:37:00 -0000 grarpamp wrote: > Wouldn't it be better to support the obvious formal emergent standards > track protocol instead of the legacy informational one? Or to perform > both via sysctl or other arguments/defines, with the standard IPFIX > being the default mode? Have you reviewed the nProbe code for other > various ideas? Thanks for working on netflow :) And any form of v6 > is good to have finally. if you have time it sounds like you know enough about it to be useful.. get involved :-) > > ========================================================================= > > Network Working Group B. Claise, Ed. > Request for Comments: 5101 Cisco Systems, Inc. > Category: Standards Track January 2008 > Specification of the IP Flow Information Export (IPFIX) Protocol > for the Exchange of IP Traffic Flow Information > This document specifies an Internet standards track protocol for the > Internet community > > > Network Working Group J. Quittek > Request for Comments: 5102 NEC > Category: Standards Track S. Bryant > B. Claise > P. Aitken > Cisco Systems, Inc. > J. Meyer > PayPal > January 2008 > Information Model for IP Flow Information Export > This document specifies an Internet standards track protocol for the > Internet community > > > Network Working Group B. Trammell > Request for Comments: 5103 CERT/NetSA > Category: Standards Track E. Boschi > Hitachi Europe > January 2008 > Bidirectional Flow Export Using IP Flow Information Export (IPFIX) > This document specifies an Internet standards track protocol for the > Internet community > > > Network Working Group E. Boschi > Request for Comments: 5153 Hitachi Europe > Category: Informational L. Mark > Fraunhofer FOKUS > J. Quittek > M. Stiemerling > NEC > P. Aitken > Cisco Systems, Inc. > April 2008 > IP Flow Information Export (IPFIX) Implementation Guidelines > This memo provides information for the Internet community. It does > not specify an Internet standard of any kind. > > > Network Working Group S. Leinen > Request for Comments: 3955 SWITCH > Category: Informational October 2004 > Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX) > This memo provides information for the Internet community. It does > not specify an Internet standard of any kind. > > > Network Working Group J. Quittek > Request for Comments: 3917 NEC Europe Ltd. > Category: Informational T. Zseby > Fraunhofer FOKUS > B. Claise > Cisco Systems > S. Zander > Swinburne University > October 2004 > Requirements for IP Flow Information Export (IPFIX) > This memo provides information for the Internet community. It does > not specify an Internet standard of any kind. > > > And others... > > ========================================================================= > > Network Working Group B. Claise, Ed. > Request for Comments: 3954 Cisco Systems > Category: Informational October 2004 > Cisco Systems NetFlow Services Export Version 9 > This RFC documents the NetFlow services export protocol Version 9 as > it was when submitted to the IETF as a basis for further work in the > IPFIX WG. > This RFC itself is not a candidate for any level of Internet > Standard. The IETF disclaims any knowledge of the fitness of this > RFC for any purpose... > > And others... > > ========================================================================= > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-isp@FreeBSD.ORG Mon Sep 7 07:55:07 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12BEF1065676 for ; Mon, 7 Sep 2009 07:55:07 +0000 (UTC) (envelope-from dyr@homelink.ru) Received: from relay.eltel.net (relay.eltel.net [81.9.101.72]) by mx1.freebsd.org (Postfix) with ESMTP id B47AA8FC15 for ; Mon, 7 Sep 2009 07:55:06 +0000 (UTC) Received: from mail.homelink.ru ([81.9.33.123] helo=eltel.net) by relay.eltel.net with esmtp (Exim 4.63) (envelope-from ) id 1MkZ43-0001fS-HW; Mon, 07 Sep 2009 11:55:03 +0400 Received: from [85.249.167.249] (account dyr@homelink.ru HELO localhost) by eltel.net (CommuniGate Pro SMTP 4.2.10) with ESMTP-TLS id 29073082; Mon, 07 Sep 2009 11:55:03 +0400 Date: Mon, 7 Sep 2009 11:54:55 +0400 From: Dennis Yusupoff X-Mailer: The Bat! (v4.0.24) Professional Organization: Severen-Home ISP X-Priority: 3 (Normal) Message-ID: <1116202669.20090907115455@homelink.ru> To: freebsd-isp@freebsd.org, freebsd-net@freebsd.org In-Reply-To: <4AA46103.9000108@elischer.org> References: <4AA46103.9000108@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 3.3 X-Spam-Report: Mail system is relay.eltel.net Contact postmaster@eltel.net for details. Content analysis details: (3.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 2.8 UNWANTED_LANGUAGE_BODY BODY: Message written in an undesired language 0.5 BAYES_50 BODY: Bayesian spam probability is 40 to 60% [score: 0.5000] X-Spam-Flag: NO Cc: freebsd-isp@freebsd.org, freebsd-net@freebsd.org, grarpamp Subject: Re[2]: Call for testers: ng_netflow with v9 and IPv6 support X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2009 07:55:07 -0000 By the way, what's about this one: --- The ng_netflow node type does not fill in AS numbers. This is due to = the lack of necessary information in the kernel routing table. However, t= his information can be injected into the kernel from a routing daemon such= as GNU Zebra. This functionality may become available in future releases. --- Is any chance to see it in, for example, 8.0? --=20 =D1 =F3=E2=E0=E6=E5=ED=E8=E5=EC, =F1=E8=F1=F2=E5=EC=ED=FB=E9 =E0=E4=EC=E8=ED=E8=F1=F2=F0=E0=F2=EE=F0=20 Ozerki.Net/Cifracom.Ru =DE=F1=F3=EF=EE=E2 =C4=E5=ED=E8=F1 mailto:dyr@ho= melink.ru From owner-freebsd-isp@FreeBSD.ORG Mon Sep 7 21:03:34 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A6C91065693; Mon, 7 Sep 2009 21:03:34 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from fallbackmx10.syd.optusnet.com.au (fallbackmx10.syd.optusnet.com.au [211.29.132.251]) by mx1.freebsd.org (Postfix) with ESMTP id A146E8FC0A; Mon, 7 Sep 2009 21:03:33 +0000 (UTC) Received: from mail13.syd.optusnet.com.au (mail13.syd.optusnet.com.au [211.29.132.194]) by fallbackmx10.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n87JufNe023084; Tue, 8 Sep 2009 05:56:41 +1000 Received: from server.vk2pj.dyndns.org (c122-106-217-45.belrs3.nsw.optusnet.com.au [122.106.217.45]) by mail13.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n87Jubsq004145 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Sep 2009 05:56:38 +1000 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id n87JuagG041181; Tue, 8 Sep 2009 05:56:37 +1000 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id n87Jua3g041180; Tue, 8 Sep 2009 05:56:36 +1000 (EST) (envelope-from peter) Date: Tue, 8 Sep 2009 05:56:36 +1000 From: Peter Jeremy To: Dennis Yusupoff Message-ID: <20090907195636.GB35275@server.vk2pj.dyndns.org> References: <4AA46103.9000108@elischer.org> <1116202669.20090907115455@homelink.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TRYliJ5NKNqkz5bu" Content-Disposition: inline In-Reply-To: <1116202669.20090907115455@homelink.ru> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-isp@freebsd.org, freebsd-net@freebsd.org Subject: Re: Re[2]: Call for testers: ng_netflow with v9 and IPv6 support X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2009 21:03:34 -0000 --TRYliJ5NKNqkz5bu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2009-Sep-07 11:54:55 +0400, Dennis Yusupoff wrote: >By the way, what's about this one: >--- > The ng_netflow node type does not fill in AS numbers. This is due to= the =2E.. >--- > >Is any chance to see it in, for example, 8.0? New feature cutoff for 8.0 was several months ago. Once the code exists, it may be a candidate for inclusion in a future 8.x release. --=20 Peter Jeremy --TRYliJ5NKNqkz5bu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (FreeBSD) iEYEARECAAYFAkqlZXQACgkQ/opHv/APuIdndwCgmdtSQk8X9j31YuseFEIL1q22 +F4AnjifLddBcdFKs4BbSMxr5O1D3cVi =xh8+ -----END PGP SIGNATURE----- --TRYliJ5NKNqkz5bu-- From owner-freebsd-isp@FreeBSD.ORG Tue Sep 8 16:15:15 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DA691065670 for ; Tue, 8 Sep 2009 16:15:15 +0000 (UTC) (envelope-from vandj@securenet.net) Received: from mail.securenet.net (ms.securenet.net [205.236.147.20]) by mx1.freebsd.org (Postfix) with ESMTP id 63E8D8FC22 for ; Tue, 8 Sep 2009 16:15:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.securenet.net (Postfix) with ESMTP id EC59FC436 for ; Tue, 8 Sep 2009 11:59:10 -0400 (EDT) X-Virus-Scanned: by f-prot anti-virus, and clamav anti-virus at SecureNet Inc. Received: from mail.securenet.net ([127.0.0.1]) by localhost (ms.securenet.net [127.0.0.1]) (amavisd-new, port 10025) with ESMTP id 0eYHi42-HsCI for ; Tue, 8 Sep 2009 11:59:10 -0400 (EDT) Received: from office.securenet.net (office.securenet.net [205.236.147.3]) by mail.securenet.net (Postfix) with ESMTP id 8ABE3C2E5 for ; Tue, 8 Sep 2009 11:59:10 -0400 (EDT) X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 08 Sep 2009 11:58:53 -0400 To: freebsd-isp@freebsd.org From: "Jean M. Vandette" Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Message-Id: <20090908155910.8ABE3C2E5@mail.securenet.net> Subject: Looking for News feed X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Sep 2009 16:15:15 -0000 Greetings all Seems that both our upstream providers who were feeding us news had retired there news machines, as a result we have a news server which is not getting any feeds. Would there be anyone out there that can offer us full news feed Thanks in advance Regards Jean M. Vandette ************************************************************************************* *SecureNet Information Services Inc. 100 Alexis Nihon Blvd., Suite 283* *(514) 744-4242 Vox (514) 744-1552 Fax St. Laurent, Quebec H4M 2N7* ********** Providing Quality Public Internet access since 1994*************** From owner-freebsd-isp@FreeBSD.ORG Tue Sep 8 18:39:37 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 30EB71065670; Tue, 8 Sep 2009 18:39:37 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from no.spam.no.ddos.ru (no.spam.no.ddos.ru [77.73.233.132]) by mx1.freebsd.org (Postfix) with ESMTP id B1A0E8FC16; Tue, 8 Sep 2009 18:39:36 +0000 (UTC) Received: from ws.ipfw.ru (secured.by.ipfw.ru [81.200.11.182]) by no.spam.no.ddos.ru (Postfix) with ESMTPA id 171A837B1D8; Tue, 8 Sep 2009 18:39:32 +0000 (UTC) Message-ID: <4AA6A4C8.4090200@ipfw.ru> Date: Tue, 08 Sep 2009 22:39:04 +0400 From: "Alexander V. Chernikov" User-Agent: Thunderbird 2.0.0.22 (X11/20090818) MIME-Version: 1.0 To: grarpamp References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-isp@freebsd.org, freebsd-net@freebsd.org Subject: Re: Call for testers: ng_netflow with v9 and IPv6 support X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Sep 2009 18:39:37 -0000 grarpamp wrote: > Wouldn't it be better to support the obvious formal emergent standards > track protocol instead of the legacy informational one? Or to perform > both via sysctl or other arguments/defines, with the standard IPFIX > being the default mode? Have you reviewed the nProbe code for other > various ideas? Thanks for working on netflow :) And any form of v6 > is good to have finally. > Thanks for pointing out those RFCs. We're still using v5 for accounting, so my behavior was quite straightforward: can v5 count ipv6 ? No, what's next netflow version can ? v9? Ok, let's implement v9. However, IPFIX seems to be very much like v9 at a first glance. Moreover, ng_ksocket can handle udp/tcp/sctp exports. > ========================================================================= > > Network Working Group B. Claise, Ed. > Request for Comments: 5101 Cisco Systems, Inc. > Category: Standards Track January 2008 > Specification of the IP Flow Information Export (IPFIX) Protocol > for the Exchange of IP Traffic Flow Information > This document specifies an Internet standards track protocol for the > Internet community > > > Network Working Group J. Quittek > Request for Comments: 5102 NEC > Category: Standards Track S. Bryant > B. Claise > P. Aitken > Cisco Systems, Inc. > J. Meyer > PayPal > January 2008 > Information Model for IP Flow Information Export > This document specifies an Internet standards track protocol for the > Internet community > > > Network Working Group B. Trammell > Request for Comments: 5103 CERT/NetSA > Category: Standards Track E. Boschi > Hitachi Europe > January 2008 > Bidirectional Flow Export Using IP Flow Information Export (IPFIX) > This document specifies an Internet standards track protocol for the > Internet community > > > Network Working Group E. Boschi > Request for Comments: 5153 Hitachi Europe > Category: Informational L. Mark > Fraunhofer FOKUS > J. Quittek > M. Stiemerling > NEC > P. Aitken > Cisco Systems, Inc. > April 2008 > IP Flow Information Export (IPFIX) Implementation Guidelines > This memo provides information for the Internet community. It does > not specify an Internet standard of any kind. > > > Network Working Group S. Leinen > Request for Comments: 3955 SWITCH > Category: Informational October 2004 > Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX) > This memo provides information for the Internet community. It does > not specify an Internet standard of any kind. > > > Network Working Group J. Quittek > Request for Comments: 3917 NEC Europe Ltd. > Category: Informational T. Zseby > Fraunhofer FOKUS > B. Claise > Cisco Systems > S. Zander > Swinburne University > October 2004 > Requirements for IP Flow Information Export (IPFIX) > This memo provides information for the Internet community. It does > not specify an Internet standard of any kind. > > > And others... > > ========================================================================= > > Network Working Group B. Claise, Ed. > Request for Comments: 3954 Cisco Systems > Category: Informational October 2004 > Cisco Systems NetFlow Services Export Version 9 > This RFC documents the NetFlow services export protocol Version 9 as > it was when submitted to the IETF as a basis for further work in the > IPFIX WG. > This RFC itself is not a candidate for any level of Internet > Standard. The IETF disclaims any knowledge of the fitness of this > RFC for any purpose... > > And others... > > ========================================================================= > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-isp@FreeBSD.ORG Wed Sep 9 02:05:15 2009 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 919C21065672; Wed, 9 Sep 2009 02:05:15 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-fx0-f210.google.com (mail-fx0-f210.google.com [209.85.220.210]) by mx1.freebsd.org (Postfix) with ESMTP id F37D58FC17; Wed, 9 Sep 2009 02:05:14 +0000 (UTC) Received: by fxm6 with SMTP id 6so2936005fxm.43 for ; Tue, 08 Sep 2009 19:05:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=noSEXORYk0fqpv880QgYGo33AVnJQC+JytP50QbDLMk=; b=WF+E9WDfezTAmFvbfpoU3aofr1XuW2HLtW98ZTYMJnNFKbGeQS79HlxEt9IOSrgMPQ cwAY6INhztXvksDTSLax7DdIFli15Q87fx97xX4zcyoOb74rV4Zpj5ivP+FkwuE7orjH Rl77v//cw+CPJ1DomLoxVc3QmoVXG7STIRy9A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=sqa56HRdipYgsB+ecviBupimlKg/tKwbu8xxkqksP2BvGJNFzopl3BS5ux7Bji3u1h /GG8T17gXuy6Q9bDu4WfZa1VkidQ1Q2IDy80yVlzw1fnOxcAd2pryY+enn9OOLT52o1Z 0isDH5iy9s1Rv6BnQCCR8boW3De7HeO3Vnkfg= MIME-Version: 1.0 Received: by 10.223.1.10 with SMTP id 10mr6374037fad.94.1252461913243; Tue, 08 Sep 2009 19:05:13 -0700 (PDT) In-Reply-To: <4AA6A4C8.4090200@ipfw.ru> References: <4AA6A4C8.4090200@ipfw.ru> Date: Tue, 8 Sep 2009 22:05:13 -0400 Message-ID: From: grarpamp To: freebsd-net@freebsd.org, freebsd-isp@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: Call for testers: ng_netflow with v9 and IPv6 support X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Sep 2009 02:05:15 -0000 > Thanks for pointing out those RFCs. Sure. There are more I probably missed. Search rfc-editor or ietf for netflow or ipfix. > can v5 count ipv6 ? No, what's next netflow version can ? v9? Ok, let's > implement v9. Yep, ipv6 is becoming really important, definitely on backbones. nProbe has had it for a while. nProbe is a bpf flow sniffer and exporter. I don't know about any other open source ones that do, besides your patch :) > However, IPFIX seems to be very much like v9 at a first glance. v9 was the first that came out, mostly a defacto cisco proprietary thing as was v5. Netflow needed to be opened up / standardized. So ipfix group was started. cisco v9 was submitted to ietf ipfix working group as one of proposed ipfix methods. v9 was then modified in the group a bit and called IPFIX. Some of this is in rfc 3955. Ipfix is probably the way to go. Of course others on the lists could input whether they see more of v9 or ipfix now and in future.