From owner-svn-src-head@FreeBSD.ORG Mon Jun 18 13:56:36 2012 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B2DBC1065673; Mon, 18 Jun 2012 13:56:36 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 9C6B78FC16; Mon, 18 Jun 2012 13:56:36 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q5IDua2h082646; Mon, 18 Jun 2012 13:56:36 GMT (envelope-from melifaro@svn.freebsd.org) Received: (from melifaro@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q5IDuaIf082641; Mon, 18 Jun 2012 13:56:36 GMT (envelope-from melifaro@svn.freebsd.org) Message-Id: <201206181356.q5IDuaIf082641@svn.freebsd.org> From: "Alexander V. Chernikov" Date: Mon, 18 Jun 2012 13:56:36 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r237227 - in head: share/man/man4 sys/netgraph/netflow X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jun 2012 13:56:36 -0000 Author: melifaro Date: Mon Jun 18 13:56:36 2012 New Revision: 237227 URL: http://svn.freebsd.org/changeset/base/237227 Log: Make radix lookup on src and dst flow addresses optional and configurable on per-interface basis. Remove __inline__ for several functions being called once per flow (e.g once per 10-20 packets on common traffic flows). Update manual page to simplify search for BPF data link types. Sponsored by Yandex LLC Reviewed by: glebius Approved by: ae(mentor) MFC after: 2 weeks Modified: head/share/man/man4/ng_netflow.4 head/sys/netgraph/netflow/netflow.c head/sys/netgraph/netflow/ng_netflow.c head/sys/netgraph/netflow/ng_netflow.h Modified: head/share/man/man4/ng_netflow.4 ============================================================================== --- head/share/man/man4/ng_netflow.4 Mon Jun 18 13:50:41 2012 (r237226) +++ head/share/man/man4/ng_netflow.4 Mon Jun 18 13:56:36 2012 (r237227) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd March 2, 2011 +.Dd June 16, 2012 .Dt NG_NETFLOW 4 .Os .Sh NAME @@ -112,7 +112,18 @@ The hook number is passed as an argument Sets data link type on the .Va iface Ns Ar N hook. -Currently, supported types are raw IP datagrams and Ethernet. +Currently, supported types are +.Cm DLT_RAW +(raw IP datagrams) and +.Cm DLT_EN10MB +(Ethernet). +DLT_ definitions can be found in +.In net/bpf.h +header. +Currently used values are 1 for +.Cm DLT_EN10MB +and 12 for +.Cm DLT_RAW . This message type uses .Vt "struct ng_netflow_setdlt" as an argument: @@ -180,18 +191,36 @@ struct ng_netflow_setconfig { #define NG_NETFLOW_CONF_EGRESS 2 #define NG_NETFLOW_CONF_ONCE 4 #define NG_NETFLOW_CONF_THISONCE 8 +#define NG_NETFLOW_CONF_NOSRCLOOKUP 16 +#define NG_NETFLOW_CONF_NODSTLOOKUP 32 }; .Ed .Pp Configuration is a bitmask of several options. Option NG_NETFLOW_CONF_INGRESS enabled by default enables ingress NetFlow generation (for data coming from -ifaceX hook). Option NG_NETFLOW_CONF_EGRESS enables egress NetFlow (for data -coming from outX hook). Option NG_NETFLOW_CONF_ONCE defines that packet should -be accounted only once if it several times passes via netflow node. Option -NG_NETFLOW_CONF_THISONCE defines that packet should be accounted only once -if it several times passes via exactly this netflow node. Last two options are -important to avoid duplicate accounting when both ingress and egress NetFlow -are enabled. +ifaceX hook). +Option +.Va NG_NETFLOW_CONF_EGRESS +enables egress NetFlow (for data coming from outX hook). +Option +.Va NG_NETFLOW_CONF_ONCE +defines that packet should be accounted only once if it several times passes +via netflow node. +Option +.Va NG_NETFLOW_CONF_THISONCE +defines that packet should be accounted only once if it several times passes +via exactly this netflow node. +These two options are important to avoid duplicate accounting when both ingress +and egress NetFlow are enabled. +Option +.Va NG_NETFLOW_CONF_NOSRCLOOKUP +skips radix lookup on flow source address used to fill in network mask. +Option +.Va NG_NETFLOW_CONF_NODSTLOOKUP +skips radix lookup on destination (which fills egress interface id, destination +mask and gateway). +If one doesn't need data provided by lookups, he/she can disable them, to reduce +load on routers. .It Dv NGM_NETFLOW_SETTEMPLATE Sets various timeouts to announce data flow templates (NetFlow v9-specific). This message requires Modified: head/sys/netgraph/netflow/netflow.c ============================================================================== --- head/sys/netgraph/netflow/netflow.c Mon Jun 18 13:50:41 2012 (r237226) +++ head/sys/netgraph/netflow/netflow.c Mon Jun 18 13:56:36 2012 (r237227) @@ -98,9 +98,9 @@ MALLOC_DEFINE(M_NETFLOW_HASH, "netflow_h static int export_add(item_p, struct flow_entry *); static int export_send(priv_p, fib_export_p, item_p, int); -static int hash_insert(priv_p, struct flow_hash_entry *, struct flow_rec *, int, uint8_t); +static int hash_insert(priv_p, struct flow_hash_entry *, struct flow_rec *, int, uint8_t, uint8_t); #ifdef INET6 -static int hash6_insert(priv_p, struct flow_hash_entry *, struct flow6_rec *, int, uint8_t); +static int hash6_insert(priv_p, struct flow_hash_entry *, struct flow6_rec *, int, uint8_t, uint8_t); #endif static __inline void expire_flow(priv_p, fib_export_p, struct flow_entry *, int); @@ -325,9 +325,9 @@ ng_netflow_copyinfo(priv_p priv, struct * as this was done in previous version. Need to test & profile * to be sure. */ -static __inline int +static int hash_insert(priv_p priv, struct flow_hash_entry *hsh, struct flow_rec *r, - int plen, uint8_t tcp_flags) + int plen, uint8_t flags, uint8_t tcp_flags) { struct flow_entry *fle; struct sockaddr_in sin; @@ -358,44 +358,48 @@ hash_insert(priv_p priv, struct flow_has * First we do route table lookup on destination address. So we can * fill in out_ifx, dst_mask, nexthop, and dst_as in future releases. */ - bzero(&sin, sizeof(sin)); - sin.sin_len = sizeof(struct sockaddr_in); - sin.sin_family = AF_INET; - sin.sin_addr = fle->f.r.r_dst; - rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r->fib); - if (rt != NULL) { - fle->f.fle_o_ifx = rt->rt_ifp->if_index; - - if (rt->rt_flags & RTF_GATEWAY && - rt->rt_gateway->sa_family == AF_INET) - fle->f.next_hop = - ((struct sockaddr_in *)(rt->rt_gateway))->sin_addr; - - if (rt_mask(rt)) - fle->f.dst_mask = bitcount32(((struct sockaddr_in *) - rt_mask(rt))->sin_addr.s_addr); - else if (rt->rt_flags & RTF_HOST) - /* Give up. We can't determine mask :( */ - fle->f.dst_mask = 32; + if ((flags & NG_NETFLOW_CONF_NODSTLOOKUP) == 0) { + bzero(&sin, sizeof(sin)); + sin.sin_len = sizeof(struct sockaddr_in); + sin.sin_family = AF_INET; + sin.sin_addr = fle->f.r.r_dst; + rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r->fib); + if (rt != NULL) { + fle->f.fle_o_ifx = rt->rt_ifp->if_index; + + if (rt->rt_flags & RTF_GATEWAY && + rt->rt_gateway->sa_family == AF_INET) + fle->f.next_hop = + ((struct sockaddr_in *)(rt->rt_gateway))->sin_addr; + + if (rt_mask(rt)) + fle->f.dst_mask = bitcount32(((struct sockaddr_in *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) + /* Give up. We can't determine mask :( */ + fle->f.dst_mask = 32; - RTFREE_LOCKED(rt); + RTFREE_LOCKED(rt); + } } /* Do route lookup on source address, to fill in src_mask. */ - bzero(&sin, sizeof(sin)); - sin.sin_len = sizeof(struct sockaddr_in); - sin.sin_family = AF_INET; - sin.sin_addr = fle->f.r.r_src; - rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r->fib); - if (rt != NULL) { - if (rt_mask(rt)) - fle->f.src_mask = bitcount32(((struct sockaddr_in *) - rt_mask(rt))->sin_addr.s_addr); - else if (rt->rt_flags & RTF_HOST) - /* Give up. We can't determine mask :( */ - fle->f.src_mask = 32; + if ((flags & NG_NETFLOW_CONF_NOSRCLOOKUP) == 0) { + bzero(&sin, sizeof(sin)); + sin.sin_len = sizeof(struct sockaddr_in); + sin.sin_family = AF_INET; + sin.sin_addr = fle->f.r.r_src; + rt = rtalloc1_fib((struct sockaddr *)&sin, 0, 0, r->fib); + if (rt != NULL) { + if (rt_mask(rt)) + fle->f.src_mask = bitcount32(((struct sockaddr_in *) + rt_mask(rt))->sin_addr.s_addr); + else if (rt->rt_flags & RTF_HOST) + /* Give up. We can't determine mask :( */ + fle->f.src_mask = 32; - RTFREE_LOCKED(rt); + RTFREE_LOCKED(rt); + } } /* Push new flow at the and of hash. */ @@ -410,10 +414,10 @@ hash_insert(priv_p priv, struct flow_has bitcount32((x).__u6_addr.__u6_addr32[1]) + \ bitcount32((x).__u6_addr.__u6_addr32[2]) + \ bitcount32((x).__u6_addr.__u6_addr32[3]) -/* XXX: Do we need inline here ? */ -static __inline int +#define RT_MASK6(x) (ipv6_masklen(((struct sockaddr_in6 *)rt_mask(x))->sin6_addr)) +static int hash6_insert(priv_p priv, struct flow_hash_entry *hsh6, struct flow6_rec *r, - int plen, uint8_t tcp_flags) + int plen, uint8_t flags, uint8_t tcp_flags) { struct flow6_entry *fle6; struct sockaddr_in6 *src, *dst; @@ -445,49 +449,55 @@ hash6_insert(priv_p priv, struct flow_ha * First we do route table lookup on destination address. So we can * fill in out_ifx, dst_mask, nexthop, and dst_as in future releases. */ - bzero(&rin6, sizeof(struct route_in6)); - dst = (struct sockaddr_in6 *)&rin6.ro_dst; - dst->sin6_len = sizeof(struct sockaddr_in6); - dst->sin6_family = AF_INET6; - dst->sin6_addr = r->dst.r_dst6; - - rin6.ro_rt = rtalloc1_fib((struct sockaddr *)dst, 0, 0, r->fib); - - if (rin6.ro_rt != NULL) { - rt = rin6.ro_rt; - fle6->f.fle_o_ifx = rt->rt_ifp->if_index; - - if (rt->rt_flags & RTF_GATEWAY && - rt->rt_gateway->sa_family == AF_INET6) - fle6->f.n.next_hop6 = - ((struct sockaddr_in6 *)(rt->rt_gateway))->sin6_addr; - - if (rt_mask(rt)) - fle6->f.dst_mask = ipv6_masklen(((struct sockaddr_in6 *)rt_mask(rt))->sin6_addr); - else - fle6->f.dst_mask = 128; + if ((flags & NG_NETFLOW_CONF_NODSTLOOKUP) == 0) + { + bzero(&rin6, sizeof(struct route_in6)); + dst = (struct sockaddr_in6 *)&rin6.ro_dst; + dst->sin6_len = sizeof(struct sockaddr_in6); + dst->sin6_family = AF_INET6; + dst->sin6_addr = r->dst.r_dst6; + + rin6.ro_rt = rtalloc1_fib((struct sockaddr *)dst, 0, 0, r->fib); + + if (rin6.ro_rt != NULL) { + rt = rin6.ro_rt; + fle6->f.fle_o_ifx = rt->rt_ifp->if_index; + + if (rt->rt_flags & RTF_GATEWAY && + rt->rt_gateway->sa_family == AF_INET6) + fle6->f.n.next_hop6 = + ((struct sockaddr_in6 *)(rt->rt_gateway))->sin6_addr; - RTFREE_LOCKED(rt); + if (rt_mask(rt)) + fle6->f.dst_mask = RT_MASK6(rt); + else + fle6->f.dst_mask = 128; + + RTFREE_LOCKED(rt); + } } - /* Do route lookup on source address, to fill in src_mask. */ - bzero(&rin6, sizeof(struct route_in6)); - src = (struct sockaddr_in6 *)&rin6.ro_dst; - src->sin6_len = sizeof(struct sockaddr_in6); - src->sin6_family = AF_INET6; - src->sin6_addr = r->src.r_src6; - - rin6.ro_rt = rtalloc1_fib((struct sockaddr *)src, 0, 0, r->fib); - - if (rin6.ro_rt != NULL) { - rt = rin6.ro_rt; - - if (rt_mask(rt)) - fle6->f.src_mask = ipv6_masklen(((struct sockaddr_in6 *)rt_mask(rt))->sin6_addr); - else - fle6->f.src_mask = 128; + if ((flags & NG_NETFLOW_CONF_NODSTLOOKUP) == 0) + { + /* Do route lookup on source address, to fill in src_mask. */ + bzero(&rin6, sizeof(struct route_in6)); + src = (struct sockaddr_in6 *)&rin6.ro_dst; + src->sin6_len = sizeof(struct sockaddr_in6); + src->sin6_family = AF_INET6; + src->sin6_addr = r->src.r_src6; + + rin6.ro_rt = rtalloc1_fib((struct sockaddr *)src, 0, 0, r->fib); + + if (rin6.ro_rt != NULL) { + rt = rin6.ro_rt; - RTFREE_LOCKED(rt); + if (rt_mask(rt)) + fle6->f.src_mask = RT_MASK6(rt); + else + fle6->f.src_mask = 128; + + RTFREE_LOCKED(rt); + } } /* Push new flow at the and of hash. */ @@ -495,6 +505,8 @@ hash6_insert(priv_p priv, struct flow_ha return (0); } +#undef ipv6_masklen +#undef RT_MASK6 #endif @@ -651,7 +663,7 @@ ng_netflow_cache_flush(priv_p priv) /* Insert packet from into flow cache. */ int ng_netflow_flow_add(priv_p priv, fib_export_p fe, struct ip *ip, caddr_t upper_ptr, uint8_t upper_proto, - uint8_t is_frag, unsigned int src_if_index) + uint8_t flags, unsigned int src_if_index) { register struct flow_entry *fle, *fle1; struct flow_hash_entry *hsh; @@ -770,7 +782,7 @@ ng_netflow_flow_add(priv_p priv, fib_exp } } } else /* A new flow entry. */ - error = hash_insert(priv, hsh, &r, plen, tcp_flags); + error = hash_insert(priv, hsh, &r, plen, flags, tcp_flags); mtx_unlock(&hsh->mtx); @@ -781,7 +793,7 @@ ng_netflow_flow_add(priv_p priv, fib_exp /* Insert IPv6 packet from into flow cache. */ int ng_netflow_flow6_add(priv_p priv, fib_export_p fe, struct ip6_hdr *ip6, caddr_t upper_ptr, uint8_t upper_proto, - uint8_t is_frag, unsigned int src_if_index) + uint8_t flags, unsigned int src_if_index) { register struct flow_entry *fle = NULL, *fle1; register struct flow6_entry *fle6; @@ -811,7 +823,7 @@ ng_netflow_flow6_add(priv_p priv, fib_ex #if 0 r.r_tos = ip->ip_tos; #endif - if (is_frag == 0) { + if ((flags & NG_NETFLOW_IS_FRAG) == 0) { switch(upper_proto) { case IPPROTO_TCP: { @@ -896,7 +908,7 @@ ng_netflow_flow6_add(priv_p priv, fib_ex } } } else /* A new flow entry. */ - error = hash6_insert(priv, hsh, &r, plen, tcp_flags); + error = hash6_insert(priv, hsh, &r, plen, flags, tcp_flags); mtx_unlock(&hsh->mtx); Modified: head/sys/netgraph/netflow/ng_netflow.c ============================================================================== --- head/sys/netgraph/netflow/ng_netflow.c Mon Jun 18 13:50:41 2012 (r237226) +++ head/sys/netgraph/netflow/ng_netflow.c Mon Jun 18 13:56:36 2012 (r237227) @@ -560,7 +560,7 @@ ng_netflow_rcvdata (hook_p hook, item_p struct ip6_hdr *ip6 = NULL; struct m_tag *mtag; int pullup_len = 0, off; - uint8_t acct = 0, bypass = 0, is_frag = 0, upper_proto = 0; + uint8_t acct = 0, bypass = 0, flags = 0, upper_proto = 0; int error = 0, l3_off = 0; unsigned int src_if_index; caddr_t upper_ptr = NULL; @@ -619,6 +619,9 @@ ng_netflow_rcvdata (hook_p hook, item_p } } + /* Import configuration flags related to flow creation */ + flags = iface->info.conf & NG_NETFLOW_FLOW_FLAGS; + NGI_GET_M(item, m); m_old = m; @@ -759,7 +762,7 @@ ng_netflow_rcvdata (hook_p hook, item_p } } else if (ip != NULL) { /* Nothing to save except upper layer proto, since this is packet fragment */ - is_frag = 1; + flags |= NG_NETFLOW_IS_FRAG; upper_proto = ip->ip_p; if ((ip->ip_v != IPVERSION) || ((ip->ip_hl << 2) < sizeof(struct ip))) @@ -821,7 +824,7 @@ ng_netflow_rcvdata (hook_p hook, item_p upper_proto = ip6f->ip6f_nxt; hdr_off = sizeof(struct ip6_frag); off += hdr_off; - is_frag = 1; + flags |= NG_NETFLOW_IS_FRAG; goto loopend; #if 0 @@ -886,10 +889,10 @@ loopend: } if (ip != NULL) - error = ng_netflow_flow_add(priv, fe, ip, upper_ptr, upper_proto, is_frag, src_if_index); + error = ng_netflow_flow_add(priv, fe, ip, upper_ptr, upper_proto, flags, src_if_index); #ifdef INET6 else if (ip6 != NULL) - error = ng_netflow_flow6_add(priv, fe, ip6, upper_ptr, upper_proto, is_frag, src_if_index); + error = ng_netflow_flow6_add(priv, fe, ip6, upper_ptr, upper_proto, flags, src_if_index); #endif else goto bypass; Modified: head/sys/netgraph/netflow/ng_netflow.h ============================================================================== --- head/sys/netgraph/netflow/ng_netflow.h Mon Jun 18 13:50:41 2012 (r237226) +++ head/sys/netgraph/netflow/ng_netflow.h Mon Jun 18 13:56:36 2012 (r237227) @@ -111,10 +111,16 @@ struct ng_netflow_settimeouts { uint32_t active_timeout; /* flow active timeout */ }; -#define NG_NETFLOW_CONF_INGRESS 1 -#define NG_NETFLOW_CONF_EGRESS 2 -#define NG_NETFLOW_CONF_ONCE 4 -#define NG_NETFLOW_CONF_THISONCE 8 +#define NG_NETFLOW_CONF_INGRESS 0x01 /* Account on ingress */ +#define NG_NETFLOW_CONF_EGRESS 0x02 /* Account on egress */ +#define NG_NETFLOW_CONF_ONCE 0x04 /* Add tag to account only once */ +#define NG_NETFLOW_CONF_THISONCE 0x08 /* Account once in current node */ +#define NG_NETFLOW_CONF_NOSRCLOOKUP 0x10 /* No radix lookup on src */ +#define NG_NETFLOW_CONF_NODSTLOOKUP 0x20 /* No radix lookup on dst */ + +#define NG_NETFLOW_IS_FRAG 0x01 +#define NG_NETFLOW_FLOW_FLAGS (NG_NETFLOW_CONF_NOSRCLOOKUP|\ + NG_NETFLOW_CONF_NODSTLOOKUP) /* This structure is passed to NGM_NETFLOW_SETCONFIG */ struct ng_netflow_setconfig {