From owner-freebsd-net@FreeBSD.ORG Mon Jul 1 12:02:21 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D994D425 for ; Mon, 1 Jul 2013 12:02:21 +0000 (UTC) (envelope-from syuu@dokukino.com) Received: from mail-pb0-x234.google.com (mail-pb0-x234.google.com [IPv6:2607:f8b0:400e:c01::234]) by mx1.freebsd.org (Postfix) with ESMTP id B4C82179A for ; Mon, 1 Jul 2013 12:02:21 +0000 (UTC) Received: by mail-pb0-f52.google.com with SMTP id xa12so4692974pbc.39 for ; Mon, 01 Jul 2013 05:02:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokukino.com; s=google; h=mime-version:from:date:message-id:subject:to:content-type; bh=skOb+qm7fUcZhlY8tCz+dzyBArmBVD9a86aWttQzThY=; b=NOyqOLFva+HoUzXx9sMHpd23g92SYOrKMMAWIoLFv3R09FXP9XOlutxXYwevkZwczJ qEKogI1ACZbTz7JAJEsLr/4V2vmD6lvV9VPythk/9N7ODDjnmyR+eOtadvSpUpKhtC/I 0R9OHR5uC+YdSoDJnhUKguLkjmKorL4Mqk91w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type :x-gm-message-state; bh=skOb+qm7fUcZhlY8tCz+dzyBArmBVD9a86aWttQzThY=; b=jtYfiAfvCJfe7mfaEFk2CqS0GavycmqTs3AOo4prH3VUjaLuagLZg5hOw//Ir59Kba 7/nv+ZU1v6Yn2Dtu1Jg/q/U/e1m5FLw7S5SZ6ElnfZGeuar3dIRFgUfeoKoDJdkZiOe9 m3wv+lD6I3rYf1V6Mkh7hmitc5HMjHpEn8fMu68vewR2HORtLnJw46M8aIugEFLX5awU aJ/twjU+oNnrtdSwijPn7ZdhMPs/Tkf3c9HsCxGBq4KjKDs/crON8eT0bUXxZvojM6A+ svdKTnVFj+vFZDrGxyOpp910YGSyK4BHdsgKhh/xuKJ+yAQZssvBHq/j5gEDh6m9OjIQ hoxQ== X-Received: by 10.68.160.99 with SMTP id xj3mr13892110pbb.189.1372680141462; Mon, 01 Jul 2013 05:02:21 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.222.34 with HTTP; Mon, 1 Jul 2013 05:01:41 -0700 (PDT) From: Takuya ASADA Date: Mon, 1 Jul 2013 21:01:41 +0900 Message-ID: Subject: Multiqueue support for bpf To: FreeBSD Net X-Gm-Message-State: ALoCoQkq++TgVmBMEIPt86v/R6XuAjVsjirdczD/hBp+lqQz8dIbP0vAqSFphdWYJ8//yGhpr6Hc Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jul 2013 12:02:21 -0000 Hi all, I'd like to propose multiqueue support for bpf. It's result of GSoC'11, and proposed on freebsd-net at Aug.2011 but not yet merged: http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/syuu1228/1 http://lists.freebsd.org/pipermail/freebsd-net/2011-August/029585.html The objectives of the patch is to support multiqueue NICs on BPF, and provide interfaces for multithreaded packet processing using BPF. To get optimal performance and to reduce lock contention, multiqueue BPF provides a feature to specify hardware queues. Following is basic usage of multiqueue BPF: void *bpf_thread(void *arg) { cpu = (int)arg; CPU_ZERO(&cpuset); CPU_SET(cpu, &cpuset); cpuset_setaffinity(CPU_LEVEL_WHICH, CPU_WHICH_TID, -1, sizeof(cpuset), &cpuset); fd = open("/dev/bpf0", O_RDWR); ioctl(fd, BIOCSTRXQMASK, &cpu); ioctl(fd, BIOCSTTXQMASK, &cpu); /* actual works */ } int main(void) { for (i = 0; i < maxcpus; i++) pthread_create(&threads[i], NULL, bpf_thread, (void *)i); for (i = 0; i < maxcpus; i++) pthread_join(&threads[i], NULL); } In this example, threads[n] bind to CPUn, receives packets from rxqueue n and txqueue n. To implement it, the patch modifies bpf_*tap*() and extends struct ifnet, struct mbuf to notify hardware queue information. The changes are in this branch: http://svnweb.freebsd.org/base/user/syuu/mq_bpf/ API descriptions and benchmark results are on previous post: http://lists.freebsd.org/pipermail/freebsd-net/2011-August/029585.html Benchmark program is on this repository: https://github.com/syuu1228/mq_bpf_test