From owner-freebsd-isp  Mon Nov 18 20:22:15 1996
Return-Path: owner-isp
Received: (from root@localhost)
          by freefall.freebsd.org (8.7.5/8.7.3) id UAA12828
          for isp-outgoing; Mon, 18 Nov 1996 20:22:15 -0800 (PST)
Received: from red.jnx.com (red.jnx.com [208.197.169.254])
          by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id UAA12814
          for <isp@FreeBSD.ORG>; Mon, 18 Nov 1996 20:22:06 -0800 (PST)
Received: from chimp.jnx.com (chimp.jnx.com [208.197.169.246]) by red.jnx.com (8.8.2/8.8.2) with ESMTP id UAA03018; Mon, 18 Nov 1996 20:20:57 -0800 (PST)
Received: (from tli@localhost) by chimp.jnx.com (8.7.6/8.7.3) id UAA14723; Mon, 18 Nov 1996 20:20:49 -0800 (PST)
Date: Mon, 18 Nov 1996 20:20:49 -0800 (PST)
Message-Id: <199611190420.UAA14723@chimp.jnx.com>
From: Tony Li <tli@jnx.com>
To: jgreco@brasil.moneng.mei.com
CC: dennis@etinc.com, isp@FreeBSD.ORG
In-reply-to: <199611190357.VAA03823@brasil.moneng.mei.com> (message from Joe
	Greco on Mon, 18 Nov 1996 21:57:49 -0600 (CST))
Subject: Re: changed to: Frac T3?
Sender: owner-isp@FreeBSD.ORG
X-Loop: FreeBSD.org
Precedence: bulk


   I always thought packet routing was handled as a fairly high priority
   thing.  Packet arrives, generates interrupt, kernel processes, queues
   for output, and sends it.

   User processes should have no way to compete for time spent basically
   dealing with an interrupt.

Yes, exactly.  The problem is that routing protocols (e.g., routed) run as
processes and exhibit this behavior:

   As "sy" reached its peak value of 100, the system's interactive response
   became virtually dead and as the number of packets routed per second
   continued to grow, the vmstat "froze".  I couldn't coax any response
   from any daemons on the system, although I was allowed to connect.  When
   the pps being routed dropped, suddenly the system "sprang" to life
   again.

This is enough to trash most protocols.  Note that ongoing competition from
other process can also be a problem.

   However, less clear to me is that your argument against
   having any "significant non-routing load" on a router applies to UNIX
   based routers.  Maybe someone like David or Bruce could clarify this.

Simple to demonstrate.  Take a compute bound process which generates and
receives packets periodically.  This emulates your routing protocol.  Start
a kernel build.  You'll notice a non-trivial performance hit in interactive
response and computation by your routing protocol.  Now renice your kernel
build.  Things get better, but that process still consumes bus cycles,
physical memory, and gets some share of the CPU.  Add more load until
processes start paging.  You'll note that the progress of your 'routing
protocol' degrades further.  Pretty soon, your timers start to slip.
You're now on the brink of collapse.

The result is simple: routing protocols need a guaranteed minimal amount of
CPU and I/O to survive.  Lacking strict resource controls under Unix, the
best way of delivering that guarantee is to exclude other load from the
box.  For a seriously robust router, you'd also need to put a cap on the
amount of time spent in interrupt processing.

Tony