Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 13 Nov 1995 15:51:48 +0200
From:      Heikki Suonsivu <hsu@cs.hut.fi>
To:        current@FreeBSD.org
Cc:        freebsd-current@freefall.FreeBSD.org
Subject:   ISP state their FreeBSD concerns
Message-ID:  <199511131351.PAA20135@shadows.cs.hut.fi>
In-Reply-To: owner-freebsd-current@FreeBSD.org's message of 12 Nov 1995 07:35:31 %2B0200

next in thread | raw e-mail | index | archive | help

   These are generally ordered with the highest concerns first:

   1.	A concern that FreeBSD tends to "bind" for brief periods when
	   loaded. Here is how it was described to me:  You will be doing
	   something (like skimming news articles in trn or tin) that is
	   hitting the hard disk perhaps once every five seconds and you
	   are getting instant response.  Then something else starts up, like
	   tind, and the hard disk is now extremely busy (it was practically
	   idle before tind started).  Now, your disk requests aren't getting

Doing a sync on our news server causes something like this, the system
gets really sticky for a couple of seconds.  Of course update is doing a
sync every n seconds...

We use -o async for the news spool, so there probably are lots of dirty
pages in memory.

	   There was a lessor complaint of mystery panics and lockups but
	   this only came from someone who played with 2.0.0.

We get panics, sometimes lockups, at pace of 1-2 machines per day (from a
pool of approximately ~5 servers and ~15 routers or terminal servers built
on top of FreeBSD.  I'm running -current.  On servers the problems seem
often to be scsi related.  On routers panics seem very rare, on terminal
(modem) servers and routers with serial ports panic about once a week.

   2.	A concern that disk I/O on large (4GB or bigger) hard disks or
	   large numbers of hard disks is not reliable, particularly on

The troubling disks for us seem to be seagate hawks, but I have also had
problems with an ibm 0662 (but it could be seagates in the same bus).  I
have tried a 2G barracuda and 4G hawk for news spool, both with same
results, disk gets confused and scsi driver returns I/O errors for all
accesses or the system panics.

   5.	Not at all obvious what system resources to increase for
	   large and/or heavily loaded systems, and what ratios of
	   parameter settings are best.  Apparently SUN, SCO and even Linux
	   have a lot of documentation in this area and FreeBSDs is
	   limited, missing or not obvious.  I know SCO used to have
	   little tables that said "if you increase this, this other
	   value should be probably increased according to this formula...".
	   FreeBSD is different enough apparently that the rules of
	   thumb for BSD 4.x aren't valid here.

Tatu Yl|nen filed a PR about this a while ago.

Better than document this would be make it easy to configure and defaults
high enough.  Tuning maxusers should tune everything necessary, and
everything which you don't know for sure, oversize it.  Now it seems that
installing a FreeBSD system has defaults so small that it can't be used
even for a single-user system, not to speak of a large server.

   6.	Multi-port serial support and even single-port serial support.

cyclades driver sort-of works, if a panic approximately once a week is
tolerated (could be something else than cyclades driver).

	   Seems they feel hardware flow control doesn't work or isn't enabled
	   when it should be (or can't be) or both.  This may actually be
	   an issue with the 16550 drivers not setting silo depth to a
	   reasonable level.   Most of these guys use terminal servers for
	   the actual users where these local serial ports are used for UPS,
	   router control, serial printers, and other in-house controls.  They
	   complain of seeing problems with lost data outbound when flow
	   control was in use, including during UUCP sessions.  

I have complained about this several times, but gave up after I realized
noone was listening.  I have been including these in all our kernels:

*** sys/i386/isa/sio.c.orig     Thu Aug 24 21:56:48 1995
--- sys/i386/isa/sio.c  Thu Aug 24 21:57:13 1995
***************
*** 71,77 ****
--- 71,79 ----
  
  #define       LOTS_OF_EVENTS  64      /* helps separate urgent events from input */
  #define       RB_I_HIGH_WATER (TTYHOG - 2 * RS_IBUFSIZE)
+ #ifndef RS_IBUFSIZE
  #define       RS_IBUFSIZE     256
+ #endif
  
  #define       CALLOUT_MASK            0x80
  #define       CONTROL_MASK            0x60
*** sys/i386/isa/cy.c.orig      Thu Aug 24 21:57:50 1995
--- sys/i386/isa/cy.c   Thu Aug 24 21:58:17 1995
***************
*** 153,159 ****
--- 153,161 ----
  
  #define       LOTS_OF_EVENTS  64      /* helps separate urgent events from input */
  #define       RB_I_HIGH_WATER (TTYHOG - 2 * RS_IBUFSIZE)
+ #ifndef RS_IBUFSIZE
  #define       RS_IBUFSIZE     256
+ #endif
  
  #define       CALLOUT_MASK            0x80
  #define       CONTROL_MASK            0x60

and these in my configuration files

maxusers        128
options "NMBCLUSTERS=4096"
options "TTYHOG=8192"
options "RS_IBUFSIZE=2048"

Without increasing TTYHOG I could not get even 38.4k through reliably.
This was on 386-40, 486-40 and a 386-16.  Curiously the 486-40 was loosing
the most data, 386-40 was the best.  I have oversized these, I got good
results at 1k/4k, so I decided that doubling them should be enough.  We use
AST4's with sio (one per machine) and cyclades 16 port cards (one or two
per machine).

Notice that the problems don't show up with dialup modems, only with leased
line stuff which really does not have any flow control.  You are slowing up
normal dialup modems also, but it does not show up in other than occasional
inability to receive data at full 115.2k speed.

Growing these values also reduces number of crashes, I do not know why.

	   feel FreeBSD has to have a "stable" version followed by an unstable
	   one, just because the next "." number is even and someone
	   did releases that way accidentally ten years ago.

The primary reason I use FreeBSD in our servers is better release scheme
and more controlled management, gnats, cvs and such things.  Also I do have
the knowledge to install the missing pieces, which Linux distributions
usually include.  I also can deal with supping, make worlding and such
things.  I can also deal with panics and bugs.  But I can't see how our
computer-illiterated customers could do that, not even close.  They want
something which is complete, includes everything in sight, and never
crashes. 

	   On the other hand, I understand why these ISPs want a stable core.
	   Tweaking a few microseconds out of memory allocation or something
	   is not interesting or relevant to them since they can get a faster
	   system by spending money on faster hardware.  What they want from
	   the software is robustness.  Code speed is secondary.  
	   Also, apps, utilities and higher-level stuff that is flakey can be
	   worked-around, but only if that stuff exists at all.

Yes!

   8.	File creation (particularly directories) appears to be slow compared
	   to other BSD-like systems.  They say the stats for INN and CNEWS
	   for articles processed per second are quite a bit lower than that
	   on some "other" systems.  They say that file deletion seems to be
	   a bit slower than BSDI, but not by much.  I think they are talking
	   2.0.5 on this item, although one ISP was experimenting with 1026 SNAP.

Generic BSD fs problem, to my knowledge.  Has anyone tried to use the
ext2fs in -current for this heavy problem?  Could help a lot, but is it
stable enough?

   9.	A concern that man pages don't stay formatted.  They would rather
	   chew disk space than have newbies constantly reformatting 'ls' and
	   'cat'.  They seem to want the old BSD-style arrangement where the
	   first time a page is accessed it gets formatted and then that copy
	   hangs around for future reference.  This seemed like something
	   they could fix themselves, but they responded that there are
	   dozens of things like this that they would have to re-tweak
	   everytime the system was updated. 

	   To me this didn't seem to important, but they mentioned it.
	   Listen to your customer...

Any work is costly.  You need to pro to find out why the manual pages
aren't formatted, and even he will spend half an hour for every little
thing like this.  They probably would like to keep the pro's doing things
they can bill customers at $100-$200 per hour.

Another irritating thing are the des libraries, it really should work, no
matter where you live.

The "geometry is incorrect" when installing is something which I know would
scare most unexperienced users away.  Correct way to report this is "Your
disk drive geometry will not work with FreeBSD.  Please use values
ccc/hh/ss instead." or better "Your disk drive geometry will not work with
FreeBSD.  Do you wish to replace the values with ccc/hh/ss (y/n) ?". 

BTW, Out-of-box Slackware installation doesn't format the manual pages
either, at least it was like that some time ago.

   10.	More support for high-end hardware.  I put this last because
	   it is one of the harder things for FreeBSD to do much about
	   since hardware vendors don't always want to tell us the tricks
	   to making their hardware work.  They aren't just talking about
	   plug-in cards.  These guys are interested in multiple-processors,
	   things that make load scale nicely, extremely fast interconnects
	   between systems, shared disk farms, Video-delivery bandwidths, etc.
	   Having Support the latest whizzo video cards aren't a big deal to
	   them, but support for disk controllers, CPUs and network interfaces
	   are.  This makes their priorities different in general from those
	   that the average FreeBSD user probably has.  Most of us aren't
	   ready to pop down and buy a four-processor P6 system with 256Meg
	   of RAM and three 100Mbit network interfaces, but these guys are.

I agree with Jordan; this is an issue where ISP's (in addition to
manufacturers) should come forward to support things by either providing
hardware or, better yet, also providing money to support creating free
drivers.

BTW, donating for "generic BSD/Linux driver" is a lot easier to get through
in management than donating for a "FreeBSD driver".  But don't call it
donating, call it "custom driver development" or something like that :-)

A commercial support provider for *BSD* would also do a lot of good.  I
don't mean one-man shops.  There are people who will never buy anything
without a support contract.

-- 
Heikki Suonsivu, T{ysikuu 10 C 83/02210 Espoo/FINLAND,
hsu@cs.hut.fi  home +358-0-8031121 work -4513377 fax -4555276  riippu SN



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199511131351.PAA20135>