Date: Fri, 10 Jul 1998 19:54:08 -0400 (EDT) From: Bill Paul <wpaul@skynet.ctr.columbia.edu> To: current@FreeBSD.ORG Subject: Device driver weirdness Message-ID: <199807102354.TAA23827@skynet.ctr.columbia.edu>
next in thread | raw e-mail | index | archive | help
I have a question for other driver developers writing/maintaining things
for -current. I've been trying to track down a problem I've been having
with the ThunderLAN driver and today I finally found it, but I'm not
sure how to fix it.
The problem seems to manifest itself mostly when running at 10Mb/s mode.
Here's how I reproduce it:
- system is a Compaq Deskpro 4000 or Proliant server with embedded
ThunderLAN chip, 64MB RAM, Pentium Pro 300Mhz CPU with 512K cache
- connect system to 10Mb/s network or otherwise force the chip to work
at 10Mb/s, half-duplex
- run ping -f some.machine.on.local.net
- While the first ping -f is running, start another ping -f to the same
host (on another virtual console)
The first ping -f, if left alone, goes on about its way without any
problems, until you start the second ping -f. After a few seconds, the
chip generates an adapter check interrupt with a status code indicating
that either EOF interrupts are being underserviced or a TX-GO command was
issued before an outstanting TXEOC interrupt was acknowledged. Once an
adapter check interrupt hits, you have to reset the chip.
Basically, the chip is complaining that EOF or EOC interrupts aren't being
acknowledged. It turns out that the reason this happens is that I wrote
the driver to use memory mapped I/O. PCI devices can be accessed using
either programmed I/O or memory mapped I/O. In PIO mode, you have to use
inb/outb and friends and go through the I/O ports returned by the PCI
autoconfig. With MMIO, the chip's registers are mapped into the system's
memory and you access them simply by reading or writing to the right
memory addresses.
The problem, I think, is the cache. Interrupts are acknowledged by writing
an ACK command (and some other info) to the ThunderLAN's host command
register, which is 32 bits wide. In order for this to work, the 32-bit
word has to be written out to main memory rather than getting written
to the cache and then flushed out to memory later, otherwise it might
happen that the interrupt acknowledge command is delayed long enough
for the driver to do something else that it isn't supposed to do until
after the ack has been sent.
For the moment, I've fixed this by changing code like this:
csr->tl_host_cmd = TL_CMD_ACK;
to this:
CMD_PUT(csr, TL_CMD_ACK);
where CMD_PUT() is a macro that looks like this:
#define CMD_PUT(c, x) { \
u_int32_t f; \
c->tl_host_cmd = (u_int32_t)x;\
f = c->tl_host_cmd; \
}
The macro basically reads back the command register after it's been
written. This seems to defeat the cache and forces the interrupt
acknowledgement to be written out.
What I don't understand is why I have to do this. I have declared
the csr structure like this:
struct tl_csr {
volatile u_int32_t tl_host_cmd;
volatile u_int32_t tl_ch_parm;
volatile u_int16_t tl_dio_addr;
volatile u_int16_t tl_host_int;
union {
volatile u_int32_t tl_dio_data;
volatile struct tl_regwords tl_dio_words;
volatile struct tl_regbytes tl_dio_bytes;
} u;
};
I was under the impression that the volatile keyword was supposed to
help avoid this problem, but even with 'volatile' in there I still need
to use the macros to force the cache flush in order for commands to
be written to the chip on time. I had the same problem with the code
that read the EEPROM and used the same macro workaround, but I stupidly
forgot the lesson I'd learned and didn't think I might have the same
problem with the other registers.
The two other ThunderLAN drivers I've seen (one from Linux and one from
NetBSD) both use programmed I/O. Consequently, they don't have these
problems. I thought I'd be clever by using memory mapped I/O, and
naturally Murphy just had to zonk me for it. Can anybody tell me the
'politically correct' way to work around this problem? I'm prepared to
update the code to use the macro hack, but I want to see if there's a
better way first.
-Bill
--
=============================================================================
-Bill Paul (212) 854-6020 | System Manager, Master of Unix-Fu
Work: wpaul@ctr.columbia.edu | Center for Telecommunications Research
Home: wpaul@skynet.ctr.columbia.edu | Columbia University, New York City
=============================================================================
"It is not I who am crazy; it is I who am mad!" - Ren Hoek, "Space Madness"
=============================================================================
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199807102354.TAA23827>