Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 08 Apr 2015 14:46:29 -0400
From:      John Baldwin <jhb@freebsd.org>
To:        Gleb Smirnoff <glebius@freebsd.org>
Cc:        svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org
Subject:   Re: svn commit: r281188 - head/usr.bin/netstat
Message-ID:  <4894607.y8WAnnWEI6@ralph.baldwin.cx>
In-Reply-To: <20150408182426.GT64665@FreeBSD.org>
References:  <201504070550.t375oklh014315@svn.freebsd.org> <6969546.Tk7XnsGrVd@ralph.baldwin.cx> <20150408182426.GT64665@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday, April 08, 2015 09:24:26 PM Gleb Smirnoff wrote:
>   John,
> 
> On Wed, Apr 08, 2015 at 12:24:48PM -0400, John Baldwin wrote:
> J> On Tuesday, April 07, 2015 05:50:46 AM Gleb Smirnoff wrote:
> J> > Author: glebius
> J> > Date: Tue Apr  7 05:50:45 2015
> J> > New Revision: 281188
> J> > URL: https://svnweb.freebsd.org/changeset/base/281188
> J> > 
> J> > Log:
> J> >   Like it was already done for 'netstat -i', drop the kvm(3) support
> J> >   in 'netstat -r'.
> J> >   
> J> >   The netstat/route.c was the last abuser of struct ifnet and struct
> J> >   rtentry in the tree. With this change if_var.h can become kernel
> J> >   only include, _WANT_RTENTRY can go away and projects/ifnet and
> J> >   projects/routing can go forward.
> J> >   
> J> >   Differential Revision:	https://reviews.freebsd.org/D2242
> J> >   Reviewed by:		melifaro, gnn
> J> >   Sponsored by:		Nginx, Inc.
> J> >   Sponsored by:		Netflix
> J> 
> J> This breaks netstat -r on a crashdump.  Just because you don't find this
> J> information useful during post-mortem debugging doesn't mean that other folks
> J> do not.  The easiest solution is to update crashinfo to remove the netstat
> J> commands it runs.
> 
> I do analyze cores often, and always find gdb the only useful tool. My opinion
> is seconded by melifaro@, who was quite a lot into the routing debugging for
> the last years.

It can be useful to look at this information when debugging non-routing
problems as well.  Please understand that having more information available
to debug kernel panics (which are sometimes triggered by specific behavior of
user applications) is a feature, not a bug.  I didn't really appreciate this
until I worked in an environment with thousands of machines where there were
on the order of 100's of panics per day that could only be debugged via
post-mortem.  Just because you haven't needed some bit of info in the past
doesn't mean that someone else hasn't (or won't in the future).

> J> However, I don't understand the fear that system binaries can peek inside
> J> the kernel for debugging info.  It's ok for it to be a fragile ABI that
> J> isn't exposed to normal applications and to require people to keep a matching
> J> netstat with their kernel (along with the other tools crashinfo calls).  Just
> J> update the code in netstat to cope with the new data structures.  That should
> J> be much easier to test than having to convert N device drivers.
> 
> You mentioned only the smallest problem - the need to routinely update
> the netstat, as the fragile ABI changes.
> 
> But there are two more problems:
> 
> 1) If userland application knows kernel structures, then we can not merge
> to stable/X changes that change these structures. For example, since
> netstat in stable/10 still uses kvm(3), we can't merge r256525 and r262763,
> which improve performance and make counters precise. Yes, exactly the counters
> that netstat is meant to report. We can't fix the netstat output due to netstat!

No.  This is what I meant about it being a fragile ABI.  I mean specifically
that we do not support this as a public ABI and that users have to have a
matching netstat binary to examine crash dumps in this manner.  That requirement
isn't very hard to achieve in practice.  We already have this in spades inside
of libkvm and kgdb: structures that they know about that aren't part of the
supported ABI for non-system tools.

> 2) If we expose kernel structures to userland, people would write code that
> use them. And the code won't go into src/tools only, it will also go into
> ports. Albeit we have a stable API to read interface addresses and counters,
> in the ports there were 15 applications, that were based on copy-paste from
> netstat/if.c, using kvm(3) instead of API. And I had to fix them all.

No, you are assuming that just because we expose it under __WANT_FOO means
we support it as a public ABI.  I'm saying we do not.

I think the copy and paste doesn't quite work as an argument either.  In
the olden days, kvm was the only way to get this data, so that was the
only thing to copy and paste.  Several tools that have old origins copied
that.  Just as we have fixed netstat to not use kvm aside from crash dumps,
those tools also have to be fixed.  However, new tools aren't going to copy
crash dump code.  They are going to copy what netstat does now for new
kernels.  Thus, I don't think we will have future kvm(3) copy/paste problems,
only existing ones to deal with.

> Finally, when we start to expose kernel structures to userland this leads
> to endless header dependencies, and may yield in expose all policy, but
> usually yields in dirty hacks like hiding second half of a structure
> under #ifdef _KERNEL.

We've managed to expose bits to tools like netstat for 20 years.  History
does not support your argument of this being impossible.

-- 
John Baldwin



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4894607.y8WAnnWEI6>