Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 29 Apr 2012 11:16:32 -0400
From:      Alejandro Imass <ait@p2ee.org>
To:        Polytropon <freebsd@edvax.de>
Cc:        perryh@pluto.rain.com, freebsd-questions@freebsd.org
Subject:   Re: UFS Crash and directories now missing
Message-ID:  <CAHieY7QCpV0Tz-mJHyNuObnF%2BN%2BnAXGpnxr4D1f8s_2E_No%2BHw@mail.gmail.com>
In-Reply-To: <20120429103740.aa7df743.freebsd@edvax.de>
References:  <201204281731.q3SHVaiM061997@mail.r-bonomi.com> <CAHieY7Tcv%2Bo-KbmLtPVHWXSphJX7b5f0QMO46yM-DOju4S9S7Q@mail.gmail.com> <20120428200116.b2f5820e.freebsd@edvax.de> <CAHieY7TpWsCbm8LZFMboWHgXJ2M79TbcK7Jse3=MoVUR2XB5Ow@mail.gmail.com> <4f9ced3a.f7WBDlsMkhxvy%2BeF%perryh@pluto.rain.com> <20120429103740.aa7df743.freebsd@edvax.de>

next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Apr 29, 2012 at 4:37 AM, Polytropon <freebsd@edvax.de> wrote:
> On Sun, 29 Apr 2012 00:26:50 -0700, perryh@pluto.rain.com wrote:
>> Alejandro Imass <ait@p2ee.org> wrote:
>>
>> > 3) the directories were moved at reboot by journal recovery,
>> > fsck or something else
>>
>> I think it's *extremely* unlikely that fsck was involved, because
>> it just doesn't do things like that.
>
> The point is: fsck moving directories "looks different". In
> case inodes get "de-connected" (their reference entries on
> level n-1 are gone, their data on level n is still present),
> fsck will access the lost+found/ directory in the corresponding
> partition's root directory (or create it, if not present) and
> write _new_ directory entries with the inode as their name,
> because that's the only naming information possible (as the
> original names on n-1 aren't accessible anymore). So those
> directories will have names like #177628676/ and they _can_
> contain subtrees full of data, _including_ names from levels
> n+1 and onward. Files also are named #4767667892 and their
> names can _maybe_ identified from their content (the "file"
> command is helpful, and if they are textfiles containing
> a CVS or other revision control system data tag, it's possible
> to find out what they've been in their previous life).
>
> However, as it has been explained, fsck will _not_ do so
> unless being _allowed explicitely_ to do that kind of
> MODIFICATION to the file system. Flags like -yf can do
> that, but they are _not_ the default. This is due to the
> fact that _any_ critical modification of file systems
> requires the _responsible administrator_ to give permission.
>

OK, so fsck couldn't have done this. Besides fsck reported the fs as
clean so I have to conclude as others have commented that it must have
been a mv

I've been looking at the logs very carefully and trying to make sense
of this. There is a possibility that it could have been an attack
because we enabled ftp.proxy so that some clients could upload stuff
to their jails using ftp. So I was initially wrong in my assessment
because on this particular server we are running a service outside of
jails and it's this ftp.proxy that was suppose to be a temporary
solution but I guess we never got around to fixing this.

The ftp.proxy is started via inetd like so:
ftp    stream tcp      nowait nobody /usr/local/sbin/ftp.proxy ftp.proxy -e

And there was a log of a couple of ftp connections the same day this
happened, the ONLY 3 messages before the reboot at about 6 pm and they
were NOT from any of our customers. Here are the log entries:

Apr 27 05:54:37 nune ftp.proxy[2726]: connected to client:
host-46-50-183-5.bbcustomer.zsttk.net, interface= 207.158.52.74:21
Apr 27 05:54:37 nune ftp.proxy[2726]: info: monitor mode: off, ccp: <unset>
Apr 27 05:54:38 nune ftp.proxy[2726]: -ERR: missing hostname
Apr 27 18:55:42 nune syslogd: kernel boot file is /boot/kernel/kernel

OK. So let's suppose ftp.proxy is the culprit is there any way the
could have done the mv by cracking ftp and ftp.proxy ??

I have of course disabled the ftp and I am now thinking that another
possibility or combination by also using the ftp proxy on the
http-proxy jail, that is, the jail that swallowed the other jails. The
http-proxy jails was also running apache ftp proxy.

So the question now becomes: could a break in ftp.proxy coupled with
Apache ftp proxy have caused the http-proxy jails to have swallowed
all the other jails into it's configuration directory??

-- 
Alejandro Imass



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHieY7QCpV0Tz-mJHyNuObnF%2BN%2BnAXGpnxr4D1f8s_2E_No%2BHw>