From owner-freebsd-questions@FreeBSD.ORG Tue Jul 19 10:54:43 2011 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A627E106566B for ; Tue, 19 Jul 2011 10:54:43 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 413A78FC0C for ; Tue, 19 Jul 2011 10:54:42 +0000 (UTC) Received: by wyg24 with SMTP id 24so3564705wyg.13 for ; Tue, 19 Jul 2011 03:54:41 -0700 (PDT) Received: by 10.216.237.205 with SMTP id y55mr564304weq.49.1311072881173; Tue, 19 Jul 2011 03:54:41 -0700 (PDT) Received: from dfleuriot-at-hi-media.com ([83.167.62.196]) by mx.google.com with ESMTPS id fc2sm4243793wbb.18.2011.07.19.03.54.39 (version=SSLv3 cipher=OTHER); Tue, 19 Jul 2011 03:54:40 -0700 (PDT) Message-ID: <4E25626E.3080509@my.gd> Date: Tue, 19 Jul 2011 12:54:38 +0200 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: "C. P. Ghost" References: <201107190549.p6J5n6sP028960@mail.r-bonomi.com> <4E252119.3030208@esiee.fr> <89EB5E14-AA8E-4265-9C5D-22641ECC1C37@my.gd> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Frank Bonnet , "freebsd-questions@freebsd.org" Subject: Re: Tools to find "unlegal" files ( videos , music etc ) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jul 2011 10:54:43 -0000 On 7/19/11 11:06 AM, C. P. Ghost wrote: > On Tue, Jul 19, 2011 at 8:55 AM, Damien Fleuriot wrote: >> On 19 Jul 2011, at 08:15, Frank Bonnet wrote: >>> In France it's illegal and I have my boss's instruction : >>> >>> - find and delete the files that's all. >> >> Bon courage then... >> >> A file can not be illegal per se, so you won't be able to detect >> these by looking up names or contents. > >> Even then, if a file is labeled as personal, privacy protection >> applies and it is *unlawful* for you to process it. > >> (That is in the same way that your employer is strictly forbidden >> from peeking inside your email messages clearly labeled as personal, >> even if they were received on your work mailbox.) > > Exactly! > > Speaking with my university sysadmin hat on: you're NOT allowed to > peek inside personal files of your users, UNLESS the user has waived > his/her rights to privacy by explicitly agreeing to the TOS and > there's legal language in the TOS that allows staff to inspect files > (and then staff needs to abide by those rules in a very strict and > cautious manner). So unless the TOS are very explicit, a sysadmin or > an IT head can get in deep trouble w.r.t. privacy laws. > The poorly written IT TOS of a company can never bypass the law, regardless of anything you agreed to in your company's TOS. It *is* unlawful for them to even open your files as long as they are clearly labeled as private. To open them, they would require a judge's injunction, for example in cases of pedo pornography or the like. >> You may want to look for files that are unusually large. >> They could possibly be ISOs, dvdrips, HD movie dumps... > > Not to forget encrypted RAR files (which btw. could contain anything, > including legitimate content, so be careful here). > It would be unlawful to try to brute force the files' password ;) >> We have the same problem here with users sharing movies on the file >> servers, and what makes it worse is some of their movie files are >> legit because they're, for example, official trailers that are >> reworked and redistributed to our customers. >> >> You won't win this, tell your boss it can not be done. > > What can technically be done is that the copyright owner provides a > list of hashes for his files, and requests that you traverse your > filesystems, looking for files that match those hashes. AND, even > then, all you can do is flag the files, and you'll have to check with > the user that he/she doesn't own a license permitting him/her to own > that file! > Not to mention that: 1/ you'll be doing the copyright protection company's job. This is going to cost you time = money, while it's going to cost them nothing and earn them the copyright holder's money. 2/ as you rightfully pointed out, hashes are exceedingly far from foolproof, for example a downloaded movie could be gzipped, or just tared, and that would completely change its md5/sha hashes. Hell, even stripping 1 second from the ending credits would make the hash fail. > Better talk with your users and resolve the problem using > non-technical means. Inventive users WILL always outsmart any > technical solution that you implement: this is a race you absolutely > can't win. > Head CP's advice Frank, you can't win this, for real. Today you try to find suspicious files, tomorrow your inventive/tech-savvy users will rename their files so they look private, they'll encrypt them with GPG, they'll pack them in a password-protected rar, they'll embed their MP3s in powerpoint documents... You can not and will not win this, you'll only be wasting your time and your company's money. Again and still out of curiosity, are the files available on the internet ? If not, you REALLY shouldn't be bothering. Delete the files from your file server, next they'll swap them on external drives and USB keys. Even using DPI (deep packet inspection) technologies is a flawed approach to this. You can detect that a file matches a given pattern/hash, you will *ABSOLUTELY NEVER* detect that the file is lawful or unlawful.