Date: Fri, 30 Nov 2001 10:10:28 +0200 From: "Patrick O'Reilly" <patrick@mip.co.za> To: "Joseph Wright" <JWRIGHT@mbakercorp.com>, <freebsd-questions@FreeBSD.ORG> Subject: RE: Mirroring apache web sites Message-ID: <NDBBIMKICMDGDMNOOCAIMEFNEAAA.patrick@mip.co.za> In-Reply-To: <sc065319.020@mbakercorp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> From: Joseph Wright > Sent: 29 November 2001 22:24 > > I am wondering what is the best way to mirror two websites. > I have a website inside my network on a freebsd box(this is > where we develop) and what I want to do is everynight push > the changes to a website I have on my dmz. I would like the > process to be automatic probably through cron. Should this be > done with ftp or rsync or ????. Any ideas > Joseph, I have scripted a little solution which uses ftp. One issue I had to deal with, which I suspect you will have to deal with too, is that an automated synchronisation may take across stuff that is still being tested, or has not yet been approved. Here is my solution, be it crude, but it works fine for me. 1) On the development web server I set up an FTP account which has the same root as the DocumentRoot of the www stuff. I'll leave it up to you to apply whatever security you find necessary. You probably already have a suitable ftp account which is used by the html editing tool the web developers use to access the web documents. 2) I require the developers to simply add filenames of any newly changed or added files to a file in that same root. A command as simple as: # echo "mynewfile.html" >> _UPLOADS.txt does the trick. You might script that to ensure no one accidentally does it with just one > :) Naturally, they should only add the file name to the list when it is ready to publish. 3) On the LIVE web server I created the script (below), and added it to cron to run periodically (in my case it is "30 * * * *"). Since this script only fetches across the changes (based on _UPLOADS.txt), not the whole website, the load is very low. Therefore, if you want to run it much more frequently it will not really place a heavy burden on either server. The script looks like this: --------------------------------------- #!/bin/bash # Script to load updated web site for genesis (gmed) # this is just a place to do some work cd /var/mip # first, fetch the _UPLOADS.txt list ( echo "ascii" echo "get _UPLOADS.txt" echo "bye" ) | ftp -i devwww.mydomain.com # if there is a list, then go to it! if [ -f ./_UPLOADS.txt ] then # build the full FTP script on the fly ( # use binary in case there are images, etc echo "bin" # get each file specified for file in `cat ./_UPLOADS.txt` do # get the files and put them in a temp # directory locally echo "get $file ./gmed/$file" done # remove the list off the dev server echo "delete _UPLOADS.txt" # be polite echo "bye" ) | ftp -i devwww.mydomain.com # remove the list locally too now that we are done rm ./_UPLOADS.txt # move the whole lot from the temp directory to # the LIVE DocumentRoot directory cp -r ./gmed/* /www/html/gmed # and clean up after yourself rm -f ./gmed/* fi # That's all folks! --------------------------------------- Of course, you will need a .netrc file to enable the FTP login to work: --------------------------------------- machine devwww.mydomain.com login ftpsync password sesame --------------------------------------- Note that the script does not check for existence of the directories in the temporary workspace - you can add automated directory building into the script, or just make sure they exist before you use the script. Like I said, crude, but effective ;) I hope this helps. BTW: If you need to keep an audit of when changes have happened, you could simply replace: --------------------------------------- # remove the list locally too now that we are done rm ./_UPLOADS.txt --------------------------------------- with: --------------------------------------- # copy the _UPLOADS.txt file to the audit directory timestamp=`date +%Y%m%d%H%M` mv ./_UPLOADS.txt ./audit/$timestamp_UPLOADS.txt --------------------------------------- Have fun, Patrick. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?NDBBIMKICMDGDMNOOCAIMEFNEAAA.patrick>