[plug] ftp scripting?
Brad Hill
brad at marketcast.com.au
Wed May 21 17:47:17 WST 2003
sorry to the rsync and unison guys, but it has to be ftp.
I've used Russel's suggestion so far but i'm wondering about the limitations
of the amount of files that may wind up in the touch directory.... What's
linux's limit of files per directory... ?
this is what i've got so far:
while true
do
# The files for each day are in a folder named as the date
# The basedir never changes, but the ftp url contains a (sensitive)
# password hence the "-i url" option to wget
url=`cat baseurl``date +%Y%m%d`
echo $url/ > url
# We don't want anyone else reading this file
chmod 600 url
# I'm sure there's a better sed (or awk) substitution to handle
this...
for file in `wget -O - --quiet -i url | sed "s/.*\">//" | sed
"s/<.*//"`
do
if [ ! -e touch/$file ]
then
echo $url/$file > url
chmod 600 url
wget -i url && touch touch/$file && mv $file
downloaded
# downloaded/$file will get cleaned out, but
touch/$file wont
fi
done
rm url
sleep 60
done
Still have to handle clearing the touch folder out somehow, but since the
directory listing is in fact in seperate folders each day, and i want to be
able to handle if they miss a file yesterday and sent it instead today i
need to keep more than just today's files in the touch folder.
More information about the plug
mailing list