[plug] ftp scripting?

Brad Hill brad at marketcast.com.au
Wed May 21 17:47:17 WST 2003


sorry to the rsync and unison guys, but it has to be ftp.

I've used Russel's suggestion so far but i'm wondering about the limitations
of the amount of files that may wind up in the touch directory.... What's
linux's limit of files per directory... ?

this is what i've got so far:

while true
do
        # The files for each day are in a folder named as the date
        # The basedir never changes, but the ftp url contains a (sensitive)
        # password hence the "-i url" option to wget
        url=`cat baseurl``date +%Y%m%d`
        echo $url/ > url
        # We don't want anyone else reading this file
        chmod 600 url
        # I'm sure there's a better sed (or awk) substitution to handle
this...
        for file in `wget -O - --quiet -i url | sed "s/.*\">//" | sed
"s/<.*//"`
       do
                if [ ! -e touch/$file ]
                then
                        echo $url/$file > url
                        chmod 600 url
                        wget -i url && touch touch/$file && mv $file
downloaded
                        # downloaded/$file will get cleaned out, but
touch/$file wont
                fi
        done
        rm url
        sleep 60
done

Still have to handle clearing the touch folder out somehow, but since the
directory listing is in fact in seperate folders each day, and i want to be
able to handle if they miss a file yesterday and sent it instead today i
need to keep more than just today's files in the touch folder.



More information about the plug mailing list