[plug] "Cluster" Management. Job Queue & network commands

Tim weirdit at gmail.com
Fri Sep 3 15:17:27 WST 2010


I have a lot of video encoding I need to do. Many years ago, it was
ether mp3 encoding or video encoding, I discovered a nice CLI program
that would run a queue and farm out the jobs to my other machines,
similar to distcc. I'm looking for something similar now as I can't
remember what it was I used.
I've seen and tried ppss which the author of used with Handbrake
(which I'm using) to run his cluster, but I've not been successful in
getting it all working.
I have SSH keys all setup so that you can login between all the
machines in the "cluster" without passwords. They all have NFS mounts
to the same point so essentially it just needs to SSH into the remote
nodes, and start a command and wait for it to finish and then when it
does finish, let the master node know so it can be taken out of the
queue.
Any ideas short of a bash script?
I've looked at ppss, parallel, pexec. I know dvd::rip has cluster
functionality, but I don't want to use that if I can avoid it. I've
got gearman but haven't had a chance to try and work it out.

Thanks

Tim

-- 
Timothy White - Somewhere in Australia



More information about the plug mailing list