[plug] "Cluster" Management. Job Queue & network commands

[William Kenworthy]

openmosix?

BillK

On Fri, 2010-09-03 at 17:17 +1000, Tim wrote:
> I have a lot of video encoding I need to do. Many years ago, it was
> ether mp3 encoding or video encoding, I discovered a nice CLI program
> that would run a queue and farm out the jobs to my other machines,
> similar to distcc. I'm looking for something similar now as I can't
> remember what it was I used.
> I've seen and tried ppss which the author of used with Handbrake
> (which I'm using) to run his cluster, but I've not been successful in
> getting it all working.
> I have SSH keys all setup so that you can login between all the
> machines in the "cluster" without passwords. They all have NFS mounts
> to the same point so essentially it just needs to SSH into the remote
> nodes, and start a command and wait for it to finish and then when it
> does finish, let the master node know so it can be taken out of the
> queue.
> Any ideas short of a bash script?
> I've looked at ppss, parallel, pexec. I know dvd::rip has cluster
> functionality, but I don't want to use that if I can avoid it. I've
> got gearman but haven't had a chance to try and work it out.
> 
> Thanks
> 
> Tim
> 

-- 
William Kenworthy <billk at iinet.net.au>
Home in Perth!