[plug] Tracing memory leaks in python(!) programs.

Shayne O'Neill shayneo at bestflights.com.au
Fri Oct 13 15:45:44 WST 2006


I've got a fairly perplexing problem

I've got a multi threaded data processing app that churns thru
aproximately 1 cubic ziggatonne of xml files.

Normally with about 6 threads (threaded for IO purposes. Threading
doesn't really help any other performance in python) it should take
about ten minutes to process. Lately that jumped to about a day. The
thing starts slow and gradually slows down to an almost glacial speed

I've noted now its memory footprint swells to around 2 1/2 gb. Ugh.

So , does anyone know a good way to profile and debug a multi threaded
python app to look for a memory leak?



More information about the plug mailing list