[plug] Query: Very problematic memory behaviour with SSH/Debian

[James Devenish]

Hi,

This is a query (ahem, looking for free advice!) about runaway memory
usage that a friend and colleague is experiencing with his new Debian
box.

(Note to Trevor Phillips: the machine in question is one you recently
had some dealings with--the user advises that you are the one who
compiled the nvidia driver on it--so you may be more aware of its
configuration than any of us apart from Jeremy Phillips who, I am
lead to believe, did most of the installation. I'm intentionally not
mentioning the user's name in this public forum, so sorry about the
indirect description. Pretend I am mailing from an address @murdoch)

The behaviour relates to wild memory usage when large numbers of files
have been created. When the (woody) machine starts up, there are about
50 processes and ~128 MB is in use (no dramas). It is a new computer
and has an Intel processor of some sort. The user uses KDE and his prior
Linux experience is with R__H__. The memory problem has manifested itself
while doing various file transfers via OpenSSH. I am not sure if the
symptoms appear in other circumstances and we did not get time to test
this out before the close of business (this was on Christmas eve).
Unfortuntely, I haven't been able to look through his log files to
date (with the universities' Christmas holidays and all).

A simple way to induce his problem is to use scp to copy directory
contents. As the copy proceeds, the amount of memory in use increases.
However, this is not identified by ps and top and belonging to any
particular process (e.g.  the ssh processes). The 'cached' memory
reading increases in the same way. Eventually all real memory (about 900
MB) is exhausted and the machine begins thrashing its disk. At no time
does it indicate more an a few MB of swap in use.  All regular queries
(vmstat, top, /proc, &c) report the same overall memory stats. The
processes behave normally from the user's point of view (the copy
procedes, the processes finish, etc).  Killing sshd does not release the
memory in use.  In a rudimentary sense it is like file descriptors were
never being closed and memory allocated in the kernel was not being
released (but tens of thousands of files are involved and there has
never been any 'out of fds' error). The only action that seemed to have
any effect was to reboot. We didn't find any applications other than
file transfers with ssh that could recreate this problem, but I can say
that both scp and rsync-over-ssh produce the effect. Because of these
things, the system is very unresponsive during SSH transfers and ends up
unusable even after the transfer is complete.

Another issue is that the X server (XFree86) is reported as having a
memory "size" of 256 MB and a resident size of about 17 MB, at startup,
even though the total memory usage is only 128 MB at startup. Is this
sort of contradiction normal in the sort of system I described? Be aware
that I don't really appreciate the way Linux handles process reporting
and I could be oblivious to some obvious diagnostic tools.

Does this ring a bell for anyone? Or can anyone think of some good
search terms to use ;)

Regards,
James.

PS. I am subscribed to this list for the time being.