[plug] Strange process listening on random ports

Alexander Hartner alex at j2anywhere.com
Tue Jun 4 04:44:56 UTC 2013


Hi all,

I am currently investigating a rather strange issue where every couple of minute a new port is being listened on. To trace this issue I created a quick shell script to dump me all processes and their open ports to a file:

#!/bin/bash
echo "==============================================" >> /tmp/traceports.log
date >> /tmp/traceports.log
echo "==============================================" >> /tmp/traceports.log
/usr/sbin/lsof -i -P >> /tmp/traceports.log
echo "==============================================" >> /tmp/traceports.log
netstat -tulpn >> /tmp/traceports.log
echo "==============================================" >> /tmp/traceports.log
echo "Done" >> /tmp/traceports.log

At on point for example port 36281 was listened on. However the output from the log file didn't provide much insight into the source or purpose. LSOF didn't even list this port / process while NETSTAT indicated that the process was "-" (tcp        0      0 0.0.0.0:36281               0.0.0.0:*                   LISTEN      -   )

Any idea how I can investigate this further and isolate the process or application which is listening on these ports.

Thanks in advance for any pointers.
Alex

==============================================
Tue Jun  4 04:40:02 UTC 2013
==============================================
COMMAND     PID     USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
master     1119     root   12u  IPv4    11310      0t0  TCP localhost:25 (LISTEN)
master     1119     root   13u  IPv6    11312      0t0  TCP localhost:25 (LISTEN)
rpcbind    3402      rpc    8u  IPv4    33084      0t0  UDP *:111 
rpcbind    3402      rpc    9u  IPv4    33088      0t0  UDP *:609 
rpcbind    3402      rpc   10u  IPv4    33089      0t0  TCP *:111 (LISTEN)
rpcbind    3402      rpc   11u  IPv6    33091      0t0  UDP *:111 
rpcbind    3402      rpc   12u  IPv6    33093      0t0  UDP *:609 
rpcbind    3402      rpc   13u  IPv6    33094      0t0  TCP *:111 (LISTEN)
rpc.statd  4050  rpcuser    5u  IPv4    42097      0t0  UDP *:834 
rpc.statd  4050  rpcuser    8u  IPv4    42104      0t0  UDP *:38979 
rpc.statd  4050  rpcuser    9u  IPv4    42108      0t0  TCP *:48234 (LISTEN)
rpc.statd  4050  rpcuser   10u  IPv6    42112      0t0  UDP *:42649 
rpc.statd  4050  rpcuser   11u  IPv6    42116      0t0  TCP *:34077 (LISTEN)
sshd       4315     root    3u  IPv4    45281      0t0  TCP *:22 (LISTEN)
sshd       4315     root    4u  IPv6    45283      0t0  TCP *:22 (LISTEN)
ntpd       4447      ntp   16u  IPv4    45597      0t0  UDP *:123 
ntpd       4447      ntp   17u  IPv6    45598      0t0  UDP *:123 
ntpd       4447      ntp   18u  IPv6    45602      0t0  UDP localhost:123 
ntpd       4447      ntp   19u  IPv6    45603      0t0  UDP [fe80::5477:49ff:fe7d:d451]:123 
ntpd       4447      ntp   20u  IPv4    45604      0t0  UDP localhost:123 
ntpd       4447      ntp   21u  IPv4    45605      0t0  UDP th-dc03-con01.one.local:123 
osad       4487     root    3u  IPv4    45703      0t0  TCP th-dc03-con01.one.local:52769->th-dc03-space01.one.local:5222 (ESTABLISHED)
sshd       7216     root    3u  IPv4 20723320      0t0  TCP th-dc03-con01.one.local:22->10.104.97.54:34982 (ESTABLISHED)
sshd       7218 ahartner    3u  IPv4 20723320      0t0  TCP th-dc03-con01.one.local:22->10.104.97.54:34982 (ESTABLISHED)
zabbix_ag 19024   zabbix    4u  IPv4  1169196      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19024   zabbix    8u  IPv6  1169197      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19025   zabbix    4u  IPv4  1169196      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19025   zabbix    8u  IPv6  1169197      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19026   zabbix    4u  IPv4  1169196      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19026   zabbix    8u  IPv6  1169197      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19027   zabbix    4u  IPv4  1169196      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19027   zabbix    8u  IPv6  1169197      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19028   zabbix    4u  IPv4  1169196      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19028   zabbix    8u  IPv6  1169197      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19029   zabbix    4u  IPv4  1169196      0t0  TCP *:10050 (LISTEN)
zabbix_ag 19029   zabbix    8u  IPv6  1169197      0t0  TCP *:10050 (LISTEN)
java      20573      lpm   10u  IPv6 15562206      0t0  TCP *:48082 (LISTEN)
java      20573      lpm   11u  IPv6 15562353      0t0  TCP *:48081 (LISTEN)
java      20573      lpm   12u  IPv6 16327122      0t0  TCP localhost:48081->localhost:52568 (ESTABLISHED)
java      20573      lpm   14u  IPv6 15562358      0t0  TCP *:5435 (LISTEN)
java      20573      lpm   20u  IPv6 16327138      0t0  TCP localhost:48081->localhost:52569 (ESTABLISHED)
java      20573      lpm   21u  IPv6 16327149      0t0  TCP localhost:48081->localhost:52570 (ESTABLISHED)
java      20582      lpm   40u  IPv6 15562469      0t0  TCP *:8080 (LISTEN)
java      20582      lpm   41u  IPv6 15562476      0t0  TCP *:8443 (LISTEN)
java      20582      lpm   42u  IPv6 16327137      0t0  TCP localhost:52569->localhost:48081 (ESTABLISHED)
java      20582      lpm   87u  IPv6 16327121      0t0  TCP localhost:52568->localhost:48081 (ESTABLISHED)
java      20582      lpm   88u  IPv6 16327148      0t0  TCP localhost:52570->localhost:48081 (ESTABLISHED)
java      20582      lpm   92u  IPv6 17443300      0t0  TCP th-dc03-con01.one.local:33654->10.103.45.97:8443 (CLOSE_WAIT)
java      20582      lpm   96u  IPv6 17443330      0t0  TCP th-dc03-con01.one.local:35346->10.103.45.105:8443 (CLOSE_WAIT)
java      20582      lpm   99u  IPv6 17443309      0t0  TCP th-dc03-con01.one.local:36762->10.103.45.102:8443 (CLOSE_WAIT)
java      20582      lpm  101u  IPv6 17443310      0t0  TCP th-dc03-con01.one.local:36763->10.103.45.102:8443 (CLOSE_WAIT)
java      20582      lpm  103u  IPv6 17443312      0t0  TCP th-dc03-con01.one.local:51976->10.103.45.99:8443 (CLOSE_WAIT)
java      20582      lpm  105u  IPv6 17443314      0t0  TCP th-dc03-con01.one.local:51977->10.103.45.99:8443 (CLOSE_WAIT)
java      20582      lpm  109u  IPv6 17443320      0t0  TCP th-dc03-con01.one.local:52449->10.103.45.103:8443 (CLOSE_WAIT)
java      20582      lpm  114u  IPv6 17443325      0t0  TCP th-dc03-con01.one.local:44779->10.103.45.104:8443 (CLOSE_WAIT)
java      20582      lpm  115u  IPv6 17443326      0t0  TCP th-dc03-con01.one.local:44780->10.103.45.104:8443 (CLOSE_WAIT)
java      20582      lpm  117u  IPv6 17443328      0t0  TCP th-dc03-con01.one.local:35345->10.103.45.105:8443 (CLOSE_WAIT)
.vasd     22532   daemon   14u  IPv4 20755165      0t0  TCP th-dc03-con01.one.local:49161->th-dc03-ad02.one.local:389 (ESTABLISHED)
ossec-age 29917    ossec   16u  IPv4  1505336      0t0  UDP th-dc03-con01.one.local:48001->th-dc03-hids01.one.local:1514 
==============================================
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      3402/rpcbind        
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      4315/sshd           
tcp        0      0 0.0.0.0:36281               0.0.0.0:*                   LISTEN      -                   
tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      1119/master         
tcp        0      0 0.0.0.0:10050               0.0.0.0:*                   LISTEN      19024/zabbix_agentd 
tcp        0      0 0.0.0.0:48234               0.0.0.0:*                   LISTEN      4050/rpc.statd      
tcp        0      0 :::111                      :::*                        LISTEN      3402/rpcbind        
tcp        0      0 :::8080                     :::*                        LISTEN      20582/java          
tcp        0      0 :::48081                    :::*                        LISTEN      20573/java          
tcp        0      0 :::48082                    :::*                        LISTEN      20573/java          
tcp        0      0 :::22                       :::*                        LISTEN      4315/sshd           
tcp        0      0 ::1:25                      :::*                        LISTEN      1119/master         
tcp        0      0 :::35675                    :::*                        LISTEN      -                   
tcp        0      0 :::8443                     :::*                        LISTEN      20582/java          
tcp        0      0 :::5435                     :::*                        LISTEN      20573/java          
tcp        0      0 :::34077                    :::*                        LISTEN      4050/rpc.statd      
tcp        0      0 :::10050                    :::*                        LISTEN      19024/zabbix_agentd 
udp        0      0 0.0.0.0:111                 0.0.0.0:*                               3402/rpcbind        
udp        0      0 10.103.20.29:123            0.0.0.0:*                               4447/ntpd           
udp        0      0 127.0.0.1:123               0.0.0.0:*                               4447/ntpd           
udp        0      0 0.0.0.0:123                 0.0.0.0:*                               4447/ntpd           
udp        0      0 0.0.0.0:834                 0.0.0.0:*                               4050/rpc.statd      
udp        0      0 0.0.0.0:38979               0.0.0.0:*                               4050/rpc.statd      
udp        0      0 0.0.0.0:42334               0.0.0.0:*                               -                   
udp        0      0 0.0.0.0:609                 0.0.0.0:*                               3402/rpcbind        
udp        0      0 :::111                      :::*                                    3402/rpcbind        
udp        0      0 fe80::5477:49ff:fe7d:d45:123 :::*                                    4447/ntpd           
udp        0      0 ::1:123                     :::*                                    4447/ntpd           
udp        0      0 :::123                      :::*                                    4447/ntpd           
udp        0      0 :::42649                    :::*                                    4050/rpc.statd      
udp        0      0 :::45386                    :::*                                    -                   
udp        0      0 :::609                      :::*                                    3402/rpcbind        
==============================================
Done
[root@




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20130604/c1259f4c/attachment.html>


More information about the plug mailing list