[plug] web server questions

Sat Sep 20 11:51:03 WST 2003

What I'm asking, is there a way to detect HTTP traffic and only allow traffic through that is accessing the virtual web sites on the webserver.  Two weeks ago I found out that spamming was done using HTTP traffic to disguise it's real intent and Matt discovered and fixed it.  Also what I'm asking is since it's possible to have a filtering list attached to a mail server (e.g check if the address is a know spam address)  to validate  that the sender is a known spammer, is there a similar filtering mechanism for HTTP, DNS and ICMP traffic.  Yes I know that some of the http, dns and ICMP traffic is legit, I want to filter out the illegitimate traffic.  Surely it's possible to do a similar filtering system.  Since the traffic that is being disguise as 
Is this better handled by number of packets/sec in a firewall rule?
IDS system only detects they do not act on this detection unless someone knows of one that does. If so, I would be interested in such a package.

As for the hoax mail since the mail is coming through SMTP, it be further checked.  Since all SMTP connection are allowed I guess the only  available method is blocking attachments, and content filtering.  But this has it's drawbacks as some of the mail may be valid.  Either way the spammer gets his traffic to and through to a certain point.  This of course causes traffic to slow down due to the volume.
see JLM> for comments

Jon L. Miller, MCNE, CNS
Director/Sr Systems Consultant
MMT Networks Pty Ltd
http://www.mmtnetworks.com.au

"I don't know the key to success, but the key to failure
 is trying to please everybody." -Bill Cosby

>>> devenish at guild.uwa.edu.au 10:33:41 AM 20/09/2003 >>>
In message <sf6c28e2.069 at mmtnetworks.com.au>
on Sat, Sep 20, 2003 at 10:15:55AM +0800, Jon  Miller wrote:
> while viewing the logs (/var/log/httpd/access.log) and seeing a lot MS
> hoax e-mails being deleted by MailMonitor I'm wondering is it possible
> to block certain sites from accessing the web server.

I'm confused: you are receiving lots of hoax e-mails. Okay. What on
earth does this have to do with your web server?

> Unlike mail servers where one can setup blacklist/blackholes/rbl list
> is there such a service for web servers?

Absolutely. There are many ways of doing this. For example:

- packet and connection filters (e.g. ipchains, tcpwrappers)
- web server configuration (consult documentation for your web server)

Apache has directives such as Allow and Deny. It is possible to make it
much more sophisticated than that, though.
JLM> You stated absolutely to the web server question using ipchains, etc, but wouldn't this have to be constantly updated with new IP addresses as they become available?

> I've noticed the following:
> 
> /var/log/httpd/error.log
> [Sat Sep 20 10:01:12 2003] [error] [client 61.139.60.84] File does not exist: /var/www/html/tmpad/banner/itrack.asp
> [Sat Sep 20 10:01:13 2003] [error] [client 61.139.60.84] File does not exist: /var/www/html/a.htm
> [Sat Sep 20 10:01:22 2003] [error] [client 210.83.18.98] File does not exist: /var/www/html/search.php
> [Sat Sep 20 10:01:35 2003] [error] [client 61.139.60.84] File does not exist: /var/www/html/Affiliate/SB/search1.js

So what? Does this bother you in some way? Could you elaborate?
JLM> Yes as this just is a small amount, in the log files this goes on for hours on end, thus limiting our services. We use a 2M/@M connection and at times it feels like a a 56kb connection.  The logs are flooded with these errors.

> /var/log/httpd/access.log
> 221.pool0.dsltokyo.att.ne.jp - - [20/Sep/2003:10:08:26 +0800] "GET / HTTP/1.1" 200 9515
> public2-runc2-5-cust118.manc.broadband.ntl.com - - [20/Sep/2003:10:08:26 +0800] "GET / HTTP/1.1" 200 9515
[...]
> These may or may not be legit entries, is there a way to tell other than bringing those site up.

Huh? What do you mean "legit entires"? They are log entries of pages
served by your web server, correct? So...they are simply a record of
what was happening. From the information that you've presented so far,
it looks like two remote users accessed a home page that is served by
your web server. What is the problem with that? Many websites have
hundred of thousands or millions of accesses to their home pages every
day. The two remote hosts are probably user machines...what do you mean
"bringing those sites up"?

JLM> Yes, these may be legitimate entries meaning they are looking at the clients web pages, no problem here, but the client tracks the number of hits to their site, these we know will count, but does the one such as these below count also?
61.139.60.84 - - [20/Sep/2003:11:43:26 +0800] "GET http://www.uccinema.com/a.htm HTTP/1.0" 404 199
220.113.15.29 - - [20/Sep/2003:11:43:33 +0800] "GET http://a.as-eu.falkag.net/dat/dlv/aslmain.js HTTP/1.1" 404 224
61.139.60.84 - - [20/Sep/2003:11:43:35 +0800] "GET http://ad.trafficmp.com/tmpad/banner/itrack.asp?rv=1.2&id=2873 HTTP/1.0" 404 217

Since they are 404 codes I know they are not completing their GET command because the pages or files do not exists, but the traffic they consume is immense.

So what I'm trying to understand is there must be a way to get our bandwidth back and eliminate this type of traffic from consuming the bandwidth or am I batting my head up against a hard wall?

Thanks
_______________________________________________
plug mailing list
plug at plug.linux.org.au
http://mail.plug.linux.org.au/cgi-bin/mailman/listinfo/plug
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20030920/79c9ea8d/attachment.htm>