[plug] [OT] http: different results from browser and telnet

Carl Gherardi gherardi at cs.curtin.edu.au
Thu Oct 25 15:52:20 WST 2001


On Thu, 25 Oct 2001, Leon Blackwell wrote:

> On Thu, Oct 25, 2001 at 02:49:44PM +0800, Carl Gherardi wrote:
> > Question still stands, why two different pages?
>
> I give you this (somewhat familiar sounding) answer):
>
> > On Thu, 25 Oct 2001, Leon Brooks wrote:
> > > Name-based virtual hosting. The base host is pr0n, badlinks is name-based on
> > > the same IP. Try:
> > >
> > > Host: www.badlinks.com
> > > GET /
>
> :)
>
> Basically, the webserver is hosting a number of sites on the one IP
> address and uses the site's name to decide what content to server.
> Since you didn't specify the hostname when you used Lynx, it used the
> default virtual host.
>

Ok bit more of a description is in order, i'm writing a program in c which
get the results of a web search by directly querying a search engine then
iteratively fetching and stripping links out of resulting pages. (see
gherardi.cs.curtin.edu.au for more details - it doesn't do anything
atm...)

Basically my program gets the name www.badlinks.com from a query to
dmoz.org (Open Directory "redhat linux") and then tries to fetch / from
that hostname.

My program returns the pr0n result(which i was emulating using telnet),
netscape, lynx get one page and dmoz has indexed another(probably out
of date) page.

How can I get my page to return the same as netscape?

Thanks

Carl




More information about the plug mailing list