[plug] Fwd: Re: Anyone know PHP + cURL?
Shanon Loughton
autobot at iinet.net.au
Thu Dec 9 09:46:05 WST 2010
Still no go with this, Im sure my POST variables are correct, including a
JSESSIONID that looks like it is needed.
POST array looks like this:
$cmd="cat /tmp/cookie | tail -1 | awk '{print $7;}'"; // From a prior
curl_exec I get the jsessionid
$sid=exec($cmd);
$post_data['username'] = 'USERNAME';
$post_data['password'] = 'PASSWORD';
$post_data['JSESSIONID'] = $sid;
$post_data['smauthreason'] = 0;
$post_data['TARGET'] = '';
$post_data['error_target'] = 'http%3A%2F%2Fwww.telstrabusiness.com
%2Fbusiness%2Flogin.jsp%3Fmode%3Derror';
$post_data['final_target'] = 'http%3A%2F%2Fwww.telstrabusiness.com
%2Fbusiness%2Fsecure%2FssoLoginProtected';
$post_data['postpreservationdata'] = '';
$post_data['generallogondata'] =
'SMLogonVersion%3D1.0%3BDebug%3Dfalse%3BReferer%3Dhttp%253A%2F%
2Fwww.telstrabusiness.com%2Fbusiness%2Flogin.jsp%3B';
foreach ( $post_data as $key => $value) {
$post_items[] = $key . '=' . $value;
}
//create the final string to be posted using implode()
$post_string = implode ('&', $post_items);
$options = array(
CURLOPT_COOKIEJAR => "/tmp/cookie",
CURLOPT_COOKIEFILE => "/tmp/cookie",
CURLOPT_COOKIESESSION => true,
CURLOPT_RETURNTRANSFER => true, // return web page rather than
printing it
CURLOPT_HEADER => true, // return headers
CURLOPT_VERBOSE => true,
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_SSL_VERIFYPEER => false, // accept any server certificate
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_USERAGENT => "Mozilla/4.0 (compatible; MSIE 6.0;
Windows NT 5.1)", // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_POST => true, // Enable POST
CURLOPT_POSTFIELDS => $post_string //POST data
);
I notice that Im not using CURLOPT_HTTPHEADER to return any headers. Should
I be?
thanks
Shanon
On Thu, Dec 2, 2010 at 2:12 PM, Lucas van Staden <lvs at dedmeet.com> wrote:
> HA
>
> and then even still dumber, I again sent it as the non member email!
> I think I need to stay away from email today!
>
> On 02/12/10 14:09, Lucas van Staden wrote:
>
> Ok, being really dumb today.
> .
> First sent reply as non member email, then sent it to wrong list (sorry for
> any duplicated posts to those belonging to off-topic list as well)
>
> -Lucas
>
> -------- Original Message -------- Subject: Re: [plug] Anyone know PHP +
> cURL? Date: Thu, 02 Dec 2010 14:02:18 +0800 From: Lucas van Staden
> <lvs at dedmeet.com> <lvs at dedmeet.com> Reply-To: lvs at dedmeet.com To: plug
> off-topic <off-topic at plug.org.au> <off-topic at plug.org.au>
>
> Hi,
>
> Not sure what you have tried, but here are some ideas to bounce.
>
> Are you 100% your post cars are correctly formatted?
> Are you building the postvars yourself that is used in curl?
>
> My original attempt was to build the actual post vars in the code, but that
> failed.
> Just passing a cut and pasted string from tamper data plugin worked for me.
>
> Install Firefox plug Tamper Data.
> Start it.
> Then login to the page normally
> Tamper Data will give a lot of results.
> The one you are interested should be the first or second, holding the
> actual post vars as they are formatted by the form.
> Cut and paste that to your curl (remember to make any @ signs back to be @
> (they will be url encoded)
>
> See if you can then post/login using what you got from Tamper Data.
>
> Also, as mentioned before, I could not login, unless I had once loaded the
> homepage (and saved the cookie from there first)
> Not sure why, but it is most likely just that site I scrape's manner in
> detecting cookies is available.
>
> As to what comes back, just dump the result into a file (or use a debugger
> like xdebug) to read the vars values in realtime.
> Solving this type of issue is very handy with a debugger and step through
> teh code in 'real-time'
>
> (untested)
>
> $data = curl_exec($ch);
> $fp = fopen("/tmp/test",'w+');
> fputs($fp, $data);
> close($fp);
>
> If the result is False (bool) then it is most likely not even getting to
> post to the page, or you are being diverted to another page, and curl has
> been set not to follow.
>
> I am not sure on curl verbose level but the verbose option in the curel I
> posted previously will place all headers to stdout.
> Read through them carefully, as they will most likely give you a hint.
>
> Also make sure you set a CURLOPT_USERAGENT and a CURLOPT_REFERER (they
> could be detecing the scripting, and blocking you, or testing the referer
> (make that the same as the login url, so the post looks like it came from
> that page.)
>
> no further ideas.
>
> -Lucas
>
>
>
> -Lucas
>
>
>
> On 02/12/10 10:09, Shanon Loughton wrote:
>
> Thanks Daniel for the suggestion of Python, and I would like to try that
> eventually. But there must be a way with cURL because all examples online
> are about logging into sites.
>
> Lucas, your code makes sense. I was essentially doing the same thing.
> However Im unsure that my post variables are actually being 'posted' or
> rather what the server reply is, apart from redirect to the login page
> again. Where do I look for curl logs? the Apache ones dont say much, can I
> increase curl verbosity? Is all the info I need in the header response for
> each curl exec?
>
> thanks
> Shanon
>
>
> On Wed, Dec 1, 2010 at 9:42 PM, <home at oranges.id.au> wrote:
>
>> On 1 December 2010 13:45, Shanon Loughton <autobot at iinet.net.au> wrote:
>> > Im trying to extract our monthly telstra bill from the hideous portal
>> > https://www.telstrabusiness.com/business/login.jsp
>>
>> Once you get some success, and once I manage to get a login to said
>> hideous portal, I'd love to get a snapshot of how you do it.
>>
>> Greg.
>> _______________________________________________
>> PLUG discussion list: plug at plug.org.au
>> http://www.plug.org.au/mailman/listinfo/plug
>> Committee e-mail: committee at plug.linux.org.au
>>
>
>
> _______________________________________________
> PLUG discussion list: plug at plug.org.auhttp://www.plug.org.au/mailman/listinfo/plug
> Committee e-mail: committee at plug.linux.org.au
>
>
>
> --
> Regards
> Lucas van Staden
> http://www.proxiblue.com.au/ (Consumer Electronics Retail Shop)http://www.vanstaden.com.au/ (Linux)
>
>
>
>
> --
> Regards
> Lucas van Staden
> http://www.proxiblue.com.au/ (Consumer Electronics Retail Shop)http://www.vanstaden.com.au/ (Linux)
>
>
> _______________________________________________
> PLUG discussion list: plug at plug.org.au
> http://www.plug.org.au/mailman/listinfo/plug
> Committee e-mail: committee at plug.linux.org.au
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20101209/c6df0edb/attachment.html>
More information about the plug
mailing list