[plug] Fwd: Re: Anyone know PHP + cURL?

Thu Dec 2 14:12:04 WST 2010

HA

and then even still dumber, I again sent it as the non member email!
I think I need to stay away from email today!

On 02/12/10 14:09, Lucas van Staden wrote:
> Ok, being really dumb today.
> .
> First sent reply as non member email, then sent it to wrong list 
> (sorry for any duplicated posts to those belonging to off-topic list 
> as well)
>
> -Lucas
>
> -------- Original Message --------
> Subject: 	Re: [plug] Anyone know PHP + cURL?
> Date: 	Thu, 02 Dec 2010 14:02:18 +0800
> From: 	Lucas van Staden <lvs at dedmeet.com>
> Reply-To: 	lvs at dedmeet.com
> To: 	plug off-topic <off-topic at plug.org.au>
>
>
>
> Hi,
>
> Not sure what you have tried, but here are some ideas to bounce.
>
> Are you 100% your post cars are correctly formatted?
> Are you building the postvars yourself that is used in curl?
>
> My original attempt was to build the actual post vars in the code, but 
> that failed.
> Just passing a cut and pasted string from tamper data plugin worked 
> for me.
>
> Install Firefox plug Tamper Data.
> Start it.
> Then login to the page normally
> Tamper Data will give a lot of  results.
> The one you are interested should be the first or second, holding the 
> actual post vars as they are formatted by the form.
> Cut and paste that to your curl (remember to make any @ signs back to 
> be @ (they will be url encoded)
>
> See if you can then post/login using what you got from Tamper Data.
>
> Also, as mentioned before, I could not login, unless I had once loaded 
> the homepage (and saved the cookie from there first)
> Not sure why, but it is most likely just that site I scrape's manner 
> in detecting cookies is available.
>
> As to what comes back, just dump the result into a file (or use a 
> debugger like xdebug) to read the vars values in realtime.
> Solving this type of issue is very handy with a debugger and step 
> through teh code in 'real-time'
>
> (untested)
>
> $data = curl_exec($ch);
> $fp = fopen("/tmp/test",'w+');
> fputs($fp, $data);
> close($fp);
>
> If the result is False (bool) then it is most likely not even getting 
> to post to the page, or you are being diverted to another page, and 
> curl has been set not to follow.
>
> I am not sure on curl verbose level but the verbose option in the 
> curel I posted previously will place all headers to stdout.
> Read through them carefully, as they will most likely give you a hint.
>
> Also make sure you set a CURLOPT_USERAGENT and a CURLOPT_REFERER (they 
> could be detecing the scripting, and blocking you, or testing the 
> referer (make that the same as the login url, so the post looks like 
> it came from that page.)
>
> no further ideas.
>
> -Lucas
>
>
>
> -Lucas
>
>
> On 02/12/10 10:09, Shanon Loughton wrote:
>> Thanks Daniel for the suggestion of Python, and I would like to try 
>> that eventually. But there must be a way with cURL because all 
>> examples online are about logging into sites.
>>
>> Lucas, your code makes sense. I was essentially doing the same thing. 
>> However Im unsure that my post variables are actually being 'posted' 
>> or rather what the server reply is, apart from redirect to the login 
>> page again.  Where do I look for curl logs? the Apache ones dont say 
>> much, can I increase curl verbosity?  Is all the info I need in the 
>> header response for each curl exec?
>>
>> thanks
>> Shanon
>>
>>
>> On Wed, Dec 1, 2010 at 9:42 PM, <home at oranges.id.au 
>> <mailto:home at oranges.id.au>> wrote:
>>
>>     On 1 December 2010 13:45, Shanon Loughton <autobot at iinet.net.au
>>     <mailto:autobot at iinet.net.au>> wrote:
>>     > Im trying to extract our monthly telstra bill from the hideous
>>     portal
>>     > https://www.telstrabusiness.com/business/login.jsp
>>
>>     Once you get some success, and once I manage to get a login to said
>>     hideous portal, I'd love to get a snapshot of how you do it.
>>
>>     Greg.
>>     _______________________________________________
>>     PLUG discussion list: plug at plug.org.au <mailto:plug at plug.org.au>
>>     http://www.plug.org.au/mailman/listinfo/plug
>>     Committee e-mail: committee at plug.linux.org.au
>>     <mailto:committee at plug.linux.org.au>
>>
>>
>>
>> _______________________________________________
>> PLUG discussion list:plug at plug.org.au
>> http://www.plug.org.au/mailman/listinfo/plug
>> Committee e-mail:committee at plug.linux.org.au
>>    
>
>
> -- 
> Regards
> Lucas van Staden
>
> http://www.proxiblue.com.au/  (Consumer Electronics Retail Shop)
> http://www.vanstaden.com.au/  (Linux)
>    

-- 
Regards
Lucas van Staden

http://www.proxiblue.com.au/ (Consumer Electronics Retail Shop)
http://www.vanstaden.com.au/ (Linux)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20101202/25ef3920/attachment.html>