Friday, May 23, 2008

Grab website content using cURL

A website content can grab easily using cURL by following the steps below:
  • Enable cURL
  • Copy following codes in to PHP file
  • Change target URL
  • Run the page from webserver
$target_url = "http://www.morshed-alam.com";

$options = array(
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_HEADER => false,
    CURLOPT_FOLLOWLOCATION => true,
    CURLOPT_ENCODING => "",
    CURLOPT_USERAGENT => "spider",
    CURLOPT_AUTOREFERER => true,
    CURLOPT_CONNECTTIMEOUT => 120,
    CURLOPT_TIMEOUT => 120,
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_SSL_VERIFYPEER => false,
    CURLOPT_SSL_VERIFYHOST => false,
    CURLOPT_URL => $target_url
);

$ch = curl_init($url);

curl_setopt_array($ch, $options);

$content = curl_exec($ch);

curl_close($ch);
Click here to see the cURL Manual
Extract href links from a website link
Demo page to show extracted href links from a website

No comments:

Post a Comment