Morshed Alam
Morshed Alam Software Architect
← Back to Blog

Extract Urls from a remote webpage using PHP

• By Morshed Alam • parsing
Scraping data from website is extremely popular now a days. I have written a simple website parser class to grab all the urls from a website. Shared the class below for all to see and fun.

We will use the parser class below to extract all image sources and hyper links from a website.
Uses:
Create an instance of WebsiteParser class with a website url to get all the urls from their. And, then call getHrefLinks() and getImageSources() method like below to extract hyper links and image sources respectively.

View Demo :: Try it out and rate on phpclasses.org



Tags: #Web Scraping #PHP
← Previous Article Dirty checking to warn for unsaved changes using jQuery Next Article → ActionView::Template::Error Couldn't find file 'jquery-ui' with Rails 3.2

Comments (4)

Unknown
i add the proxy values to curl_options array:<br /><br />properties in the class:<br /> /**<br /> * Proxy Address<br /> * @var int<br /> */<br /> private $proxy;<br /> <br /> /**<br /> * Proxy Port<br /> * @var int<br /> */<br /> private $proxy_port;<br /><br />constructor:<br /> /**<br /> * Class constructor<br /> * @param string $url Target Url to parse<br /> * @param string $link_type Link type to grab<br /> * @param int $proxy proxy address<br /> * @param int $proxy_port proxy port<br /> */<br /> function __construct($url, $link_type = &#39;all&#39;, $proxy = 0, $proxy_port = 0) {<br /> $this-&gt;target_url = $url;<br /> $this-&gt;setUrls();<br /> $this-&gt;setLinksType($link_type);<br /> if ($proxy &gt; 0 &amp;&amp; $proxy_port &gt; 0) {<br /> $this-&gt;proxy = $proxy;<br /> $this-&gt;proxy_port = $proxy_port;<br /> $this-&gt;curl_options = array_merge (<br /> $this-&gt;curl_options,<br /> array(&#39;CURLOPT_PROXY&#39; =&gt; $this-&gt;proxy, &#39;CURLOPT_PROXYPORT&#39; =&gt; $this-&gt;proxy_port));<br /> }<br /> } //__construct()<br />
Unknown
ups sorry, property &quot;proxy&quot; is a string type!
Morshed Alam Author
Thank you for your comment and using the class. You can update that in to https://github.com/morshedalam/url-scraper-php. Can checkout the class from - http://www.phpclasses.org/package/8113-PHP-Parse-and-extract-links-and-images-from-Web-pages.html as well.
Morshed Alam Author
Thank you for your comment and using the class. <br />You can update that in to https://github.com/morshedalam/url-scraper-php. <br /><br />Can checkout the class from - http://www.phpclasses.org/package/8113-PHP-Parse-and-extract-links-and-images-from-Web-pages.html as well.

Leave a Comment