strip away html tags from extracted links

[Joel Goldstick]

On Fri, Nov 29, 2013 at 12:33 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote:

> On 29/11/2013 16:56, Max Cuban wrote:
>
>> I have the following code to extract certain links from a webpage:
>>
>> from bs4 import BeautifulSoup
>> import urllib2, sys
>> import re
>>
>> def tonaton():
>>      site = "http://tonaton.com/en/job-vacancies-in-ghana"
>>      hdr = {'User-Agent' : 'Mozilla/5.0'}
>>      req = urllib2.Request(site, headers=hdr)
>>      jobpass = urllib2.urlopen(req)
>>      invalid_tag = ('h2')
>>      soup = BeautifulSoup(jobpass)
>>      print soup.find_all('h2')
>>
>> The links are contained in the 'h2' tags so I get the links as follows:
>>
>> <h2><a href="/en/cashiers-accra">cashiers </a></h2>
>> <h2><a href="/en/cake-baker-accra">Cake baker</a></h2>
>> <h2><a href="/en/automobile-technician-accra">Automobile
>> Technician</a></h2>
>> <h2><a href="/en/marketing-officer-accra-4">Marketing Officer</a></h2>
>>
>> But I'm interested in getting rid of all the 'h2' tags so that I have
>> links only in this manner:
>>
>> <a href="/en/cashiers-accra">cashiers </a>
>> <a href="/en/cake-baker-accra">Cake baker</a>
>> <a href="/en/automobile-technician-accra">Automobile Technician</a>
>> <a href="/en/marketing-officer-accra-4">Marketing Officer</a>
>>
>>
>> This is more a beautiful soup question than python.  Have you gone
>> through their tutorial.  Check here:
>>
>
They have an example that looks close here:
http://www.crummy.com/software/BeautifulSoup/bs4/doc/

One common task is extracting all the URLs found within a page’s <a> tags:

for link in soup.find_all('a'):
    print(link.get('href'))
# http://example.com/elsie
# http://example.com/lacie
# http://example.com/tillie

In your case, you want the href values for the child of the h2 refences.

So this might be close (untested)

for link in soup.find_all('a'):
    print (link.a.get('href'))
# http://example.com/elsie
# http://example.com/lacie
# http://example.com/tillie






-- 
Joel Goldstick
http://joelgoldstick.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20131129/76d9b746/attachment.html>