download all mib files from a web page

powah wong_powah at yahoo.ca
Wed May 27 12:29:27 EDT 2009


I want to download all mib files from the web page:
http://www.juniper.net/techpubs/software/junos/junos94/swconfig-net-mgmt/juniper-specific-mibs-junos-nm.html#jN18E19

All mib filenames are of this format:
www.juniper.net/techpubs ... .txt

I write this program but has the following error.
Please help.
Thanks.

[code]
#!/usr/bin/env python
import urllib2,os,urlparse
url="http://www.juniper.net/techpubs/software/junos/junos94/swconfig-
net-mgmt/juniper-specific-mibs-junos-nm.html#jN18E19"
page=urllib2.urlopen(url)
f=0
links=[]
data=page.read().split("\n")
for item in data:
    if "www.juniper.net/techpubs" in item:
        httpind=item.index("www.juniper.net/techpubs")
        item=item[httpind:]
        #print "item " + item
        ind=item.index("<")
        links.append(item[:ind]) #grab all links
# download all links
for link in links:
    print "link " + link
    filename=link.split("/")[-1]
    print "downloading ... " + filename
    u=urllib2.urlopen(link)
    p=u.read()
    open(filename,"w").write(p)
[/code]

$ ~/python/downloadjuniper.py
link www.juniper.net/techpubs/software/junos/junos94/swconfig-net-mgmt/mib-jnx-user-aaa.txt
downloading ... mib-jnx-user-aaa.txt
Traceback (most recent call last):
  File "/home/powah/python/downloadjuniper.py", line 20, in ?
    u=urllib2.urlopen(link)
  File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.4/urllib2.py", line 350, in open
    protocol = req.get_type()
  File "/usr/lib/python2.4/urllib2.py", line 233, in get_type
    raise ValueError, "unknown url type: %s" % self.__original
ValueError: unknown url type:
www.juniper.net/techpubs/software/junos/junos94/swconfig-net-mgmt/mib-jnx-user-aaa.txt



$ python
Python 2.4.4 (#1, Oct 23 2006, 13:58:00)
[GCC 4.1.1 20061011 (Red Hat 4.1.1-30)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

My computer is FC6 linux.



More information about the Python-list mailing list