Unable to Debug

[Deborah Swanson]

Aritra Bhattacharjee wrote, on January 02, 2017 1:05 AM:
> I am new to python programming. I wrote a code to search for 
> the product names on a page of snapdeal.com .
> 
> Code:
> import urllib.request
> from bs4 import BeautifulSoup as BS
> 
> url = 
> 'https://www.snapdeal.com/products/electronics>
-headphones?sort=plrty'
> 
> response = urllib.request.urlopen(url).read()
> soup = BS(response, 'html.parser')
> 
> #for dataprice in soup.find_all('span', class_="lfloat 
> product-price"):
> #    print(dataprice)
> product_name={}
> i=0
> for title in soup.find_all('p', class_="product-title"):
>         product_name[i]=title.string
>         i += 1
> for i in range(1,21):
>     print(product_name[i])
> 
> 
> Output:
> Traceback (most recent call last):
> Motorola Pulse 2 Over Ear Wired Headphones With Mic (Black) 
> Bose SoundLink On-Ear Bluetooth Headphones - Black & Blue 
> Sony MDR-ZX110A Headphones Without Mic (White) Philips 
> SBCHL140/98 Over Ear Headphone Without Mic
>   File "C:/Users/Aritra 
> Bhattacharjee/PycharmProjects/PythonWebModules/Web 
> Scraper.py", line 17, in <module> Intex Desire BT Over Ear 
> Wired With Mic Headphone Black
>     print(product_name[i])
> JBL T450 On Ear Wired Headphones With Mic Black
> KeyError: 20
> Motorola Pulse Max Over Ear Wired Headphones With Mic (Black) 
> Philips SHB7250WT/00 Over Ear Wireless Headphones With Mic 
> White Sony MDR-XB650BT On-Ear Extra Bass(XB) Headphones with 
> Bluetooth & NFC (Black) Intex JAZZ Over Ear Wired With Mic 
> Headphone Black Skullcandy S5GBW-J539 On Ear Wireless 
> Headphones With Mic Black JBL C300SI Over Ear Wired Without 
> Mic Headphone Black Zoook Rocker iFit Bluetooth Wireless 
> Headphones With Mic Black Signature VM-46 Over Ear Wired 
> Headphone Without Mic White Sony MDR-G45 Over Ear Wired 
> Without Mic Headphone- Black Motorola Pulse Max Over Ear 
> Wired Headphones With Mic (White) Bose SoundTrue Around-Ear 
> Headphones with Mic (Navy Blue) for Samsung and Android 
> Devices JBL T450 On Ear Wired Headphones With Mic Blue 
> Motorola Pulse 2 Over Ear Wired Headphones With Mic (White)
> 
> 
> The Output shows some error that I could not understand 
> why...Thanks in Advance

This looks like Beautiful Soup. I've only used it once, and can just
barely count it as a successful use.

But what stands out to me is the "KeyError: 20", which closely follows
after the "print(product_name[i])" statement. 

If product_name is a dictionary (and I see that it is), the key error
says that none of product_name's keys has the value that was in 'i' when
the error was thrown, ie. the key equal to 20 isn't in the dictionary.

Frequently this occurs in a loop that continues beyond the length of the
dictionary. 

The error was thrown in:

for i in range(1,21):
     print(product_name[i])

so my guess is that you didn't have 20 keys in product_name, and the
error was thrown when you tried to print product_name[20].

You could try just editting the loop so it only goes up to 20, since all
the keys up to 20 were found (and range will stop the loop at one less
than the given upper bound):

for i in range(1,20):
     print(product_name[i])

although I wonder why it isn't:

for i in range(0,19):
     print(product_name[i])

since the first item in the dictionary is index 0. Unless you don't want
the first one to print for some reason.

Hope this helps.
D.