I wonder if I would be able to collect data from such page using Python

Terry Reedy tjreedy at udel.edu
Wed Aug 21 15:18:19 EDT 2013


On 8/21/2013 1:52 PM, Joel Goldstick wrote:
> On Wed, Aug 21, 2013 at 1:41 PM, Comment Holder <commentholder at gmail.com> wrote:

>> Many thanks for your help - I think I shall start with this way and see how it goes. My concerns were if the task can be accomplished with Python, and from your posts, I guess it can - so I shall give it a try :).

CM: You still seem a bit doubtful. If you are wondering why no one else 
has answered, it is because Joel has given you a really good answer that 
cannot be beat without writing your code for you.

> You're welcome.  One thought popped into my mind.  Since the site
> seems to be from the Wall Street Journal, you may want to look into
> whether they have an api for searching and retrieving articles.  If
> they do, this would be simpler and probably safer than parsing web
> pages.  From time to time, websites change their layout, which would
> probably break your program.  However APIs are more stable

Including this suggestion, which I did not think of.

-- 
Terry Jan Reedy




More information about the Python-list mailing list