Reading *.json from URL - json.loads() versus urllib.urlopen.readlines()

[Fábio Santos]

On 28 May 2013 04:19, "Bryan Britten" <britten.bryan at gmail.com> wrote:
> I'm not familiar with using read(4096), I'll have to look into that. When
I tried to just save the file, my computer just sat in limbo for some time
and didn't seem to want to process the command.

That's just file.read with an integer argument. You can read a file by
chunks by repeatedly calling that function until you get the empty string.

> Based on my *extremely* limited knowledge of JSON, that's definitely the
type of file this is. Here is a snippet of what is seen when you log in:
...
That's json. It's pretty big, but not big enough to stall a slow computer
more than half a second.

-

I've looked for documentation on that method on twitter.

It seems that it's part of the twitter streaming api.

https://dev.twitter.com/docs/streaming-apis

What this means is that the requests aren't supposed to end. They are
supposed to be read gradually, using the lines to split the response into
meaningful chunks. That's why you can't read the data and why your browser
never gets around to download it. Both urlopen and your browser block while
waiting for the request to end.

Here's more info on streaming requests on their docs:

https://dev.twitter.com/docs/streaming-apis/processing

For streaming requests in python, I would point you to the requests
library, but I am not sure it handles streaming requests.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20130528/f9fb2872/attachment.html>