how to handle response data that is streaming and chunked?

Artie Ziff artie.ziff at gmail.com
Thu Mar 21 16:12:46 EDT 2019


Hello,

I am trying to learn how to do use response library in the least confusing
way possible. My data source is streaming.

The sample I share below looks more complicated that it needs to be. I do
not have enough experience with this to know better. Hence why I came here
for guidance and direction. Comments are very welcome as well as pointers
to reading material. I want to learn this.

I expected that I was going to be able to use the JSON methods such as load
or loads. After all, this is JSON and should be able to slurp into builtin
python objects and data structures, I'd imagine.

Does this mailing list use a paste site or is better to paste code in-line
for future?

Thanks for reading!
--Art


import os, sys, json, requests, signal
from requests_oauthlib import OAuth1

api_key = 'HIDDEN'
api_secret = 'HIDDEN'
user_token_key = 'HIDDEN'
user_token_secret = 'HIDDEN'

authorization = OAuth1(api_key, api_secret, user_token_key,
user_token_secret)
session = requests.Session()
session.auth = authorization
finis = 0


def handler(signum, frame):
    global finis
    finis = 1


signal.signal(signal.SIGINT, handler)

url = "https://stream.tradeking.com/v1/market/quotes.json?symbols=AAPL,DJT"
resp = session.get(url, stream=True)
lines = ''
for chunk in resp.iter_content(None, decode_unicode=True):
    if finis:
        break
    lines += chunk.decode('utf-8')
    while lines.find('}}') > 0:
        line, lines = lines.split('}}', 1)
        print(line)



More information about the Python-list mailing list