trying to retrieve comments with activated API key

[Chris Angelico]

On Sat, Mar 9, 2019 at 6:14 AM Drake Gossi <drake.gossi at gmail.com> wrote:
>
> Hi everyone,
>
> I'm further along than I was last time. I've installed python and am
> running this in spyder. This is the code I'm working with:
>
> import requests
> import csv
> import time
> import sys
> api_key = 'my api key'
> docket_id = 'ED-2018-OCR-0064'
> total_docs = 32068
> docs_per_page = 1000

This is all just setup. You're importing some useful modules and
setting up some variables.

> runfile('/Users/susan/.spyder-py3/temp.py', wdir='/Users/susan/.spyder-py3')

Not sure what's going on here; is that something you ran? Did you
create the file in a hidden directory?

> But I feel like I'm missing something super important. Like, for instance,
> how is python being told to go to the right website? Again, I'm trying to
> retrieve these comments
> <https://www.regulations.gov/docketBrowser?rpp=25&so=DESC&sb=commentDueDate&po=0&dct=PS&D=ED-2018-OCR-0064>
> off of regulations.gov. I don't know if this helps, but the interactive API
> interface is here <https://regulationsgov.github.io/developers/console/>.

> Help! At the end of the day, I'm trying to use python to get the comments
> from regulations.gov into a csv file so that I can analyze them in R.

I'm not sure about comments, but based on the docs you linked to, I
would try to retrieve some documents. The first block in that page
says:

GET /documents.{response_format}

The response_format will be JSON (you mentioned wanting to use JSON),
so that means you're trying to get /documents.json. Try out the
interactive parameter setting, and then use the button at the bottom
to get a URL. As a bare minimum, it'll be something like this:

https://api.data.gov:443/regulations/v3/documents.json?api_key=DEMO_KEY

(No, I've no idea why it says ":443" in there.)

To do that from Python, make a call like this:

r = requests.get("https://api.data.gov:443/regulations/v3/documents.json",
params={
    "api_key": api_key,
    "dktid": docket_id,
})

Add whichever parameters you need in there; it's a plain Python
dictionary, so you can create it any way you like.

That should give you back a *response* object. The easiest thing to do
from there is to turn HTTP errors into Python exceptions, and then get
the results as JSON:

r.raise_for_status()
data = r.json()

At this point, 'data' should be a dict or list with the content that
you want. It's up to you to delve into that.

Have fun! Hopefully that can get you started.

ChrisA