JSON Object to CSV Question

[Saran Ahluwalia]

Hi Steven,

Okay, I understand now how convoluted this sounds. Here is a more focused
issue (I hope) referencing the following sample JSON object within a file:


   1. "PAC": {
   2. "Account": [{
   3. "PC": "0",
   4. "CMC": "0",
   5. "WC": "0",
   6. "DLA": "0",
   7. "CN": null,
   8. "FC": {
   9. "Int32": ["0",
   10. "0",
   11. "0",
   12. "0",
   13. "0"]
   14. },
   15. "F": {
   16. "Description": null,
   17. "Code": "0"
   18. }
   19.

The rationale for the following: the out put will need to be a CSV. All of
the keys should be headers in the CSV. The values are all corresponding
with the headers. However, Int32 should be deleted and the key reassigned
to FC. Account should be deleted and reassigned to PAC.

Essentially, where all of the instances where you have a key with a list of
dictionary, that outer key should be deleted and reassigned to the
outermost key.

Finally (in this example) the "0"s should be all concatenated in the output
CSV so accessing the lists will need to be addressed first. I know that the
following:

def hook(obj):
    return obj
def flatten(obj):
    for k, v in obj:
        if isinstance(v, list):
            yield from flatten(v)
        else:
            yield k, v
if __name__ == "__main__":
    with open("somefileneame.json") as f:
        data = json.load(f, object_pairs_hook=hook)
    pairs = list(flatten(data))
    writer = csv.writer(sys.stdout)
    header = writer.writerow([k for k, v in pairs])
    row = writer.writerow([v for k, v in pairs]) #writer.writerows for
any other iterable object

is not the best strategy. The list objects cannot be accessed (hence, the
"ValueError: too many values to unpack") is caught at the function
json.loads(). I am wondering how to add the exception when 1) when one
encounters the list and 2) how to delete and re-asssign the keys (as
described above).

Thank you for your guidance.

On Fri, Jun 19, 2015 at 10:47 AM, Steven D'Aprano <steve at pearwood.info>
wrote:

> On Fri, 19 Jun 2015 07:50 pm, Saran Ahluwalia wrote:
>
> > If you read the most recent thread that I just posted it states far more
> > information
>
> The problem is, you are posting *too much* of the *wrong* information.
>
> The exception you are getting appears to be a simple one: you are getting
> the exception
>
> ValueError: too many values to unpack
>
>
> but since you don't show the traceback, only the error message, we have no
> idea *where* that exception is happening. It could be a bug in your code,
> in the JSON library, or who knows where.
>
> Python provides lots of useful debugging information in the traceback, but
> you are ignoring it. You should look at the complete traceback, everything
> from the initial line:
>
>     Traceback
>
> to the end of the error message, not just the error message itself. That
> will show you the series of function calls that lead to the error, and
> usually also the actual line of code that caused the problem. That alone
> might be enough to solve the problem. If not, it is one extra clue. Either
> way, when asking for help, you should always post the complete traceback
> (unless it is hundreds of lines long, in which case, ask first).
>
>
> Unfortunately you have dumped over 100 lines of code in our lap, from two
> different files. To solve this problem for you, we would have to copy your
> code, install the libraries you use, somehow get a copy of your XML file:
>
> C:\\Users\\wynsa2\\Desktop\\Python
> Folder\\PCSU\\Trial2_PCSU\\2-Response.xml
>
> (don't ask me how we can possibly do that!), run your code, see what the
> full traceback says, and debug the code for you. That is simply not going
> to happen, not unless you pay someone.
>
> The most critical skill a programmer can have is to learn how to focus on
> what is essential when troubleshooting, and not get lost in a sea of
> irrelevant code. The way to do that is to narrow the problem down to the
> smallest possible code sample which demonstrates the same error, and see
> whether that shows you the solution. If it does, you have your solution. If
> not, you can gradually add more complexity until you have your answer.
>
> You can read more about this here:
>
> http://sscce.org/
>
> Although it is written for Java, the principles apply equally to Python.
>
> Here is the minimum code needed to give your same error:
>
> py> x, y, z = (1, 2, 3, 4)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> ValueError: too many values to unpack
>
>
> Here I am trying to assign four values into three variables. The solution
> is
> obvious:
>
> py> a, x, y, z = (1, 2, 3, 4)
>
> will work. Does that help you see the solution you need? If not, we will
> need a SSCCE to work with.
>
>
> Some other things to consider.
>
> You can avoid all those doubled backslashes by using forward slashes.
> Windows accepts both:
>
> C:/Users/wynsa2/Desktop/Python Folder/PCSU/Trial2_PCSU/2-Response.xml
>
> which makes for much neater and easier file names.
>
> Also, it doesn't help to ask multiple unrelated questions in the same post.
> If they are directly related, that is okay, but you are asking (at least)
> two questions:
>
> - how to fix the ValueError exception;
>
> - something about "I am wondering if there is a function or methodology
> that
> would allow me to remove such nested keys and reassign the new keys to the
> outer key..."
>
> The second question is very vague, and vague questions get vague answers
> (or
> no answers at all.) Yes, there is such a methodology. It is
> called "programming".
>
> Again, simplify what you are trying to do. Show us a *minimal* example of
> the data you start with, and the result you hope to end up with. I'm afraid
> I don't understand your question at all, simplifying it and showing an
> example will help a lot.
>
> You should post these in separate threads. People interested in one thread
> may not be interested in the other. By posting them together, you only get
> responses from people interested in both.
>
>
> --
> Steven
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150619/90544184/attachment.html>