JSON Object to CSV File Troubleshooting

[Joonas Liik]

On 21 June 2015 at 17:38, Sahlusar <ahlusar.ahluwalia at gmail.com> wrote:
>
> [snip]
> I do agree with you Denis that this is an unconventional approach. I was wondering then that perhaps I should add additional functionality at the XML to JSON step? So far, with JSON objects without nested lists (as values) I have been successful with this (the following is rather lengthy):
> [snip]


> ##JSON sample:
>
> data2 = {
>     "OTF": "0",
>     "F": "False",
>     "F": {
>         "Int32": ["0",
>         "0",
>         "0",
>         "0"]
>     },
> [snip]
>                 "PBDS": {
>                     "DateTime": ["1/1/0001 12:00:00 AM",
>                     "1/1/0001 12:00:00 AM",
>                     "1/1/0001 12:00:00 AM",
>                     "1/1/0001 12:00:00 AM"]
>                 },
>                 "PBDS": {
>                     "Double": ["0",
>                     "0",
>                     "0"]
>                 },
>                 "SCS": {
>                     "String": ["1",
>                     "2"]
>                 }
>             }
>
> The result:

and compare those closely now....

>
> {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
>  'F_Int32': ['0',
>   '0',
>   '0',
>   '0'],
>  'OTF': '0',
>  'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
>  'SCS_String': ['1', '2']}
>
Notice in the original text you have 2 entries under the name F and
later 2 entiries under the name PBDS. in the result you are missing
the first entry of each.
you say you have succeeded in generating json, unless you meant to
throw away huge swafts of data i would say... nope..



[snip]
>
> I know that this is alot of sequential steps. I am wondering if I could insert or conditionally pass these functions when originally parsing the XML, so that the JSON is formatted for more recursive reading of the JSON dictionary and then writing to CSV? I welcome constructive feedback for refactoring....

theres things you could do to fix up the generated json .. tho really,

stop generating json when you need to generate csv.
you are winning nothing. you are losing.. well pretty much .. a little
of everything .. by doing this

there are fundemental properties of xml and json you fail to grasp,
you are touting code claiming that it works when the output it
produces is horribly deformed :(

In xml for instance this is valid:

<a>
 <b>1</b>
</a>
.. and so is this:
<a>
 <b>1</b>
 <b>2</b>
</a>

a naive translatio n of the first might yield
{"a":
 {"b":1}
}
but this will not work with the second example, it would emit
{"a":
 {"b":1,"b":2}
}
which really means
{"a":
 {"b":2}
}

if you insist on emitting json as an intermediate step you need to
take care of these inconsistencies somehow.
you need to decide which behaviour you want and be explicit about it.
is it desireable that the last entry overrites the previous one? (you
have this now, i doubt this is what you want)
would you like some mergine behaviour? (some config file might work
well with this, or not)
would you like to have every entry be a list? (this is simple, but you
will end up with a lot of junk like {a:[{b:[1]}]}
do you wrap some things in list but not others?

and the conversion from json to CSV has similar issues ofc.