JSON Object to CSV File Troubleshooting

[Sahlusar]

On Sunday, June 21, 2015 at 5:56:00 PM UTC-4, Waffle wrote:
> On 21 June 2015 at 17:38, Sahlusar <ahlusar.ahluwalia at gmail.com> wrote:
> >
> > [snip]
> > I do agree with you Denis that this is an unconventional approach. I was wondering then that perhaps I should add additional functionality at the XML to JSON step? So far, with JSON objects without nested lists (as values) I have been successful with this (the following is rather lengthy):
> > [snip]
> 
> 
> > ##JSON sample:
> >
> > data2 = {
> >     "OTF": "0",
> >     "F": "False",
> >     "F": {
> >         "Int32": ["0",
> >         "0",
> >         "0",
> >         "0"]
> >     },
> > [snip]
> >                 "PBDS": {
> >                     "DateTime": ["1/1/0001 12:00:00 AM",
> >                     "1/1/0001 12:00:00 AM",
> >                     "1/1/0001 12:00:00 AM",
> >                     "1/1/0001 12:00:00 AM"]
> >                 },
> >                 "PBDS": {
> >                     "Double": ["0",
> >                     "0",
> >                     "0"]
> >                 },
> >                 "SCS": {
> >                     "String": ["1",
> >                     "2"]
> >                 }
> >             }
> >
> > The result:
> 
> and compare those closely now....
> 
> >
> > {'D_B': ['0', '0', '0', '0', '0', '0', '0', '0', '0', '0', '0'],
> >  'F_Int32': ['0',
> >   '0',
> >   '0',
> >   '0'],
> >  'OTF': '0',
> >  'PBDS_Double': ['0', '0', '0', '0', '0', '0', '0', '0'],
> >  'SCS_String': ['1', '2']}
> >
> Notice in the original text you have 2 entries under the name F and
> later 2 entiries under the name PBDS. in the result you are missing
> the first entry of each.
> you say you have succeeded in generating json, unless you meant to
> throw away huge swafts of data i would say... nope..
> 
> 
> 
> [snip]
> >
> > I know that this is alot of sequential steps. I am wondering if I could insert or conditionally pass these functions when originally parsing the XML, so that the JSON is formatted for more recursive reading of the JSON dictionary and then writing to CSV? I welcome constructive feedback for refactoring....
> 
> theres things you could do to fix up the generated json .. tho really,
> 
> stop generating json when you need to generate csv.
> you are winning nothing. you are losing.. well pretty much .. a little
> of everything .. by doing this
> 
> there are fundemental properties of xml and json you fail to grasp,
> you are touting code claiming that it works when the output it
> produces is horribly deformed :(
> 
> In xml for instance this is valid:
> 
> <a>
>  <b>1</b>
> </a>
> .. and so is this:
> <a>
>  <b>1</b>
>  <b>2</b>
> </a>
> 
> a naive translatio n of the first might yield
> {"a":
>  {"b":1}
> }
> but this will not work with the second example, it would emit
> {"a":
>  {"b":1,"b":2}
> }
> which really means
> {"a":
>  {"b":2}
> }
> 
> if you insist on emitting json as an intermediate step you need to
> take care of these inconsistencies somehow.
> you need to decide which behaviour you want and be explicit about it.
> is it desireable that the last entry overrites the previous one? (you
> have this now, i doubt this is what you want)
> would you like some mergine behaviour? (some config file might work
> well with this, or not)
> would you like to have every entry be a list? (this is simple, but you
> will end up with a lot of junk like {a:[{b:[1]}]}
> do you wrap some things in list but not others?
> 
> and the conversion from json to CSV has similar issues ofc.


I am rethinking my strategy and going back to the drawing board. The data is malformed. I am quite familiar with JSON and XML. I am trying to work within the parameters given to me. I will rethink this. Thank you for giving me a reality check with a splash of water. Sometimes it is hard to get one's head out of the ground....especially when one is new to enterprise and commercial development.