How to expand and flatten a nested of list of dictionaries of varied lengths?

[dn]

On 19/10/2020 09:09, Shaozhong SHI wrote:
> Even worse is that, in some cases, an addition called serviceRatings as a
> key occur with new data unexpectedly.

"Even worse" than what?

Do you need to keep a list of acceptable/applicable/available keys?
(and reject or deal with others in some alternate fashion)


> How to produce a robust Python/Panda script to coping with all these?

> u'historicRatings': [{u'overall': {u'keyQuestionRatings': [{u'name':
> u'Safe', u'rating': u'Requires improvement'}, {u'name': u'Well-led',
> u'rating': u'Requires improvement'}], u'rating': u'Requires improvement'},
> u'reportDate': u'2019-10-04', u'reportLinkId':
> u'63ff05ec-4d31-406e-83de-49a271cfdc43'}, {u'overall':
> {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Good'}, {u'name':
> u'Well-led', u'rating': u'Good'}, {u'name': u'Caring', u'rating': u'Good'},
> {u'name': u'Responsive', u'rating': u'Good'}, {u'name': u'Effective',
> u'rating': u'Requires improvement'}], u'rating': u'Good'}, u'reportDate':
> u'2017-09-08', u'reportLinkId': u'4f20da40-89a4-4c45-a7f9-bfd52b48f286'},
> {u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating':
> u'Requires improvement'}, {u'name': u'Well-led', u'rating': u'Requires
> improvement'}, {u'name': u'Caring', u'rating': u'Requires improvement'},
> {u'name': u'Responsive', u'rating': u'Requires improvement'}, {u'name':
> u'Effective', u'rating': u'Good'}], u'rating': u'Requires improvement'},
> u'reportDate': u'2016-06-11', u'reportLinkId':
> u'0cc4226b-401e-4f0f-ba35-062cbadffa8f'}, {u'overall':
> {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Good'}, {u'name':
> u'Well-led', u'rating': u'Good'}, {u'name': u'Caring', u'rating': u'Good'},
> {u'name': u'Responsive', u'rating': u'Requires improvement'}, {u'name':
> u'Effective', u'rating': u'Good'}], u'rating': u'Good'}, u'reportDate':
> u'2015-01-12', u'reportLinkId': u'a11c1e52-ddfd-4cd8-8b56-1b96ac287c96'}]

You may find it helpful to use the pprint ("pretty printing" library to 
print data-structures in a more readable/structured format).

To "flatten" a dictionary, you must first be sure that there will be no 
keys that will clash (else the second entry will completely replace the 
first, without trace).

Thus, we will need to understand more about this particular definition 
of "flatten" in relation to the range of incoming data. Perhaps explain 
them in English first...
-- 
Regards =dn