[BangPypers] JSON PARSER

[Prashant Gaur]

Hello Lokesh ,

as we know json.loads return data in form of dict and dict can never have
same keys .

so we can do one thing and that is to return a list of all values which are
having same names.

import simplejson as json
from collections import defaultdict

def duplicate_key_lookup(ordered_pairs):
    """Convert duplicate keys values to lists."""
    # read all values into lists
     d = defaultdict(list)
     for k, v in ordered_pairs:
         d[k].append(v)
         # unpack lists that have only 1 item
     for k, v in d.items():
         if len(v) == 1:
             d[k] = v[0]
      return dict(d)

>>> json.loads('''{"test":"hi","test":"bye", "test1": "tata"}''',
object_pairs_hook=multidict)
{'test': ['hi', 'bye'], 'test1': 'tata'}


On Fri, Mar 21, 2014 at 3:22 PM, lokesh bobby <lokeshbobbys at yahoo.com>wrote:

> Hi Jayanth/Prashant,
>
> Either "the duplicates to be loaded" or "An ouput of all the duplicate key
> names in a JSON file" should be fine for me :-)
>
> NOTE: JSON file consits of more than 2 lakhs LOC
>
> Lokesh
>
>
>   On Friday, 21 March 2014 3:17 PM, Jayanth Koushik <jnkoushik at gmail.com>
> wrote:
>  Hi Prashant
>
> I think he wants the duplicates to be loaded, not cause an exception.
>
> Jayanth
>
>
> On Fri, Mar 21, 2014 at 3:15 PM, Prashant Gaur <91prashantgaur at gmail.com>wrote:
>
> Hi Lokesh,
>
> we can pass lookup while parsing your json file which will make sure that
> name is repetitive or not .
>
> import json
>
> def duplicate_checking_hook(pairs):
>     ''' lookup for duplicate names'''
>     result = dict()
>     for key, val in pairs:
>         if key in result:
>             raise KeyError('Duplicate Key specified: %s % key)
>         result[key] = val
>     return result
>
> >>> json.loads('''{"test":"hi","test":"bye"}''',
> object_pairs_hook=duplicate_checking_hook)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
>     return cls(encoding=encoding, **kw).decode(s)
>   File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
>     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>   File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
>     obj, end = self.scan_once(s, idx)
>   File "<stdin>", line 5, in duplicate_checking_hook
> KeyError: u'Duplicate key specified: a'
>
> While
>
> >>> json.loads('''{"test":"hi","test1":"bye"}''',
> object_pairs_hook=duplicate_checking_hook)
> {u'test': u'hi', u'test1': u'bye'}
> >>>
>
>
>
> def dupe_checking_hook(pairs):
>     result = dict()
>     for key,val in pairs:
>         if key in result:
>             raise KeyError("Duplicate key specified: %s" % key)
>         result[key] = val
>     return result
>
>
>
>
>
> On Fri, Mar 21, 2014 at 2:59 PM, lokesh bobby <lokeshbobbys at yahoo.com>wrote:
>
> Hi Jayanth,
>
> Ideally speaking a JSON shouldn't be created with repetitive key names.
> But manually it is possible that a proper JSON file can be appended with a
> duplicate key. We need to catch that duplicate key. If we are going to use
> json.load(), the repetitive keys of the JSON file wont get loaded. Only the
> last occurrence of the key will be there in that loaded data.
>
> Instead I want that entire data in JSON file should be loaded. Hoping that
> you got the point what I want :-)
>
> Thanks,
> Lokesh.
>
>
>
>
>
> On Friday, 21 March 2014 2:45 PM, Jayanth Koushik <jnkoushik at gmail.com>
> wrote:
>
> Hi Lokesh
>
>
> The 'problem' that you talk about isn't really a problem. Since the JSON
> specification does not say what is to be done for repeated names, it is up
> to the implementation to decide. What is your requirement for handling
> repeated names?
>
>
> Jayanth
>
>
>
>
> On Fri, Mar 21, 2014 at 2:30 PM, lokesh bobby <lokeshbobbys at yahoo.com>
> wrote:
>
> Hi Noufal,
> >
> >Thanks for your reply. I am not looking for loading the JSON file. There
> is a limitation in it. Go thru the links
> >
> >
> http://docs.python.org/2/library/json.html#repeated-names-within-an-object
> >
> http://docs.python.org/3.2/library/json.html#repeated-names-within-an-object
> >
> >In order to get rid of that problem, I am looking for some JSON stream
> parsers.
> >
> >Thanks,
> >Lokesh.
> >
> >
> >
> >
> >On Friday, 21 March 2014 2:09 PM, Noufal Ibrahim KV <
> noufal at nibrahim.net.in> wrote:
> >
> >On Fri, Mar 21 2014, lokesh bobby wrote:
> >
> >
> >> Hi ALL,
> >>
> >> Can you share your thoughts on how to parse a JSON file by using
> >> python?
> >
> >import json
> >
> >with open("data.json") as f:
> >     json.load(f)
> >
> >
> >[...]
> >
> >
> >--
> >Cordially,
> >Noufal
> >http://nibrahim.net.in
> >_______________________________________________
> >BangPypers mailing list
> >BangPypers at python.org
> >https://mail.python.org/mailman/listinfo/bangpypers
> >
> _______________________________________________
> BangPypers mailing list
> BangPypers at python.org
> https://mail.python.org/mailman/listinfo/bangpypers
>
>
>
>
> --
> Prashant Gaur
>
> Mobile : +91 9717353657
> http://gaurprashant.blogspot.in/
> http://stackoverflow.com/users/1850358/prashant-gaur
> http://www.about.me/prashantgaur/
>
>
>
>
>


-- 
Prashant Gaur

Mobile : +91 9717353657
http://gaurprashant.blogspot.in/
http://stackoverflow.com/users/1850358/prashant-gaur
http://www.about.me/prashantgaur/