increasing the page size of a dbm store?

[Barry]

Maybe port to SQLite? I would not choose dbm these days.

Barry

> On 27 Nov 2019, at 01:48, Tim Chase <python.list at tim.thechases.com> wrote:
> 
> Working with the dbm module (using it as a cache), I've gotten the
> following error at least twice now:
> 
>  HASH: Out of overflow pages.  Increase page size
>  Traceback (most recent call last):
>  [snip]
>  File ".py", line 83, in get_data
>    db[key] = data
>  _dbm.error: cannot add item to database
> 
> I've read over the py3 docs on dbm
> 
> https://docs.python.org/3/library/dbm.html
> 
> but don't see anything about either "page" or "size" contained
> therein.
> 
> There's nothing particularly complex as far as I can tell.  Nothing
> more than a straightforward
> 
>  import dbm
>  with dbm.open("cache", "c") as db:
>    for thing in source:
>      key = extract_key_as_bytes(thing)
>      if key in db:
>        data = db[key]
>      else:
>        data = long_process(thing)
>        db[key] = data
> 
> The keys can get a bit large (think roughly book-title length), but
> not huge. I have 11k records so it seems like it shouldn't be
> overwhelming, but this is the second batch where I've had to nuke the
> cache and start afresh.  Fortunately I've tooled the code so it can
> work incrementally and no more than a hundred or so requests have to
> be re-performed.
> 
> How does one increas the page-size in a dbm mapping?  Or are there
> limits that I should be aware of?
> 
> Thanks,
> 
> -tkc
> 
> PS: FWIW, this is Python 3.6 on FreeBSD in case that exposes any
> germane implementation details.
> 
> 
> 
> 
> 
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>