Fast lookup of bulky "table"

[Weatherby,Gerard]

With Postgresql, one can also do pre-processing in Python. https://www.postgresql.org/docs/15/plpython.html

While it’s not as convenient to develop as client-side Python, it can be used to implement complicated constraints or implement filtering on the server side, which reduces the amount of data that has to be sent back to the client.


From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Stefan Ram <ram at zedat.fu-berlin.de>
Date: Sunday, January 15, 2023 at 5:03 PM
To: python-list at python.org <python-list at python.org>
Subject: Re: Fast lookup of bulky "table"
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

dn <PythonList at DancesWithMice.info> writes:
>Some programmers don't realise that SQL can also be used for
>calculations, eg the eponymous COUNT(), which saves (CPU-time and
>coding-effort) over post-processing in Python.

  Yes, I second that! Sometimes, people only re-invent things
  in Python because they don't know SQL well enough, or they
  do not normalize their tables because they have not properly
  learned how to do this.

  I'd always start out with normalized tables and do as many
  operations in SQL as possible. I would then hesitate to
  de-normalize anything or transfer data operations into
  the programming language unless I am very sure that this
  is really advantageous.

  Once I had the task of writing VBA code to query and analyze
  data from a Jet engine (i.e., Microsoft Access). I ended up
  writing 90 % of the code in SQL and a thin layer of 10 % in VBA.
  And it was fast.


--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kAIZWRJ3oqrlkixX-iwrGeG9VVWjooBvzuMirfp44VTP32cELWf8Dk6MkPQwK2QwWzuUT9eNPNTlN152b23eFcM$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kAIZWRJ3oqrlkixX-iwrGeG9VVWjooBvzuMirfp44VTP32cELWf8Dk6MkPQwK2QwWzuUT9eNPNTlN152b23eFcM$>