[Python-ideas] Proposal: Query language extension to Python (PythonQL)

[Brice PARENT]

Hello!

If I had to provide a unified way of dealing with data, whatever its 
source is, I would probably go with creating an standardized ORM, 
probably based on Django's or PonyORM, because:

- it doesn't require any change in the Python language

- it allows queries for both reading and writing. I didn't see a way to 
write (UPDATE, CREATE and DELETE equivalents), but maybe I didn't look 
right. Or maybe I didn't understand the purpose at all!

- it makes it easy (although not fast) to extend to any backend (files, 
databases, or any other kind of data storage)

- as a pure python object, any IDE already supports it.

- It can live as a separate module until it is stable enough to be 
integrated into standard library (if it should ever be integrated there).

- it is way easier to learn and use. Comprehensions are not the easier 
things to work with. Most of the places I worked in forbid their use 
except for really easy and short cases (single line, single loop, simple 
tests). You're adding a whole new syntax with new keywords to one of the 
most complicated and less readable (yet efficient in many cases, don't 
get me wrong) part of the language.

I'm not saying there is no need for what you're developing, there 
probably is if you did it, but maybe the solution you chose isn't the 
easier way to have it merged to the core language, and if it was, it 
would really be a long shot, as there are many new keywords and new 
syntax to discuss, implement and test.

But I like the idea of a standard API to deal with data, a nice battery 
to be included.

Side note : I might not have understood what you were doing, so if I'm 
off-topic, tell me !

-Brice


Le 24/03/17 à 16:10, Pavel Velikhov a écrit :
> Hi folks!
>
>   We started a project to extend Python with a full-blown query 
> language about a year ago. The project is call PythonQL, the links are 
> given below in the references section. We have implemented what is 
> kind of an alpha version now, and gained some experience and insights 
> about why and where this is really useful. So I’d like to share those 
> with you and gather some opinions whether you think we should try to 
> include these extensions in the Python core.
>
> *Intro*
>
>   What we have done is (mostly) extended Python’s comprehensions with 
> group by, order by, let and window clauses, which can come in any 
> order, thus comprehensions become a query language a bit cleaner and 
> more powerful than SQL. And we added a couple small convenience 
> extensions, like a  We have identified three top motivations for folks 
> to use these extensions:
>
> *Our Motivations*
>
> 1. This can become a standard for running queries against database 
> systems. Instead of learning a large number of different SQL dialects 
> (the pain point here are libraries of functions and operators that are 
> different for each vendor), the Python developer needs only to learn 
> PythonQL and he can query any SQL and NoSQL database.
>
> 2. A single PythonQL expression can integrate a number of 
> databases/files/memory structures seamlessly, with the PythonQL 
> optimizer figuring out which pieces of plans to ship to which 
> databases. This is a cool virtual database integration story that can 
> be very convenient, especially now, when a lot of data scientists use 
> Python to wrangle the data all day long.
>
> 3. Querying data structures inside Python with the full power of SQL 
> (and a bit more) is also really convenient on its own. Usually folks 
> that are well-versed in SQL have to resort to completely different 
> means when they need to run a query in Python on top of some data 
> structures.
>
> *Current Status*
>
> We have PythonQL running, its installed via pip and an encoding hack, 
> that runs our preprocessor. We currently compile PythonQL into Python 
> using our executor functions and execute Python subexpressions via 
> eval. We don’t do any optimization / rewriting of queries into 
> languages of underlying systems. And the query processor is basic too, 
> with naive implementations of operators. But we’ve build DBMS systems 
> before, so if there is a good amount of support for this project, 
> we’ll be able to build a real system here.
>
> *Your take on this*
>
> Extending Python’s grammar is surely a painful thing for the 
> community. We’re now convinced that it is well worth it, because of 
> all the wonderful functionality and convenience this extension offers. 
> We’d like to get your feedback on this and maybe you’ll suggest some 
> next steps for us.
>
> *References*
>
> PythonQL GitHub page: https://github.com/pythonql/pythonql
> PythonQL Intro and Tutorial (this is all User Documentation we have 
> right now): 
> https://github.com/pythonql/pythonql/wiki/PythonQL-Intro-and-Tutorial
> A use-case of querying Event Logs and doing Process Mining with 
> PythonQL: 
> https://github.com/pythonql/pythonql/wiki/Event-Log-Querying-and-Process-Mining-with-PythonQL
> PythonQL demo site: www.pythonql.org <http://www.pythonql.org>
>
> Best regards,
> PythonQL Team
>
>
>
>
>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170325/0020087e/attachment.html>