From anthony.tuininga at gmail.com Mon Jul 2 17:59:14 2018 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Mon, 2 Jul 2018 15:59:14 -0600 Subject: [DB-SIG] cx_Oracle 6.4 Message-ID: What is cx_Oracle? cx_Oracle is a Python extension module that enables access to Oracle Database for Python 3.x and 2.x and conforms to the Python database API 2.0 specifications with a number of enhancements. Where do I get it? https://oracle.github.io/python-cx_Oracle The easiest method to install/upgrade cx_Oracle is via pip as in python -m pip install cx_Oracle --upgrade What's new? This release adds support for grouping notifications to subscriptions and adds support for getting notifications when AQ messages are available to be dequeued. Support was also added for timed waits when acquiring sessions from a session pool and for setting the values of the timeout and maximum lifetime attributes of a pool when it is created. Cursors can now be used as context managers and a number of other enhancements and bug fixes were made. See the release notes for more information. http://cx-oracle.readthedocs.io/en/latest/releasenotes.html#version-6-4-july-2018 Please provide any feedback via GitHub issues ( https://github.com/oracle/python-cx_Oracle/issues). -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthony.tuininga at gmail.com Mon Jul 9 15:56:43 2018 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Mon, 9 Jul 2018 13:56:43 -0600 Subject: [DB-SIG] cx_Oracle 6.4.1 Message-ID: What is cx_Oracle? cx_Oracle is a Python extension module that enables access to Oracle Database for Python 3.x and 2.x and conforms to the Python database API 2.0 specifications with a number of enhancements. Where do I get it? https://oracle.github.io/python-cx_Oracle The easiest method to install/upgrade cx_Oracle is via pip as in python -m pip install cx_Oracle --upgrade What's new? This release addresses a couple of bugs. See the release notes for more information. https://cx-oracle.readthedocs.io/en/latest/releasenotes.html#version-6-4-1-july-2018 Please provide any feedback via GitHub issues ( https://github.com/oracle/python-cx_Oracle/issues). -------------- next part -------------- An HTML attachment was scrubbed... URL: From fog at dndg.it Tue Jul 24 03:37:27 2018 From: fog at dndg.it (Federico Di Gregorio) Date: Tue, 24 Jul 2018 09:37:27 +0200 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: Message-ID: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> On 06/21/2018 01:31 AM, Gerald Venzl wrote: > Hi all, > > I was wondering whether there has been any particular reason that the > specification of the Cursor object doesn?t implement a context manager > to enable for example the ?with? statement? > Being able to use the ?with? statement for database cursors, just like > other external resources like files, etc., I think makes a lot of sense: > > with conn.cursor()as c: > c.execute("SELECT 'test' from dual") > result = c.fetchall() > print(result) It does (make sense). The explanation is that context managers did not exist when the PEP was written. An update is long overdue, IMHO. federico From dlenski at gmail.com Wed Jul 25 23:30:31 2018 From: dlenski at gmail.com (Daniel Lenski) Date: Wed, 25 Jul 2018 20:30:31 -0700 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Tue, Jul 24, 2018 at 12:37 AM, Federico Di Gregorio wrote: > On 06/21/2018 01:31 AM, Gerald Venzl wrote: >> >> Hi all, >> >> I was wondering whether there has been any particular reason that the >> specification of the Cursor object doesn?t implement a context manager to >> enable for example the ?with? statement? >> Being able to use the ?with? statement for database cursors, just like >> other external resources like files, etc., I think makes a lot of sense: >> >> with conn.cursor()as c: >> c.execute("SELECT 'test' from dual") >> result = c.fetchall() >> print(result) > > > It does (make sense). The explanation is that context managers did not exist > when the PEP was written. An update is long overdue, IMHO. Indeed. contextlib.closing (https://docs.python.org/2/library/contextlib.html#contextlib.closing) can be used to emulate the same behavior, but it really ought to be part of the DBAPI specification. Python Database API Specification v3.0, anyone?? :-P Dan From daniele.varrazzo at gmail.com Thu Jul 26 07:56:55 2018 From: daniele.varrazzo at gmail.com (Daniele Varrazzo) Date: Thu, 26 Jul 2018 12:56:55 +0100 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Thu, Jul 26, 2018 at 4:30 AM, Daniel Lenski wrote: > On Tue, Jul 24, 2018 at 12:37 AM, Federico Di Gregorio wrote: >> On 06/21/2018 01:31 AM, Gerald Venzl wrote: >>> >>> Hi all, >>> >>> I was wondering whether there has been any particular reason that the >>> specification of the Cursor object doesn?t implement a context manager to >>> enable for example the ?with? statement? >>> Being able to use the ?with? statement for database cursors, just like >>> other external resources like files, etc., I think makes a lot of sense: >>> >>> with conn.cursor()as c: >>> c.execute("SELECT 'test' from dual") >>> result = c.fetchall() >>> print(result) >> >> >> It does (make sense). The explanation is that context managers did not exist >> when the PEP was written. An update is long overdue, IMHO. In 2012 the thing was discussed (https://mail.python.org/pipermail/db-sig/2012-November/thread.html). The most used semantics appears what documented here: https://github.com/psycopg/psycopg2/commit/c2f284cd3b7d3b33f4de348aaac62590feda8c78#L2R554 but it hasn't made it into the specs so far. Note: while the assumption is that cursors are closed at the end of the block, this is not true for connection, for which the most useful thing is to commit or roll back the transaction. Or at least the most useful if the connection is not autocommit). > Python Database API Specification v3.0, anyone?? :-P In case there would be work in its direction, points to touch, on top of my mind (implemented in various way by different drivers) are also likely: - unicode support - Python 3 support (likely it will be the only targeted version) - async support - multiprocess support - green threads support - autocommit (was not-autocommit by default a good idea, even?) - should execute(query) with no param go for a round of placeholders escape on the string as it would execute(query, {})? - placeholders style, of course... -- Daniele From mike_mp at zzzcomputing.com Thu Jul 26 09:18:29 2018 From: mike_mp at zzzcomputing.com (Mike Bayer) Date: Thu, 26 Jul 2018 09:18:29 -0400 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Thu, Jul 26, 2018 at 7:56 AM, Daniele Varrazzo wrote: > >> Python Database API Specification v3.0, anyone?? :-P > > In case there would be work in its direction, points to touch, on top > of my mind (implemented in various way by different drivers) are also > likely: > > - unicode support yes > - Python 3 support (likely it will be the only targeted version) yes > - async support begrudgingly yes > - multiprocess support hmm, like sharing connections over processes? > - green threads support YES, now that we are to have a new BDFL perhaps green threads are welcome again? > - autocommit (was not-autocommit by default a good idea, even?) Make real .autocommit a thing but please don't change the defaults.... > - should execute(query) with no param go for a round of placeholders > escape on the string as it would execute(query, {})? I think I know what issue you're talking about. This seems much more subtle of an issue than we would normally see in a pep-249 which is overall very terse about each point. > - placeholders style, of course... one positional and one named and that's it ....Annnnndddd: - prepared statements I don't recall if I was for or against this being in the spec. Seems like something that should be clarified though. > > > -- Daniele > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > https://mail.python.org/mailman/listinfo/db-sig From daniele.varrazzo at gmail.com Thu Jul 26 09:45:18 2018 From: daniele.varrazzo at gmail.com (Daniele Varrazzo) Date: Thu, 26 Jul 2018 14:45:18 +0100 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Thu, Jul 26, 2018 at 2:18 PM, Mike Bayer wrote: Oh hi! If there is someone who has experience in how much effort it takes to beat every different driver into submission that's you :) >> - should execute(query) with no param go for a round of placeholders >> escape on the string as it would execute(query, {})? > I think I know what issue you're talking about. This seems much > more subtle of an issue than we would normally see in a pep-249 which > is overall very terse about each point. Still a baffling thing... - HAL, retrieve the even records please. - cursor.execute("select * from students where id % 2 = 0") - Now please retrieve only the ones born after the date D. - cursor.execute("select * from students where id % 2 = 0 and dob >= %s", [D]) (some mysterious traceback) - Open the pod bay doors, HAL. I'd rather force people to specify % as %% regardless of the presence of parameters (in the case of other driver the problem would be with other characters - question marks, colons... - but it's probably the same). >> - placeholders style, of course... > > one positional and one named and that's it ....Annnnndddd: Maybe it's easy enough to use the most generic one ("%s" and "%(name)s"? "{}" and "{name}"?) and provide a conversion library to help implementing drivers using whatever different format they have to use? > - prepared statements > > I don't recall if I was for or against this being in the spec. > Seems like something that should be clarified though. Yes! -- Daniele From mike_mp at zzzcomputing.com Thu Jul 26 09:57:30 2018 From: mike_mp at zzzcomputing.com (Mike Bayer) Date: Thu, 26 Jul 2018 09:57:30 -0400 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Thu, Jul 26, 2018 at 9:45 AM, Daniele Varrazzo wrote: > On Thu, Jul 26, 2018 at 2:18 PM, Mike Bayer wrote: > > Oh hi! If there is someone who has experience in how much effort it > takes to beat every different driver into submission that's you :) > > >>> - should execute(query) with no param go for a round of placeholders >>> escape on the string as it would execute(query, {})? > >> I think I know what issue you're talking about. This seems much >> more subtle of an issue than we would normally see in a pep-249 which >> is overall very terse about each point. > > Still a baffling thing... > > - HAL, retrieve the even records please. > > - cursor.execute("select * from students where id % 2 = 0") > > - Now please retrieve only the ones born after the date D. > > - cursor.execute("select * from students where id % 2 = 0 and dob >= %s", [D]) > > (some mysterious traceback) > > - Open the pod bay doors, HAL. > > I'd rather force people to specify % as %% regardless of the presence > of parameters (in the case of other driver the problem would be with > other characters - question marks, colons... - but it's probably the > same). that's what SQLAlchemy does. if you are using raw driver-level SQL , you have to double the percent signs. but the users still complain. It is less intuitive when you are switching between SQL and DDL. > > >>> - placeholders style, of course... >> >> one positional and one named and that's it ....Annnnndddd: > > Maybe it's easy enough to use the most generic one ("%s" and > "%(name)s"? "{}" and "{name}"?) and provide a conversion library to > help implementing drivers using whatever different format they have to > use? there are so many ways to make this super easy yet I still get complaints about people wanting to support some non-standard pet parameter style their driver uses (asyncpg uses $ or something like that). My votes remain at support ? and :name (solves your percent sign issue). > > >> - prepared statements >> >> I don't recall if I was for or against this being in the spec. >> Seems like something that should be clarified though. > > Yes! > > > -- Daniele From dlenski at gmail.com Thu Jul 26 10:16:56 2018 From: dlenski at gmail.com (Daniel Lenski) Date: Thu, 26 Jul 2018 07:16:56 -0700 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Thu, Jul 26, 2018 at 6:45 AM, Daniele Varrazzo wrote: > On Thu, Jul 26, 2018 at 2:18 PM, Mike Bayer wrote: > > Oh hi! If there is someone who has experience in how much effort it > takes to beat every different driver into submission that's you :) > > >>> - should execute(query) with no param go for a round of placeholders >>> escape on the string as it would execute(query, {})? > >> I think I know what issue you're talking about. This seems much >> more subtle of an issue than we would normally see in a pep-249 which >> is overall very terse about each point. > > Still a baffling thing... > > - HAL, retrieve the even records please. > > - cursor.execute("select * from students where id % 2 = 0") > > - Now please retrieve only the ones born after the date D. > > - cursor.execute("select * from students where id % 2 = 0 and dob >= %s", [D]) > > (some mysterious traceback) > > - Open the pod bay doors, HAL. > > I'd rather force people to specify % as %% regardless of the presence > of parameters (in the case of other driver the problem would be with > other characters - question marks, colons... - but it's probably the > same). > > >>> - placeholders style, of course... >> >> one positional and one named and that's it ....Annnnndddd: > > Maybe it's easy enough to use the most generic one ("%s" and > "%(name)s"? "{}" and "{name}"?) and provide a conversion library to > help implementing drivers using whatever different format they have to > use? > This might be one of the only places where I think Oracle has made a more robust and programmer-friendly design decision than Postgres. The :param syntax of Oracle is good because it can't be confused with valid SQL or PL/SQL syntax in the absence of query parameters. Thanks, Dan From daniele.varrazzo at gmail.com Thu Jul 26 13:03:42 2018 From: daniele.varrazzo at gmail.com (Daniele Varrazzo) Date: Thu, 26 Jul 2018 18:03:42 +0100 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Thu, Jul 26, 2018 at 2:57 PM, Mike Bayer wrote: > On Thu, Jul 26, 2018 at 9:45 AM, Daniele Varrazzo >> Maybe it's easy enough to use the most generic one ("%s" and >> "%(name)s"? "{}" and "{name}"?) and provide a conversion library to >> help implementing drivers using whatever different format they have to >> use? > > there are so many ways to make this super easy yet I still get > complaints about people wanting to support some non-standard pet > parameter style their driver uses (asyncpg uses $ or > something like that). It sounds like the Postgres server-side parameters (https://www.postgresql.org/docs/current/static/libpq-exec.html#LIBPQ-PQPREPARE), which is not listed in the DBAPI 2 paramstyle choices. But that's why I think the approach of allowing the driver to pass-through placeholders the way the underlying protocol/library wants is wrong, from the PoV of designing something supposed to be portable. > My votes remain at support ? and :name > (solves your percent sign issue). But it replaces it with two different issues. ? is a valid Postgres operator for some data type (JSON), : is used in the cast operation, value::type. Also, where does "name" ends? Can I have names with an hyphen? Can I have another token concatenated after :name without space? If there is a mechanism of placeholders to replace with values there must be a mechanism to escape placeholders: I don't think there's a way around it. The approach "let's use this funny character which nobody use" is short-sighted: that character *will* be used eventually, and it's not up the driver's author to forbid it. The assumption is already weak for an interface targeting a single system, but it's entirely broken for something like DBAPI that would try to be agnostic and allow for future drivers to be written. The chars used as placeholder marker can also appear in comments and strings, so either the query author will escape them or the driver will have to tokenize the query to tell apart the ? and the : which are not parameters, which is not trivial if the database syntax is not trivial (there are nested strings, there are nested comments...): IMO not only it's not the driver's task, but it's also a totally unnecessary complication of a very simple matter. The last time there was an approach to go past DBAPI 2 we crashed on the placeholder style and didn't go further. I hope, if the matter is open again, that we don't go down the same path. It's not "I like ? more than $" bikeshedding: if the mechanism doesn't allow for escaping placeholder chars it's just a broken system and a non-starter. We may as well leave the thing to anarchy and allow drivers authors do expose whatever the underlying protocol fancy, leaving the responsibility to use the right format to SQLAlchemy and the Django ORM, which have to address the different SQL syntax anyway. I personally think the responsibility to mandate a single params style sits square in the DBAPI field, because it is there exactly to allow working with different protocols homogeneous. But if the DBAPI decides to avoid this responsibility then at least it shouldn't mandate "they should be one of these fews". The entire presence of the "paramstyle" attribute is broken by the way: it doesn't consider drivers using more than one (qmark and named, or format and pyformat), or to have different formats for different connections (such as a "native" one and a DBAPI one). I don't think the presence of the paramstyle attribute is really useful in any scenario: you will still need to read the documentation of the driver to be able to use it correctly. -- Daniele From mike_mp at zzzcomputing.com Thu Jul 26 13:27:53 2018 From: mike_mp at zzzcomputing.com (Mike Bayer) Date: Thu, 26 Jul 2018 13:27:53 -0400 Subject: [DB-SIG] Context manager support for cursors In-Reply-To: References: <12a5992d-7ee6-484d-a9b3-1010a9ce8833@dndg.it> Message-ID: On Thu, Jul 26, 2018 at 1:03 PM, Daniele Varrazzo wrote: > On Thu, Jul 26, 2018 at 2:57 PM, Mike Bayer wrote: >> On Thu, Jul 26, 2018 at 9:45 AM, Daniele Varrazzo > >>> Maybe it's easy enough to use the most generic one ("%s" and >>> "%(name)s"? "{}" and "{name}"?) and provide a conversion library to >>> help implementing drivers using whatever different format they have to >>> use? >> >> there are so many ways to make this super easy yet I still get >> complaints about people wanting to support some non-standard pet >> parameter style their driver uses (asyncpg uses $ or >> something like that). > > It sounds like the Postgres server-side parameters > (https://www.postgresql.org/docs/current/static/libpq-exec.html#LIBPQ-PQPREPARE), > which is not listed in the DBAPI 2 paramstyle choices. But that's why > I think the approach of allowing the driver to pass-through > placeholders the way the underlying protocol/library wants is wrong, > from the PoV of designing something supposed to be portable. > > >> My votes remain at support ? and :name >> (solves your percent sign issue). > > But it replaces it with two different issues. ? is a valid Postgres > operator for some data type (JSON), : is used in the cast operation, > value::type. Also, where does "name" ends? Can I have names with an > hyphen? Can I have another token concatenated after :name without > space? I'll give you the ? issue but for :name if you want spaces it comes out as :"my name", e.g. quote the identifier like any other. > > If there is a mechanism of placeholders to replace with values there > must be a mechanism to escape placeholders: I don't think there's a > way around it. The approach "let's use this funny character which > nobody use" is short-sighted: that character *will* be used > eventually, and it's not up the driver's author to forbid it. The > assumption is already weak for an interface targeting a single system, > but it's entirely broken for something like DBAPI that would try to be > agnostic and allow for future drivers to be written. The chars used as > placeholder marker can also appear in comments and strings, so either > the query author will escape them or the driver will have to tokenize > the query to tell apart the ? and the : which are not parameters, > which is not trivial if the database syntax is not trivial (there are > nested strings, there are nested comments...): IMO not only it's not > the driver's task, but it's also a totally unnecessary complication of > a very simple matter. > > The last time there was an approach to go past DBAPI 2 we crashed on > the placeholder style and didn't go further. I hope, if the matter is > open again, that we don't go down the same path. It's not "I like ? > more than $" bikeshedding: if the mechanism doesn't allow for escaping > placeholder chars it's just a broken system and a non-starter. We may > as well leave the thing to anarchy and allow drivers authors do expose > whatever the underlying protocol fancy, leaving the responsibility to > use the right format to SQLAlchemy and the Django ORM, which have to > address the different SQL syntax anyway. > > I personally think the responsibility to mandate a single params style > sits square in the DBAPI field, because it is there exactly to allow > working with different protocols homogeneous. But if the DBAPI decides > to avoid this responsibility then at least it shouldn't mandate "they > should be one of these fews". of course. if every DBAPI has to use the same single or two paramstyles and the rules for escaping etc. are very clear and unambiguous, that is all we need. > The entire presence of the "paramstyle" > attribute is broken by the way: it doesn't consider drivers using more > than one (qmark and named, or format and pyformat), or to have > different formats for different connections (such as a "native" one > and a DBAPI one). I don't think the presence of the paramstyle > attribute is really useful in any scenario: you will still need to > read the documentation of the driver to be able to use it correctly. no kidding. but if you have just two paramstyles, one is positional and one named, then you can assume the paramstyle from execute("", {}) vs. execute("", []). > > > -- Daniele