From aprotin at research.att.com Fri Jun 1 15:34:06 2007 From: aprotin at research.att.com (Art Protin) Date: Fri, 01 Jun 2007 09:34:06 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <1180646415.3382.155.camel@dot.uniqsys.com> References: <1346.22888.qm@web36213.mail.mud.yahoo.com> <1180617183.3382.37.camel@dot.uniqsys.com> <465EDF5C.2060104@research.att.com> <1180646415.3382.155.camel@dot.uniqsys.com> Message-ID: <4660204E.9020407@research.att.com> Dear folks, Carsten Haese wrote: >On Thu, 2007-05-31 at 10:44 -0400, Art Protin wrote: > > >>I guess I am expected to weigh in and make perfectly clear where I >>stand. >> >> > >"Expected" may be too strong a word, but yes, input from as many >interested parties and API module maintainers as possible is helpful to >measure consensus. > > > >>[snip...] >> >> > >If I summarized that correctly, you are in favor of requiring qmark, >named, and numeric, and dropping format and pyformat. > >I could live with that. As I said before, InformixDB already supports >all three styles, so whether named/numeric are optional or required >makes no difference to me. I'm concerned that requiring named/numeric >might encounter more resistance than requiring qmark, but so far I >haven't seen such resistance on this thread. > > > >>As for a switching requirement: how does this sound (I just though of >>it this morning) >>making the parameterstyle depend on the first character of the SQL >>statement. >>If it is a colon, remove it and the parameter style is either numeric >>or named; if it is >>not a colon, the parameter style is qmark. >> >> > >Sorry, I give that a -1. Using "magical characters" is utterly >unpythonic, and somebody reading the code would have no clue what's >going on. > >A sensible switching mechanism should look something like this, in my >opinion: Add optional 'paramstyle' keyword arguments to >module.connect(), connection.cursor(), cursor.execute() and >cursor.executemany(). In the absence of a 'paramstyle' argument, >individual executions inherit the cursor's paramstyle, cursors inherit >the connection's paramstyle, and connections use the module's read-only >paramstyle attribute as default. (As discussed before, allowing the >module-wide paramstyle to be changed is IMO a bad idea.) > > First, I must confess. I do not know how to make a module's attributes read-only. I do know how to make classes where object instances have read-only attributes. Optional keyword arguments are a very good approach & I do like the inheritance scheme. Would it make sense for paramstyle to be an attribute of connection and cursor, that could be read and written? (As for changing the module-wide paramstyle, you have persuaded me that that approach is "sub-optimum". I do expect that I will at least have to add a superior way and change the documentation to point users at the better way.) >Then again, I'm not too excited about the whole idea of manual >switching. I'd prefer auto-detection based on whether the query string >contains question marks, colon+number or colon+identifier markers >outside of string literals. Not only because that is what InformixDB >already does, but because it leads to cleaner code. > > I would like autodetection, even prefer autodetection, if I was sure that it would work correctly every time no matter what. I am getting more comfortable with that hypothesis, but have not yet developed a proof that autodetection could not get spoofed. In absense of such a proof, I prefer to have the user (even when that user is me) declare what is expected and have the module check as best it can and complain when the expectations are not met. If such a proof was included in the spec., then .paramstyle could be eliminated (and that would be good). >Best regards, > > > [[ why does my email agent seem to discard your signature ??? ]] Thank you all, Art Protin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20070601/eab0813b/attachment.htm From carsten at uniqsys.com Fri Jun 1 16:38:34 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Fri, 01 Jun 2007 10:38:34 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <4660204E.9020407@research.att.com> References: <1346.22888.qm@web36213.mail.mud.yahoo.com> <1180617183.3382.37.camel@dot.uniqsys.com> <465EDF5C.2060104@research.att.com> <1180646415.3382.155.camel@dot.uniqsys.com> <4660204E.9020407@research.att.com> Message-ID: <1180708714.3376.56.camel@dot.uniqsys.com> On Fri, 2007-06-01 at 09:34 -0400, Art Protin wrote: > First, I must confess. I do not know how to make a module's > attributes read-only. Um, apparently neither do I ;) I just tried, and informixdb allows me to change its paramstyle attribute. Then again, there are no code paths in informixdb that depend on the paramstyle setting. > Optional keyword arguments are a very good approach & I do like the > inheritance scheme. Would it make sense for paramstyle to be an > attribute > of connection and cursor, that could be read and written? I was kicking around that idea, but I discarded it because it violates the "Single Point Of Truth" principle. If the paramstyle of a cursor can be changed after the cursor is created, the execute calls on that cursor can in general not be sure that the paramstyle is still the one they need. Each execute call would then have to explicitly specify its paramstyle, which defeats the purpose of the inheritance mechanism. The problem is not as pronounced as the problem caused by allowing the user to set the module's paramstyle, because cursors are rarely if ever passed between threads, but a cursor object could conceivably be shared across functions. > I would like autodetection, even prefer autodetection, if I was sure > that it would > work correctly every time no matter what. I am getting more > comfortable with > that hypothesis, but have not yet developed a proof that autodetection > could not > get spoofed. What do you mean by spoofing in this context? Making the parser think that the paramstyle is one thing when it is in reality something else? I think we can make spoofing impossible by definition. In order to spoof the auto-detection, you'd have to make the query contain both false positives (character sequences that aren't parameter markers but are recognized as parameter markers) and false negatives (parameter markers that aren't recognized as such). An adequate query parser MUST always recognize all parameter markers that are present, therefore a parser that permits false negatives is not adequate. [If the parser has to err due to performance reasons, it must err on the side of false positives.] Hence, an adequate parser can not be spoofed. Of course, I've simply transformed the problem to "Does an adequate parser exist for your SQL dialect?" but I think we'll just have to stipulate that the answer is yes. Best regards, -- Carsten Haese http://informixdb.sourceforge.net From aprotin at research.att.com Fri Jun 1 18:09:27 2007 From: aprotin at research.att.com (Art Protin) Date: Fri, 01 Jun 2007 12:09:27 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <1180708714.3376.56.camel@dot.uniqsys.com> References: <1346.22888.qm@web36213.mail.mud.yahoo.com> <1180617183.3382.37.camel@dot.uniqsys.com> <465EDF5C.2060104@research.att.com> <1180646415.3382.155.camel@dot.uniqsys.com> <4660204E.9020407@research.att.com> <1180708714.3376.56.camel@dot.uniqsys.com> Message-ID: <466044B7.7010808@research.att.com> Dear folks, Carsten Haese wrote: >On Fri, 2007-06-01 at 09:34 -0400, Art Protin wrote: > > >>First, I must confess. I do not know how to make a module's >>attributes read-only. >> >> > >Um, apparently neither do I ;) > > Thank you!! I needed this. I laughed out loud to read your response. A nice way to end the week. >I just tried, and informixdb allows me to change its paramstyle >attribute. Then again, there are no code paths in informixdb that depend >on the paramstyle setting. > > > >>Optional keyword arguments are a very good approach & I do like the >>inheritance scheme. Would it make sense for paramstyle to be an >>attribute >>of connection and cursor, that could be read and written? >> >> > >I was kicking around that idea, but I discarded it because it violates >the "Single Point Of Truth" principle. If the paramstyle of a cursor can >be changed after the cursor is created, the execute calls on that cursor >can in general not be sure that the paramstyle is still the one they >need. Each execute call would then have to explicitly specify its >paramstyle, which defeats the purpose of the inheritance mechanism. > >The problem is not as pronounced as the problem caused by allowing the >user to set the module's paramstyle, because cursors are rarely if ever >passed between threads, but a cursor object could conceivably be shared >across functions. > > Yeah, I see the point. Set the style on object creation and make the attribute read-only. Does it need to "pass through" the connection object or can the cursor inherit its default from the module? > > > >>I would like autodetection, even prefer autodetection, if I was sure >>that it would >>work correctly every time no matter what. I am getting more >>comfortable with >>that hypothesis, but have not yet developed a proof that autodetection >>could not >>get spoofed. >> >> > >What do you mean by spoofing in this context? Making the parser think >that the paramstyle is one thing when it is in reality something else? I >think we can make spoofing impossible by definition. > >In order to spoof the auto-detection, you'd have to make the query >contain both false positives (character sequences that aren't parameter >markers but are recognized as parameter markers) and false negatives >(parameter markers that aren't recognized as such). An adequate query >parser MUST always recognize all parameter markers that are present, >therefore a parser that permits false negatives is not adequate. [If the >parser has to err due to performance reasons, it must err on the side of >false positives.] Hence, an adequate parser can not be spoofed. > >Of course, I've simply transformed the problem to "Does an adequate >parser exist for your SQL dialect?" but I think we'll just have to >stipulate that the answer is yes. > > Ah, there is the disconnect. For my interface I must ask can I code enough of a parser for my dialect of SQL in Python so that my module can properly handle parameters. There is no reasonable way for my driver to get the backend to parse the query and then tell the driver what parameters are needed. My driver must dissect the query enough to identify all the parameters and "transform them" in the query before passing the query to the backend. >Best regards, > > > Thank you all, Art Protin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20070601/3c1df820/attachment.html From carsten at uniqsys.com Fri Jun 1 19:41:34 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Fri, 01 Jun 2007 13:41:34 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <466044B7.7010808@research.att.com> References: <1346.22888.qm@web36213.mail.mud.yahoo.com> <1180617183.3382.37.camel@dot.uniqsys.com> <465EDF5C.2060104@research.att.com> <1180646415.3382.155.camel@dot.uniqsys.com> <4660204E.9020407@research.att.com> <1180708714.3376.56.camel@dot.uniqsys.com> <466044B7.7010808@research.att.com> Message-ID: <1180719694.3376.93.camel@dot.uniqsys.com> On Fri, 2007-06-01 at 12:09 -0400, Art Protin wrote: > > Of course, I've simply transformed the problem to "Does an adequate > > parser exist for your SQL dialect?" but I think we'll just have to > > stipulate that the answer is yes. > > > Ah, there is the disconnect. For my interface I must ask can I code > enough > of a parser for my dialect of SQL in Python so that my module can > properly > handle parameters. There is no reasonable way for my driver to get > the > backend to parse the query and then tell the driver what parameters > are > needed. My driver must dissect the query enough to identify all the > parameters > and "transform them" in the query before passing the query to the > backend. The same is true for InformixDB. The good news is that making a parser that locates parameter placeholders is not hard. Essentially it boils down to "Look for question marks and colons occurring outside of string literals". There's only a bit of a wrinkle if the SQL dialect allows colons outside of string literals. For example, Informix has datetime literals, a "::" cast notation, and "databasename:tablename" remote-table notation. (I don't know what standard SQL allows.) These situations can be handled by a straightforward heuristic. If a colon follows any character that's neither a colon nor an alphanumeric character, and it's followed by a valid placeholder number or name, it's a placeholder. Otherwise it's considered part of the SQL query. Best regards, -- Carsten Haese http://informixdb.sourceforge.net From paul at boddie.org.uk Sat Jun 2 20:42:37 2007 From: paul at boddie.org.uk (Paul Boddie) Date: Sat, 02 Jun 2007 20:42:37 +0200 Subject: [DB-SIG] paramstyles, again Message-ID: <200706022042.38146.paul@boddie.org.uk> Carsten Haese wrote: > > The same is true for InformixDB. The good news is that making a parser > that locates parameter placeholders is not hard. Essentially it boils > down to "Look for question marks and colons occurring outside of string > literals". I have just written tokenisers using both pyparsing and regular expressions [1] which split SQL statements into non-literal and literal regions (for character string literals, of course). I haven't checked using the various "public" specifications to see if there are any special cases in standard SQL which would affect this code, but I don't see why we couldn't just have this sort of thing in all DB-API modules, support question marks as placeholders, and move the ridiculous paramstyle situation forward. Paul [1] http://www.python.org/pypi/sqlliterals From carsten at uniqsys.com Mon Jun 4 01:08:57 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Sun, 03 Jun 2007 19:08:57 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <200706022042.38146.paul@boddie.org.uk> References: <200706022042.38146.paul@boddie.org.uk> Message-ID: <1180912137.3191.24.camel@localhost.localdomain> On Sat, 2007-06-02 at 20:42 +0200, Paul Boddie wrote: > Carsten Haese wrote: > > > > The same is true for InformixDB. The good news is that making a parser > > that locates parameter placeholders is not hard. Essentially it boils > > down to "Look for question marks and colons occurring outside of string > > literals". > > I have just written tokenisers using both pyparsing and regular expressions > [1] which split SQL statements into non-literal and literal regions (for > character string literals, of course). I haven't checked using the various > "public" specifications to see if there are any special cases in standard SQL > which would affect this code, but I don't see why we couldn't just have this > sort of thing in all DB-API modules, support question marks as placeholders, > and move the ridiculous paramstyle situation forward. As far I can tell, we have already reached consensus for making at least qmark mandatory. We also seem to have consensus on at least deprecating, possibly even eliminating format/pyformat. We still need to settle the question of what to do about numeric and named styles. There is some support for making both styles mandatory and expecting the API to automatically discern from the query string which parameter style is in use. This is the most programmer-friendly option, and at least InformixDB and sqlite3 already behave this way. (Those two I know for sure, there may be more.) Sqlite supports all styles natively, and InformixDB uses a simple FSM-based parser to translate named and numeric markers into the native qmark style. Any input to move this discussion along is appreciated. I'd also like to get some more input on the original subject of this thread, type mapping. I believe Marc-Andre still owes me a counter-proposal. ;) Best regards, -- Carsten Haese http://informixdb.sourceforge.net From carsten at uniqsys.com Mon Jun 4 01:40:29 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Sun, 03 Jun 2007 19:40:29 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <831120.61375.qm@web36203.mail.mud.yahoo.com> References: <831120.61375.qm@web36203.mail.mud.yahoo.com> Message-ID: <1180914029.3191.35.camel@localhost.localdomain> On Thu, 2007-05-31 at 01:07 -0700, Vern Cole wrote: > I am sold. +1 on requiring support of > 'qmark' and 'named' parameter styles. Could I sell you on making numeric style required, too? > Implementors, like me, who are running qmark only > databases will have to parse the dictionary for 'named' parameters and > build a qmark string > as part of the cursor.execute() method. No big deal. You mean you'd parse the query string, right? > Next question: can't I determine the paramstyle by looking at the > second parameter of c.execute()? If you're parsing the query string, you already know which parameter style the query uses. > Only if I support other (nonstandard) styles will I really ever need > to have someone specify > which to expect. What non-standard styles are you expecting to support? Regards, -- Carsten Haese http://informixdb.sourceforge.net From mal at egenix.com Mon Jun 4 10:41:46 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 04 Jun 2007 10:41:46 +0200 Subject: [DB-SIG] paramstyles, again In-Reply-To: <1180912137.3191.24.camel@localhost.localdomain> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> Message-ID: <4663D04A.3010103@egenix.com> On 2007-06-04 01:08, Carsten Haese wrote: > On Sat, 2007-06-02 at 20:42 +0200, Paul Boddie wrote: >> Carsten Haese wrote: >>> The same is true for InformixDB. The good news is that making a parser >>> that locates parameter placeholders is not hard. Essentially it boils >>> down to "Look for question marks and colons occurring outside of string >>> literals". >> I have just written tokenisers using both pyparsing and regular expressions >> [1] which split SQL statements into non-literal and literal regions (for >> character string literals, of course). I haven't checked using the various >> "public" specifications to see if there are any special cases in standard SQL >> which would affect this code, but I don't see why we couldn't just have this >> sort of thing in all DB-API modules, support question marks as placeholders, >> and move the ridiculous paramstyle situation forward. > > As far I can tell, we have already reached consensus for making at least > qmark mandatory. We also seem to have consensus on at least deprecating, > possibly even eliminating format/pyformat. > > We still need to settle the question of what to do about numeric and > named styles. There is some support for making both styles mandatory and > expecting the API to automatically discern from the query string which > parameter style is in use. I don't think that this is such a good idea and I'm also not sure how you'd do that: what if a programmer mixes the two (or three) styles by mistake ? How would the programmer test whether a module supports this automatic detection scheme ? I'd suggest to make the current module read-only paramstyle attribute and writable connection/cursor attribute which gets inherited from connections to cursors. There's also the problem of parameter type if we allow named style. The parameters would then have to be dictionaries for named and tuples for qmark/numeric. So you effectively change the signature of the .execute() methods based on a parameter which is not good style. > This is the most programmer-friendly option, > and at least InformixDB and sqlite3 already behave this way. (Those two > I know for sure, there may be more.) Sqlite supports all styles > natively, and InformixDB uses a simple FSM-based parser to translate > named and numeric markers into the native qmark style. > > Any input to move this discussion along is appreciated. I'd also like to > get some more input on the original subject of this thread, type > mapping. I believe Marc-Andre still owes me a counter-proposal. ;) Yeah, right :-) It's not really a counter-proposal, but only a slightly different approach. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 04 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carsten at uniqsys.com Mon Jun 4 13:17:46 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 4 Jun 2007 07:17:46 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <4663D04A.3010103@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> Message-ID: <20070604110700.M68756@uniqsys.com> On Mon, 04 Jun 2007 10:41:46 +0200, M.-A. Lemburg wrote > > We still need to settle the question of what to do about numeric and > > named styles. There is some support for making both styles mandatory and > > expecting the API to automatically discern from the query string which > > parameter style is in use. > > I don't think that this is such a good idea and I'm also not sure > how you'd do that: what if a programmer mixes the two (or three) > styles by mistake ? How would the programmer test whether a > module supports this automatic detection scheme ? If the programmer mixes incompatible parameter styles, the implementation can choose to handle it gracefully (sqlite3) or raise an exception (InformixDB). InformixDB actually allows mixing qmark and numeric in a straightforward way: The first question mark is equivalent to :1, the second question mark is equivalent to :2, etc. Only mixing positional markers and named markers raises an exception. I'd be okay with throwing an exception in any case of mixing, but I don't know how that would sit with sqlite3. The test for whether the module supports the automatic detection is module.apilevel=='3.0'. > There's also the problem of parameter type if we allow named > style. The parameters would then have to be dictionaries for > named and tuples for qmark/numeric. So you effectively change > the signature of the .execute() methods based on a parameter > which is not good style. I disagree. It's duck-typing. -- Carsten Haese http://informixdb.sourceforge.net From carsten at uniqsys.com Mon Jun 4 14:43:59 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 08:43:59 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <4663D04A.3010103@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> Message-ID: <1180961039.3388.6.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 10:41 +0200, M.-A. Lemburg wrote: > I'd suggest to make the current module read-only paramstyle > attribute and writable connection/cursor attribute which gets > inherited from connections to cursors. > > There's also the problem of parameter type if we allow named > style. The parameters would then have to be dictionaries for > named and tuples for qmark/numeric. So you effectively change > the signature of the .execute() methods based on a parameter > which is not good style. In addition to what I said before, you're proposing that the .execute() signature depend on a connection/cursor attribute. I'm not sure that that's much better than having it depend directly on the query string. Regards, -- Carsten Haese http://informixdb.sourceforge.net From mal at egenix.com Mon Jun 4 15:00:15 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 04 Jun 2007 15:00:15 +0200 Subject: [DB-SIG] paramstyles, again In-Reply-To: <1180961039.3388.6.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <1180961039.3388.6.camel@dot.uniqsys.com> Message-ID: <46640CDF.8020804@egenix.com> On 2007-06-04 14:43, Carsten Haese wrote: > On Mon, 2007-06-04 at 10:41 +0200, M.-A. Lemburg wrote: >> I'd suggest to make the current module read-only paramstyle >> attribute and writable connection/cursor attribute which gets >> inherited from connections to cursors. >> >> There's also the problem of parameter type if we allow named >> style. The parameters would then have to be dictionaries for >> named and tuples for qmark/numeric. So you effectively change >> the signature of the .execute() methods based on a parameter >> which is not good style. > > In addition to what I said before, you're proposing that the .execute() > signature depend on a connection/cursor attribute. I'm not sure that > that's much better than having it depend directly on the query string. Well, no: I'm saying that it's not a good idea to have the signature depend on a parameter. OTOH, you need such a parameter to allow programmers to query the current paramstyle setting, if you do allow multiple param styles. This creates a conflict. The only way around it would be to say that the 'parameters' parameter in .execute() must have a __getitem__ compatible interface which is then used to either get the positional argument (via an integer key) or the named argument (via a string or Unicode object). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 04 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carsten at uniqsys.com Mon Jun 4 15:24:31 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 09:24:31 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <46640CDF.8020804@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <1180961039.3388.6.camel@dot.uniqsys.com> <46640CDF.8020804@egenix.com> Message-ID: <1180963471.3388.21.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 15:00 +0200, M.-A. Lemburg wrote: > Well, no: I'm saying that it's not a good idea to have the > signature depend on a parameter. > > OTOH, you need such a parameter to allow programmers to query > the current paramstyle setting, if you do allow multiple param > styles. > > This creates a conflict. > > The only way around it would be to say that the 'parameters' > parameter in .execute() must have a __getitem__ compatible > interface which is then used to either get the positional > argument (via an integer key) or the named argument (via > a string or Unicode object). I'm not sure I follow. The proposal on the table is to require all interfaces to support qmark, numeric, and named, and to automatically detect from the query string which style the query uses. Since the programmer controls both the query string and the corresponding parameters, there is no use case for inspecting the current paramstyle setting. To me, there is no practical difference between saying "the parameters argument is a sequence or a mapping as determined by the query's parameter style" and "the parameters argument may be any object with a __getitem__ method that provides appropriate keys", so I'm fine with either one as long as that gives us auto-detection of parameter styles. -- Carsten Haese http://informixdb.sourceforge.net From mal at egenix.com Mon Jun 4 15:32:05 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 04 Jun 2007 15:32:05 +0200 Subject: [DB-SIG] paramstyles, again In-Reply-To: <20070604110700.M68756@uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> Message-ID: <46641455.4070803@egenix.com> On 2007-06-04 13:17, Carsten Haese wrote: > On Mon, 04 Jun 2007 10:41:46 +0200, M.-A. Lemburg wrote >>> We still need to settle the question of what to do about numeric and >>> named styles. There is some support for making both styles mandatory and >>> expecting the API to automatically discern from the query string which >>> parameter style is in use. >> I don't think that this is such a good idea and I'm also not sure >> how you'd do that: what if a programmer mixes the two (or three) >> styles by mistake ? How would the programmer test whether a >> module supports this automatic detection scheme ? > > If the programmer mixes incompatible parameter styles, the implementation can > choose to handle it gracefully (sqlite3) or raise an exception (InformixDB). > InformixDB actually allows mixing qmark and numeric in a straightforward way: > The first question mark is equivalent to :1, the second question mark is > equivalent to :2, etc. Only mixing positional markers and named markers raises > an exception. > > I'd be okay with throwing an exception in any case of mixing, but I don't know > how that would sit with sqlite3. > > The test for whether the module supports the automatic detection is > module.apilevel=='3.0'. Do you really think it's reasonable to have the module parse every single SQL statement just to detect the param style ? What happens if you're in qmark style mode and the SQL statement contains substrings which look like named style (but are not intended that way, e.g. think stored procedures) ? This approach also contradicts the explicit is better than implicit Zen rule. Just so that there's no misunderstanding: you're free to implement this sort of extension in your module(s), but making this mandatory in the spec feels wrong. I tend to view this whole discussion as a bit off the track: In an API specification, you normally define one way which all implementations have to follow, not multiple ways of doing the same thing in slightly different ways. >> There's also the problem of parameter type if we allow named >> style. The parameters would then have to be dictionaries for >> named and tuples for qmark/numeric. So you effectively change >> the signature of the .execute() methods based on a parameter >> which is not good style. > > I disagree. It's duck-typing. Following you proposal to require auto-detection, this would mean more than just duck-typing. You wouldn't know what type to expect until you've parsed the SQL statement string - ie. when you're already inside the .execute() method and passed the argument parsing step. That's a bit much too dynamic and too implicit for an interface specification. So I'm: * +1 on making support one param style mandatory for all implementations * +1 on requiring an interface to adjust the param style on a per connection or even per cursor basis, so that it's easy to setup an implementation to the mandatory param style (in case the module defaults to another param style for backwards compatibility reasons) This interface would have to support at least the mandatory param style. * -1 on doing auto-detection of param styles at .execute() call time based on parsing the SQL statement -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 04 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carl at personnelware.com Mon Jun 4 15:47:30 2007 From: carl at personnelware.com (Carl Karsten) Date: Mon, 04 Jun 2007 08:47:30 -0500 Subject: [DB-SIG] paramstyles, again In-Reply-To: <1180963471.3388.21.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <1180961039.3388.6.camel@dot.uniqsys.com> <46640CDF.8020804@egenix.com> <1180963471.3388.21.camel@dot.uniqsys.com> Message-ID: <466417F2.9060601@personnelware.com> Carsten Haese wrote: > On Mon, 2007-06-04 at 15:00 +0200, M.-A. Lemburg wrote: >> Well, no: I'm saying that it's not a good idea to have the >> signature depend on a parameter. >> >> OTOH, you need such a parameter to allow programmers to query >> the current paramstyle setting, if you do allow multiple param >> styles. >> >> This creates a conflict. >> >> The only way around it would be to say that the 'parameters' >> parameter in .execute() must have a __getitem__ compatible >> interface which is then used to either get the positional >> argument (via an integer key) or the named argument (via >> a string or Unicode object). > > I'm not sure I follow. The proposal on the table is to require all > interfaces to support qmark, numeric, and named, and to automatically > detect from the query string which style the query uses. Since the > programmer controls both the query string and the corresponding > parameters, there is no use case for inspecting the current paramstyle > setting. > > To me, there is no practical difference between saying "the parameters > argument is a sequence or a mapping as determined by the query's > parameter style" and "the parameters argument may be any object with a > __getitem__ method that provides appropriate keys", so I'm fine with > either one as long as that gives us auto-detection of parameter styles. > Is auto-detection that important? So far it seems the main (only?) reason for 'auto' is because no one has come up with a good way of specifying it as a parameter. I think I saw a good reason for supporting multiple paramstyles, but maybe the agreed upon points is should be put on a wiki. I have this 'feeling' that if the multiple paramstyles 'requries' auto detection, it isn't worth it and would be better if we just picked a single paramstyle. I can't really see why any one single paramstyle would not be useable in all cases. Of qmark, numeric, and named, I think named is the best of all worlds. I think if that was the only choice, life would be pretty good. Would adding the others make life any better? especially considering the 'cost' (complexity: more code in dbapi modules, more than one way to do things, more docs, more time in dbapi tutorials...) I vote for named only and get rid of the whole paramstyle issue. Carl K From carsten at uniqsys.com Mon Jun 4 16:00:21 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 10:00:21 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <46641455.4070803@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> Message-ID: <1180965621.3388.42.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 15:32 +0200, M.-A. Lemburg wrote: > Do you really think it's reasonable to have the module parse > every single SQL statement just to detect the param style ? Well, at least I don't think it's unreasonable to do so. This parsing is not expensive and only happens when the query is prepared, so the performance impact is minimal. > What happens if you're in qmark style mode and the SQL statement > contains substrings which look like named style (but are not > intended that way, e.g. think stored procedures) ? Do you have a concrete example? > This approach also contradicts the explicit is better than > implicit Zen rule. Well, one might argue that the paramstyle is explicitly visible in the format of the query string. > Just so that there's no misunderstanding: you're free to implement > this sort of extension in your module(s), but making this > mandatory in the spec feels wrong. Fair enough. > Following you proposal to require auto-detection, this would mean > more than just duck-typing. You wouldn't know what type to expect > until you've parsed the SQL statement string - ie. when you're > already inside the .execute() method and passed the argument > parsing step. > > That's a bit much too dynamic and too implicit for an interface > specification. I thought being dynamic was Pythonic ;) Anyway, I do think we can reach a compromise here: > So I'm: > > * +1 on making support one param style mandatory for all > implementations Any one? Let's make named mandatory, then. > * +1 on requiring an interface to adjust the param style on a > per connection or even per cursor basis, so that it's easy > to setup an implementation to the mandatory param style > (in case the module defaults to another param style for backwards > compatibility reasons) > > This interface would have to support at least the mandatory > param style. +1, let's discuss what that interface should look like. > * -1 on doing auto-detection of param styles at .execute() > call time based on parsing the SQL statement I now agree with -1 on making auto-detection mandatory as long as there is a way to turn it on. How about making a special paramstyle called auto? -- Carsten Haese | Phone: (419) 861-3331 Software Engineer | Direct Line: (419) 794-2531 Unique Systems, Inc. | FAX: (419) 893-2840 1687 Woodlands Drive | Cell: (419) 343-7045 Maumee, OH 43537 | Email: carsten at uniqsys.com From mal at egenix.com Mon Jun 4 16:32:18 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 04 Jun 2007 16:32:18 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1180965621.3388.42.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> Message-ID: <46642272.8090501@egenix.com> On 2007-06-04 16:00, Carsten Haese wrote: >> So I'm: >> >> * +1 on making support one param style mandatory for all >> implementations > > Any one? Let's make named mandatory, then. I intentionally left this open :-) We should have a vote on it. Let's gather votes for say two weeks and then see what the outcome is. Here's my vote: +1 on qmark +0 on named +0 on numeric -1 on format -1 on pyformat >> * +1 on requiring an interface to adjust the param style on a >> per connection or even per cursor basis, so that it's easy >> to setup an implementation to the mandatory param style >> (in case the module defaults to another param style for backwards >> compatibility reasons) >> >> This interface would have to support at least the mandatory >> param style. > > +1, let's discuss what that interface should look like. We basically have two options: * use an attribute which is inherited from connections to cursors * use a get/set method approach with much the same meaning For simplicity, I'd be in favor of adding a settable .paramstyle attribute. >> * -1 on doing auto-detection of param styles at .execute() >> call time based on parsing the SQL statement > > I now agree with -1 on making auto-detection mandatory as long as there > is a way to turn it on. How about making a special paramstyle called > auto? Sounds like a plan :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 04 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From roger4867 at gmail.com Mon Jun 4 16:46:22 2007 From: roger4867 at gmail.com (Roger Doger) Date: Mon, 4 Jun 2007 09:46:22 -0500 Subject: [DB-SIG] trouble with list formatting Message-ID: The following use of the API with MySQL works fine > items = [1, 2] > execute('SELECT * from table1 WHERE field1 IN %s', (items,)) resultant query string: SELECT * from table1 WHERE field1 IN ('1', '2') By contrast, the following yields the message "error in your SQL syntax" > items = [1] > execute('SELECT * from table1 WHERE field1 IN %s', (items,)) resulting query string: SELECT * from table1 WHERE field1 IN ('1',) Thus it seems the only way to manage this situation is to use this stilted construction: > if len(items)==1: > execute('SELECT * from table1 WHERE field1 IN (%s)', items[0]) > else: > execute('SELECT * from table1 WHERE field1 IN %s', (items,)) Is this just a shortcoming in how the API handles formatting of lists or is there a cleaner way to handle this? From carl at personnelware.com Mon Jun 4 17:14:14 2007 From: carl at personnelware.com (Carl Karsten) Date: Mon, 04 Jun 2007 10:14:14 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46642272.8090501@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> Message-ID: <46642C46.3020707@personnelware.com> M.-A. Lemburg wrote: > On 2007-06-04 16:00, Carsten Haese wrote: >>> So I'm: >>> >>> * +1 on making support one param style mandatory for all >>> implementations >> Any one? Let's make named mandatory, then. > > I intentionally left this open :-) > > We should have a vote on it. Let's gather votes for say two weeks > and then see what the outcome is. Is there a wiki where I can start cataloging the cases for the various cases? I am sure I saw a good case for qmark, but I can't for the life of me remember what it was. I will eventually dig though the archives, but as long as I am doing that I may as well list what I find. Also, I would like to add to the ballot: * support multiple styles * something about auto. * something about specifying the format I am currently against multiple, so someone who is for it will have to figure out how to phrase those 2nd two. :) btw - before this topic came up, I was a big fan of pyformat, but the cases presented against it were enough to sway me. I kinda remember them, but would rather quote the original text. Carl K From carsten at uniqsys.com Mon Jun 4 17:17:25 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 11:17:25 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46642272.8090501@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> Message-ID: <1180970245.3388.65.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 16:32 +0200, M.-A. Lemburg wrote: > On 2007-06-04 16:00, Carsten Haese wrote: > >> So I'm: > >> > >> * +1 on making support one param style mandatory for all > >> implementations > > > > Any one? Let's make named mandatory, then. > > I intentionally left this open :-) > > We should have a vote on it. Let's gather votes for say two weeks > and then see what the outcome is. Here's my vote: +1 on qmark +1 on named +0 on numeric -1 on format -1 on pyformat > > +1, let's discuss what that interface should look like. > > We basically have two options: > > * use an attribute which is inherited from connections to > cursors > > * use a get/set method approach with much the same meaning > > For simplicity, I'd be in favor of adding a settable > .paramstyle attribute. +1 on connection.paramstyle and cursor.paramstyle attributes. We should also discuss what is to happen to the .paramstyle module attribute. Naturally, the attribute would reflect the module's default style (and this default should not be settable module wide). However, we might want to convey which paramstyles the module can accept. I see two options: * Invent a new module attribute called paramstyles that is a list or set of supported paramstyles. * Simply raise ValueError if the programmer requests an unsupported parameter style. I think the second one should be done regardless, but I'd be in favor of also having the explicit paramstyles metadata. Best regards, -- Carsten Haese http://informixdb.sourceforge.net From carsten at uniqsys.com Mon Jun 4 17:34:37 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 11:34:37 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46642C46.3020707@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <46642C46.3020707@personnelware.com> Message-ID: <1180971277.3388.77.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 10:14 -0500, Carl Karsten wrote: > Is there a wiki where I can start cataloging the cases for the various cases? There is http://wiki.python.org/moin/ > I am sure I saw a good case for qmark, but I can't for the life of me remember > what it was. I will eventually dig though the archives, but as long as I am > doing that I may as well list what I find. I don't know if that qualifies as a good case for qmark, but qmark is SQL standard. > Also, I would like to add to the ballot: > * support multiple styles > * something about auto. > * something about specifying the format > > I am currently against multiple, so someone who is for it will have to figure > out how to phrase those 2nd two. :) In the interest of programmer freedom, module implementers should have the option of allowing multiple styles. In the interest of ease of implementation, this should be optional, not mandatory. The exact same case can be made for auto-detection. The case against format and pyformat goes like this: It looks too much like string interpolation, which may lead to bad code, and it requires percent signs in queries to be doubled up, which is an ugly wart. Regards, -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Mon Jun 4 18:00:19 2007 From: carl at personnelware.com (Carl Karsten) Date: Mon, 04 Jun 2007 11:00:19 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1180971277.3388.77.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <46642C46.3020707@personnelware.com> <1180971277.3388.77.camel@dot.uniqsys.com> Message-ID: <46643713.5080407@personnelware.com> Carsten Haese wrote: > On Mon, 2007-06-04 at 10:14 -0500, Carl Karsten wrote: >> Is there a wiki where I can start cataloging the cases for the various cases? > > There is http://wiki.python.org/moin/ Hey look, much of this is already there: Unified Parameter Style http://wiki.python.org/moin/DbApi3?highlight=%28dbapi%29#head-88df6649e674265578517ee6b9fd421e18b87af1 > >> I am sure I saw a good case for qmark, but I can't for the life of me remember >> what it was. I will eventually dig though the archives, but as long as I am >> doing that I may as well list what I find. > > I don't know if that qualifies as a good case for qmark, but qmark is > SQL standard. That may have been it. I'll be sure to add it to the list. > >> Also, I would like to add to the ballot: >> * support multiple styles >> * something about auto. >> * something about specifying the format >> >> I am currently against multiple, so someone who is for it will have to figure >> out how to phrase those 2nd two. :) > > In the interest of programmer freedom, module implementers should have > the option of allowing multiple styles. In the interest of ease of > implementation, this should be optional, not mandatory. > > The exact same case can be made for auto-detection. I will begrudgingly add that too :) > > The case against format and pyformat goes like this: It looks too much > like string interpolation, which may lead to bad code, and it requires > percent signs in queries to be doubled up, which is an ugly wart. doubled up percent signs - that was it. Also there is no point in specifying a data type, right? Carl K From dieter at handshake.de Mon Jun 4 18:58:48 2007 From: dieter at handshake.de (Dieter Maurer) Date: Mon, 4 Jun 2007 18:58:48 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46642272.8090501@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> Message-ID: <18020.17608.35709.968306@gargle.gargle.HOWL> M.-A. Lemburg wrote at 2007-6-4 16:32 +0200: > ... >> Any one? Let's make named mandatory, then. > >I intentionally left this open :-) > >We should have a vote on it. Let's gather votes for say two weeks >and then see what the outcome is. > >Here's my vote: > > +1 on qmark > +0 on named > +0 on numeric > -1 on format > -1 on pyformat I favour "named" or something equivalent (I do not mind "pyformat" as well). -- Dieter From mike_mp at zzzcomputing.com Mon Jun 4 19:05:48 2007 From: mike_mp at zzzcomputing.com (Michael Bayer) Date: Mon, 4 Jun 2007 13:05:48 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <18020.17608.35709.968306@gargle.gargle.HOWL> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <18020.17608.35709.968306@gargle.gargle.HOWL> Message-ID: <5C8EE4E4-E7BB-47C8-B393-DCB5FFCC0BE7@zzzcomputing.com> +1 on qmark +1 on named -1 on numeric -1 on format -1 on pyformat also +1 on a 'paramstyle' attribute that is settable on connection as well as cursor +1 on raising an error if the given statement/arguments dont match the 'paramstyle'. explicit over implicit, refuse temptation to guess. From mal at egenix.com Mon Jun 4 19:09:35 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 04 Jun 2007 19:09:35 +0200 Subject: [DB-SIG] paramstyles, again In-Reply-To: <1180965621.3388.42.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> Message-ID: <4664474F.3050409@egenix.com> On 2007-06-04 16:00, Carsten Haese wrote: >> What happens if you're in qmark style mode and the SQL statement >> contains substrings which look like named style (but are not >> intended that way, e.g. think stored procedures) ? > > Do you have a concrete example? Yes: In SAP DB and Max DB triggers use ":old" and ":new" markers to provide a reference mechanism to the updated/deleted row. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 04 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carl at personnelware.com Mon Jun 4 19:13:52 2007 From: carl at personnelware.com (Carl Karsten) Date: Mon, 04 Jun 2007 12:13:52 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1180971277.3388.77.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <46642C46.3020707@personnelware.com> <1180971277.3388.77.camel@dot.uniqsys.com> Message-ID: <46644850.2060003@personnelware.com> Carsten Haese wrote: > On Mon, 2007-06-04 at 16:32 +0200, M.-A. Lemburg wrote: >> On 2007-06-04 16:00, Carsten Haese wrote: >>>> So I'm: >>>> >>>> * +1 on making support one param style mandatory for all >>>> implementations >>> Any one? Let's make named mandatory, then. >> I intentionally left this open :-) >> >> We should have a vote on it. Let's gather votes for say two weeks >> and then see what the outcome is. > > Here's my vote: > > +1 on qmark > +1 on named > +0 on numeric > -1 on format > -1 on pyformat > >>> +1, let's discuss what that interface should look like. >> We basically have two options: >> >> * use an attribute which is inherited from connections to >> cursors >> >> * use a get/set method approach with much the same meaning >> >> For simplicity, I'd be in favor of adding a settable >> .paramstyle attribute. > > +1 on connection.paramstyle and cursor.paramstyle attributes. > > We should also discuss what is to happen to the .paramstyle module > attribute. Naturally, the attribute would reflect the module's default > style (and this default should not be settable module wide). However, we Hmm, default style brings up a new issue: If there will be more than one style, should there be a common default? > might want to convey which paramstyles the module can accept. I see two > options: > > * Invent a new module attribute called paramstyles that is a list or set > of supported paramstyles. > * Simply raise ValueError if the programmer requests an unsupported > parameter style. I kinda think it been decided that the set of styles will be required of all modules. But given that it was never voted on, add it to the ballot. Carl K From carsten at uniqsys.com Mon Jun 4 19:24:39 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 13:24:39 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46644850.2060003@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <46642C46.3020707@personnelware.com> <1180971277.3388.77.camel@dot.uniqsys.com> <46644850.2060003@personnelware.com> Message-ID: <1180977879.3388.99.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 12:13 -0500, Carl Karsten wrote: > Hmm, default style brings up a new issue: > If there will be more than one style, should there be a common default? No, the default will be implementation-defined for backwards compatibility. There will be a common way of switching to the common mandatory paramstyle. > > might want to convey which paramstyles the module can accept. I see two > > options: > > > > * Invent a new module attribute called paramstyles that is a list or set > > of supported paramstyles. > > * Simply raise ValueError if the programmer requests an unsupported > > parameter style. > > I kinda think it been decided that the set of styles will be required of all > modules. But given that it was never voted on, add it to the ballot. I think we're more leaning towards making only one common style mandatory, and allowing other styles as optional extensions. -- Carsten Haese http://informixdb.sourceforge.net From fabien.coutant at neuf.fr Mon Jun 4 19:29:43 2007 From: fabien.coutant at neuf.fr (Fabien COUTANT) Date: Mon, 04 Jun 2007 19:29:43 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46642272.8090501@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> Message-ID: <20070604192943.e492530f.fabien.coutant@neuf.fr> On Mon, 04 Jun 2007 16:32:18 +0200, M.-A. Lemburg wrote: > We should have a vote on it. Let's gather votes for say two weeks > and then see what the outcome is. Here's my application programmer's vote: -1 on qmark +1 on named 0 on numeric -1 on format +1 on pyformat I was initially set at +1 on numeric and changed my mind. I consider it's only marginally better than format and qmark, and less good than named and pyformat. For me, named and pyformat are the only styles that allow writing maintanable request code when you have more than 2 parameters/request and hundreds of/long requests. W.r.t. sql string parsing matter: I want to add that using a pythonic parameter markup such as pyformat is less confusing, if needed, to a parser within a SQL string, because they're coming from 2 different worlds. I hope it's unlikely that %(foo)s is valid SQL in any database... Now, acceptably, there's the confusion with Python formatting :-/ -- Hope this helps, Fabien. From Chris.Clark at ingres.com Mon Jun 4 19:30:41 2007 From: Chris.Clark at ingres.com (Chris Clark) Date: Mon, 04 Jun 2007 10:30:41 -0700 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46642272.8090501@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> Message-ID: <46644C41.4040802@ingres.com> +1 on qmark +0 on named +0 on numeric -1 on format -1 on pyformat -- wish I could vote -2 ;-) From carsten at uniqsys.com Mon Jun 4 19:30:56 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 13:30:56 -0400 Subject: [DB-SIG] paramstyles, again In-Reply-To: <4664474F.3050409@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <4664474F.3050409@egenix.com> Message-ID: <1180978256.3388.106.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 19:09 +0200, M.-A. Lemburg wrote: > On 2007-06-04 16:00, Carsten Haese wrote: > >> What happens if you're in qmark style mode and the SQL statement > >> contains substrings which look like named style (but are not > >> intended that way, e.g. think stored procedures) ? > > > > Do you have a concrete example? > > Yes: In SAP DB and Max DB triggers use ":old" and ":new" markers to > provide a reference mechanism to the updated/deleted row. Interesting. Then again, CREATE TRIGGER statements probably shouldn't be using parameter markers of any kind, but I see your point that parsing an SQL query for parameter markers may be made harder by database-specific quirks. Of course, SAP DB/Max DB might have a bit of a problem if we decide to make 'named' the mandatory minimum paramstyle... -- Carsten Haese http://informixdb.sourceforge.net From aprotin at research.att.com Mon Jun 4 20:03:00 2007 From: aprotin at research.att.com (Art Protin) Date: Mon, 04 Jun 2007 14:03:00 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1180977879.3388.99.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <46642C46.3020707@personnelware.com> <1180971277.3388.77.camel@dot.uniqsys.com> <46644850.2060003@personnelware.com> <1180977879.3388.99.camel@dot.uniqsys.com> Message-ID: <466453D4.5010601@research.att.com> Dear folks, Carsten Haese wrote: >On Mon, 2007-06-04 at 12:13 -0500, Carl Karsten wrote: > > >>Hmm, default style brings up a new issue: >>If there will be more than one style, should there be a common >> >> >default? > >No, the default will be implementation-defined for backwards >compatibility. There will be a common way of switching to the common >mandatory paramstyle. > > > >>>might want to convey which paramstyles the module can accept. I see >>> >>> >two > > >>>options: >>> >>>* Invent a new module attribute called paramstyles that is a list or >>> >>> >set > > >>>of supported paramstyles. >>>* Simply raise ValueError if the programmer requests an unsupported >>>parameter style. >>> >>> >>I kinda think it been decided that the set of styles will be required >> >> >of all > > >>modules. But given that it was never voted on, add it to the ballot. >> >> > >I think we're more leaning towards making only one common style >mandatory, and allowing other styles as optional extensions. > > > The collective may be leaning toward a more status quo with allowing other styles as extensions, BUT I for one, strongly endorse forbidding whatever is not required in the the way of parameter styles (or at the very least dropping any mention of any parameter style that is not required). I do not care much about how easy it is to implement. I care more about how easy it is for users of the interfaces to understand and be able to "transfer" that understanding to another interface. The users should be able to know how to get the parameter style that they want to use without having to switch the DB backend and interface module to get it. To me, that is the only reason for a shared API, to allow the users to code to common standards. If users can not code to a common standard with Python, they will code to a common standard with JAVA & JDBC. Thank you all, Art Protin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20070604/363c2ef6/attachment.html From carsten at uniqsys.com Mon Jun 4 20:20:32 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Mon, 04 Jun 2007 14:20:32 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <466453D4.5010601@research.att.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <46642C46.3020707@personnelware.com> <1180971277.3388.77.camel@dot.uniqsys.com> <46644850.2060003@personnelware.com> <1180977879.3388.99.camel@dot.uniqsys.com> <466453D4.5010601@research.att.com> Message-ID: <1180981232.3388.115.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 14:03 -0400, Art Protin wrote: > If users can not code to a common standard with Python, they will > code to a common > standard with JAVA & JDBC. Shoehorning all databases into one true interface may be fine for the Bondage And Domination world of Java, but here in the Python community we realize that no two database engines are the same. Cross-platform development and inter-API knowledge transfer are served by having one mandatory minimum paramstyle. Database-specific needs and optimizations are served by allowing optional extensions. IMHO this is the best of both worlds. -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Mon Jun 4 20:35:14 2007 From: carl at personnelware.com (Carl Karsten) Date: Mon, 04 Jun 2007 13:35:14 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <466453D4.5010601@research.att.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> <46642C46.3020707@personnelware.com> <1180971277.3388.77.camel@dot.uniqsys.com> <46644850.2060003@personnelware.com> <1180977879.3388.99.camel@dot.uniqsys.com> <466453D4.5010601@research.att.com> Message-ID: <46645B62.8070601@personnelware.com> Art Protin wrote: > Dear folks, > Carsten Haese wrote: > >> On Mon, 2007-06-04 at 12:13 -0500, Carl Karsten wrote: >> >> >>> Hmm, default style brings up a new issue: >>> If there will be more than one style, should there be a common >>> >> default? >> >> No, the default will be implementation-defined for backwards >> compatibility. There will be a common way of switching to the common >> mandatory paramstyle. >> >> >> >>>> might want to convey which paramstyles the module can accept. I see >>>> >> two >> >> >>>> options: >>>> >>>> * Invent a new module attribute called paramstyles that is a list or >>>> >> set >> >> >>>> of supported paramstyles. >>>> * Simply raise ValueError if the programmer requests an unsupported >>>> parameter style. >>>> >>> I kinda think it been decided that the set of styles will be required >>> >> of all >> >>> modules. But given that it was never voted on, add it to the ballot. >>> >> >> I think we're more leaning towards making only one common style >> mandatory, and allowing other styles as optional extensions. >> >> >> > The collective may be leaning toward a more status quo with allowing > other styles > as extensions, BUT I for one, strongly endorse forbidding whatever is > not required > in the the way of parameter styles (or at the very least dropping any > mention of any > parameter style that is not required). I do not care much about how > easy it is to > implement. I care more about how easy it is for users of the interfaces > to understand da! I have been meaning to bring that up: The module should make the users life easier. "because it will be easier for the module developer" should not be a reason for anything. Also, on the point of backwards computability - I think that can be addressed with a 'backwards compatibility module" that is a shim between legacy application code and a dbv3 module. It would basically be the v2 code that would use the v3 module. it would contain the who's use case for V3 is backwards computability. It would simplify V3 module code and use, and would help V2 applications use V3 modules. It would help channel new development of all code to the simpler V3. Carl K From mal at egenix.com Tue Jun 5 10:57:54 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 05 Jun 2007 10:57:54 +0200 Subject: [DB-SIG] trouble with list formatting In-Reply-To: References: Message-ID: <46652592.1050406@egenix.com> On 2007-06-04 16:46, Roger Doger wrote: > The following use of the API with MySQL works fine > >> items = [1, 2] >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) > resultant query string: SELECT * from table1 WHERE field1 IN ('1', '2') > > By contrast, the following yields the message "error in your SQL syntax" > >> items = [1] >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) > resulting query string: SELECT * from table1 WHERE field1 IN ('1',) > > Thus it seems the only way to manage this situation is to use this > stilted construction: > >> if len(items)==1: >> execute('SELECT * from table1 WHERE field1 IN (%s)', items[0]) >> else: >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) > > Is this just a shortcoming in how the API handles formatting of lists > or is there a cleaner way to handle this? Binding parameter are usually only meant to bind data items to an SQL statement, e.g. to provide data for INSERT, UPDATE, etc. Lists are usually not supported as datatype, only scalars. The fact that it works in the above case is merely a side effect of Python using a formatting that is similar to SQL. However, this won't always work, since Python uses double quotes to quote strings which include single quotes (the SQL string literal quoting character): >>> l = ["'a'", "b"] >>> print str(l) ["'a'", 'b'] PS: Yet another reason to avoid format param style :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 05 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From dieter at handshake.de Tue Jun 5 19:21:05 2007 From: dieter at handshake.de (Dieter Maurer) Date: Tue, 5 Jun 2007 19:21:05 +0200 Subject: [DB-SIG] trouble with list formatting In-Reply-To: References: Message-ID: <18021.39809.848761.153110@gargle.gargle.HOWL> Roger Doger wrote at 2007-6-4 09:46 -0500: >The following use of the API with MySQL works fine > >> items = [1, 2] >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) >resultant query string: SELECT * from table1 WHERE field1 IN ('1', '2') > >By contrast, the following yields the message "error in your SQL syntax" > >> items = [1] >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) >resulting query string: SELECT * from table1 WHERE field1 IN ('1',) A bug in your Python-database bridge. It should handle sequences correctly. -- Dieter From carsten at uniqsys.com Tue Jun 5 19:40:18 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Tue, 05 Jun 2007 13:40:18 -0400 Subject: [DB-SIG] trouble with list formatting In-Reply-To: <18021.39809.848761.153110@gargle.gargle.HOWL> References: <18021.39809.848761.153110@gargle.gargle.HOWL> Message-ID: <1181065218.3387.1.camel@dot.uniqsys.com> On Tue, 2007-06-05 at 19:21 +0200, Dieter Maurer wrote: > Roger Doger wrote at 2007-6-4 09:46 -0500: > >The following use of the API with MySQL works fine > > > >> items = [1, 2] > >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) > >resultant query string: SELECT * from table1 WHERE field1 IN ('1', '2') > > > >By contrast, the following yields the message "error in your SQL syntax" > > > >> items = [1] > >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) > >resulting query string: SELECT * from table1 WHERE field1 IN ('1',) > > A bug in your Python-database bridge. It should handle sequences correctly. Says who? As Marc-Andre pointed out, parameter binding is only for scalar values. -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Wed Jun 6 00:24:06 2007 From: carl at personnelware.com (Carl Karsten) Date: Tue, 05 Jun 2007 17:24:06 -0500 Subject: [DB-SIG] qmark is standard SQL? Message-ID: <4665E286.7080804@personnelware.com> Didn't someone here say that qmark was part of the some standard? If so, got a reference? I just skimmed and searched the first 380 pages of the SQL 92 spec: http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt and the only mention of parameters I found was: 4.18.2 Data parameters A data parameter is a parameter that is used to either assume or supply the value of data exchanged between a host program and an SQL-implementation. It may be on page 500, but I got tired of looking. Carl K From mal at egenix.com Wed Jun 6 09:42:04 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 06 Jun 2007 09:42:04 +0200 Subject: [DB-SIG] qmark is standard SQL? In-Reply-To: <4665E286.7080804@personnelware.com> References: <4665E286.7080804@personnelware.com> Message-ID: <4666654C.6080703@egenix.com> Carl Karsten wrote: > Didn't someone here say that qmark was part of the some standard? If so, got a > reference? It's part of the SQL CLI standard which again is the basis for ODBC. http://en.wikipedia.org/wiki/Call_Level_Interface > I just skimmed and searched the first 380 pages of the SQL 92 spec: > > http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt > > and the only mention of parameters I found was: > > 4.18.2 Data parameters A data parameter is a parameter that is used > to either assume or supply the value of data exchanged between a host > program and an SQL-implementation. > > > It may be on page 500, but I got tired of looking. I don't have a reference handy, sorry. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 06 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carsten at uniqsys.com Wed Jun 6 13:44:53 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 6 Jun 2007 07:44:53 -0400 Subject: [DB-SIG] qmark is standard SQL? In-Reply-To: <4665E286.7080804@personnelware.com> References: <4665E286.7080804@personnelware.com> Message-ID: <20070606114052.M20261@uniqsys.com> On Tue, 05 Jun 2007 17:24:06 -0500, Carl Karsten wrote > Didn't someone here say that qmark was part of the some standard? > If so, got a reference? > > I just skimmed and searched the first 380 pages of the SQL 92 spec: > > http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt You missed this on page 63: """ In SQL-statements that are executed dynamically, the parameters are called dynamic parameters (s) and are represented in SQL language by a (?). """ -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Wed Jun 6 16:32:05 2007 From: carl at personnelware.com (Carl Karsten) Date: Wed, 06 Jun 2007 09:32:05 -0500 Subject: [DB-SIG] qmark is standard SQL? In-Reply-To: <20070606114052.M20261@uniqsys.com> References: <4665E286.7080804@personnelware.com> <20070606114052.M20261@uniqsys.com> Message-ID: <4666C565.3090702@personnelware.com> Carsten Haese wrote: > On Tue, 05 Jun 2007 17:24:06 -0500, Carl Karsten wrote >> Didn't someone here say that qmark was part of the some standard? >> If so, got a reference? >> >> I just skimmed and searched the first 380 pages of the SQL 92 spec: >> >> http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt > > You missed this on page 63: > > """ > In SQL-statements that are executed > dynamically, the parameters are called dynamic parameters ( parameter specification>s) and are represented in SQL language by a > (?). > """ > oh jeez - why didn't I search for 'question' ?! Thanks. From dieter at handshake.de Wed Jun 6 20:55:11 2007 From: dieter at handshake.de (Dieter Maurer) Date: Wed, 6 Jun 2007 20:55:11 +0200 Subject: [DB-SIG] trouble with list formatting In-Reply-To: <1181065218.3387.1.camel@dot.uniqsys.com> References: <18021.39809.848761.153110@gargle.gargle.HOWL> <1181065218.3387.1.camel@dot.uniqsys.com> Message-ID: <18023.783.198863.60803@gargle.gargle.HOWL> Carsten Haese wrote at 2007-6-5 13:40 -0400: >On Tue, 2007-06-05 at 19:21 +0200, Dieter Maurer wrote: >> Roger Doger wrote at 2007-6-4 09:46 -0500: >> >The following use of the API with MySQL works fine >> > >> >> items = [1, 2] >> >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) >> >resultant query string: SELECT * from table1 WHERE field1 IN ('1', '2') >> > >> >By contrast, the following yields the message "error in your SQL syntax" >> > >> >> items = [1] >> >> execute('SELECT * from table1 WHERE field1 IN %s', (items,)) >> >resulting query string: SELECT * from table1 WHERE field1 IN ('1',) >> >> A bug in your Python-database bridge. It should handle sequences correctly. > >Says who? Me. >As Marc-Andre pointed out, parameter binding is only for >scalar values. Then it must raise an exception but not behave inconsistently (such as adding a ',' for one element lists but not adding a comma for lists with more than 1 element). -- Dieter From carsten at uniqsys.com Wed Jun 6 20:58:52 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 06 Jun 2007 14:58:52 -0400 Subject: [DB-SIG] trouble with list formatting In-Reply-To: <18023.783.198863.60803@gargle.gargle.HOWL> References: <18021.39809.848761.153110@gargle.gargle.HOWL> <1181065218.3387.1.camel@dot.uniqsys.com> <18023.783.198863.60803@gargle.gargle.HOWL> Message-ID: <1181156332.3362.17.camel@dot.uniqsys.com> On Wed, 2007-06-06 at 20:55 +0200, Dieter Maurer wrote: > >As Marc-Andre pointed out, parameter binding is only for > >scalar values. > > Then it must raise an exception but not behave inconsistently (such > as adding a ',' for one element lists but not adding a comma for lists > with more than 1 element). Please read Marc-Andre's response to the original poster. -- Carsten Haese http://informixdb.sourceforge.net From carsten at uniqsys.com Wed Jun 20 22:51:34 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 20 Jun 2007 16:51:34 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46642272.8090501@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <1180912137.3191.24.camel@localhost.localdomain> <4663D04A.3010103@egenix.com> <20070604110700.M68756@uniqsys.com> <46641455.4070803@egenix.com> <1180965621.3388.42.camel@dot.uniqsys.com> <46642272.8090501@egenix.com> Message-ID: <1182372694.3383.30.camel@dot.uniqsys.com> On Mon, 2007-06-04 at 16:32 +0200, M.-A. Lemburg wrote: > On 2007-06-04 16:00, Carsten Haese wrote: > >> So I'm: > >> > >> * +1 on making support one param style mandatory for all > >> implementations > > > > Any one? Let's make named mandatory, then. > > I intentionally left this open :-) > > We should have a vote on it. Let's gather votes for say two weeks > and then see what the outcome is. Well, the two weeks are up. If I counted correctly, six votes were cast and the totals are: qmark 3 named 4 numeric -1 format -5 pyformat -2 So the winner seems to be "named" unless six votes aren't enough to constitute consensus. -- Carsten Haese http://informixdb.sourceforge.net From paul at boddie.org.uk Wed Jun 20 23:15:54 2007 From: paul at boddie.org.uk (Paul Boddie) Date: Wed, 20 Jun 2007 23:15:54 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1182372694.3383.30.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> Message-ID: <200706202315.55192.paul@boddie.org.uk> On Wednesday 20 June 2007 22:51, Carsten Haese wrote: > > If I counted correctly, six votes were cast and the totals are: > > qmark 3 > named 4 > numeric -1 > format -5 > pyformat -2 > > So the winner seems to be "named" unless six votes aren't enough to > constitute consensus. Well, as was pointed out already, qmark is part of the SQL standard and should feature as part of the absolute minimum level of support for parameters in the DB-API. I've seen named and numeric in Oracle environments only, and I'm not convinced that in my last excursion into Oracle JDBC territory those paramtypes were even permitted, even though Oracle likes to break standards and conventions wherever it feels like it. Paul From mal at egenix.com Wed Jun 20 23:34:21 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 20 Jun 2007 23:34:21 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <200706202315.55192.paul@boddie.org.uk> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> Message-ID: <46799D5D.10109@egenix.com> On 2007-06-20 23:15, Paul Boddie wrote: > On Wednesday 20 June 2007 22:51, Carsten Haese wrote: >> If I counted correctly, six votes were cast and the totals are: >> >> qmark 3 >> named 4 >> numeric -1 >> format -5 >> pyformat -2 >> >> So the winner seems to be "named" unless six votes aren't enough to >> constitute consensus. > > Well, as was pointed out already, qmark is part of the SQL standard and should > feature as part of the absolute minimum level of support for parameters in > the DB-API. I've seen named and numeric in Oracle environments only, and I'm > not convinced that in my last excursion into Oracle JDBC territory those > paramtypes were even permitted, even though Oracle likes to break standards > and conventions wherever it feels like it. How about making both qmark and named the minimum required parameter styles ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 20 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2007-07-09: EuroPython 2007, Vilnius, Lithuania 18 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carl at personnelware.com Wed Jun 20 23:59:07 2007 From: carl at personnelware.com (Carl Karsten) Date: Wed, 20 Jun 2007 16:59:07 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46799D5D.10109@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> Message-ID: <4679A32B.60806@personnelware.com> M.-A. Lemburg wrote: > On 2007-06-20 23:15, Paul Boddie wrote: >> On Wednesday 20 June 2007 22:51, Carsten Haese wrote: >>> If I counted correctly, six votes were cast and the totals are: >>> >>> qmark 3 >>> named 4 >>> numeric -1 >>> format -5 >>> pyformat -2 >>> >>> So the winner seems to be "named" unless six votes aren't enough to >>> constitute consensus. >> Well, as was pointed out already, qmark is part of the SQL standard and should >> feature as part of the absolute minimum level of support for parameters in >> the DB-API. I've seen named and numeric in Oracle environments only, and I'm >> not convinced that in my last excursion into Oracle JDBC territory those >> paramtypes were even permitted, even though Oracle likes to break standards >> and conventions wherever it feels like it. > > How about making both qmark and named the minimum required > parameter styles ?! > Forget minimum. Make qmark and named requried. Make paramstyle something that is set/passed in as part of the init, and is ReadOnly after that. It should default to the same thing in all implementations. I think we can make it super simple, and it will be way better than the current implementation. For backwards compatibility, It should be pretty easy to wrap all the v3 modules in a v2 conversion layer that will add the extra 'features' like pyformat. Carl K From carsten at uniqsys.com Thu Jun 21 02:19:48 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 20 Jun 2007 20:19:48 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <46799D5D.10109@egenix.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> Message-ID: <1182385188.3256.2.camel@localhost.localdomain> On Wed, 2007-06-20 at 23:34 +0200, M.-A. Lemburg wrote: > On 2007-06-20 23:15, Paul Boddie wrote: > > On Wednesday 20 June 2007 22:51, Carsten Haese wrote: > >> If I counted correctly, six votes were cast and the totals are: > >> > >> qmark 3 > >> named 4 > >> numeric -1 > >> format -5 > >> pyformat -2 > >> > >> So the winner seems to be "named" unless six votes aren't enough to > >> constitute consensus. > > > > Well, as was pointed out already, qmark is part of the SQL standard and should > > feature as part of the absolute minimum level of support for parameters in > > the DB-API. I've seen named and numeric in Oracle environments only, and I'm > > not convinced that in my last excursion into Oracle JDBC territory those > > paramtypes were even permitted, even though Oracle likes to break standards > > and conventions wherever it feels like it. > > How about making both qmark and named the minimum required > parameter styles ?! Sounds good to me. -- Carsten Haese http://informixdb.sourceforge.net From carsten at uniqsys.com Thu Jun 21 02:35:41 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Wed, 20 Jun 2007 20:35:41 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <4679A32B.60806@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> Message-ID: <1182386141.3256.12.camel@localhost.localdomain> On Wed, 2007-06-20 at 16:59 -0500, Carl Karsten wrote: > M.-A. Lemburg wrote: > > How about making both qmark and named the minimum required > > parameter styles ?! > > > > Forget minimum. > Make qmark and named requried. > Make paramstyle something that is set/passed in as part of the init, and is > ReadOnly after that. It should default to the same thing in all implementations. > > I think we can make it super simple, and it will be way better than the current > implementation. > > For backwards compatibility, It should be pretty easy to wrap all the v3 > modules in a v2 conversion layer that will add the extra 'features' like pyformat. And how is bolting on a conversion layer "super simple" compared to not breaking backwards compatibility in the first place? -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Thu Jun 21 05:59:18 2007 From: carl at personnelware.com (Carl Karsten) Date: Wed, 20 Jun 2007 22:59:18 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1182386141.3256.12.camel@localhost.localdomain> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> Message-ID: <4679F796.80706@personnelware.com> Carsten Haese wrote: > On Wed, 2007-06-20 at 16:59 -0500, Carl Karsten wrote: >> M.-A. Lemburg wrote: >> > How about making both qmark and named the minimum required >> > parameter styles ?! >> > >> >> Forget minimum. >> Make qmark and named requried. >> Make paramstyle something that is set/passed in as part of the init, and is >> ReadOnly after that. It should default to the same thing in all implementations. >> >> I think we can make it super simple, and it will be way better than the current >> implementation. >> >> For backwards compatibility, It should be pretty easy to wrap all the v3 >> modules in a v2 conversion layer that will add the extra 'features' like pyformat. > > And how is bolting on a conversion layer "super simple" compared to not > breaking backwards compatibility in the first place? > "super simple" would be the new api. pretty easy is the wrapper for backwards computability. As complex as that set would be, it wouldn't be much worse than the current mess. "worse" would be a small price to pay for a much better API. Carl K From carsten at uniqsys.com Thu Jun 21 06:38:25 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 00:38:25 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <4679F796.80706@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> Message-ID: <20070621041545.M40830@uniqsys.com> On Wed, 20 Jun 2007 22:59:18 -0500, Carl Karsten wrote > "super simple" would be the new api. pretty easy is the wrapper > for backwards computability. > > As complex as that set would be, it wouldn't be much worse than the > current mess. That's why we're discussing an improvement. The current proposal on the table is this: * Every API implements at least qmark and named. * APIs are allowed to implement other styles. * Connection and Cursor objects have a writable paramstyle attribute to choose which paramstyle to use. The default is implementation-defined. Possible settings are implementation-defined but must include at least qmark and named. The advantages are clear: * Application code written against this spec can be sure that named/qmark styles are available, and there is a common API for choosing which one to use. That means less API dependent code, and the overhead for choosing qmark or named is minimal at one line of code per connection. * APIs aren't forced to abandon "legacy" parameter styles so existing code written for existing APIs won't break. * No backwards compatibility wrapper is necessary. You're welcome to try to come up with a simpler solution that has the same advantages. -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Thu Jun 21 14:31:28 2007 From: carl at personnelware.com (Carl Karsten) Date: Thu, 21 Jun 2007 07:31:28 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <20070621041545.M40830@uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> Message-ID: <467A6FA0.5060502@personnelware.com> Carsten Haese wrote: > On Wed, 20 Jun 2007 22:59:18 -0500, Carl Karsten wrote >> "super simple" would be the new api. pretty easy is the wrapper >> for backwards computability. >> >> As complex as that set would be, it wouldn't be much worse than the >> current mess. > > That's why we're discussing an improvement. The current proposal on the table > is this: > > * Every API implements at least qmark and named. > > * APIs are allowed to implement other styles. > > * Connection and Cursor objects have a writable paramstyle attribute to choose > which paramstyle to use. The default is implementation-defined. Possible > settings are implementation-defined but must include at least qmark and named. > > The advantages are clear: > > * Application code written against this spec can be sure that named/qmark > styles are available, and there is a common API for choosing which one to use. > That means less API dependent code, and the overhead for choosing qmark or > named is minimal at one line of code per connection. > > * APIs aren't forced to abandon "legacy" parameter styles so existing code > written for existing APIs won't break. > > * No backwards compatibility wrapper is necessary. I do not see the last 2 points as a clear advantage, given the cost. > > You're welcome to try to come up with a simpler solution that has the same > advantages. I did before: "only allow one style of parameter." But it got out voted, (or not even on the ballot) and so I won't push for it anymore. >>> numeric -1 >>> format -5 >>> pyformat -2 To me that reads "remove them." 0 would have been "tolerate them." However, I can see that if there was at least one parmstyle that was consistent across all APIs (like we are all in agreement on), no one in their right mind would use one of the non-standard ones. Carl K From carsten at uniqsys.com Thu Jun 21 14:51:31 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 08:51:31 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <467A6FA0.5060502@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> <467A6FA0.5060502@personnelware.com> Message-ID: <1182430291.3376.14.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 07:31 -0500, Carl Karsten wrote: > Carsten Haese wrote: > > The advantages are clear: > > > > * Application code written against this spec can be sure that named/qmark > > styles are available, and there is a common API for choosing which one to use. > > That means less API dependent code, and the overhead for choosing qmark or > > named is minimal at one line of code per connection. > > > > * APIs aren't forced to abandon "legacy" parameter styles so existing code > > written for existing APIs won't break. > > > > * No backwards compatibility wrapper is necessary. > > I do not see the last 2 points as a clear advantage, given the cost. What cost? The one line of code per connection to explicitly choose qmark or named style? And since when is compatibility with existing code not an advantage? > > > > You're welcome to try to come up with a simpler solution that has the same > > advantages. > > I did before: "only allow one style of parameter." But it got out voted, (or > not even on the ballot) and so I won't push for it anymore. > > >>> numeric -1 > >>> format -5 > >>> pyformat -2 > > To me that reads "remove them." 0 would have been "tolerate them." Then you misunderstood the point of the vote. We voted on which paramstyle should be required, not which paramstyles should exist at all. We could have a separate vote on which parameter styles to deprecate, and I bet a lot of people would want to remove format and pyformat. However, there is a lot of MySQL and PostgreSQL code out there that would break if those styles were removed outright. > However, I can see that if there was at least one parmstyle that was consistent > across all APIs (like we are all in agreement on), no one in their right mind > would use one of the non-standard ones. Indeed. The practical benefit comes from having common ground to stand on. There is no practical benefit in breaking backwards compatibility by forcing everybody onto this common ground. -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Thu Jun 21 16:14:28 2007 From: carl at personnelware.com (Carl Karsten) Date: Thu, 21 Jun 2007 09:14:28 -0500 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1182430291.3376.14.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> <467A6FA0.5060502@personnelware.com> <1182430291.3376.14.camel@dot.uniqsys.com> Message-ID: <467A87C4.8050600@personnelware.com> Carsten Haese wrote: > On Thu, 2007-06-21 at 07:31 -0500, Carl Karsten wrote: >> Carsten Haese wrote: >>> The advantages are clear: >>> >>> * Application code written against this spec can be sure that named/qmark >>> styles are available, and there is a common API for choosing which one to use. >>> That means less API dependent code, and the overhead for choosing qmark or >>> named is minimal at one line of code per connection. >>> >>> * APIs aren't forced to abandon "legacy" parameter styles so existing code >>> written for existing APIs won't break. >>> >>> * No backwards compatibility wrapper is necessary. >> I do not see the last 2 points as a clear advantage, given the cost. > > What cost? More code in the dbapi module. More complex API. More complex application code. Application code that uses non portable paramstyles, > The one line of code per connection to explicitly choose > qmark or named style? no. in fact I don't think that line of code is in the backwards compatibility category. backwards compatibility should use the dbapi v2 api, where paramstyle is read only. I envision changing "import MySQLdb" to "import MySQLdb_v2 as MySQLdb" > > And since when is compatibility with existing code not an advantage? > I said "clear advantage" because you did, and I disagree with the clearness. >>> You're welcome to try to come up with a simpler solution that has the same >>> advantages. >> I did before: "only allow one style of parameter." But it got out voted, (or >> not even on the ballot) and so I won't push for it anymore. >> >> >>> numeric -1 >> >>> format -5 >> >>> pyformat -2 >> >> To me that reads "remove them." 0 would have been "tolerate them." > > Then you misunderstood the point of the vote. We voted on which > paramstyle should be required, not which paramstyles should exist at > all. We could have a separate vote on which parameter styles to > deprecate, and I bet a lot of people would want to remove format and > pyformat. "it" was voted on and "it" was not really well defined. This is the closest thing I can find to what was voted on: >>> * +1 on making support one param style mandatory for all >>> > >> implementations >> > > >> > > Any one? Let's make named mandatory, then. > > > > I intentionally left this open :-) > > > > We should have a vote on it. Let's gather votes for say two weeks > > and then see what the outcome is. Given that, I think what was decided via the vote was "make named mandatory" but I would not be surprised to see 5 posts saying "I thought I was voting on Xn" Personally, even though I favor the counts (?/name positive, rest negative) I would like to see those votes discarded a more comprehensive, well defined round of voting. > However, there is a lot of MySQL and PostgreSQL code out there > that would break if those styles were removed outright. backwards compatibility wrapper. Or just name the new module something new. What is the down side to adding a dbapi version number to the module names? > >> However, I can see that if there was at least one parmstyle that was consistent >> across all APIs (like we are all in agreement on), no one in their right mind >> would use one of the non-standard ones. > > Indeed. The practical benefit comes from having common ground to stand > on. There is no practical benefit in breaking backwards compatibility by > forcing everybody onto this common ground. > "BUT I for one, strongly endorse forbidding whatever is not required in the the way of parameter styles (or at the very least dropping any mention of any parameter style that is not required)" Art Protin But his position may have changed, much like how strongly I feel about it. Carl K From carsten at uniqsys.com Thu Jun 21 17:01:22 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 11:01:22 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <467A87C4.8050600@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> <467A6FA0.5060502@personnelware.com> <1182430291.3376.14.camel@dot.uniqsys.com> <467A87C4.8050600@personnelware.com> Message-ID: <1182438082.3376.50.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 09:14 -0500, Carl Karsten wrote: > Carsten Haese wrote: > > On Thu, 2007-06-21 at 07:31 -0500, Carl Karsten wrote: > >> Carsten Haese wrote: > >>> The advantages are clear: > >>> > >>> * Application code written against this spec can be sure that named/qmark > >>> styles are available, and there is a common API for choosing which one to use. > >>> That means less API dependent code, and the overhead for choosing qmark or > >>> named is minimal at one line of code per connection. > >>> > >>> * APIs aren't forced to abandon "legacy" parameter styles so existing code > >>> written for existing APIs won't break. > >>> > >>> * No backwards compatibility wrapper is necessary. > >> I do not see the last 2 points as a clear advantage, given the cost. > > > > What cost? > > More code in the dbapi module. > > More complex API. > > More complex application code. Compared to what? What's your proposal? > Application code that uses non portable paramstyles, So what? Application code may also use non-portable SQL language. Python programmers are consenting adults, and if they want to write non-portable code, let them. > > The one line of code per connection to explicitly choose > > qmark or named style? > > no. in fact I don't think that line of code is in the backwards compatibility > category. backwards compatibility should use the dbapi v2 api, where paramstyle > is read only. I envision changing "import MySQLdb" to "import MySQLdb_v2 as > MySQLdb" I never said this line of code was for backwards compatibility. Under the proposal on the table, backwards compatibility is achieved by doing nothing to existing applications. Forward compatibility is achieved by explicitly stating which parameter style the application uses. That's that one line of code I'm talking about, and, true, it will have to be inserted into every new application, but your proposal requires changing every existing application, and somehow you call that backwards compatible. That's like saying diesel is compatible with a gasoline car as long as you install a diesel engine into the gasoline car. >[...] > "it" was voted on and "it" was not really well defined. This is the closest > thing I can find to what was voted on: > > > >>> * +1 on making support one param style mandatory for all > >>> > >> implementations > >> > > > >> > > Any one? Let's make named mandatory, then. > > > > > > I intentionally left this open :-) > > > > > > We should have a vote on it. Let's gather votes for say two weeks > > > and then see what the outcome is. > > Given that, I think what was decided via the vote was "make named mandatory" but > I would not be surprised to see 5 posts saying "I thought I was voting on Xn" The context made it clear to me that the "it" we're voting for was which paramstyle to make mandatory. > Personally, even though I favor the counts (?/name positive, rest negative) I > would like to see those votes discarded a more comprehensive, well defined round > of voting. Sure, let's keep on voting until you are happy with the outcome. > > However, there is a lot of MySQL and PostgreSQL code out there > > that would break if those styles were removed outright. > > backwards compatibility wrapper. And how does this backwards compatibility wrapper look? How does it work? How does it achieve backwards compatibility without requiring every existing application to be modified? > Or just name the new module something new. What is the down side to adding a > dbapi version number to the module names? Existing application code would automatically and gratuitously lose forward compatibility with future versions. > "BUT I for one, strongly endorse forbidding whatever is not required > in the the way of parameter styles (or at the very least dropping any mention of > any parameter style that is not required)" Art Protin > > But his position may have changed, much like how strongly I feel about it. It's one man's opinion, and he has a right to his opinion, but it is a very unpythonic opinion. -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Thu Jun 21 17:02:06 2007 From: carl at personnelware.com (Carl Karsten) Date: Thu, 21 Jun 2007 10:02:06 -0500 Subject: [DB-SIG] what is: backwards compatibility Message-ID: <467A92EE.5070207@personnelware.com> When we talk about backwards compatibility, I am wondering exactly what that means. I think my question is: should code that does not follow the dbapi2 spec be guaranteed to work? For instance, I know some of the dbapiv2 modules support more than one paramstyle, yet the spec really only allows the one defined by .paramstyle. So if application code uses some other paramstyle, does backwards compatibility mean the application code should still work? To me, the dbapi2 paramstyle 'spec' was so poorly defined that it is impossible to be compatible with it and implement some of the changes we are considering, regardless of how big the backwards compatibility umbrella is. I tried to write code once that made use of paramstyle: x='MySQLdb' dbapi=__import__(x) if dbapi.paramstyle= ... and gave up. Granted I was trying to work with existing application code that had methods like def getPKWhare(self,x): return "%s = %s" % (self.PKFieldName, self.SqlString(x)) The experience of working with code like that may have scared me into thinking the dbapi2 .paramstyle thing is worse than it really is, but others have agreed that it is pretty bad. I am wondering if there is really _any_ code out there that can survive paramstyle changes. I also wouldn't be surprised if 100% of the people who we are trying to 'help' would rather re-write their code to be simpler. Carl K From mal at egenix.com Thu Jun 21 17:17:34 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 21 Jun 2007 17:17:34 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <467A87C4.8050600@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> <467A6FA0.5060502@personnelware.com> <1182430291.3376.14.camel@dot.uniqsys.com> <467A87C4.8050600@personnelware.com> Message-ID: <467A968E.4020306@egenix.com> [Please calm down a bit.] On 2007-06-21 16:14, Carl Karsten wrote: > Carsten Haese wrote: >>>> >>>> * APIs aren't forced to abandon "legacy" parameter styles so existing code >>>> written for existing APIs won't break. >>>> >>>> * No backwards compatibility wrapper is necessary. >>> I do not see the last 2 points as a clear advantage, given the cost. >> What cost? > > More code in the dbapi module. > > More complex API. > > More complex application code. > > Application code that uses non portable paramstyles, The whole point of the voting was to determine which paramstyles people like most. To that extent we've determined qmark and named to be the paramstyles which won the vote. The vote makes a lot of sense: * qmark is a SQL standard and widely used in the SQL CLI world. * named is preferred by programmers who have to deal with lots of parameters. Now, the purpose of DB-API 3.0 is not to break existing code, only to make it possible to write new code in a more portable way. By requiring support for at least qmark and named, we make this possible. Being explicit about the choice of paramstyle in the application by setting it on the connection or cursor improves readability of the code. There's really no point in forcing breakage by insisting on dropping support for any other styles. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 21 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2007-07-09: EuroPython 2007, Vilnius, Lithuania 17 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carsten at uniqsys.com Thu Jun 21 17:37:55 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 11:37:55 -0400 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <467A92EE.5070207@personnelware.com> References: <467A92EE.5070207@personnelware.com> Message-ID: <1182440275.3376.71.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 10:02 -0500, Carl Karsten wrote: > When we talk about backwards compatibility, I am wondering exactly what that means. A module is backwards compatible if code using it won't break if the module is replaced with a newer version. > I think my question is: should code that does not follow the dbapi2 spec be > guaranteed to work? That's a completely different question. > For instance, I know some of the dbapiv2 modules support more than one > paramstyle, yet the spec really only allows the one defined by .paramstyle. So > if application code uses some other paramstyle, does backwards compatibility > mean the application code should still work? Yes. The module-wide paramstyle attribute only means "If you use this paramstyle, it's guaranteed to work." It doesn't mean "This is the only paramstyle that works." > To me, the dbapi2 paramstyle 'spec' was so poorly defined that it is impossible > to be compatible with it and implement some of the changes we are considering, > regardless of how big the backwards compatibility umbrella is. > > I tried to write code once that made use of paramstyle: > > x='MySQLdb' > dbapi=__import__(x) > if dbapi.paramstyle= ... > > and gave up. Well, the new version will allow you to pick either qmark or named and run with it. x = 'informixdb' dbapi=__import__(x) conn = dbapi.connect(dbname) conn.paramstyle = "named" # ... Without the explicit paramstyle assignment, the connection will just default to whatever the v2 implementation used so that applications remain compatible. > I am wondering if there is really _any_ code out there that can survive > paramstyle changes. If we make the change as proposed, and the modules implement the change correctly, ALL v2-based application code out there will survive the change. After all, that's what backwards compatibility means. -- Carsten Haese http://informixdb.sourceforge.net From anthony.tuininga at gmail.com Thu Jun 21 18:33:17 2007 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Thu, 21 Jun 2007 10:33:17 -0600 Subject: [DB-SIG] A slightly different paramstyle suggestion Message-ID: <703ae56b0706210933t79a0b99epf7cba3bfbcf419c5@mail.gmail.com> I've been reading the arguments going back and forth over parameter styles and I have a different suggestion which might help us out of this morass -- or perhaps provide additional fodder for the flame war that appears to be starting.... -) Define the method for cursor.execute() as follows: def execute(self, statement, *args, **kwargs): if args and kwargs: raise "Please use a sequence of parameters or a mapping of parameters, not both" if len(args) == 1: if isinstance(args[0], dict): kwargs = args[0] elif isinstance(args[0], (list, tuple)): args = args[0] if args: self._execute_with_sequence(statement, args) else: self._execute_with_mapping(statement, args) First, as an aside to this discussion, I would like to suggest adding the syntactic sugar (*args, *kwargs) in order to avoid having to build the list, tuple or dictionary when passing arguments to the execute method. For backwards compatibility you can check the list of parameters and if the length is one and the first argument is a list, tuple or dictionary change the arguments to the new style on the fly. I believe this is more convenient and improves the readability of the code when large numbers of parameters are passed. You would want to do something similar for setinputsizes() of course. Second, and more to the point, this recognizes that there are actually __two__ parameter styles that can possibly be in effect in a DB API module -- one that requires a sequence of parameters and another that requires a mapping -- and these can co-exist without any difficulty. In other words, the single parameter style that is defined by the module is inadequate and should be expanded to two at least (one for sequences and one for mappings). With this understanding in mind we can then mandate that qmark and named are the parameter styles that are used -- qmark for the sequence and named for the mapping. For those who don't care to implement one or the other they can raise a "NotImplemented" exception. I recognize that (for example) Oracle does not support qmark natively which means parsing the SQL and rewriting it to use the numeric parameter style which is native to Oracle. To avoid the overhead (and possible bugs) we may wish to add a boolean attribute to connections and cursors called "nativeparamstyle" which is defaulted to False. The cursor value would be "inherited" from the connection, of course. For cx_Oracle, this would mean that when nativeparamstyle is False, "qmark" and "named" are the parameter styles expected but when nativeparamstyle is True, "numeric" and "named" are the parameter styles expected. If I were lazy and didn't want to implement qmark, I would simply disallow the use of sequences when nativeparamstyle was False. :-) I believe this clarifies the preferred parameter styles for database independent work and allows for database dependent work where desired. I believe (someone correct me if I'm wrong) that each native database API uses one parameter style for sequences and one parameter style for named parameters, with the possibility that one or the other is not actually supported. Thus there is no need for the "paramstyle" module attribute. The DB API will indicate what the parameter styles are by default (qmark, named) and the DB API module documentation can indicate what the native parameter styles are if the user cares to go that way. Comments? Anthony From anthony.tuininga at gmail.com Thu Jun 21 18:41:44 2007 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Thu, 21 Jun 2007 10:41:44 -0600 Subject: [DB-SIG] DB API extension suggestion Message-ID: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> I have been recently playing with context managers and the new "with" statement added in Python 2.5. I would like to suggest the addition of the following methods to connections: def __enter__(self): return self def __exit__(self, excType, excValue, excTraceback): if excType is None and excValue is None and excTraceback is None: self.commit() else: self.rollback() This allows the following code: from __future__ import with_statement connection = ..... with connection: cursor = connection.cursor() cursor.execute("update SomeTable set SomeColumn = "SomeValue") cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) rather than try: cursor = connection.cursor() cursor.execute("update SomeTable set SomeColumn = "SomeValue") cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) connection.commit() except: connection.rollback() I've implemented this in cx_Oracle and it appears to work quite nicely. Thoughts? Anthony From carl at personnelware.com Thu Jun 21 18:51:44 2007 From: carl at personnelware.com (Carl Karsten) Date: Thu, 21 Jun 2007 11:51:44 -0500 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <1182440275.3376.71.camel@dot.uniqsys.com> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> Message-ID: <467AACA0.8060606@personnelware.com> Carsten Haese wrote: > On Thu, 2007-06-21 at 10:02 -0500, Carl Karsten wrote: >> When we talk about backwards compatibility, I am wondering exactly what that means. > > A module is backwards compatible if code using it won't break if the > module is replaced with a newer version. Even if it relies on implementation details? If so, then I don't think "backwards compatibility" is something worth bothering with, and we need to figure out a term for what we do want to be bothered with. > >> I think my question is: should code that does not follow the dbapi2 spec be >> guaranteed to work? > > That's a completely different question. Given your above definition of backwards compatible, it seems to be the same. How is it different? How would you answer it? > >> For instance, I know some of the dbapiv2 modules support more than one >> paramstyle, yet the spec really only allows the one defined by .paramstyle. So >> if application code uses some other paramstyle, does backwards compatibility >> mean the application code should still work? > > Yes. How would the application and/or developer know what other styles will work? > The module-wide paramstyle attribute only means "If you use this > paramstyle, it's guaranteed to work." It doesn't mean "This is the only > paramstyle that works." paramstyle String constant stating the type of parameter marker formatting expected by the interface. To me "expected by the interface" means "use this." I will admit that one line does not define it well, but that fuzzyness makes it hard if not impossible to guarantee compatibility. guarantee would mean "compatible with anyones interpretation.' I considered "reasonable interpretation' but realized that hinges on someone defining reasonable, which does not seem to fit with "guarantee." I am pretty sure in the end, reasonable compromises will have to be made, which implies "maybe we are breaking compatibility." > >> To me, the dbapi2 paramstyle 'spec' was so poorly defined that it is impossible >> to be compatible with it and implement some of the changes we are considering, >> regardless of how big the backwards compatibility umbrella is. >> >> I tried to write code once that made use of paramstyle: >> >> x='MySQLdb' >> dbapi=__import__(x) >> if dbapi.paramstyle= ... >> >> and gave up. > > Well, the new version will allow you to pick either qmark or named and > run with it. > > x = 'informixdb' > dbapi=__import__(x) > conn = dbapi.connect(dbname) > conn.paramstyle = "named" > # ... > > Without the explicit paramstyle assignment, the connection will just > default to whatever the v2 implementation used so that applications > remain compatible. Where does it say a modules paramstyle could not be changed across revisions? I am starting to think paramstyle has been used like documentation, and there is something just wrong about that. I do not think that code that relies on paramstyle being a certain value should be expected to work. > >> I am wondering if there is really _any_ code out there that can survive >> paramstyle changes. > > If we make the change as proposed, Can you tell me exactly what is proposed? I'm not sure the various posts to the list can be cited as a proposal. I really think we need to make use of some wiki pages. Or maybe I am the only one that thinks things are far from crystal clear. > and the modules implement the change > correctly, ALL v2-based application code out there will survive the > change. After all, that's what backwards compatibility means. > My point was some definitions of backwards compatibility may not be a worthwhile goal. Carl K From fumanchu at amor.org Thu Jun 21 19:20:49 2007 From: fumanchu at amor.org (Robert Brewer) Date: Thu, 21 Jun 2007 10:20:49 -0700 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> Message-ID: <435DF58A933BA74397B42CDEB8145A860D16318F@ex9.hostedexchange.local> Anthony Tuininga wrote: > I have been recently playing with context managers and the new "with" > statement added in Python 2.5. I would like to suggest the addition of > the following methods to connections: > > def __enter__(self): > return self > > def __exit__(self, excType, excValue, excTraceback): > if excType is None and excValue is None and excTraceback is None: > self.commit() > else: > self.rollback() > > This allows the following code: > > from __future__ import with_statement > > connection = ..... > with connection: > cursor = connection.cursor() > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > cursor.execute("delete from SomeOtherTable where > SomeOtherColumn = 5) > > > rather than > > try: > cursor = connection.cursor() > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > cursor.execute("delete from SomeOtherTable where > SomeOtherColumn = 5) > connection.commit() > except: > connection.rollback() > > I've implemented this in cx_Oracle and it appears to work quite > nicely. Thoughts? I've been doing this in Dejavu and it works just fine [1]. Granted, I'm doing it in a 'sandbox' object (kind of like SQLAlchemy's UnitOfWork), but the connection object is fine for the DB-API. However, it's easy to write a context manager wrapper, too. So if the API doesn't grow this natively (out of scope due to API design constraints, or maybe implementers just don't want to support it), it should be easy to do outside of the API. Robert Brewer System Architect Amor Ministries fumanchu at amor.org [1] http://projects.amor.org/dejavu/browser/trunk/sandboxes.py#L541 From mal at egenix.com Thu Jun 21 19:40:54 2007 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 21 Jun 2007 19:40:54 +0200 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> Message-ID: <467AB826.4040104@egenix.com> On 2007-06-21 18:41, Anthony Tuininga wrote: > I have been recently playing with context managers and the new "with" > statement added in Python 2.5. I would like to suggest the addition of > the following methods to connections: > > def __enter__(self): > return self > > def __exit__(self, excType, excValue, excTraceback): > if excType is None and excValue is None and excTraceback is None: > self.commit() > else: > self.rollback() Do you really think this is such a good idea ? You normally want to catch the exception and then act on it rather than just rollback the transaction and these actions usually depend on what the application is doing. Likewise, you typically want to be explicit about a commit on a connection. OTOH, a programmer could simply choose not to use "with" to implement a different coding strategy. > This allows the following code: > > from __future__ import with_statement > > connection = ..... > with connection: > cursor = connection.cursor() > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) > > rather than > > try: > cursor = connection.cursor() > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) > connection.commit() > except: > connection.rollback() > > I've implemented this in cx_Oracle and it appears to work quite > nicely. Thoughts? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 21 2007) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2007-07-09: EuroPython 2007, Vilnius, Lithuania 17 days to go :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From carsten at uniqsys.com Thu Jun 21 19:53:27 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 13:53:27 -0400 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <467AACA0.8060606@personnelware.com> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> <467AACA0.8060606@personnelware.com> Message-ID: <1182448407.3376.100.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 11:51 -0500, Carl Karsten wrote: > Carsten Haese wrote: > > On Thu, 2007-06-21 at 10:02 -0500, Carl Karsten wrote: > >> When we talk about backwards compatibility, I am wondering exactly what that means. > > > > A module is backwards compatible if code using it won't break if the > > module is replaced with a newer version. > > Even if it relies on implementation details? Yes, if those details are documented. Otherwise, no. > >> I think my question is: should code that does not follow the dbapi2 spec be > >> guaranteed to work? > > > > That's a completely different question. > > Given your above definition of backwards compatible, it seems to be the same. > > How is it different? > > How would you answer it? The question is incomplete. Code that does not follow the spec may or may not be guaranteed to work depending on whether it uses implementation details that are documented. For example, InformixDB allows the programmer to freely mix parameter styles. The spec doesn't say that this is allowed, and it doesn't say that it isn't allowed. The feature is outside the spec, but the documentation for InformixDB says that it is allowed, and programmers using the module have a reasonable expectation that it will continue to be allowed in later versions of InformixDB. > > > >> For instance, I know some of the dbapiv2 modules support more than one > >> paramstyle, yet the spec really only allows the one defined by .paramstyle. So > >> if application code uses some other paramstyle, does backwards compatibility > >> mean the application code should still work? > > > > Yes. > > How would the application and/or developer know what other styles will work? The application wouldn't need to know this, and the developer would know this at least from the module documentation. We may decide on introducing another module attribute called paramstyles or whatever that lists the allowed styles. > > The module-wide paramstyle attribute only means "If you use this > > paramstyle, it's guaranteed to work." It doesn't mean "This is the only > > paramstyle that works." > > paramstyle > String constant stating the type of parameter marker > formatting expected by the interface. > > To me "expected by the interface" means "use this." Exactly. It doesn't say "use ONLY this." > I will admit that one line > does not define it well, but that fuzzyness makes it hard if not impossible to > guarantee compatibility. guarantee would mean "compatible with anyones > interpretation.' Well, if your interpretation is "I MUST use this style", you're not going wrong. If you use the style that's indicated, it's guaranteed to work. Whether other styles are accepted is implementation defined. > I considered "reasonable interpretation' but realized that > hinges on someone defining reasonable, which does not seem to fit with > "guarantee." I am pretty sure in the end, reasonable compromises will have to > be made, which implies "maybe we are breaking compatibility." We have reached a compromise, and it doesn't involve breaking backwards compatibility. Backwards compatibility should only be broken if there are very convincing reasons to do so. > >> To me, the dbapi2 paramstyle 'spec' was so poorly defined that it is impossible > >> to be compatible with it and implement some of the changes we are considering, > >> regardless of how big the backwards compatibility umbrella is. > >> > >> I tried to write code once that made use of paramstyle: > >> > >> x='MySQLdb' > >> dbapi=__import__(x) > >> if dbapi.paramstyle= ... > >> > >> and gave up. > > > > Well, the new version will allow you to pick either qmark or named and > > run with it. > > > > x = 'informixdb' > > dbapi=__import__(x) > > conn = dbapi.connect(dbname) > > conn.paramstyle = "named" > > # ... > > > > Without the explicit paramstyle assignment, the connection will just > > default to whatever the v2 implementation used so that applications > > remain compatible. > > Where does it say a modules paramstyle could not be changed across revisions? It doesn't, but programmers have a reasonable expectation that their code works from one version of a module to the next. > I am starting to think paramstyle has been used like documentation, and there is > something just wrong about that. In database-specific code, paramstyle isn't used at all. The programmer just knows what parameter style to use. Database-independent code needs to inspect paramstyle and build queries to fit. > > I do not think that code that relies on paramstyle being a certain value should > be expected to work. I have never seen code that relies on paramstyle being one certain value, so you're talking about an empty set. I agree with the statement in so far as I also agree that flying pigs shouldn't eat cheese. > > > > >> I am wondering if there is really _any_ code out there that can survive > >> paramstyle changes. > > > > If we make the change as proposed, > > Can you tell me exactly what is proposed? I'm not sure the various posts to the > list can be cited as a proposal. I really think we need to make use of some > wiki pages. Or maybe I am the only one that thinks things are far from crystal > clear. Please allow me to repeat verbatim from a post I made earlier today: """ The current proposal on the table is this: * Every API implements at least qmark and named. * APIs are allowed to implement other styles. * Connection and Cursor objects have a writable paramstyle attribute to choose which paramstyle to use. The default is implementation-defined. Possible settings are implementation-defined but must include at least qmark and named. """ You responded to it, so I assumed that you have read it. > > and the modules implement the change > > correctly, ALL v2-based application code out there will survive the > > change. After all, that's what backwards compatibility means. > > > > My point was some definitions of backwards compatibility may not be a worthwhile > goal. Applications not breaking randomly when a module is upgraded is a worthwhile goal. I don't know what other definition of backwards compatibility you're using that makes backwards compatibility not a worthwhile goal. -- Carsten Haese http://informixdb.sourceforge.net From carsten at uniqsys.com Thu Jun 21 21:24:44 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 15:24:44 -0400 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> Message-ID: <1182453884.3376.123.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 10:41 -0600, Anthony Tuininga wrote: > I have been recently playing with context managers and the new "with" > statement added in Python 2.5. I would like to suggest the addition of > the following methods to connections: > > def __enter__(self): > return self > > def __exit__(self, excType, excValue, excTraceback): > if excType is None and excValue is None and excTraceback is None: > self.commit() > else: > self.rollback() > > This allows the following code: > > from __future__ import with_statement > > connection = ..... > with connection: > cursor = connection.cursor() > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) > > > rather than > > try: > cursor = connection.cursor() > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) > connection.commit() > except: > connection.rollback() > > I've implemented this in cx_Oracle and it appears to work quite > nicely. Thoughts? It's a good idea, but there is a conflicting use case "in the wild" for having a connection behave as a context manager. InformixDB allows you to use connections and cursors as context managers that close themselves, analogous to what file objects do: with api.connect(dbname) as conn: with conn.cursor() as cur: cur.execute(stmt) To disambiguate these use cases and to improve code-readability, I think a context manager that encapsulates a transaction should have a name that makes this clear. Something like this: with conn.transaction(): do_stuff() I think we'd also have to discuss the semantics of how this with-managed transaction coexists with the transaction that is already active on the connection. Should that transaction be committed, rolled back, or continued? If the engine underneath supports nested transactions, should the with-statement's transaction enter a nested transaction? Maybe the transaction() function should grow a parameter for specifying this. Regards, -- Carsten Haese http://informixdb.sourceforge.net From carl at personnelware.com Thu Jun 21 21:30:05 2007 From: carl at personnelware.com (Carl Karsten) Date: Thu, 21 Jun 2007 14:30:05 -0500 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <1182448407.3376.100.camel@dot.uniqsys.com> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> <467AACA0.8060606@personnelware.com> <1182448407.3376.100.camel@dot.uniqsys.com> Message-ID: <467AD1BD.9060508@personnelware.com> Carsten Haese wrote: > On Thu, 2007-06-21 at 11:51 -0500, Carl Karsten wrote: >> Carsten Haese wrote: >>> On Thu, 2007-06-21 at 10:02 -0500, Carl Karsten wrote: >>>> When we talk about backwards compatibility, I am wondering exactly what that means. >>> A module is backwards compatible if code using it won't break if the >>> module is replaced with a newer version. >> Even if it relies on implementation details? > > Yes, if those details are documented. Otherwise, no. I totally disagree, and have a feeling this needs to be voted on before either of us can continue this discussion. I propose that pep 249 is the only document that we should be considering. What happens if docs and pep 249 conflict? > >>>> I think my question is: should code that does not follow the dbapi2 spec be >>>> guaranteed to work? >>> That's a completely different question. >> Given your above definition of backwards compatible, it seems to be the same. >> >> How is it different? >> >> How would you answer it? > > The question is incomplete. > Code that does not follow the spec may or > may not be guaranteed to work depending on whether it uses > implementation details that are documented. That seems inconsistent with your definition of backwards compatible. "won't break" is not "may or may not be guaranteed to work" I am glad that you are willing to let some code "not be guaranteed to work" or am I misunderstanding? > > For example, InformixDB allows the programmer to freely mix parameter > styles. The spec doesn't say that this is allowed, and it doesn't say > that it isn't allowed. The feature is outside the spec, but the > documentation for InformixDB says that it is allowed, and programmers > using the module have a reasonable expectation that it will continue to > be allowed in later versions of InformixDB. > >>>> For instance, I know some of the dbapiv2 modules support more than one >>>> paramstyle, yet the spec really only allows the one defined by .paramstyle. So >>>> if application code uses some other paramstyle, does backwards compatibility >>>> mean the application code should still work? >>> Yes. >> How would the application and/or developer know what other styles will work? > > The application wouldn't need to know this, and the developer would know > this at least from the module documentation. We may decide on > introducing another module attribute called paramstyles or whatever that > lists the allowed styles. > Do you consider the module source code part of the documentation? >>> The module-wide paramstyle attribute only means "If you use this >>> paramstyle, it's guaranteed to work." It doesn't mean "This is the only >>> paramstyle that works." >> paramstyle >> String constant stating the type of parameter marker >> formatting expected by the interface. >> >> To me "expected by the interface" means "use this." > > Exactly. It doesn't say "use ONLY this." It doesn't need to. "use this" does not include the option of "use something else." > >> I will admit that one line >> does not define it well, but that fuzzyness makes it hard if not impossible to >> guarantee compatibility. guarantee would mean "compatible with anyones >> interpretation.' > > Well, if your interpretation is "I MUST use this style", you're not > going wrong. If you use the style that's indicated, it's guaranteed to > work. Whether other styles are accepted is implementation defined. > My single interpretation isn't whats important. The super set of possible interpretations is what needs to be considered when trying to maintain compatibility. My point is that the pep, and probably some of the docs you want to rely on, isn't crystal clear, so compatibility may be impossible. >> I considered "reasonable interpretation' but realized that >> hinges on someone defining reasonable, which does not seem to fit with >> "guarantee." I am pretty sure in the end, reasonable compromises will have to >> be made, which implies "maybe we are breaking compatibility." > > We have reached a compromise, and it doesn't involve breaking backwards > compatibility. That doesn't mean we can't compromise more. > Backwards compatibility should only be broken if there > are very convincing reasons to do so. I believe there are very convincing reasons to do so. >>>> To me, the dbapi2 paramstyle 'spec' was so poorly defined that it is impossible >>>> to be compatible with it and implement some of the changes we are considering, >>>> regardless of how big the backwards compatibility umbrella is. >>>> >>>> I tried to write code once that made use of paramstyle: >>>> >>>> x='MySQLdb' >>>> dbapi=__import__(x) >>>> if dbapi.paramstyle= ... >>>> >>>> and gave up. >>> Well, the new version will allow you to pick either qmark or named and >>> run with it. >>> >>> x = 'informixdb' >>> dbapi=__import__(x) >>> conn = dbapi.connect(dbname) >>> conn.paramstyle = "named" >>> # ... >>> >>> Without the explicit paramstyle assignment, the connection will just >>> default to whatever the v2 implementation used so that applications >>> remain compatible. >> Where does it say a modules paramstyle could not be changed across revisions? > > It doesn't, but programmers have a reasonable expectation that their > code works from one version of a module to the next. Only if they follow the spec. I bet you won't find anything that says a programmer can rely on paramstyle not changing. Which means all code, even db specific code, should be using paramstyle when creating parameters, and should be able to function using any of the 5 v2 values. > >> I am starting to think paramstyle has been used like documentation, and there is >> something just wrong about that. > > In database-specific code, paramstyle isn't used at all. The programmer > just knows what parameter style to use. Database-independent code needs > to inspect paramstyle and build queries to fit. > I am pretty sure we should not try to rely on what "The programmer just knows" You may want to reword that using words like pep 249 and "look at the modules source code." >> I do not think that code that relies on paramstyle being a certain value should >> be expected to work. > > I have never seen code that relies on paramstyle being one certain > value, so you're talking about an empty set. I agree with the statement > in so far as I also agree that flying pigs shouldn't eat cheese. > I think you mis understood what I meant. >>> import MySQLdb >>> MySQLdb.paramstyle 'format' What if in the next version of MySQLdb stopped using 'format' and on used 'qmark'? I am sure lots of code would fail. I know lots of my code would fail. Personally, I think the pep allows it to change. >>>> I am wondering if there is really _any_ code out there that can survive >>>> paramstyle changes. >>> If we make the change as proposed, >> Can you tell me exactly what is proposed? I'm not sure the various posts to the >> list can be cited as a proposal. I really think we need to make use of some >> wiki pages. Or maybe I am the only one that thinks things are far from crystal >> clear. > > Please allow me to repeat verbatim from a post I made earlier today: > > """ > The current proposal on the table is this: > > * Every API implements at least qmark and named. > > * APIs are allowed to implement other styles. > > * Connection and Cursor objects have a writable paramstyle attribute to > choose > which paramstyle to use. The default is implementation-defined. Possible > settings are implementation-defined but must include at least qmark and > named. > """ > > You responded to it, so I assumed that you have read it. > >>> and the modules implement the change >>> correctly, ALL v2-based application code out there will survive the >>> change. After all, that's what backwards compatibility means. I can write code that will work today and break under your proposal: .paramstyle='named' .execute( "select * from foo where fid = ?", (1,) ) I have no desire for code like this to continue working. Do you? Admittedly, this probably does not exist, but I think it is enough to show that "ALL v2-based application code out there" needs to be better defined. Unless you somehow think the proposal needs to be updated to somehow allow this. >>> >> My point was some definitions of backwards compatibility may not be a worthwhile >> goal. > > Applications not breaking randomly when a module is upgraded is a > worthwhile goal. I don't know what other definition of backwards > compatibility you're using that makes backwards compatibility not a > worthwhile goal. > Backwards compatibility: Applications that follow the official specs don't break. Pretty sure Pep 249 is the only spec I would consider official, but I am open to suggestions. Carl K From carsten at uniqsys.com Thu Jun 21 21:37:03 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 15:37:03 -0400 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <467AD1BD.9060508@personnelware.com> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> <467AACA0.8060606@personnelware.com> <1182448407.3376.100.camel@dot.uniqsys.com> <467AD1BD.9060508@personnelware.com> Message-ID: <1182454623.3376.130.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 14:30 -0500, Carl Karsten wrote: > Carsten Haese wrote: > > On Thu, 2007-06-21 at 11:51 -0500, Carl Karsten wrote: > >> Carsten Haese wrote: > >>> On Thu, 2007-06-21 at 10:02 -0500, Carl Karsten wrote: > >>>> When we talk about backwards compatibility, I am wondering exactly what > that means. > >>> A module is backwards compatible if code using it won't break if the > >>> module is replaced with a newer version. > >> Even if it relies on implementation details? > > > > Yes, if those details are documented. Otherwise, no. > > I totally disagree, and have a feeling this needs to be voted on before either > of us can continue this discussion. Fine. Let's vote. Please define what we're voting on. -- Carsten Haese http://informixdb.sourceforge.net From mkleehammer at gmail.com Thu Jun 21 21:43:56 2007 From: mkleehammer at gmail.com (Michael Kleehammer) Date: Thu, 21 Jun 2007 14:43:56 -0500 Subject: [DB-SIG] A slightly different paramstyle suggestion Message-ID: I like the idea of using *args and **kwargs a lot. Not having to build a sequence is nice and is an option I already provide in pyodbc: cursor.execute("insert into t(?,?)", a, b) Using the two items to differentiate between qmark and named seems like a simple idea and does not require connection or cursor variables. I also like the suggestion of *not* making either one required. ODBC drivers (and ODBC libraries themselves) already parse and rebuild SQL to replace ? with whatever is native. 1) I am very leery of modifying application SQL. 2) pyodbc is C/C++; if the community wants to provide SQL rewriting code to switch formats, I would want it in C/C++. I've seen it mentioned multiple times that it would be easy and I'm guessing the assumption is it would always be in Python. +1 for reducing the param styles to ? and named +1 for using *args and **kwargs to differentiate between them. -1 on requiring both (not yet, at least) Michael Kleehammer pyodbc maintainer http://pyodbc.sourceforge.net From mkleehammer at gmail.com Thu Jun 21 21:48:35 2007 From: mkleehammer at gmail.com (Michael Kleehammer) Date: Thu, 21 Jun 2007 14:48:35 -0500 Subject: [DB-SIG] DB API extension suggestion Message-ID: I'm not sure if there is a need for context support at the moment. Almost all databases automatically rollback transactions if they are not committed before a connection is closed. In the pyodbc connection finalizer, I automatically issue a rollback if a commit didn't occur. Since CPython uses reference counting (and will for the foreseeable future), this allows you to just use the following: def function(): cnxn = get_connection() cursor = connection.cursor() cursor.execute("update ...") cursor.execute("delete from ...") cursor.commit() If an exception is raised before the commit, everything is rolled back. +0 Michael Kleehammer pyodbc maintainer http://pyodbc.sourceforge.net From anthony.tuininga at gmail.com Thu Jun 21 22:26:49 2007 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Thu, 21 Jun 2007 14:26:49 -0600 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <1182453884.3376.123.camel@dot.uniqsys.com> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> <1182453884.3376.123.camel@dot.uniqsys.com> Message-ID: <703ae56b0706211326w416f1f3ejd9603cfe368818e8@mail.gmail.com> On 6/21/07, Carsten Haese wrote: > On Thu, 2007-06-21 at 10:41 -0600, Anthony Tuininga wrote: > > I have been recently playing with context managers and the new "with" > > statement added in Python 2.5. I would like to suggest the addition of > > the following methods to connections: > > > > def __enter__(self): > > return self > > > > def __exit__(self, excType, excValue, excTraceback): > > if excType is None and excValue is None and excTraceback is None: > > self.commit() > > else: > > self.rollback() > > > > This allows the following code: > > > > from __future__ import with_statement > > > > connection = ..... > > with connection: > > cursor = connection.cursor() > > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > > cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) > > > > > > rather than > > > > try: > > cursor = connection.cursor() > > cursor.execute("update SomeTable set SomeColumn = "SomeValue") > > cursor.execute("delete from SomeOtherTable where SomeOtherColumn = 5) > > connection.commit() > > except: > > connection.rollback() > > > > I've implemented this in cx_Oracle and it appears to work quite > > nicely. Thoughts? > > It's a good idea, but there is a conflicting use case "in the wild" for > having a connection behave as a context manager. InformixDB allows you > to use connections and cursors as context managers that close > themselves, analogous to what file objects do: > > with api.connect(dbname) as conn: > with conn.cursor() as cur: > cur.execute(stmt) That's interesting but is it of much use? Connections and cursors close themselves when they go out of scope. I rarely use cursor.close() or connection.close() for that reason. > To disambiguate these use cases and to improve code-readability, I think > a context manager that encapsulates a transaction should have a name > that makes this clear. Something like this: > > with conn.transaction(): > do_stuff() > > I think we'd also have to discuss the semantics of how this with-managed > transaction coexists with the transaction that is already active on the > connection. Should that transaction be committed, rolled back, or > continued? If the engine underneath supports nested transactions, should > the with-statement's transaction enter a nested transaction? > > Maybe the transaction() function should grow a parameter for specifying > this. Good question on the nested transaction stuff. That might prove useful -- but its also considerably more complicated than what I was suggesting.... :-) > Regards, > > -- > Carsten Haese > http://informixdb.sourceforge.net > > > From cito at online.de Thu Jun 21 22:57:49 2007 From: cito at online.de (Christoph Zwerschke) Date: Thu, 21 Jun 2007 22:57:49 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <467A6FA0.5060502@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> <467A6FA0.5060502@personnelware.com> Message-ID: <467AE64D.7020607@online.de> Carl Karsten schrieb: > >>> numeric -1 > >>> format -5 > >>> pyformat -2 > > To me that reads "remove them." 0 would have been "tolerate them." It seems the "format" and "pyformat" syntax will be abandoned in Python 3.0 anyway (in favor of yet another syntax) so that would be another reason to get rid of them in DB API v3 as well. see http://www.python.org/dev/peps/pep-3101/ and http://www.artima.com/weblogs/viewpost.jsp?thread=208549 under "Printing and Formatting" -- Christoph From carsten at uniqsys.com Thu Jun 21 23:06:00 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 17:06:00 -0400 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <703ae56b0706211326w416f1f3ejd9603cfe368818e8@mail.gmail.com> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> <1182453884.3376.123.camel@dot.uniqsys.com> <703ae56b0706211326w416f1f3ejd9603cfe368818e8@mail.gmail.com> Message-ID: <1182459960.3376.162.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 14:26 -0600, Anthony Tuininga wrote: > On 6/21/07, Carsten Haese wrote: > > with api.connect(dbname) as conn: > > with conn.cursor() as cur: > > cur.execute(stmt) > > That's interesting but is it of much use? Connections and cursors > close themselves when they go out of scope. I rarely use > cursor.close() or connection.close() for that reason. True, but you're at the mercy of when and how that happens. Sometimes you need to be certain that the resources are deallocated exactly when you need them to be deallocated. I don't know if any of my users make use of this with-block feature, but it's out there and documented, so I can't just take it away or replace the semantics of what happens when a connection object controls a with block. For what it's worth, I brought up the above use case back in September 2006, and the response was somewhere between indifferent and positive. Nobody said it was a terrible idea, so I implemented it. Is your transaction-controlling implementation of connection.__enter__/__exit__ already released, i.e. is it too late to get you to change the API to this feature? > Good question on the nested transaction stuff. That might prove useful > -- but its also considerably more complicated than what I was > suggesting.... :-) Of course, but it couldn't hurt to think about it for future extensibility. Then again, it was mostly an academic questions since Informix doesn't support nested transactions ;) -- Carsten Haese http://informixdb.sourceforge.net From anthony.tuininga at gmail.com Fri Jun 22 00:39:01 2007 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Thu, 21 Jun 2007 16:39:01 -0600 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <1182459960.3376.162.camel@dot.uniqsys.com> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> <1182453884.3376.123.camel@dot.uniqsys.com> <703ae56b0706211326w416f1f3ejd9603cfe368818e8@mail.gmail.com> <1182459960.3376.162.camel@dot.uniqsys.com> Message-ID: <703ae56b0706211539j530f220are8e3667a963424d2@mail.gmail.com> On 6/21/07, Carsten Haese wrote: > On Thu, 2007-06-21 at 14:26 -0600, Anthony Tuininga wrote: > > On 6/21/07, Carsten Haese wrote: > > > with api.connect(dbname) as conn: > > > with conn.cursor() as cur: > > > cur.execute(stmt) > > > > That's interesting but is it of much use? Connections and cursors > > close themselves when they go out of scope. I rarely use > > cursor.close() or connection.close() for that reason. > > True, but you're at the mercy of when and how that happens. Sometimes > you need to be certain that the resources are deallocated exactly when > you need them to be deallocated. Its fairly rare but it does happen. I guess its a question of what makes the most sense. Its probably a good thing I asked the question. :-) > I don't know if any of my users make use of this with-block feature, but > it's out there and documented, so I can't just take it away or replace > the semantics of what happens when a connection object controls a with > block. Not without annoying your users, of course. :-) > For what it's worth, I brought up the above use case back in September > 2006, and the response was somewhere between indifferent and positive. > Nobody said it was a terrible idea, so I implemented it. > > Is your transaction-controlling implementation of > connection.__enter__/__exit__ already released, i.e. is it too late to > get you to change the API to this feature? I have not released this publicly yet, no. It would be a good idea to have agreement on this rather than have conflicting implementations. Anyone else have an opinion on this?? > > Good question on the nested transaction stuff. That might prove useful > > -- but its also considerably more complicated than what I was > > suggesting.... :-) > > Of course, but it couldn't hurt to think about it for future > extensibility. Then again, it was mostly an academic questions since > Informix doesn't support nested transactions ;) > > -- > Carsten Haese > http://informixdb.sourceforge.net > > > From dieter at handshake.de Thu Jun 21 19:48:53 2007 From: dieter at handshake.de (Dieter Maurer) Date: Thu, 21 Jun 2007 19:48:53 +0200 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <467A6FA0.5060502@personnelware.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> <467A6FA0.5060502@personnelware.com> Message-ID: <18042.47621.9742.444186@gargle.gargle.HOWL> Carl Karsten wrote at 2007-6-21 07:31 -0500: > ... >I did before: "only allow one style of parameter." But it got out voted, (or >not even on the ballot) and so I won't push for it anymore. > > >>> numeric -1 > >>> format -5 > >>> pyformat -2 > >To me that reads "remove them." 0 would have been "tolerate them." That would not be good -- at least if module authors would follow the new API. Some current implementations support only param styles you want to remove. As a consequence, there are lots of code that use them (because they must). Not good (for the acceptance of a new API version) when it breaks large amount of existing code. -- Dieter From carsten at uniqsys.com Fri Jun 22 05:28:19 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Thu, 21 Jun 2007 23:28:19 -0400 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <703ae56b0706211539j530f220are8e3667a963424d2@mail.gmail.com> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> <1182453884.3376.123.camel@dot.uniqsys.com> <703ae56b0706211326w416f1f3ejd9603cfe368818e8@mail.gmail.com> <1182459960.3376.162.camel@dot.uniqsys.com> <703ae56b0706211539j530f220are8e3667a963424d2@mail.gmail.com> Message-ID: <1182482899.3264.18.camel@localhost.localdomain> On Thu, 2007-06-21 at 16:39 -0600, Anthony Tuininga wrote: > I have not released this publicly yet, no. It would be a good idea to > have agreement on this rather than have conflicting implementations. Indeed. I see two alternatives, modulo naming: 1) A connection method: with conn.transaction(): ... 2) A module function: with api.transaction(conn): ... Implementing 2) would require nothing more than an almost verbatim implementation of example 3 from pep-0343, but I personally would prefer 1) because it looks cleaner in application code. As far as the interaction with existing transactions is concerned, I think that continuing an existing transaction is the easiest and least surprising behavior. If the module supports autocommit and the connection is in autocommit mode, it should leave autocommit mode upon entering the transaction block and reenter autocommit mode upon leaving the transaction block. -- Carsten Haese http://informixdb.sourceforge.net From anthony.tuininga at gmail.com Fri Jun 22 15:57:00 2007 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Fri, 22 Jun 2007 07:57:00 -0600 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <1182482899.3264.18.camel@localhost.localdomain> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> <1182453884.3376.123.camel@dot.uniqsys.com> <703ae56b0706211326w416f1f3ejd9603cfe368818e8@mail.gmail.com> <1182459960.3376.162.camel@dot.uniqsys.com> <703ae56b0706211539j530f220are8e3667a963424d2@mail.gmail.com> <1182482899.3264.18.camel@localhost.localdomain> Message-ID: <703ae56b0706220657i515e78ceib9ef989d6a817a8a@mail.gmail.com> Hmm, I've thought about this a bit more. If you look at the module contextlib in the Python 2.5 standard library you can see that making connections and cursors context managers that simply close themselves is redundant. You can do this instead: with contextlib.closing(connection.cursor()) as cursor: do_stuff() I think it makes more sense to define database connections as context managers that deal with transactions since that is the primary purpose of a database connection -- not opening and closing itself! On 6/21/07, Carsten Haese wrote: > On Thu, 2007-06-21 at 16:39 -0600, Anthony Tuininga wrote: > > I have not released this publicly yet, no. It would be a good idea to > > have agreement on this rather than have conflicting implementations. > > Indeed. I see two alternatives, modulo naming: > > 1) A connection method: > > with conn.transaction(): ... > > 2) A module function: > > with api.transaction(conn): ... > > Implementing 2) would require nothing more than an almost verbatim > implementation of example 3 from pep-0343, but I personally would prefer > 1) because it looks cleaner in application code. > > As far as the interaction with existing transactions is concerned, I > think that continuing an existing transaction is the easiest and least > surprising behavior. > > If the module supports autocommit and the connection is in autocommit > mode, it should leave autocommit mode upon entering the transaction > block and reenter autocommit mode upon leaving the transaction block. > > -- > Carsten Haese > http://informixdb.sourceforge.net > > > From aprotin at research.att.com Fri Jun 22 17:14:02 2007 From: aprotin at research.att.com (Art Protin) Date: Fri, 22 Jun 2007 11:14:02 -0400 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <1182454623.3376.130.camel@dot.uniqsys.com> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> <467AACA0.8060606@personnelware.com> <1182448407.3376.100.camel@dot.uniqsys.com> <467AD1BD.9060508@personnelware.com> <1182454623.3376.130.camel@dot.uniqsys.com> Message-ID: <467BE73A.2080103@research.att.com> Dear folks, Carsten Haese wrote: >On Thu, 2007-06-21 at 14:30 -0500, Carl Karsten wrote: > > >>Carsten Haese wrote: >> > On Thu, 2007-06-21 at 11:51 -0500, Carl Karsten wrote: >> >> Carsten Haese wrote: >> >>> On Thu, 2007-06-21 at 10:02 -0500, Carl Karsten wrote: >> >>>> When we talk about backwards compatibility, I am wondering exactly what >>that means. >> >>> A module is backwards compatible if code using it won't break if the >> >>> module is replaced with a newer version. >> >> Even if it relies on implementation details? >> > >> > Yes, if those details are documented. Otherwise, no. >> >>I totally disagree, and have a feeling this needs to be voted on before either >>of us can continue this discussion. >> >> > >Fine. Let's vote. Please define what we're voting on. > > > I, as a developer who implements interfaces, only recognize the commitments I have explicitly made in the documentation for how the interface works. A bug is when the code does not match the documentation. If you choose to look at my code instead of my documentation, then when I change the code you are responsible to change how you use it - I never made any commitment to not change my code. I will only entertain complaints about how my code does not fulfill the commitments I made in the documentation. I define "backward compatibility" as allowing to continue to work that behavior that conforms to the previously provided documentation! If you cheated and used my code to figure out how things work, you maintain that old version and leave me and my new work alone. Thus, I think I side with Carsten on this issue. Thank you all, Art Protin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20070622/fb91f95b/attachment.htm From aprotin at research.att.com Fri Jun 22 18:01:12 2007 From: aprotin at research.att.com (Art Protin) Date: Fri, 22 Jun 2007 12:01:12 -0400 Subject: [DB-SIG] paramstyles, again (and now voting) In-Reply-To: <1182438082.3376.50.camel@dot.uniqsys.com> References: <200706022042.38146.paul@boddie.org.uk> <46642272.8090501@egenix.com> <1182372694.3383.30.camel@dot.uniqsys.com> <200706202315.55192.paul@boddie.org.uk> <46799D5D.10109@egenix.com> <4679A32B.60806@personnelware.com> <1182386141.3256.12.camel@localhost.localdomain> <4679F796.80706@personnelware.com> <20070621041545.M40830@uniqsys.com> <467A6FA0.5060502@personnelware.com> <1182430291.3376.14.camel@dot.uniqsys.com> <467A87C4.8050600@personnelware.com> <1182438082.3376.50.camel@dot.uniqsys.com> Message-ID: <467BF248.5030102@research.att.com> Dear folks, Carsten Haese wrote: >On Thu, 2007-06-21 at 09:14 -0500, Carl Karsten wrote: > > >>Carsten Haese wrote: >> >> >>>On Thu, 2007-06-21 at 07:31 -0500, Carl Karsten wrote: >>> >>> >>>>Carsten Haese wrote: >>>> >>>> >>>>>The advantages are clear: >>>>> >>>>>* Application code written against this spec can be sure that named/qmark >>>>>styles are available, and there is a common API for choosing which one to use. >>>>>That means less API dependent code, and the overhead for choosing qmark or >>>>>named is minimal at one line of code per connection. >>>>> >>>>>* APIs aren't forced to abandon "legacy" parameter styles so existing code >>>>>written for existing APIs won't break. >>>>> >>>>>* No backwards compatibility wrapper is necessary. >>>>> >>>>> >>>>I do not see the last 2 points as a clear advantage, given the cost. >>>> >>>> >>>What cost? >>> >>> >>More code in the dbapi module. >> >> So, we exist to write code. >>More complex API. >> >> We should write more complex implementations if that means that the applications for our code can be made less complex. >>More complex application code. >> >> How can it be any more complex than what we already have, namely, having to adjust the SQL queries to match the supported paramstyle. > >Compared to what? What's your proposal? > > > >>Application code that uses non portable paramstyles, >> >> > >So what? Application code may also use non-portable SQL language. Python >programmers are consenting adults, and if they want to write >non-portable code, let them. > > > I believe that there is more wisdom in enabling others to do the "right thing" than in requiring them to do the "right thing" . Furthermore, I do not see how users of my interface will be able to write much portable code. Our SQL does not have any support for querying "system tables", the specification does not provide methods for defining persistant named queries (like those invoked by callproc()), and the specification does not provide any support for non-SQL queries, so I have to extend the interface far beyond what the specification covers. Maybe, API v8 will permit truely portable code or then again maybe not until v18. Going to requiring some set of paramstyles is another step toward enabling applications to be written portably. >> > The one line of code per connection to explicitly choose >> > qmark or named style? >> >>no. in fact I don't think that line of code is in the backwards compatibility >>category. backwards compatibility should use the dbapi v2 api, where paramstyle >>is read only. I envision changing "import MySQLdb" to "import MySQLdb_v2 as >>MySQLdb" >> >> > >I never said this line of code was for backwards compatibility. Under >the proposal on the table, backwards compatibility is achieved by doing >nothing to existing applications. > >Forward compatibility is achieved by explicitly stating which parameter >style the application uses. That's that one line of code I'm talking >about, and, true, it will have to be inserted into every new >application, but your proposal requires changing every existing >application, and somehow you call that backwards compatible. > >That's like saying diesel is compatible with a gasoline car as long as >you install a diesel engine into the gasoline car. > > > >>[...] >>"it" was voted on and "it" was not really well defined. This is the closest >>thing I can find to what was voted on: >> >> >> >>> * +1 on making support one param style mandatory for all >> >>> > >> implementations >> >> > > >> >> > > Any one? Let's make named mandatory, then. >> > > >> > > I intentionally left this open :-) >> > > >> > > We should have a vote on it. Let's gather votes for say two weeks >> > > and then see what the outcome is. >> >>Given that, I think what was decided via the vote was "make named mandatory" but >> I would not be surprised to see 5 posts saying "I thought I was voting on Xn" >> >> > >The context made it clear to me that the "it" we're voting for was which >paramstyle to make mandatory. > > > Maybe not initially but that seemed to me to be to be all that it came to cover. >>Personally, even though I favor the counts (?/name positive, rest negative) I >>would like to see those votes discarded a more comprehensive, well defined round >>of voting. >> >> > >Sure, let's keep on voting until you are happy with the outcome. > > > >> > However, there is a lot of MySQL and PostgreSQL code out there >> >> >>>that would break if those styles were removed outright. >>> >>> >>backwards compatibility wrapper. >> >> > >And how does this backwards compatibility wrapper look? How does it >work? How does it achieve backwards compatibility without requiring >every existing application to be modified? > > > >>Or just name the new module something new. What is the down side to adding a >>dbapi version number to the module names? >> >> > >Existing application code would automatically and gratuitously lose >forward compatibility with future versions. > > > >>"BUT I for one, strongly endorse forbidding whatever is not required >>in the the way of parameter styles (or at the very least dropping any mention of >>any parameter style that is not required)" Art Protin >> >>But his position may have changed, much like how strongly I feel about it. >> >> > >It's one man's opinion, and he has a right to his opinion, but it is a >very unpythonic opinion. > > I am sorry to have to say that I frequently overstate my case. Forbidding paramstyles that were not required would be better in a new interface, however, breaking application code that uses the current interface is too much to pay for such an "upgrade". (I hope that this helps refine my position and moves us closer to "harmony".) Thank you all, Art Protin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20070622/166fc01e/attachment.htm From fumanchu at amor.org Fri Jun 22 18:12:23 2007 From: fumanchu at amor.org (Robert Brewer) Date: Fri, 22 Jun 2007 09:12:23 -0700 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <703ae56b0706220657i515e78ceib9ef989d6a817a8a@mail.gmail.com> Message-ID: <435DF58A933BA74397B42CDEB8145A860D1F1487@ex9.hostedexchange.local> Anthony Tuininga wrote: > Hmm, I've thought about this a bit more. If you look at the module > contextlib in the Python 2.5 standard library you can see that making > connections and cursors context managers that simply close themselves > is redundant. You can do this instead: > > with contextlib.closing(connection.cursor()) as cursor: > do_stuff() > > I think it makes more sense to define database connections as context > managers that deal with transactions since that is the primary purpose > of a database connection -- not opening and closing itself! Wha? There are plenty of good DB architectures that don't use transactions. They still need connections. Let's not conflate the two any more than the DB-API already does. Note also that, "[conn.rollback] is optional since not all databases provide transaction support." What happens to "with conn" (as transaction manager) if the DB has no transaction support? +1 for "with connection" implementing close(), and "with connection.transaction()" implementing start+commit/rollback if the DB supports it. Robert Brewer System Architect Amor Ministries fumanchu at amor.org From dieter at handshake.de Fri Jun 22 20:07:40 2007 From: dieter at handshake.de (Dieter Maurer) Date: Fri, 22 Jun 2007 20:07:40 +0200 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <467AD1BD.9060508@personnelware.com> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> <467AACA0.8060606@personnelware.com> <1182448407.3376.100.camel@dot.uniqsys.com> <467AD1BD.9060508@personnelware.com> Message-ID: <18044.4076.538186.713976@gargle.gargle.HOWL> Carl Karsten wrote at 2007-6-21 14:30 -0500: > ... >I propose that pep 249 is the only document that we should be considering. -1 >What happens if docs and pep 249 conflict? The more specific wins. Let's look at an example: We are using "psycopg". It uses "pyformat" as "paramStyle" and documents that it does so. As a consequence, our SQLs are using "pyformat". Guess our curses when "psycopg" decides to no longer support "pyformat". -- Dieter From dieter at handshake.de Fri Jun 22 20:11:09 2007 From: dieter at handshake.de (Dieter Maurer) Date: Fri, 22 Jun 2007 20:11:09 +0200 Subject: [DB-SIG] A slightly different paramstyle suggestion In-Reply-To: References: Message-ID: <18044.4285.715628.24956@gargle.gargle.HOWL> Michael Kleehammer wrote at 2007-6-21 14:43 -0500: > ... >+1 for reducing the param styles to ? and named -1 >+1 for using *args and **kwargs to differentiate between them. +0 We should think about "executemany" as well. >-1 on requiring both (not yet, at least) +1 (we should have both) -- Dieter From carsten at uniqsys.com Fri Jun 22 20:24:49 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Fri, 22 Jun 2007 14:24:49 -0400 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <18044.4076.538186.713976@gargle.gargle.HOWL> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> <467AACA0.8060606@personnelware.com> <1182448407.3376.100.camel@dot.uniqsys.com> <467AD1BD.9060508@personnelware.com> <18044.4076.538186.713976@gargle.gargle.HOWL> Message-ID: <1182536689.3392.15.camel@dot.uniqsys.com> On Fri, 2007-06-22 at 20:07 +0200, Dieter Maurer wrote: > Carl Karsten wrote at 2007-6-21 14:30 -0500: > > ... > >I propose that pep 249 is the only document that we should be considering. > > -1 > > > >What happens if docs and pep 249 conflict? > > The more specific wins. The question is vacuous. The documentation for a DB-API v2 implementation should never conflict with PEP 249, because that would mean that either the documentation is incorrect or the implementation is not compliant. Of course, PEP 249 is only the mandatory minimum. Implementations may and do implement more functionality than what PEP 249 mandates. In those cases, the implementation documentation is the authoritative source. We'd create a problem if we introduced language into the v3 spec that contradicts existing v2 implementations, because that would force the v2 implementation to be either non-compliant or not backwards compatible. That would not be good, which is why additions to the spec are being discussed this thoroughly. -- Carsten Haese http://informixdb.sourceforge.net From dieter at handshake.de Fri Jun 22 20:27:07 2007 From: dieter at handshake.de (Dieter Maurer) Date: Fri, 22 Jun 2007 20:27:07 +0200 Subject: [DB-SIG] DB API extension suggestion In-Reply-To: <703ae56b0706211539j530f220are8e3667a963424d2@mail.gmail.com> References: <703ae56b0706210941g4d11de54g1d93431f3158e335@mail.gmail.com> <1182453884.3376.123.camel@dot.uniqsys.com> <703ae56b0706211326w416f1f3ejd9603cfe368818e8@mail.gmail.com> <1182459960.3376.162.camel@dot.uniqsys.com> <703ae56b0706211539j530f220are8e3667a963424d2@mail.gmail.com> Message-ID: <18044.5243.157829.860249@gargle.gargle.HOWL> Anthony Tuininga wrote at 2007-6-21 16:39 -0600: > ... >> Is your transaction-controlling implementation of >> connection.__enter__/__exit__ already released, i.e. is it too late to >> get you to change the API to this feature? > >I have not released this publicly yet, no. It would be a good idea to >have agreement on this rather than have conflicting implementations. >Anyone else have an opinion on this?? I agree with Mark. It never happens that in case of an exception I only want to rollback the transaction. I do always want the exception logged and usually I want it propagated. -- Dieter From carsten at uniqsys.com Fri Jun 22 20:43:32 2007 From: carsten at uniqsys.com (Carsten Haese) Date: Fri, 22 Jun 2007 14:43:32 -0400 Subject: [DB-SIG] A slightly different paramstyle suggestion In-Reply-To: <703ae56b0706210933t79a0b99epf7cba3bfbcf419c5@mail.gmail.com> References: <703ae56b0706210933t79a0b99epf7cba3bfbcf419c5@mail.gmail.com> Message-ID: <1182537812.3392.26.camel@dot.uniqsys.com> On Thu, 2007-06-21 at 10:33 -0600, Anthony Tuininga wrote: > I would like to suggest adding > the syntactic sugar (*args, *kwargs) in order to avoid having to build > the list, tuple or dictionary when passing arguments to the execute > method. For backwards compatibility you can check the list of > parameters and if the length is one and the first argument is a list, > tuple or dictionary change the arguments to the new style on the fly. -1 As much as I'd like to be able to do this, I don't think this can be done in a backwards compatible way that is guaranteed to do the right thing in all cases. If the query has one placeholder and the parameter is a one-element sequence, it is impossible to know for certain whether the sequence itself or its only contained element should be bound as the parameter. I think we should refuse the temptation to guess and go with the established semantics. -- Carsten Haese http://informixdb.sourceforge.net From dieter at handshake.de Sat Jun 23 13:47:41 2007 From: dieter at handshake.de (Dieter Maurer) Date: Sat, 23 Jun 2007 13:47:41 +0200 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <1182536689.3392.15.camel@dot.uniqsys.com> References: <467A92EE.5070207@personnelware.com> <1182440275.3376.71.camel@dot.uniqsys.com> <467AACA0.8060606@personnelware.com> <1182448407.3376.100.camel@dot.uniqsys.com> <467AD1BD.9060508@personnelware.com> <18044.4076.538186.713976@gargle.gargle.HOWL> <1182536689.3392.15.camel@dot.uniqsys.com> Message-ID: <18045.2141.494322.90607@gargle.gargle.HOWL> Carsten Haese wrote at 2007-6-22 14:24 -0400: > ... >> >What happens if docs and pep 249 conflict? >> >> The more specific wins. > >The question is vacuous. The documentation for a DB-API v2 >implementation should never conflict with PEP 249, because that would >mean that either the documentation is incorrect or the implementation is >not compliant. > >Of course, PEP 249 is only the mandatory minimum. Implementations may >and do implement more functionality than what PEP 249 mandates. In those >cases, the implementation documentation is the authoritative source. Fine. That what I meant with the "the more specific wins". Descriptions (such as PEPs) usually are quite abstract and let some freedom. Using this freedom and documenting the taken decisions creates a more specific work. -- Dieter From carl at personnelware.com Sun Jun 24 19:20:29 2007 From: carl at personnelware.com (Carl Karsten) Date: Sun, 24 Jun 2007 12:20:29 -0500 Subject: [DB-SIG] what is: backwards compatibility In-Reply-To: <467A92EE.5070207@personnelware.com> References: <467A92EE.5070207@personnelware.com> Message-ID: <467EA7DD.6090105@personnelware.com> I'm still trying to come up with a set of possible meanings for backwards compatibility, something like doesn't break: any existing code any conceivable code reasonable code code that conforms to any documentation found, including module code. code that conforms to the docs bundled with the module code that conforms to the docs found under python.org code that conforms to pep 249 But I have a feeling I am missing something. I think the problem is separating it from how much of it is reasonable for dbapi v3. also, Anyone have a list of what dbapi v2 modules exist, and therefor need to be accommodated? Carl K From anthony.tuininga at gmail.com Mon Jun 25 15:45:40 2007 From: anthony.tuininga at gmail.com (Anthony Tuininga) Date: Mon, 25 Jun 2007 07:45:40 -0600 Subject: [DB-SIG] A slightly different paramstyle suggestion In-Reply-To: <1182537812.3392.26.camel@dot.uniqsys.com> References: <703ae56b0706210933t79a0b99epf7cba3bfbcf419c5@mail.gmail.com> <1182537812.3392.26.camel@dot.uniqsys.com> Message-ID: <703ae56b0706250645x15322b83g27c0d1d39859e27f@mail.gmail.com> On 6/22/07, Carsten Haese wrote: > On Thu, 2007-06-21 at 10:33 -0600, Anthony Tuininga wrote: > > I would like to suggest adding > > the syntactic sugar (*args, *kwargs) in order to avoid having to build > > the list, tuple or dictionary when passing arguments to the execute > > method. For backwards compatibility you can check the list of > > parameters and if the length is one and the first argument is a list, > > tuple or dictionary change the arguments to the new style on the fly. > > -1 > > As much as I'd like to be able to do this, I don't think this can be > done in a backwards compatible way that is guaranteed to do the right > thing in all cases. If the query has one placeholder and the parameter > is a one-element sequence, it is impossible to know for certain whether > the sequence itself or its only contained element should be bound as the > parameter. I agree that this is possible but it is a low enough occurrence that it can be explicitly warned against -- backwards compatibility is not perfect but it is reasonable considering that the DB API doesn't even mention sequences (arrays) as possible parameter values; it only deals with scalar parameter values. > I think we should refuse the temptation to guess and go with the > established semantics. I suppose the other option is to come up with a new name for the new syntax. Taking a page from some of the other modules that Python has, perhaps cursor.executev(statement, *args, *kwargs) Maybe that is more palatable? > -- > Carsten Haese > http://informixdb.sourceforge.net > > >