From miked at dewhirst.com.au  Sun Apr  2 08:48:42 2006
From: miked at dewhirst.com.au (Mike Dewhirst)
Date: Sun, 02 Apr 2006 16:48:42 +1000
Subject: [DB-SIG] decimal datatypes
In-Reply-To: <mailman.0.1143885602.5098.db-sig@python.org>
References: <mailman.0.1143885602.5098.db-sig@python.org>
Message-ID: <442F73CA.7090506@dewhirst.com.au>

Chris

> I hope I have the right list for this question...
> 
> According to the homepage for pyodbc, pyodbc supports decimal data types,
> but when I try to copy a row from one table to another, I'm getting:
> 
> pyodbc.NotSupportedError: ('Python type Decimal not supported.  param=26',
> 'HY097')
> 

try this ...

from decimal import Decimal

or

from decimal import *

I could be way off-beam with your problem but it sounds like the decimal 
module might be what you want. It contains the Decimal class.

OTOH - it is possible or even more likely that you need a bespoke 
mapping from your actual data type to one supported by pyodbc (ie., 
decimal).

OTTH - it is also possible that you spelled decimal incorrectly. It may 
actually start with a lower case d

Good luck

Mike

From mkleehammer at gmail.com  Mon Apr  3 22:17:21 2006
From: mkleehammer at gmail.com (Michael Kleehammer)
Date: Mon, 3 Apr 2006 15:17:21 -0500
Subject: [DB-SIG] decimal datatypes (pyodbc)
Message-ID: <cafbb4e50604031317y47d21be4t6e04bf452531527a@mail.gmail.com>

> According to the homepage for pyodbc, pyodbc supports decimal data types,
> but when I try to copy a row from one table to another, I'm getting:
>
> pyodbc.NotSupportedError: ('Python type Decimal not supported.  param=26',
> 'HY097')

Mike Dewhirst,

Thanks for reporting this.

Apparently the Decimal support wasn't complete.  They were returned
when selecting DECIMAL and NUMERIC columns but couldn't be used as
parameters.

I've implemented the necessary code in 2.0.18 which is ready for download.

Thanks again,
Michael Kleehammer

PS,
While we do monitor this newsgroup, if you create a bug report on
sourceforge.net
(http://sourceforge.net/tracker/?group_id=162557&atid=824254) it will
send us an email right away.

From szybalski at gmail.com  Fri Apr 14 19:00:09 2006
From: szybalski at gmail.com (Lukasz Szybalski)
Date: Fri, 14 Apr 2006 12:00:09 -0500
Subject: [DB-SIG] mysql string length?
Message-ID: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>

Hello,
I am trying to implement an database program but I am having these
problems for which i don't seem to find an answer.

1. One of my columns in table_x is :
body text not null

when i use python to create sql :
INSERT INTO table_x( body)VALUES( '%s')" %  (body)

this body is a string that varies in size. I keep getting an error if
the size of body is longer then 255, and if its smaller everything
goes smooth. Is this syntax correct? should '%s' be something else?

2. I get an error when i close() a connection and then reopen it in
the same program. Is there a way around it. Here is more details on
what i do.

main :
do function 1
do function 2

function 1:
open connection to db
get cursor
close cursor
close connection

function 2:
open connection to db
get cursor
close cursor
close connection
go to sleep for 6 hours
go to function 1

I need to disconnect and then connect to db. Is there a reason i get
(0, '') as an exception error? I read in docs that you can't
disconnect and then connect, is that true? And if so is there a way
around it? The easiest way to do it would be to keep the connection
alive but that would mean the database will be filled with inactive
connection, that might disconnect.

Thank You

--
Lukasz Szybalski

From andychambers2002 at yahoo.co.uk  Sat Apr 15 14:01:41 2006
From: andychambers2002 at yahoo.co.uk (Andrew Chambers)
Date: Sat, 15 Apr 2006 12:01:41 +0000
Subject: [DB-SIG] mysql string length?
In-Reply-To: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
Message-ID: <20060415120141.GA12186@yapgi.hopto.org>

* Lukasz Szybalski <szybalski at gmail.com> [2006-04-14 12:00:09 -0500]:

> INSERT INTO table_x( body)VALUES( '%s')" %  (body)
> 
> this body is a string that varies in size. I keep getting an error if
> the size of body is longer then 255, and if its smaller everything
> goes smooth. Is this syntax correct? should '%s' be something else?

What is the datatype of body?  It sounds like it is CHAR(255).  Can you
change this to be TEXT?

Regards,
Andy

Send instant messages to your online friends http://uk.messenger.yahoo.com 

From szybalski at gmail.com  Sun Apr 16 00:24:40 2006
From: szybalski at gmail.com (Lukasz Szybalski)
Date: Sat, 15 Apr 2006 17:24:40 -0500
Subject: [DB-SIG] mysql string length?
In-Reply-To: <20060415120141.GA12186@yapgi.hopto.org>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
Message-ID: <1145139880.8861.9.camel@localhost.localdomain>

On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote:
> * Lukasz Szybalski <szybalski at gmail.com> [2006-04-14 12:00:09 -0500]:
> 
> > INSERT INTO table_x( body)VALUES( '%s')" %  (body)
> > 
> > this body is a string that varies in size. I keep getting an error if
> > the size of body is longer then 255, and if its smaller everything
> > goes smooth. Is this syntax correct? should '%s' be something else?
> 
> What is the datatype of body?  It sounds like it is CHAR(255).  Can you
> change this to be TEXT?
> 
Yes it is text. 
mysql> describe table_x;
+----------+---------+------+-----+---------+-------+
| Field    | Type    | Null | Key | Default | Extra |
+----------+---------+------+-----+---------+-------+
| id | int(11) |      | PRI | 0       |       |
| body     | text    | YES  |     | NULL    |       |
+----------+---------+------+-----+---------+-------+

Would some kind of characters that are passed in had something to do
with the error, maybe EOF or something similar?

The string that is passed in is from xml node. For debugging i made it
return str(body), but that didn't change anything. 

Are there any other database/%s related requirements?

Lukasz 


> Regards,
> Andy
> 
> Send instant messages to your online friends http://uk.messenger.yahoo.com 
> _______________________________________________
> DB-SIG maillist  -  DB-SIG at python.org
> http://mail.python.org/mailman/listinfo/db-sig


From andy47 at halfcooked.com  Sun Apr 16 12:42:09 2006
From: andy47 at halfcooked.com (Andy Todd)
Date: Sun, 16 Apr 2006 20:42:09 +1000
Subject: [DB-SIG] mysql string length?
In-Reply-To: <1145139880.8861.9.camel@localhost.localdomain>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
Message-ID: <44421F81.4080702@halfcooked.com>

Lukasz Szybalski wrote:
> On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote:
>> * Lukasz Szybalski <szybalski at gmail.com> [2006-04-14 12:00:09 -0500]:
>>
>>> INSERT INTO table_x( body)VALUES( '%s')" %  (body)
>>>
>>> this body is a string that varies in size. I keep getting an error if
>>> the size of body is longer then 255, and if its smaller everything
>>> goes smooth. Is this syntax correct? should '%s' be something else?
>> What is the datatype of body?  It sounds like it is CHAR(255).  Can you
>> change this to be TEXT?
>>
> Yes it is text. 
> mysql> describe table_x;
> +----------+---------+------+-----+---------+-------+
> | Field    | Type    | Null | Key | Default | Extra |
> +----------+---------+------+-----+---------+-------+
> | id | int(11) |      | PRI | 0       |       |
> | body     | text    | YES  |     | NULL    |       |
> +----------+---------+------+-----+---------+-------+
> 
> Would some kind of characters that are passed in had something to do
> with the error, maybe EOF or something similar?
> 
> The string that is passed in is from xml node. For debugging i made it
> return str(body), but that didn't change anything. 
> 
> Are there any other database/%s related requirements?
> 
> Lukasz 
> 
> 
>> Regards,
>> Andy


My observation is that with MySQLdb %s is a parameter substitution value 
not a string substitution indicator.

What happens when you try something like this;

 >>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
 >>> cursor.execute(stmt, (body,))

[1] note that there are no quote marks around the %s

Regards,
Andy
-- 
--------------------------------------------------------------------------------
 From the desk of Andrew J Todd esq - http://www.halfcooked.com/

From engelbert.gruber at ssg.co.at  Sun Apr 16 15:53:09 2006
From: engelbert.gruber at ssg.co.at (engelbert.gruber at ssg.co.at)
Date: Sun, 16 Apr 2006 15:53:09 +0200 (CEST)
Subject: [DB-SIG] mysql string length?
In-Reply-To: <44421F81.4080702@halfcooked.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
Message-ID: <Pine.LNX.4.64.0604161508390.3514@lx3.local>

On Sun, 16 Apr 2006, Andy Todd wrote:

> Lukasz Szybalski wrote:
>> On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote:
>>> * Lukasz Szybalski <szybalski at gmail.com> [2006-04-14 12:00:09 -0500]:
>>>
>>>> INSERT INTO table_x( body)VALUES( '%s')" %  (body)
>>>>
>>>> this body is a string that varies in size. I keep getting an error if
>>>> the size of body is longer then 255, and if its smaller everything
>>>> goes smooth. Is this syntax correct? should '%s' be something else?
>>> What is the datatype of body?  It sounds like it is CHAR(255).  Can you
>>> change this to be TEXT?
>>>
>> Yes it is text.
>> mysql> describe table_x;
>> +----------+---------+------+-----+---------+-------+
>> | Field    | Type    | Null | Key | Default | Extra |
>> +----------+---------+------+-----+---------+-------+
>> | id | int(11) |      | PRI | 0       |       |
>> | body     | text    | YES  |     | NULL    |       |
>> +----------+---------+------+-----+---------+-------+
>>
>> Would some kind of characters that are passed in had something to do
>> with the error, maybe EOF or something similar?
>>
>> The string that is passed in is from xml node. For debugging i made it
>> return str(body), but that didn't change anything.
>>
>> Are there any other database/%s related requirements?

not the way you pass it (as i understand it python first does "%" 
replacement and then passes the string.

> My observation is that with MySQLdb %s is a parameter substitution value
> not a string substitution indicator.
>
> What happens when you try something like this;
>
> >>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
> >>> cursor.execute(stmt, (body,))
>
> [1] note that there are no quote marks around the %s

this works here (debian testing, python 2.3.5, 4.0.21) 
::

import MySQLdb
"""
create table table_x (
   id int auto_increment primary key,
   body text
   )
"""
db = MySQLdb.connect(passwd="",db="test")
c = db.cursor()
body = "0....5...."*28
sql = "INSERT INTO table_x (body) VALUES('%s')"
c.execute( sql % (body))
c.execute("select body from table_x")
for row in c.fetchall():
     print len(row[0])

i wont bet on fetching 1Mb.

cheers
-- 

From andy47 at halfcooked.com  Mon Apr 17 11:45:46 2006
From: andy47 at halfcooked.com (Andy Todd)
Date: Mon, 17 Apr 2006 19:45:46 +1000
Subject: [DB-SIG] mysql string length?
In-Reply-To: <Pine.LNX.4.64.0604161508390.3514@lx3.local>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
Message-ID: <444363CA.2030304@halfcooked.com>

engelbert.gruber at ssg.co.at wrote:
> On Sun, 16 Apr 2006, Andy Todd wrote:
> 
>> Lukasz Szybalski wrote:
>>> On Sat, 2006-04-15 at 12:01 +0000, Andrew Chambers wrote:
>>>> * Lukasz Szybalski <szybalski at gmail.com> [2006-04-14 12:00:09 -0500]:
>>>>
>>>>> INSERT INTO table_x( body)VALUES( '%s')" %  (body)
>>>>>
>>>>> this body is a string that varies in size. I keep getting an error if
>>>>> the size of body is longer then 255, and if its smaller everything
>>>>> goes smooth. Is this syntax correct? should '%s' be something else?
>>>> What is the datatype of body?  It sounds like it is CHAR(255).  Can you
>>>> change this to be TEXT?
>>>>
>>> Yes it is text.
>>> mysql> describe table_x;
>>> +----------+---------+------+-----+---------+-------+
>>> | Field    | Type    | Null | Key | Default | Extra |
>>> +----------+---------+------+-----+---------+-------+
>>> | id | int(11) |      | PRI | 0       |       |
>>> | body     | text    | YES  |     | NULL    |       |
>>> +----------+---------+------+-----+---------+-------+
>>>
>>> Would some kind of characters that are passed in had something to do
>>> with the error, maybe EOF or something similar?
>>>
>>> The string that is passed in is from xml node. For debugging i made it
>>> return str(body), but that didn't change anything.
>>>
>>> Are there any other database/%s related requirements?
> 
> not the way you pass it (as i understand it python first does "%" 
> replacement and then passes the string.
> 
>> My observation is that with MySQLdb %s is a parameter substitution value
>> not a string substitution indicator.
>>
>> What happens when you try something like this;
>>
>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
>>>>> cursor.execute(stmt, (body,))
>> [1] note that there are no quote marks around the %s
> 
> this works here (debian testing, python 2.3.5, 4.0.21) 
> ::
> 
> import MySQLdb
> """
> create table table_x (
>    id int auto_increment primary key,
>    body text
>    )
> """
> db = MySQLdb.connect(passwd="",db="test")
> c = db.cursor()
> body = "0....5...."*28
> sql = "INSERT INTO table_x (body) VALUES('%s')"
> c.execute( sql % (body))
> c.execute("select body from table_x")
> for row in c.fetchall():
>      print len(row[0])
> 
> i wont bet on fetching 1Mb.
> 
> cheers

You've just repeated the original poster's code. As I suggested the 
problem is that he's using string substitution when he should be using 
parameter substitution. In these cases %s means two completely different 
things.

Please read the sections on 'paramstyle' and the cursor '.execute' 
method in the DB-API 2.0 definition [1].

Regards,
Andy

[1] http://www.python.org/dev/peps/pep-0249/
-- 
--------------------------------------------------------------------------------
 From the desk of Andrew J Todd esq - http://www.halfcooked.com/

From engelbert.gruber at ssg.co.at  Mon Apr 17 14:56:11 2006
From: engelbert.gruber at ssg.co.at (engelbert.gruber at ssg.co.at)
Date: Mon, 17 Apr 2006 14:56:11 +0200 (CEST)
Subject: [DB-SIG] mysql string length?
In-Reply-To: <444363CA.2030304@halfcooked.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
Message-ID: <Pine.LNX.4.64.0604171448250.7805@lx3.local>

On Mon, 17 Apr 2006, Andy Todd wrote:

>>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
>>>>>> cursor.execute(stmt, (body,))
>>> [1] note that there are no quote marks around the %s
>> 
>> this works here (debian testing, python 2.3.5, 4.0.21) ::
>> 
>> import MySQLdb
>> """
>> create table table_x (
>>    id int auto_increment primary key,
>>    body text
>>    )
>> """
>> db = MySQLdb.connect(passwd="",db="test")
>> c = db.cursor()
>> body = "0....5...."*28
>> sql = "INSERT INTO table_x (body) VALUES('%s')"
>> c.execute( sql % (body))
>> c.execute("select body from table_x")
>> for row in c.fetchall():
>>      print len(row[0])
>> 
>> i wont bet on fetching 1Mb.
>> 
>> cheers
>
> You've just repeated the original poster's code. As I suggested the problem 
> is that he's using string substitution when he should be using parameter 
> substitution. In these cases %s means two completely different things.

yes but it works here, even with ``body = "0....5...."*280`` so i cannot 
blame it on not using 'paramstyle', or what ?

> Please read the sections on 'paramstyle' and the cursor '.execute' method in 
> the DB-API 2.0 definition [1].

of course you are right it is ::

   c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body))

or paramstyle without quotes ::

   c.execute(INSERT INTO table_x (body) VALUES(%s)", (body))

but both work here with bodylength up to 65535 bytes not 255.

sorry for dupplication, but i fail to see the point.


-- 

From farcepest at gmail.com  Tue Apr 18 17:40:05 2006
From: farcepest at gmail.com (Andy Dustman)
Date: Tue, 18 Apr 2006 11:40:05 -0400
Subject: [DB-SIG] mysql string length?
In-Reply-To: <Pine.LNX.4.64.0604171448250.7805@lx3.local>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
Message-ID: <9826f3800604180840p78b8c32cy7565d4dae8d8fda1@mail.gmail.com>

On 4/17/06, engelbert.gruber at ssg.co.at <engelbert.gruber at ssg.co.at> wrote:
> On Mon, 17 Apr 2006, Andy Todd wrote:
>
> >>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
> >>>>>> cursor.execute(stmt, (body,))
> >>> [1] note that there are no quote marks around the %s
> >>
> >> this works here (debian testing, python 2.3.5, 4.0.21) ::
> >>
> >> import MySQLdb
> >> """
> >> create table table_x (
> >>    id int auto_increment primary key,
> >>    body text
> >>    )
> >> """
> >> db = MySQLdb.connect(passwd="",db="test")
> >> c = db.cursor()
> >> body = "0....5...."*28
> >> sql = "INSERT INTO table_x (body) VALUES('%s')"
> >> c.execute( sql % (body))
> >> c.execute("select body from table_x")
> >> for row in c.fetchall():
> >>      print len(row[0])
> >>
> >> i wont bet on fetching 1Mb.
> >>
> >> cheers
> >
> > You've just repeated the original poster's code. As I suggested the problem
> > is that he's using string substitution when he should be using parameter
> > substitution. In these cases %s means two completely different things.
>
> yes but it works here, even with ``body = "0....5...."*280`` so i cannot
> blame it on not using 'paramstyle', or what ?
>
> > Please read the sections on 'paramstyle' and the cursor '.execute' method in
> > the DB-API 2.0 definition [1].
>
> of course you are right it is ::
>
>    c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body))

Don't do that. It'll work right up until you have a string with quotes
or \x00 bytes in it. Then bad things will happen. How bad depends on
where body comes from (think SQL injection attacks).

> or paramstyle without quotes ::
>
>    c.execute(INSERT INTO table_x (body) VALUES(%s)", (body))

Do that instead. Special characters will be escaped.

> but both work here with bodylength up to 65535 bytes not 255.

TEXT columns are not the same as CHAR or VARCHAR. TEXT may be very
large (they are non-binary BLOBs, but LOB sounds silly). CHAR/VARCHAR
may be at most 255 characters. However, there are really four types of
TEXT: TINYTEXT, TEXT, MEDIUMTEXT, and LONGTEXT. TEXT supports up to
65535 characters (16-bit length). If a 255-character limit is being
hit, it's most likely either CHAR/VARCHAR or possibly TINYTEXT.

--
The Pythonic Principle: Python works the way it does
because if it didn't, it wouldn't be Python.

From szybalski at gmail.com  Tue Apr 18 17:46:45 2006
From: szybalski at gmail.com (Lukasz Szybalski)
Date: Tue, 18 Apr 2006 10:46:45 -0500
Subject: [DB-SIG] mysql string length?
In-Reply-To: <Pine.LNX.4.64.0604171448250.7805@lx3.local>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
Message-ID: <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>

On 4/17/06, engelbert.gruber at ssg.co.at <engelbert.gruber at ssg.co.at> wrote:
> On Mon, 17 Apr 2006, Andy Todd wrote:
>
> >>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
> >>>>>> cursor.execute(stmt, (body,))
> >>> [1] note that there are no quote marks around the %s
> >>
> >> this works here (debian testing, python 2.3.5, 4.0.21) ::
> >>
> >> import MySQLdb
> >> """
> >> create table table_x (
> >>    id int auto_increment primary key,
> >>    body text
> >>    )
> >> """
> >> db = MySQLdb.connect(passwd="",db="test")
> >> c = db.cursor()
> >> body = "0....5...."*28
> >> sql = "INSERT INTO table_x (body) VALUES('%s')"
> >> c.execute( sql % (body))
> >> c.execute("select body from table_x")
> >> for row in c.fetchall():
> >>      print len(row[0])
> >>
> >> i wont bet on fetching 1Mb.
> >>
> >> cheers
> >
> > You've just repeated the original poster's code. As I suggested the problem
> > is that he's using string substitution when he should be using parameter
> > substitution. In these cases %s means two completely different things.
>
> yes but it works here, even with ``body = "0....5...."*280`` so i cannot
> blame it on not using 'paramstyle', or what ?
>
> > Please read the sections on 'paramstyle' and the cursor '.execute' method in
> > the DB-API 2.0 definition [1].
>
> of course you are right it is ::
>
>    c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body))
>
> or paramstyle without quotes ::
>
>    c.execute(INSERT INTO table_x (body) VALUES(%s)", (body))
>
> but both work here with bodylength up to 65535 bytes not 255.
>
> sorry for dupplication, but i fail to see the point.
>
>

Ok, I'm almost there. Thank you.
So i finally realized that the length is not a problem but the actual
body. The body includes characters such as " ' ", so ( I'm, I'll, etc)
which cause it to give an error.

>>>body="helo I'm here,I'll be there"
>>> body=body *15

When i try :
c.execute("Insert into table_x(body) VALUES(%s)",(body))
This will work but how do i make :

sql="Insert into table_x(body) VALUES(%s)",(body)
c.execute(sql)  #complains that i give it a tuple, which i did when
you look at sql

One problem i see here is that it works in one way but no the other.
So how do i add id and  make it work both ways?

c.execute("Insert into  table_x(id,body) VALUES(%d,%s)",(id),(body))  
??does not work

I'm not familiar with ('pyformat'      Python extended format codes,
e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use
"insert into table_x(id,body)VALUES('%d','%s')" % (id,body)

-How to account for " I'll " in a body?
-When would i HAVE to use pyformat? Can everything be done in
'format'        ANSI C printf format codes,  e.g. '...WHERE name=%s'

Thanks,
Lukasz

From szybalski at gmail.com  Tue Apr 18 20:00:34 2006
From: szybalski at gmail.com (Lukasz Szybalski)
Date: Tue, 18 Apr 2006 13:00:34 -0500
Subject: [DB-SIG] mysql string length?
In-Reply-To: <804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
Message-ID: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>

On 4/18/06, Lukasz Szybalski <szybalski at gmail.com> wrote:
> On 4/17/06, engelbert.gruber at ssg.co.at <engelbert.gruber at ssg.co.at> wrote:
> > On Mon, 17 Apr 2006, Andy Todd wrote:
> >
> > >>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
> > >>>>>> cursor.execute(stmt, (body,))
> > >>> [1] note that there are no quote marks around the %s
> > >>
> > >> this works here (debian testing, python 2.3.5, 4.0.21) ::
> > >>
> > >> import MySQLdb
> > >> """
> > >> create table table_x (
> > >>    id int auto_increment primary key,
> > >>    body text
> > >>    )
> > >> """
> > >> db = MySQLdb.connect(passwd="",db="test")
> > >> c = db.cursor()
> > >> body = "0....5...."*28
> > >> sql = "INSERT INTO table_x (body) VALUES('%s')"
> > >> c.execute( sql % (body))
> > >> c.execute("select body from table_x")
> > >> for row in c.fetchall():
> > >>      print len(row[0])
> > >>
> > >> i wont bet on fetching 1Mb.
> > >>
> > >> cheers
> > >
> > > You've just repeated the original poster's code. As I suggested the problem
> > > is that he's using string substitution when he should be using parameter
> > > substitution. In these cases %s means two completely different things.
> >
> > yes but it works here, even with ``body = "0....5...."*280`` so i cannot
> > blame it on not using 'paramstyle', or what ?
> >
> > > Please read the sections on 'paramstyle' and the cursor '.execute' method in
> > > the DB-API 2.0 definition [1].
> >
> > of course you are right it is ::
> >
> >    c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body))
> >
> > or paramstyle without quotes ::
> >
> >    c.execute(INSERT INTO table_x (body) VALUES(%s)", (body))
> >
> > but both work here with bodylength up to 65535 bytes not 255.
> >
> > sorry for dupplication, but i fail to see the point.
> >
> >
>
> Ok, I'm almost there. Thank you.
> So i finally realized that the length is not a problem but the actual
> body. The body includes characters such as " ' ", so ( I'm, I'll, etc)
> which cause it to give an error.
>
> >>>body="helo I'm here,I'll be there"
> >>> body=body *15
>
> When i try :
> c.execute("Insert into table_x(body) VALUES(%s)",(body))
> This will work but how do i make :
>
> sql="Insert into table_x(body) VALUES(%s)",(body)
> c.execute(sql)  #complains that i give it a tuple, which i did when
> you look at sql
>
> One problem i see here is that it works in one way but no the other.
> So how do i add id and  make it work both ways?
>
> c.execute("Insert into  table_x(id,body) VALUES(%d,%s)",(id),(body))
> ??does not work
>
> I'm not familiar with ('pyformat'      Python extended format codes,
> e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use
> "insert into table_x(id,body)VALUES('%d','%s')" % (id,body)
>
> -How to account for " I'll " in a body?
> -When would i HAVE to use pyformat? Can everything be done in
> 'format'        ANSI C printf format codes,  e.g. '...WHERE name=%s'
>
Ok guys, thanks for help. I finally search for how to scape strings and i used:

"insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body)

where "'''" on each side of s  = double quote + 3x single quote+ double quote

Thanks again
> Thanks,
> Lukasz
>


--
Lukasz Szybalski
www.lucasmanual.com

From engelbert.gruber at ssg.co.at  Tue Apr 18 20:09:56 2006
From: engelbert.gruber at ssg.co.at (engelbert.gruber at ssg.co.at)
Date: Tue, 18 Apr 2006 20:09:56 +0200 (CEST)
Subject: [DB-SIG] mysql string length?
In-Reply-To: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0604182007510.7841@lx3.local>

On Tue, 18 Apr 2006, Lukasz Szybalski wrote:

SNIP

> Ok guys, thanks for help. I finally search for how to scape strings and i used:
>
> "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body)
>
> where "'''" on each side of s  = double quote + 3x single quote+ double quote

as andy dustman said

   Don't do that. It'll work right up until you have a string with quotes
   or \x00 bytes in it. Then bad things will happen. How bad depends on
   where body comes from (think SQL injection attacks).

   >    c.execute(INSERT INTO table_x (body) VALUES(%s)", (body))

   Do that instead. Special characters will be escaped.

cheers

From python at venix.com  Tue Apr 18 23:53:29 2006
From: python at venix.com (Python)
Date: Tue, 18 Apr 2006 17:53:29 -0400
Subject: [DB-SIG] mysql string length?
In-Reply-To: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
Message-ID: <1145397209.6268.53.camel@www.venix.com>

On Tue, 2006-04-18 at 13:00 -0500, Lukasz Szybalski wrote:
> > When i try :
> > c.execute("Insert into table_x(body) VALUES(%s)",(body))
> > This will work but how do i make :
> >
> > sql="Insert into table_x(body) VALUES(%s)",(body)
> > c.execute(sql)  #complains that i give it a tuple, which i did when

c.execute( sql, (body,) )

(I'm pretty sure that with a single value this would also work:
	c.execute( sql, body )
but I do not have time to actually test that right now.)

> > you look at sql
> >
> > One problem i see here is that it works in one way but no the other.
> > So how do i add id and  make it work both ways?
> >
> > c.execute("Insert into  table_x(id,body) VALUES(%d,%s)",(id),(body))

c.execute("Insert into  table_x(id,body) VALUES(%s,%s)", (id,body) )

Notes:
	the placeholders are *ALWAYS* %s.  This is not Python string
formatting.

	There are two arguments, the SQL command string and the tuple of values
to be substituted into the database.


> > ??does not work
> 
-- 
Lloyd Kvam
Venix Corp.
1 Court Street, Suite 378
Lebanon, NH 03766-1358

voice:  603-653-8139
fax:    320-210-3409
-- 
Lloyd Kvam
Venix Corp


From andy47 at halfcooked.com  Wed Apr 19 09:45:17 2006
From: andy47 at halfcooked.com (Andy Todd)
Date: Wed, 19 Apr 2006 17:45:17 +1000
Subject: [DB-SIG] mysql string length?
In-Reply-To: <804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
Message-ID: <4445EA8D.6040806@halfcooked.com>

Lukasz Szybalski wrote:
> On 4/18/06, Lukasz Szybalski <szybalski at gmail.com> wrote:
>> On 4/17/06, engelbert.gruber at ssg.co.at <engelbert.gruber at ssg.co.at> wrote:
>>> On Mon, 17 Apr 2006, Andy Todd wrote:
>>>
>>>>>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
>>>>>>>>> cursor.execute(stmt, (body,))
>>>>>> [1] note that there are no quote marks around the %s
>>>>> this works here (debian testing, python 2.3.5, 4.0.21) ::
>>>>>
>>>>> import MySQLdb
>>>>> """
>>>>> create table table_x (
>>>>>    id int auto_increment primary key,
>>>>>    body text
>>>>>    )
>>>>> """
>>>>> db = MySQLdb.connect(passwd="",db="test")
>>>>> c = db.cursor()
>>>>> body = "0....5...."*28
>>>>> sql = "INSERT INTO table_x (body) VALUES('%s')"
>>>>> c.execute( sql % (body))
>>>>> c.execute("select body from table_x")
>>>>> for row in c.fetchall():
>>>>>      print len(row[0])
>>>>>
>>>>> i wont bet on fetching 1Mb.
>>>>>
>>>>> cheers
>>>> You've just repeated the original poster's code. As I suggested the problem
>>>> is that he's using string substitution when he should be using parameter
>>>> substitution. In these cases %s means two completely different things.
>>> yes but it works here, even with ``body = "0....5...."*280`` so i cannot
>>> blame it on not using 'paramstyle', or what ?
>>>
>>>> Please read the sections on 'paramstyle' and the cursor '.execute' method in
>>>> the DB-API 2.0 definition [1].
>>> of course you are right it is ::
>>>
>>>    c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body))
>>>
>>> or paramstyle without quotes ::
>>>
>>>    c.execute(INSERT INTO table_x (body) VALUES(%s)", (body))
>>>
>>> but both work here with bodylength up to 65535 bytes not 255.
>>>
>>> sorry for dupplication, but i fail to see the point.
>>>
>>>
>> Ok, I'm almost there. Thank you.
>> So i finally realized that the length is not a problem but the actual
>> body. The body includes characters such as " ' ", so ( I'm, I'll, etc)
>> which cause it to give an error.
>>
>>>>> body="helo I'm here,I'll be there"
>>>>> body=body *15
>> When i try :
>> c.execute("Insert into table_x(body) VALUES(%s)",(body))
>> This will work but how do i make :
>>
>> sql="Insert into table_x(body) VALUES(%s)",(body)
>> c.execute(sql)  #complains that i give it a tuple, which i did when
>> you look at sql
>>
>> One problem i see here is that it works in one way but no the other.
>> So how do i add id and  make it work both ways?
>>
>> c.execute("Insert into  table_x(id,body) VALUES(%d,%s)",(id),(body))
>> ??does not work
>>
>> I'm not familiar with ('pyformat'      Python extended format codes,
>> e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use
>> "insert into table_x(id,body)VALUES('%d','%s')" % (id,body)
>>
>> -How to account for " I'll " in a body?
>> -When would i HAVE to use pyformat? Can everything be done in
>> 'format'        ANSI C printf format codes,  e.g. '...WHERE name=%s'
>>
> Ok guys, thanks for help. I finally search for how to scape strings and i used:
> 
> "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body)
> 
> where "'''" on each side of s  = double quote + 3x single quote+ double quote
> 
> Thanks again
>> Thanks,
>> Lukasz
>>
> 
> 
> --
> Lukasz Szybalski
> www.lucasmanual.com


No, no, no, no. Do not use string substitution. I was trying to be 
subtle in my earlier messages but that didn't seem to work. In MySQLdb 
%s is a parameter substitution character NOT string substitution. Please 
repeat this thirty times and then come back and read the rest of this 
message.

...

You've done it, right?

OK. Your code examples re-written to use parameter substitution and not 
string substitution are;

 >>> c.execute("Insert into table_x(body) VALUES(%s)",(body))

This is good.

 >>> sql="Insert into table_x(body) VALUES(%s)",(body)
 >>> c.execute(sql)  #complains that i give it a tuple, which i did when 
you look at sql

this should be;

 >>> sql = "INSERT INTO table_x (body) VALUES (%s)"
 >>> c.execute(sql, (body,))

Note that you do parameter substitution when you execute your statement 
not when you assign it to a variable. Also note that the execute method 
expects a tuple hence the comma after 'body'.

 >>> c.execute("Insert into  table_x(id,body) 
VALUES(%d,%s)",(id),(body)) ??does not work

And I'm very glad that it doesn't. You could do this as;

 >>> c.execute("Insert into  table_x(id,body) VALUES(%s, %s)",(id, body))

Or

 >>> sql = "INSERT INTO table_x (id, body) VALUES (%s, %s)"
 >>> c.execute(sql, (id, body))

Note that in my versions of your code the substitution character is 
always %s. This does not mean that it's a string, MySQLdb is smart 
enough to figure out the underlying column's datatype and map whatever 
you provide (within reason of course) as part of the parameter substitution.

 >>> "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body)

This is just not very good at all. If you carry on like this your code 
will almost entirely consist of single and double quotes and you will 
likely go stark raving mad.

Once more, %s is simply a marker in the statement you provide to the 
execute method which says "substitute the next item from the tuple I've 
provided here please". It is not string substitution.

Now hands up for changing the DB-API to remove the pyformat paramstyle? 
If we're going to have options I think we should restrict them to either 
named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC).

Regards,
Andy
-- 
--------------------------------------------------------------------------------
 From the desk of Andrew J Todd esq - http://www.halfcooked.com/

From carsten at uniqsys.com  Wed Apr 19 14:12:51 2006
From: carsten at uniqsys.com (Carsten Haese)
Date: Wed, 19 Apr 2006 08:12:51 -0400
Subject: [DB-SIG] mysql string length?
In-Reply-To: <4445EA8D.6040806@halfcooked.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com>
Message-ID: <1145448771.7406.10.camel@dot.uniqsys.com>

On Wed, 2006-04-19 at 03:45, Andy Todd wrote:
> Now hands up for changing the DB-API to remove the pyformat paramstyle? 
> If we're going to have options I think we should restrict them to either 
> named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC).

I'm all for that with the clarification that 'pyformat' and 'format'
should be deprecated, leaving 'qmark', 'numeric', and 'named'. Of course
this change would have to go into a completely new version of DB-API,
since it would break existing code.

-Carsten



From mal at egenix.com  Wed Apr 19 14:24:10 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Apr 2006 14:24:10 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <4445EA8D.6040806@halfcooked.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com>
Message-ID: <44462BEA.6080006@egenix.com>

Andy Todd wrote:
> Once more, %s is simply a marker in the statement you provide to the 
> execute method which says "substitute the next item from the tuple I've 
> provided here please". It is not string substitution.
> 
> Now hands up for changing the DB-API to remove the pyformat paramstyle? 
> If we're going to have options I think we should restrict them to either 
> named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC).

Since this sort of confusion comes up again and again, I'm +1 on
removing 'pyformat' and 'format' in the next version of the DB-API
(and maybe marking it for deprecation in PEP 249 now).

I've found the qmark style to be sufficient for everything, but
then, I'm biased. For the few cases where binding variables
are not allowed in SQL (e.g. table names), I use standard Python
string formatting. Even then, things look very straight forward
in the code: %s is a Python formatting marker, ? is a bound
parameter.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 19 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From carsten at uniqsys.com  Wed Apr 19 14:48:43 2006
From: carsten at uniqsys.com (Carsten Haese)
Date: Wed, 19 Apr 2006 08:48:43 -0400
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44462BEA.6080006@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com>  <44462BEA.6080006@egenix.com>
Message-ID: <1145450923.7406.27.camel@dot.uniqsys.com>

On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
> I've found the qmark style to be sufficient for everything

I agree. Considering that many modules already support qmark, and those
that currently use 'format'/'pyformat' would have to move to something
else if these are dropped, I'd like to suggest that supporting 'qmark'
style should become a required feature. Modules would be free to also
implement 'named' and/or 'numeric', but supporting 'qmark' should be
mandatory.

-Carsten



From mal at egenix.com  Wed Apr 19 16:20:53 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Apr 2006 16:20:53 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <1145450923.7406.27.camel@dot.uniqsys.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>
	<44462BEA.6080006@egenix.com>
	<1145450923.7406.27.camel@dot.uniqsys.com>
Message-ID: <44464745.5020409@egenix.com>

Carsten Haese wrote:
> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
>> I've found the qmark style to be sufficient for everything
> 
> I agree. Considering that many modules already support qmark, and those
> that currently use 'format'/'pyformat' would have to move to something
> else if these are dropped, I'd like to suggest that supporting 'qmark'
> style should become a required feature. Modules would be free to also
> implement 'named' and/or 'numeric', but supporting 'qmark' should be
> mandatory.

I'm not sure whether that's easily possible for database modules
which rely on different marker mechanisms, e.g. some modules
probably use the implicit type information in the pyformat style
to find out which method to use for binding variables.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 19 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From szybalski at gmail.com  Wed Apr 19 16:43:11 2006
From: szybalski at gmail.com (Lukasz Szybalski)
Date: Wed, 19 Apr 2006 09:43:11 -0500
Subject: [DB-SIG] mysql string length?
In-Reply-To: <4445EA8D.6040806@halfcooked.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com>
Message-ID: <804e5c70604190743y614cf9c1lb841a94a6575e10a@mail.gmail.com>

On 4/19/06, Andy Todd <andy47 at halfcooked.com> wrote:
> Lukasz Szybalski wrote:
> > On 4/18/06, Lukasz Szybalski <szybalski at gmail.com> wrote:
> >> On 4/17/06, engelbert.gruber at ssg.co.at <engelbert.gruber at ssg.co.at> wrote:
> >>> On Mon, 17 Apr 2006, Andy Todd wrote:
> >>>
> >>>>>>>>> stmt = "INSERT INTO table_x (body) VALUES (%s)" # [1]
> >>>>>>>>> cursor.execute(stmt, (body,))
> >>>>>> [1] note that there are no quote marks around the %s
> >>>>> this works here (debian testing, python 2.3.5, 4.0.21) ::
> >>>>>
> >>>>> import MySQLdb
> >>>>> """
> >>>>> create table table_x (
> >>>>>    id int auto_increment primary key,
> >>>>>    body text
> >>>>>    )
> >>>>> """
> >>>>> db = MySQLdb.connect(passwd="",db="test")
> >>>>> c = db.cursor()
> >>>>> body = "0....5...."*28
> >>>>> sql = "INSERT INTO table_x (body) VALUES('%s')"
> >>>>> c.execute( sql % (body))
> >>>>> c.execute("select body from table_x")
> >>>>> for row in c.fetchall():
> >>>>>      print len(row[0])
> >>>>>
> >>>>> i wont bet on fetching 1Mb.
> >>>>>
> >>>>> cheers
> >>>> You've just repeated the original poster's code. As I suggested the problem
> >>>> is that he's using string substitution when he should be using parameter
> >>>> substitution. In these cases %s means two completely different things.
> >>> yes but it works here, even with ``body = "0....5...."*280`` so i cannot
> >>> blame it on not using 'paramstyle', or what ?
> >>>
> >>>> Please read the sections on 'paramstyle' and the cursor '.execute' method in
> >>>> the DB-API 2.0 definition [1].
> >>> of course you are right it is ::
> >>>
> >>>    c.execute("INSERT INTO table_x (body) VALUES('%s')"% (body))
> >>>
> >>> or paramstyle without quotes ::
> >>>
> >>>    c.execute(INSERT INTO table_x (body) VALUES(%s)", (body))
> >>>
> >>> but both work here with bodylength up to 65535 bytes not 255.
> >>>
> >>> sorry for dupplication, but i fail to see the point.
> >>>
> >>>
> >> Ok, I'm almost there. Thank you.
> >> So i finally realized that the length is not a problem but the actual
> >> body. The body includes characters such as " ' ", so ( I'm, I'll, etc)
> >> which cause it to give an error.
> >>
> >>>>> body="helo I'm here,I'll be there"
> >>>>> body=body *15
> >> When i try :
> >> c.execute("Insert into table_x(body) VALUES(%s)",(body))
> >> This will work but how do i make :
> >>
> >> sql="Insert into table_x(body) VALUES(%s)",(body)
> >> c.execute(sql)  #complains that i give it a tuple, which i did when
> >> you look at sql
> >>
> >> One problem i see here is that it works in one way but no the other.
> >> So how do i add id and  make it work both ways?
> >>
> >> c.execute("Insert into  table_x(id,body) VALUES(%d,%s)",(id),(body))
> >> ??does not work
> >>
> >> I'm not familiar with ('pyformat'      Python extended format codes,
> >> e.g. '...WHERE name=%(name)s' ), therefore i would prefer to use
> >> "insert into table_x(id,body)VALUES('%d','%s')" % (id,body)
> >>
> >> -How to account for " I'll " in a body?
> >> -When would i HAVE to use pyformat? Can everything be done in
> >> 'format'        ANSI C printf format codes,  e.g. '...WHERE name=%s'
> >>
> > Ok guys, thanks for help. I finally search for how to scape strings and i used:
> >
> > "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body)
> >
> > where "'''" on each side of s  = double quote + 3x single quote+ double quote
> >
> > Thanks again
> >> Thanks,
> >> Lukasz
> >>
> >
> >
> > --
> > Lukasz Szybalski
> > www.lucasmanual.com
>
>
> No, no, no, no. Do not use string substitution. I was trying to be
> subtle in my earlier messages but that didn't seem to work. In MySQLdb
> %s is a parameter substitution character NOT string substitution. Please
> repeat this thirty times and then come back and read the rest of this
> message.
>
> ...
>
> You've done it, right?
>
> OK. Your code examples re-written to use parameter substitution and not
> string substitution are;
>
>  >>> c.execute("Insert into table_x(body) VALUES(%s)",(body))
>
> This is good.
>
>  >>> sql="Insert into table_x(body) VALUES(%s)",(body)
>  >>> c.execute(sql)  #complains that i give it a tuple, which i did when
> you look at sql
>
> this should be;
>
>  >>> sql = "INSERT INTO table_x (body) VALUES (%s)"
>  >>> c.execute(sql, (body,))
>
> Note that you do parameter substitution when you execute your statement
> not when you assign it to a variable. Also note that the execute method
> expects a tuple hence the comma after 'body'.
>
>  >>> c.execute("Insert into  table_x(id,body)
> VALUES(%d,%s)",(id),(body)) ??does not work
>
> And I'm very glad that it doesn't. You could do this as;
>
>  >>> c.execute("Insert into  table_x(id,body) VALUES(%s, %s)",(id, body))
>
> Or
>
>  >>> sql = "INSERT INTO table_x (id, body) VALUES (%s, %s)"
>  >>> c.execute(sql, (id, body))


Sounds good. And just to make it even more clear, if i had 5 columns then :

col3='hello'
col4='bye'
col5='example'

sql = "insert into table_x( id, body, col3, col4, col5) values(%s,%s,%s,%s,%s)"
c.execute(sql, (id,body,col3,col4,col5))

Great, this object/parameter substitution works like magic!

Thanks again.
Lukasz

> Note that in my versions of your code the substitution character is
> always %s. This does not mean that it's a string, MySQLdb is smart
> enough to figure out the underlying column's datatype and map whatever
> you provide (within reason of course) as part of the parameter substitution.
>
>  >>> "insert into table(id,body) values ('%d',"'''"%s"'''")"%(id,body)
>
> This is just not very good at all. If you carry on like this your code
> will almost entirely consist of single and double quotes and you will
> likely go stark raving mad.
>
> Once more, %s is simply a marker in the statement you provide to the
> execute method which says "substitute the next item from the tuple I've
> provided here please". It is not string substitution.
>
> Now hands up for changing the DB-API to remove the pyformat paramstyle?
> If we're going to have options I think we should restrict them to either
> named parameters (a la cx_Oracle) or the '?' qmark (a la ODBC and JDBC).
>
> Regards,
> Andy
> --
> --------------------------------------------------------------------------------
>  From the desk of Andrew J Todd esq - http://www.halfcooked.com/
>

From gh at ghaering.de  Wed Apr 19 16:46:04 2006
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Wed, 19 Apr 2006 16:46:04 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44464745.5020409@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>
	<44464745.5020409@egenix.com>
Message-ID: <44464D2C.9020601@ghaering.de>

M.-A. Lemburg wrote:
> Carsten Haese wrote:
> 
>>On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
>>
>>>I've found the qmark style to be sufficient for everything
>>
>>I agree. Considering that many modules already support qmark, and those
>>that currently use 'format'/'pyformat' would have to move to something
>>else if these are dropped, I'd like to suggest that supporting 'qmark'
>>style should become a required feature. Modules would be free to also
>>implement 'named' and/or 'numeric', but supporting 'qmark' should be
>>mandatory.
>  
> I'm not sure whether that's easily possible for database modules
> which rely on different marker mechanisms, e.g. some modules
> probably use the implicit type information in the pyformat style
> to find out which method to use for binding variables.

The ones I know about (pysqlite 1.x, pyPgSQL, psycopg, MySQLdb) all only 
look at the Python values, not at the pyformat format character for 
deciding how to bind the variable.

-- Gerhard


From carsten at uniqsys.com  Wed Apr 19 16:53:52 2006
From: carsten at uniqsys.com (Carsten Haese)
Date: Wed, 19 Apr 2006 10:53:52 -0400
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44464745.5020409@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com>  <44462BEA.6080006@egenix.com>
	<1145450923.7406.27.camel@dot.uniqsys.com>
	<44464745.5020409@egenix.com>
Message-ID: <1145458432.7406.44.camel@dot.uniqsys.com>

On Wed, 2006-04-19 at 10:20, M.-A. Lemburg wrote:
> Carsten Haese wrote:
> > On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
> >> I've found the qmark style to be sufficient for everything
> > 
> > I agree. Considering that many modules already support qmark, and those
> > that currently use 'format'/'pyformat' would have to move to something
> > else if these are dropped, I'd like to suggest that supporting 'qmark'
> > style should become a required feature. Modules would be free to also
> > implement 'named' and/or 'numeric', but supporting 'qmark' should be
> > mandatory.
> 
> I'm not sure whether that's easily possible for database modules
> which rely on different marker mechanisms, e.g. some modules
> probably use the implicit type information in the pyformat style
> to find out which method to use for binding variables.

I don't follow. I thought we're voting on dropping 'pyformat' as a
parameter style. Also, how does pyformat contain implicit type
information?

-Carsten



From daniel.dittmar at sap.com  Wed Apr 19 16:02:15 2006
From: daniel.dittmar at sap.com (Dittmar, Daniel)
Date: Wed, 19 Apr 2006 16:02:15 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <1145450923.7406.27.camel@dot.uniqsys.com>
Message-ID: <AF2DBDAE7523734B8E8FDC3EAE6BCCE70330C223@dewdfe23.wdf.sap.corp>

>On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
>> I've found the qmark style to be sufficient for everything

This is more about convenience (same as Python positional arguments vs.
keyword arguments). Celko's book on SQL tree structures is full of
examples using :names, with the same name appearing sevveral times in
the same query.

But I guess it's not that difficult to write an SQL tokenizer that can
translate named to qmark and build a map from name to indices.

>
>I agree. Considering that many modules already support qmark, and those
>that currently use 'format'/'pyformat' would have to move to something
>else if these are dropped, I'd like to suggest that supporting 'qmark'
>style should become a required feature. Modules would be free to also
>implement 'named' and/or 'numeric', but supporting 'qmark' should be
>mandatory.

But how should a module support multiple notations?
- by tokenizing any SQL statement to check which paramstyle was used
- by having an option on the connection or the cursor
- by having an additional method which requires qmark placeholders (this
one would be backwards compatible)

And while we're discussing annoying differences between drivers: aren't
there some drivers that define 
def execute (stmt, *args)
instead of 
def execute (stmt, args = [])

Daniel

-- 
Daniel Dittmar
SAP Labs Berlin
daniel.dittmar at sap.com  

From szybalski at gmail.com  Wed Apr 19 16:56:30 2006
From: szybalski at gmail.com (Lukasz Szybalski)
Date: Wed, 19 Apr 2006 09:56:30 -0500
Subject: [DB-SIG] Is there a fox pro substitute for linux?
Message-ID: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>

Hello again,
Not sure if this is a right place to ask this but is there a "fox pro"
substitute in Linux or a program that would act in a similar way but
be platform independent (Linux,mac,win)?

Fox pro aka, GUI way to design layout for database results, drug and
drop, adjust size by mouse, create GUI for entering data in few
minutes, and you just have to create sql.

I assume something written in Java, or python. But I'm not sure if it
exists or not. Does anybody came across something similar?

Thx
Lukasz

From mal at egenix.com  Wed Apr 19 17:04:49 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Apr 2006 17:04:49 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44464D2C.9020601@ghaering.de>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>
	<44464D2C.9020601@ghaering.de>
Message-ID: <44465191.5090600@egenix.com>

Gerhard H?ring wrote:
> M.-A. Lemburg wrote:
>> Carsten Haese wrote:
>>
>>> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
>>>
>>>> I've found the qmark style to be sufficient for everything
>>> I agree. Considering that many modules already support qmark, and those
>>> that currently use 'format'/'pyformat' would have to move to something
>>> else if these are dropped, I'd like to suggest that supporting 'qmark'
>>> style should become a required feature. Modules would be free to also
>>> implement 'named' and/or 'numeric', but supporting 'qmark' should be
>>> mandatory.
>>  
>> I'm not sure whether that's easily possible for database modules
>> which rely on different marker mechanisms, e.g. some modules
>> probably use the implicit type information in the pyformat style
>> to find out which method to use for binding variables.
> 
> The ones I know about (pysqlite 1.x, pyPgSQL, psycopg, MySQLdb) all only 
> look at the Python values, not at the pyformat format character for 
> deciding how to bind the variable.

If that's the case, then I suppose supporting qmark style
wouldn't be too much of a problem.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 19 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Wed Apr 19 17:06:47 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Apr 2006 17:06:47 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <1145458432.7406.44.camel@dot.uniqsys.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>
	<44462BEA.6080006@egenix.com>	<44464745.5020409@egenix.com>
	<1145458432.7406.44.camel@dot.uniqsys.com>
Message-ID: <44465207.1000003@egenix.com>

Carsten Haese wrote:
> On Wed, 2006-04-19 at 10:20, M.-A. Lemburg wrote:
>> Carsten Haese wrote:
>>> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
>>>> I've found the qmark style to be sufficient for everything
>>> I agree. Considering that many modules already support qmark, and those
>>> that currently use 'format'/'pyformat' would have to move to something
>>> else if these are dropped, I'd like to suggest that supporting 'qmark'
>>> style should become a required feature. Modules would be free to also
>>> implement 'named' and/or 'numeric', but supporting 'qmark' should be
>>> mandatory.
>> I'm not sure whether that's easily possible for database modules
>> which rely on different marker mechanisms, e.g. some modules
>> probably use the implicit type information in the pyformat style
>> to find out which method to use for binding variables.
> 
> I don't follow. I thought we're voting on dropping 'pyformat' as a
> parameter style. Also, how does pyformat contain implicit type
> information?

Markers such as %i and %s could be treated as meaning: bind
these variables as integer and string. This implicit type
information might be vital to some modules.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 19 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Wed Apr 19 17:13:36 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Apr 2006 17:13:36 +0200
Subject: [DB-SIG] Is there a fox pro substitute for linux?
In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>
References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>
Message-ID: <444653A0.5020508@egenix.com>

Lukasz Szybalski wrote:
> Hello again,
> Not sure if this is a right place to ask this but is there a "fox pro"
> substitute in Linux or a program that would act in a similar way but
> be platform independent (Linux,mac,win)?
> 
> Fox pro aka, GUI way to design layout for database results, drug and
> drop, adjust size by mouse, create GUI for entering data in few
> minutes, and you just have to create sql.
> 
> I assume something written in Java, or python. But I'm not sure if it
> exists or not. Does anybody came across something similar?

You might be looking for something like theKompany's Rekall:

	http://www.thekompany.com/products/rekall/

You can script it using Python, so it's vaguely related to this
mailing list :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 19 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From plblists at iotk.com  Wed Apr 19 17:09:32 2006
From: plblists at iotk.com (Peter L. Buschman)
Date: Wed, 19 Apr 2006 17:09:32 +0200
Subject: [DB-SIG] Is there a fox pro substitute for linux?
In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com
 >
References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>
Message-ID: <7.0.1.0.2.20060419170856.03d57470@iotk.com>


Check out Dabo :)

http://dabodev.com/

--PLB

At 16:56 19.4.06, Lukasz Szybalski wrote:
>Hello again,
>Not sure if this is a right place to ask this but is there a "fox pro"
>substitute in Linux or a program that would act in a similar way but
>be platform independent (Linux,mac,win)?
>
>Fox pro aka, GUI way to design layout for database results, drug and
>drop, adjust size by mouse, create GUI for entering data in few
>minutes, and you just have to create sql.
>
>I assume something written in Java, or python. But I'm not sure if it
>exists or not. Does anybody came across something similar?
>
>Thx
>Lukasz
>_______________________________________________
>DB-SIG maillist  -  DB-SIG at python.org
>http://mail.python.org/mailman/listinfo/db-sig



From ianb at colorstudy.com  Wed Apr 19 17:21:09 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 19 Apr 2006 10:21:09 -0500
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44465191.5090600@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>
	<44465191.5090600@egenix.com>
Message-ID: <44465565.8090801@colorstudy.com>

M.-A. Lemburg wrote:
>>The ones I know about (pysqlite 1.x, pyPgSQL, psycopg, MySQLdb) all only 
>>look at the Python values, not at the pyformat format character for 
>>deciding how to bind the variable.
> 
> 
> If that's the case, then I suppose supporting qmark style
> wouldn't be too much of a problem.

AFAIK, they all use %, so you have to do execute("%s like 'foo%%'", 
(value,)); this is quite different from qmark.



-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From gh at ghaering.de  Wed Apr 19 17:38:09 2006
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Wed, 19 Apr 2006 17:38:09 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44465191.5090600@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>
	<44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com>
Message-ID: <44465961.1090904@ghaering.de>

M.-A. Lemburg wrote:
> [...]
> If that's the case, then I suppose supporting qmark style
> wouldn't be too much of a problem.

It's doable, but it requires parsing the SQL, i. e. find all the qmarks 
that are not inside SQL string literals.

Databases like MySQL, PostgreSQL (*) and SQLite 2.x don't natively 
support parameter binding, so the Python adapter will end up quoting 
parameters and stuffing them in the query string.

Which is why these modules just use (py)format now, then they only need 
to quote the parameters appropriately. The rest they get for free from 
Python.

Abolishing (py)format certainly means additional work for the module 
authors. If we do so - I haven't read the whole thread, so I don't know 
what the arguments are for it - then we should include example code in 
the DB-API for parsing qmarks out of ANSI SQL statements.

-- Gerhard

(*) Only in a stupid, nontransparent form.

From carsten at uniqsys.com  Wed Apr 19 18:13:44 2006
From: carsten at uniqsys.com (Carsten Haese)
Date: Wed, 19 Apr 2006 12:13:44 -0400
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44465961.1090904@ghaering.de>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>
	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>
	<44464D2C.9020601@ghaering.de> <44465191.5090600@egenix.com>
	<44465961.1090904@ghaering.de>
Message-ID: <1145463224.7406.83.camel@dot.uniqsys.com>

On Wed, 2006-04-19 at 11:38, Gerhard H?ring wrote:
> Abolishing (py)format certainly means additional work for the module 
> authors. If we do so - I haven't read the whole thread, so I don't know 
> what the arguments are for it - then we should include example code in 
> the DB-API for parsing qmarks out of ANSI SQL statements.

The main argument for abolishing (py)format is that it blurs the line
between parameter passing, which is good, and hand-rolling a query via
string substitution, which is bad because it invites SQL injections if
not done carefully, and it's almost never done carefully.

Especially newbies seem to have a problem with telling the two apart and
understanding why parameter binding is better than string substitution.
Abolishing %s should make it a lot easier to clearly separate the two
concepts.

Ian also brought up the point that implementations that use (py)format
have a rather ugly wart: Literal % signs in queries have to be doubled
up to prevent accidental parameter markers. This is ugly and makes
writing portable code unnecessarily hard.

I agree that if we decide to abolish (py)format, we should help out
module authors for databases that don't natively support '?' by
providing example code for performing the necessary parsing.

-Carsten



From leif at solumslekt.org  Wed Apr 19 17:22:32 2006
From: leif at solumslekt.org (Leif B. Kristensen)
Date: Wed, 19 Apr 2006 17:22:32 +0200
Subject: [DB-SIG] Is there a fox pro substitute for linux?
In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>
References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>
Message-ID: <200604191722.32386.leif@solumslekt.org>

On Wednesday 19 April 2006 16:56, Lukasz Szybalski wrote:
>Hello again,
>Not sure if this is a right place to ask this but is there a "fox pro"
>substitute in Linux or a program that would act in a similar way but
>be platform independent (Linux,mac,win)?
>
>Fox pro aka, GUI way to design layout for database results, drug and
>drop, adjust size by mouse, create GUI for entering data in few
>minutes, and you just have to create sql.
>
>I assume something written in Java, or python. But I'm not sure if it
>exists or not. Does anybody came across something similar?

Dabo <http://dabodev.com/> is probably the closest you'll get. I haven't 
tried it seriously, mostly because of the poor documentation, but they 
had a quite an active and helpful mailing list at the time I looked 
into it about half a year ago.

Personally, I prefer plain old PHP. If you know something about HTML 
forms, you can churn out a Web interface with it in no time. The 
downside is of course the inherent statelessness of the http protocol.
-- 
Leif Biberg Kristensen | Registered Linux User #338009
http://solumslekt.org/ | Cruising with Gentoo/KDE

From ianb at colorstudy.com  Wed Apr 19 18:29:03 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 19 Apr 2006 11:29:03 -0500
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <1145463224.7406.83.camel@dot.uniqsys.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>
	<44465191.5090600@egenix.com>	<44465961.1090904@ghaering.de>
	<1145463224.7406.83.camel@dot.uniqsys.com>
Message-ID: <4446654F.3020704@colorstudy.com>

Carsten Haese wrote:
> On Wed, 2006-04-19 at 11:38, Gerhard H?ring wrote:
> 
>>Abolishing (py)format certainly means additional work for the module 
>>authors. If we do so - I haven't read the whole thread, so I don't know 
>>what the arguments are for it - then we should include example code in 
>>the DB-API for parsing qmarks out of ANSI SQL statements.
> 
> 
> The main argument for abolishing (py)format is that it blurs the line
> between parameter passing, which is good, and hand-rolling a query via
> string substitution, which is bad because it invites SQL injections if
> not done carefully, and it's almost never done carefully.

It's not *so* bad, because this works:

   execute('... where x = %s', (x,))

And this will fail immediately:

   execute('... where x = %s' % (x,))

It's only repr() which really confuses things.  Well, that and cases 
where you have integers.  Anyway, it's significantly better than PHP.

> Especially newbies seem to have a problem with telling the two apart and
> understanding why parameter binding is better than string substitution.
> Abolishing %s should make it a lot easier to clearly separate the two
> concepts.
> 
> Ian also brought up the point that implementations that use (py)format
> have a rather ugly wart: Literal % signs in queries have to be doubled
> up to prevent accidental parameter markers. This is ugly and makes
> writing portable code unnecessarily hard.
> 
> I agree that if we decide to abolish (py)format, we should help out
> module authors for databases that don't natively support '?' by
> providing example code for performing the necessary parsing.

When using a higher-level SQL generator the parsing overhead would be 
unnecessary, as such a library can produce proper pyformat queries 
itself.  Using a parser and whatnot might be more reasonable if the user 
could select the format in some manner; then libraries that didn't care 
could stick with the most efficient one for the driver.

OK, crazy idea: use \x00 as a marker, which requires no quoting or 
parsing (as it can't be included in any SQL literal anyway).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From python at venix.com  Wed Apr 19 18:50:22 2006
From: python at venix.com (Python)
Date: Wed, 19 Apr 2006 12:50:22 -0400
Subject: [DB-SIG] Is there a fox pro substitute for linux?
In-Reply-To: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>
References: <804e5c70604190756m54fa317ew57e47bb29fc51624@mail.gmail.com>
Message-ID: <1145465422.6268.90.camel@www.venix.com>

On Wed, 2006-04-19 at 09:56 -0500, Lukasz Szybalski wrote:
> Hello again,
> Not sure if this is a right place to ask this but is there a "fox pro"
> substitute in Linux or a program that would act in a similar way but
> be platform independent (Linux,mac,win)?
> 
> Fox pro aka, GUI way to design layout for database results, drug and
> drop, adjust size by mouse, create GUI for entering data in few
> minutes, and you just have to create sql.
> 
> I assume something written in Java, or python. But I'm not sure if it
> exists or not. Does anybody came across something similar?

http://dabodev.com/

The key developers on this project are FoxPro veterans and are trying to
incorporate FoxPro features that they felt were important.  They've
presented at the last 2 PyCons.

> 
> Thx
> Lukasz
> _______________________________________________
> DB-SIG maillist  -  DB-SIG at python.org
> http://mail.python.org/mailman/listinfo/db-sig
-- 
Lloyd Kvam
Venix Corp


From ianb at colorstudy.com  Wed Apr 19 19:00:44 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 19 Apr 2006 12:00:44 -0500
Subject: [DB-SIG] SQL-API
Message-ID: <44466CBC.3000502@colorstudy.com>

Hi guys.

So, I've been meaning to bring this up here, and what with the param 
marker discussions...

Anyway, while doing the SQLObject 2 rewrite I wanted to extract some of 
the machinery that has nothing to do with ORM stuff.  The result is 
SQL-API: http://sqlobject.org/sqlapi/

Instead of fixing problems with DB-API, it tries more to wrap existing 
DB-API drivers and give them a more consistent, higher-level interface. 
  It's also intended to be pretty neutral about everything else, and it 
is entirely procedural.

I'd really like to move this forward so that it can serve the purposes 
of more than just SQLObject 2, so I'm interested in other consumers, 
particularly people using very different strategies from SQLObject. 
There's a lot of tricky aspects to using the DB-API that I think can 
really be solved to everyone's satisfaction once, instead of each system 
having to implement these things themselves -- things like 
configuration, logging, pooling, exceptions, and also a place to hang 
other more complex stuff like database portability information, or the 
occassional driver monkeypatch.

Both the scope and implementation of SQL-API is fairly open at this 
point, so I'm very open to suggestions or contributions.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From mal at egenix.com  Wed Apr 19 20:36:21 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 19 Apr 2006 20:36:21 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <4446654F.3020704@colorstudy.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>	<44465191.5090600@egenix.com>	<44465961.1090904@ghaering.de>	<1145463224.7406.83.camel@dot.uniqsys.com>
	<4446654F.3020704@colorstudy.com>
Message-ID: <44468325.3050007@egenix.com>

Ian Bicking wrote:
> Carsten Haese wrote:
>> I agree that if we decide to abolish (py)format, we should help out
>> module authors for databases that don't natively support '?' by
>> providing example code for performing the necessary parsing.
> 
> When using a higher-level SQL generator the parsing overhead would be 
> unnecessary, as such a library can produce proper pyformat queries 
> itself.  Using a parser and whatnot might be more reasonable if the user 
> could select the format in some manner; then libraries that didn't care 
> could stick with the most efficient one for the driver.

Actually, I don't think that parsing SQL is really necessary
at all: in all the years I've used qmark style, I've never come
across a situation where a SQL literal would include a question
mark.

In reality, it all boils down to doing a simple search for
'?' in the string - after all, you usually pass strings in via
bound parameters.

> OK, crazy idea: use \x00 as a marker, which requires no quoting or 
> parsing (as it can't be included in any SQL literal anyway).

Not sure about that one: some systems such as e.g. Zope use
\x00 to delimit multi-line SQL statements.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 19 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From Chris.Clark at ingres.com  Thu Apr 20 22:53:36 2006
From: Chris.Clark at ingres.com (Chris Clark)
Date: Thu, 20 Apr 2006 13:53:36 -0700
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44465207.1000003@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<44464745.5020409@egenix.com>	<1145458432.7406.44.camel@dot.uniqsys.com>
	<44465207.1000003@egenix.com>
Message-ID: <4447F4D0.1020303@ingres.com>

My vote - +1 for making qmark mandatory and everything else optional

My ramblings -

M.-A. Lemburg wrote:
> Markers such as %i and %s could be treated as meaning: bind
> these variables as integer and string. This implicit type
> information might be vital to some modules.
>
>   
I have to confess I couldn't figure out from the v2 spec if this was 
implied or not for pyformat - I've certainly seen dbi users (e.g. 
Django), using %s irrespective of the Python or DBMS type; relying on on 
the host DBMS to then coerce, e.g. into an integer column.

The way I read the spec was that the constructors in the "Type Objects 
and Constructors" section in pep 249 are to be used to show the 
destination type. E.g.:

    z=23
    x.execute("select ?", (dbmodule.NUMBER(z),) )

I'm definitely a +1 for making qmark mandatory and everything else 
optional. Question marks are standard in SQL across a few api's, e.g. 
ODBC, embedded (sql) C, JDBC, etc.

Gerhard suggestion of example code for parsing is a good one, but 
someone would have pony up some code :-) Some drivers are pure C (so 
they would need pure C code) and others use some Python code and they 
would prefer Python so this (apparently) simple request may not get 
fulfilled :-(


(in a later email) M.-A. Lemburg wrote:
> Actually, I don't think that parsing SQL is really necessary
> at all: in all the years I've used qmark style, I've never come
> across a situation where a SQL literal would include a question
> mark.
>
> In reality, it all boils down to doing a simple search for
> '?' in the string - after all, you usually pass strings in via
> bound parameters.
For most real world applications I think this is a reasonable 
assumption, however there is always a an edge case and I wouldn't be 
surprised if some one on the list says, "Hang on! I've got some dynamic 
queries with string literals containing question marks". But for initial 
testing:

    sql_query = "select foo from bar where a = ? and b like '%slugs'"
    new_sql_query = sql_query.replace('%', '%%') # deal with embedded 
percent signs
    new_sql_query = new_sql_query.replace('?', '%s')  # will NOT deal 
with embedded qmarks in string literals!
    ## what ever pyformat driver currently does for pyformat queries

is probably a resonable short term testing solution until a more robust 
solution is found for drivers that don't natively support parameter binding.

Chris



From dieter at handshake.de  Thu Apr 20 20:08:48 2006
From: dieter at handshake.de (Dieter Maurer)
Date: Thu, 20 Apr 2006 20:08:48 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44468325.3050007@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<20060415120141.GA12186@yapgi.hopto.org>
	<1145139880.8861.9.camel@localhost.localdomain>
	<44421F81.4080702@halfcooked.com>
	<Pine.LNX.4.64.0604161508390.3514@lx3.local>
	<444363CA.2030304@halfcooked.com>
	<Pine.LNX.4.64.0604171448250.7805@lx3.local>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com>
	<1145450923.7406.27.camel@dot.uniqsys.com>
	<44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de>
	<44465191.5090600@egenix.com> <44465961.1090904@ghaering.de>
	<1145463224.7406.83.camel@dot.uniqsys.com>
	<4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com>
Message-ID: <17479.52784.244921.297972@gargle.gargle.HOWL>

M.-A. Lemburg wrote at 2006-4-19 20:36 +0200:
> ...
>Actually, I don't think that parsing SQL is really necessary
>at all: in all the years I've used qmark style, I've never come
>across a situation where a SQL literal would include a question
>mark.

I do not think this is a good idea. '?' are perfectly legal
in string literals and there they conventionally
do *NOT* mean "insert a parameter".

>In reality, it all boils down to doing a simple search for
>'?' in the string - after all, you usually pass strings in via
>bound parameters.

Zope, at least, would be unhappy, as it does not (yet) support
bound parameters.


-- 
Dieter

From ianb at colorstudy.com  Thu Apr 20 23:15:32 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 20 Apr 2006 16:15:32 -0500
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44468325.3050007@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>	<44465191.5090600@egenix.com>	<44465961.1090904@ghaering.de>	<1145463224.7406.83.camel@dot.uniqsys.com>
	<4446654F.3020704@colorstudy.com> <44468325.3050007@egenix.com>
Message-ID: <4447F9F4.30607@colorstudy.com>

M.-A. Lemburg wrote:
>>OK, crazy idea: use \x00 as a marker, which requires no quoting or 
>>parsing (as it can't be included in any SQL literal anyway).
> 
> 
> Not sure about that one: some systems such as e.g. Zope use
> \x00 to delimit multi-line SQL statements.

\x01 then!


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Thu Apr 20 23:29:37 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 20 Apr 2006 16:29:37 -0500
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <C06D6705.824C%paul@snake.net>
References: <C06D6705.824C%paul@snake.net>
Message-ID: <4447FD41.9080506@colorstudy.com>

Paul DuBois wrote:
> On 4/20/06 13:08, "Dieter Maurer" <dieter at handshake.de> wrote:
> 
> 
>>M.-A. Lemburg wrote at 2006-4-19 20:36 +0200:
>>
>>>...
>>>Actually, I don't think that parsing SQL is really necessary
>>>at all: in all the years I've used qmark style, I've never come
>>>across a situation where a SQL literal would include a question
>>>mark.
>>
>>I do not think this is a good idea. '?' are perfectly legal
>>in string literals and there they conventionally
>>do *NOT* mean "insert a parameter".
> 
> 
> Umm ... thing is, the ? placeholder markers in a SQL string _don't_ occur
> within string literals.  That is, when you construct a SQL statement
> containing placeholder markers, you don't write the placeholders within
> quote marks.  The parameter substitution mechanism adds quotes as necessary
> when it substitutes a data value for a placeholder.

He's referring to something like this:

   def get_androgenous_users(state):
       execute("SELECT * FROM users WHERE state = ? and gender = '?',
               (state,))
       ...

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From paul at snake.net  Thu Apr 20 23:27:49 2006
From: paul at snake.net (Paul DuBois)
Date: Thu, 20 Apr 2006 16:27:49 -0500
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <17479.52784.244921.297972@gargle.gargle.HOWL>
Message-ID: <C06D6705.824C%paul@snake.net>

On 4/20/06 13:08, "Dieter Maurer" <dieter at handshake.de> wrote:

> M.-A. Lemburg wrote at 2006-4-19 20:36 +0200:
>> ...
>> Actually, I don't think that parsing SQL is really necessary
>> at all: in all the years I've used qmark style, I've never come
>> across a situation where a SQL literal would include a question
>> mark.
> 
> I do not think this is a good idea. '?' are perfectly legal
> in string literals and there they conventionally
> do *NOT* mean "insert a parameter".

Umm ... thing is, the ? placeholder markers in a SQL string _don't_ occur
within string literals.  That is, when you construct a SQL statement
containing placeholder markers, you don't write the placeholders within
quote marks.  The parameter substitution mechanism adds quotes as necessary
when it substitutes a data value for a placeholder.

If this were not true, then binding NULL to a placeholder would not be
possible.  If you wrote '?' rather than ?, NULL would incorrectly become
'NULL' and thus would itself become a literal string in the statement.


>> In reality, it all boils down to doing a simple search for
>> '?' in the string - after all, you usually pass strings in via
>> bound parameters.
> 
> Zope, at least, would be unhappy, as it does not (yet) support
> bound parameters.
> 



From farcepest at gmail.com  Fri Apr 21 07:37:06 2006
From: farcepest at gmail.com (Andy Dustman)
Date: Fri, 21 Apr 2006 01:37:06 -0400
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <44465961.1090904@ghaering.de>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>
	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>
	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>
	<4445EA8D.6040806@halfcooked.com> <44462BEA.6080006@egenix.com>
	<1145450923.7406.27.camel@dot.uniqsys.com>
	<44464745.5020409@egenix.com> <44464D2C.9020601@ghaering.de>
	<44465191.5090600@egenix.com> <44465961.1090904@ghaering.de>
Message-ID: <9826f3800604202237x2c475ed7tf60aa1513b44b247@mail.gmail.com>

On 4/19/06, Gerhard H?ring <gh at ghaering.de> wrote:
> M.-A. Lemburg wrote:
> > [...]
> > If that's the case, then I suppose supporting qmark style
> > wouldn't be too much of a problem.
>
> It's doable, but it requires parsing the SQL, i. e. find all the qmarks
> that are not inside SQL string literals.
>
> Databases like MySQL, PostgreSQL (*) and SQLite 2.x don't natively
> support parameter binding, so the Python adapter will end up quoting
> parameters and stuffing them in the query string.

-1 on making qmark mandatory, even though I am constantly having to
correct newbies who see %s and think, "I must use a string formatting
operator since I am too ignorant to read PEP-249".

On the other hand, MySQL-4.1 and newer *does* support ? as a
placeholder *iff* you are using the prepared statement API, which
MySQLdb does not implement *yet*, but that is one of the next things
to do. One complication is, not all SQL statements will work with the
prepared statements API, and there is a lot of code out there that
correctly uses %s. I am probably going to have to have some means for
indicating what paramstyle is in use by the application, so the driver
can transliterate it if it needs to, and that's one of the things I
have to figure out for MySQLdb-1.3/1.4/2.0.

--
The Pythonic Principle: Python works the way it does
because if it didn't, it wouldn't be Python.

From mal at egenix.com  Fri Apr 21 13:30:20 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 21 Apr 2006 13:30:20 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <9826f3800604202237x2c475ed7tf60aa1513b44b247@mail.gmail.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>
	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>
	<44464D2C.9020601@ghaering.de>	<44465191.5090600@egenix.com>
	<44465961.1090904@ghaering.de>
	<9826f3800604202237x2c475ed7tf60aa1513b44b247@mail.gmail.com>
Message-ID: <4448C24C.7030700@egenix.com>

Andy Dustman wrote:
> On 4/19/06, Gerhard H?ring <gh at ghaering.de> wrote:
>> M.-A. Lemburg wrote:
>>> [...]
>>> If that's the case, then I suppose supporting qmark style
>>> wouldn't be too much of a problem.
>> It's doable, but it requires parsing the SQL, i. e. find all the qmarks
>> that are not inside SQL string literals.
>>
>> Databases like MySQL, PostgreSQL (*) and SQLite 2.x don't natively
>> support parameter binding, so the Python adapter will end up quoting
>> parameters and stuffing them in the query string.
> 
> -1 on making qmark mandatory, even though I am constantly having to
> correct newbies who see %s and think, "I must use a string formatting
> operator since I am too ignorant to read PEP-249".
> 
> On the other hand, MySQL-4.1 and newer *does* support ? as a
> placeholder *iff* you are using the prepared statement API, which
> MySQLdb does not implement *yet*, but that is one of the next things
> to do. One complication is, not all SQL statements will work with the
> prepared statements API, and there is a lot of code out there that
> correctly uses %s. I am probably going to have to have some means for
> indicating what paramstyle is in use by the application, so the driver
> can transliterate it if it needs to, and that's one of the things I
> have to figure out for MySQLdb-1.3/1.4/2.0.

If we make qmark support mandatory and deprecate pyformat/format
in DB API 2.1 would that be a workable approach ?

I'd say that we schedule DB API 2.1 for later this year with
the main aim of making some of the currently already documented
standard extensions mandatory and maybe adding optional support
for defining type mappings (both input and output).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 21 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From wilk-ml at flibuste.net  Sat Apr 22 20:53:42 2006
From: wilk-ml at flibuste.net (William Dode)
Date: Sat, 22 Apr 2006 18:53:42 +0000 (UTC)
Subject: [DB-SIG] how to know the connexion is closed
Message-ID: <e2du3m$83o$1@sea.gmane.org>

Hi,

I search for a generic way (for psycopg, mysqldb, adodbapi) to
know that the connexion is closed, for example because the database
server was restarted. I mean for wich database exceptions should i ask
again for a connexion ?

thanks

-- 
William Dod? - http://flibuste.net


From mal at egenix.com  Mon Apr 24 20:30:10 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 24 Apr 2006 20:30:10 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <17479.52784.244921.297972@gargle.gargle.HOWL>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>	<44465191.5090600@egenix.com>	<44465961.1090904@ghaering.de>	<1145463224.7406.83.camel@dot.uniqsys.com>	<4446654F.3020704@colorstudy.com>	<44468325.3050007@egenix.com>
	<17479.52784.244921.297972@gargle.gargle.HOWL>
Message-ID: <444D1932.1020503@egenix.com>

Dieter Maurer wrote:
> M.-A. Lemburg wrote at 2006-4-19 20:36 +0200:
>> ...
>> Actually, I don't think that parsing SQL is really necessary
>> at all: in all the years I've used qmark style, I've never come
>> across a situation where a SQL literal would include a question
>> mark.
> 
> I do not think this is a good idea. '?' are perfectly legal
> in string literals and there they conventionally
> do *NOT* mean "insert a parameter".

Right, but if you use bound parameters, you rarely ever
have a need to use question marks in SQL literals.

>> In reality, it all boils down to doing a simple search for
>> '?' in the string - after all, you usually pass strings in via
>> bound parameters.
> 
> Zope, at least, would be unhappy, as it does not (yet) support
> bound parameters.

Would be a good incentive to change this :-)

Of course, scanning the SQL statement string for SQL literals
is safer, so this is what modules should do (at least if the
user passes in bound parameters to the .execute() method).

Does anyone know a good implementation of such a search&replace
mechanism ?

I tried coming up with something using re.sub() but it seems
this is beyond what you can do with re.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 24 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From patriciap.gu at gmail.com  Mon Apr 24 20:35:33 2006
From: patriciap.gu at gmail.com (Patty)
Date: Mon, 24 Apr 2006 18:35:33 +0000 (UTC)
Subject: [DB-SIG] MySQLdb - syntax question
Message-ID: <loom.20060424T201929-224@post.gmane.org>

I have the following code:  

UPDATE mytable SET col1 = %s, col2 = %s, col3 = %s, col4 = %s,
    col5 = %s, col6 = %s   WHERE name = %s
    """, (tuple(entries)))
In this example, entries is a list of numbers and the last element is a name of
a target.

This works fine. But, there will be times when new columns will be added based
on a dictionary I have. I've been asked not to hard code the column names (i.e.
col1, col2, etc.). I need to know if it is possible to put placeholders instead
of column names (i.e. %s=%s -> column name = column value). If so, what is the
right syntax for that?

Thanks,
Patty


From chris at cogdon.org  Mon Apr 24 20:51:19 2006
From: chris at cogdon.org (Chris Cogdon)
Date: Mon, 24 Apr 2006 11:51:19 -0700
Subject: [DB-SIG] MySQLdb - syntax question
In-Reply-To: <loom.20060424T201929-224@post.gmane.org>
References: <loom.20060424T201929-224@post.gmane.org>
Message-ID: <ff968ef86a40608eb3f4107f39dce52d@cogdon.org>


On Apr 24, 2006, at 11:35, Patty wrote:

> I have the following code:
>
> UPDATE mytable SET col1 = %s, col2 = %s, col3 = %s, col4 = %s,
>     col5 = %s, col6 = %s   WHERE name = %s
>     """, (tuple(entries)))
> In this example, entries is a list of numbers and the last element is 
> a name of
> a target.
>
> This works fine. But, there will be times when new columns will be 
> added based
> on a dictionary I have. I've been asked not to hard code the column 
> names (i.e.
> col1, col2, etc.). I need to know if it is possible to put 
> placeholders instead
> of column names (i.e. %s=%s -> column name = column value). If so, 
> what is the
> right syntax for that?

Unfortunately, using the %s syntax is going to put the name of the 
column in quotes, and so MySQL will think you're trying to assign one 
string to another, rather than a string to a column. Instead, you'll 
need to piecemeal your string from parts manually, like this:

d = { ... your dictionary of column names and values }

d_items = d.items ()   # doing it this way to ensure that we dont get 
inconsistent ordering

query_string = "update mytable set " + ",".join ( [ key + "=%s" for 
key, value in d_items ] )

cursor.execute ( query_string, tuple ( [ value for key, value in 
d_items ] ) )


-- 
    ("`-/")_.-'"``-._        Chris Cogdon <chris at cogdon.org>
     . . `; -._    )-;-,_`)
    (v_,)'  _  )`-.\  ``-'
   _.- _..-_/ / ((.'
((,.-'   ((,/   fL


From dieter at handshake.de  Mon Apr 24 21:09:43 2006
From: dieter at handshake.de (Dieter Maurer)
Date: Mon, 24 Apr 2006 21:09:43 +0200
Subject: [DB-SIG] how to know the connexion is closed
In-Reply-To: <e2du3m$83o$1@sea.gmane.org>
References: <e2du3m$83o$1@sea.gmane.org>
Message-ID: <17485.8823.318107.617005@gargle.gargle.HOWL>

William Dode wrote at 2006-4-22 18:53 +0000:
>I search for a generic way (for psycopg, mysqldb, adodbapi) to
>know that the connexion is closed

I fear there is no generic way....

-- 
Dieter

From patriciap.gu at gmail.com  Mon Apr 24 22:37:37 2006
From: patriciap.gu at gmail.com (Patty)
Date: Mon, 24 Apr 2006 20:37:37 +0000 (UTC)
Subject: [DB-SIG] MySQLdb - syntax question
References: <loom.20060424T201929-224@post.gmane.org>
	<ff968ef86a40608eb3f4107f39dce52d@cogdon.org>
Message-ID: <loom.20060424T222904-530@post.gmane.org>


Hi again,

> query_string = "update mytable set " + ",".join ( [ key + "=%s" for 
> key, value in d_items ] )
> 
> cursor.execute ( query_string, tuple ( [ value for key, value in 
> d_items ] ) )
> 


Thanks for your prompt reply. I kinda got it, but the the part that I forgot to
mention was that the values are not retrieved from the dictionary. What I
retrieve from the dictionary are the column names and the target. The values are
in a separate list, so how would it be in this case? I'm sorry if I sound silly,
but I'm new to this.
Thanks!
Patty
 





From chris at cogdon.org  Mon Apr 24 22:58:26 2006
From: chris at cogdon.org (Chris Cogdon)
Date: Mon, 24 Apr 2006 13:58:26 -0700
Subject: [DB-SIG] MySQLdb - syntax question
In-Reply-To: <loom.20060424T222904-530@post.gmane.org>
References: <loom.20060424T201929-224@post.gmane.org>
	<ff968ef86a40608eb3f4107f39dce52d@cogdon.org>
	<loom.20060424T222904-530@post.gmane.org>
Message-ID: <d195d0277a3230bb61e637d9396ea47e@cogdon.org>


On Apr 24, 2006, at 13:37, Patty wrote:

>
> Hi again,
>
>> query_string = "update mytable set " + ",".join ( [ key + "=%s" for
>> key, value in d_items ] )
>>
>> cursor.execute ( query_string, tuple ( [ value for key, value in
>> d_items ] ) )
>>
>
>
> Thanks for your prompt reply. I kinda got it, but the the part that I 
> forgot to
> mention was that the values are not retrieved from the dictionary. 
> What I
> retrieve from the dictionary are the column names and the target. The 
> values are
> in a separate list, so how would it be in this case? I'm sorry if I 
> sound silly,
> but I'm new to this.

I don't know what you mean by 'target' :)

But, that kind of fiddling is more in the python realm than the 
DB/mysql realm... ie, as far as the dbabi side is concerned, the values 
are passed in through a tuple, and connected to the query string with % 
marks, and the column names have to be manually inserted into the query 
string (which is what the + and join items did) because there's no 
dbapi support for manipulating non-literal-value entries in the query 
string.

ie, you can't do this:

colname = "column1"
value = "John"
cursor.execute ( "update mytable set %s=%s", ( colname, value ) )

Because that would result in this SQL:

update mytable set "column1"="John";   # the mysql connector might use 
single quotes, I'm not sure

And that's not what you wanted. Instead, if it was just ONE value, 
you'd do this:

cursor.execute ( "update mytable set "+colname+"=%s", ( value, ) )

Would would send this SQL:

update mytable set column1="John";


Now, my example above assumed all the column names, and values, came 
from one dictionary, and used join and list comprehensions to expand 
everything. They're just python tricks, not mysql or db-api tricks.


-- 
    ("`-/")_.-'"``-._        Chris Cogdon <chris at cogdon.org>
     . . `; -._    )-;-,_`)
    (v_,)'  _  )`-.\  ``-'
   _.- _..-_/ / ((.'
((,.-'   ((,/   fL


From patriciap.gu at gmail.com  Mon Apr 24 23:03:00 2006
From: patriciap.gu at gmail.com (Patty)
Date: Mon, 24 Apr 2006 21:03:00 +0000 (UTC)
Subject: [DB-SIG] MySQLdb - syntax question
References: <loom.20060424T201929-224@post.gmane.org>
	<ff968ef86a40608eb3f4107f39dce52d@cogdon.org>
	<loom.20060424T222904-530@post.gmane.org>
	<d195d0277a3230bb61e637d9396ea47e@cogdon.org>
Message-ID: <loom.20060424T230147-137@post.gmane.org>

Got it!! Thank you!!!!



From Chris.Clark at ingres.com  Mon Apr 24 23:56:46 2006
From: Chris.Clark at ingres.com (Chris Clark)
Date: Mon, 24 Apr 2006 14:56:46 -0700
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <444D1932.1020503@egenix.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>	<44465191.5090600@egenix.com>	<44465961.1090904@ghaering.de>	<1145463224.7406.83.camel@dot.uniqsys.com>	<4446654F.3020704@colorstudy.com>	<44468325.3050007@egenix.com>	<17479.52784.244921.297972@gargle.gargle.HOWL>
	<444D1932.1020503@egenix.com>
Message-ID: <444D499E.4030409@ingres.com>

In reference to finding question marks in SQL queries and ignoring 
string literals

M.-A. Lemburg wrote:
> Does anyone know a good implementation of such a search&replace
> mechanism ?
>   
I've not seen any existing code to deal with this BUT there are a couple 
of places to check:

    * SnakeSQL <http://www.pythonweb.org/projects/snakesql/>
    * gadfly <http://gadfly.sourceforge.net/>


as they are pure Python database engines and so have some SQL parsing 
code in them already that maybe usable.

But you can always cheat and iterate through the SQL string and just 
count (incrementing and decrementing) single quotes.
You need to remember to deal with the special case of embedded single 
quotes (by keeping a mini back buffer) like this:

    'it''s Monty Python'


Chris


From ianb at colorstudy.com  Tue Apr 25 00:19:47 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 24 Apr 2006 17:19:47 -0500
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <444D499E.4030409@ingres.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>	<44465191.5090600@egenix.com>	<44465961.1090904@ghaering.de>	<1145463224.7406.83.camel@dot.uniqsys.com>	<4446654F.3020704@colorstudy.com>	<44468325.3050007@egenix.com>	<17479.52784.244921.297972@gargle.gargle.HOWL>	<444D1932.1020503@egenix.com>
	<444D499E.4030409@ingres.com>
Message-ID: <444D4F03.6070609@colorstudy.com>

Chris Clark wrote:
> In reference to finding question marks in SQL queries and ignoring 
> string literals
> 
> M.-A. Lemburg wrote:
>>Does anyone know a good implementation of such a search&replace
>>mechanism ?
>>  
> 
> I've not seen any existing code to deal with this BUT there are a couple 
> of places to check:
> 
>     * SnakeSQL <http://www.pythonweb.org/projects/snakesql/>
>     * gadfly <http://gadfly.sourceforge.net/>
> 
> 
> as they are pure Python database engines and so have some SQL parsing 
> code in them already that maybe usable.
> 
> But you can always cheat and iterate through the SQL string and just 
> count (incrementing and decrementing) single quotes.
> You need to remember to deal with the special case of embedded single 
> quotes (by keeping a mini back buffer) like this:
> 
>     'it''s Monty Python'

Postgres and MySQL allow 'it\'s Monty Python'.  Parsing SQL for every 
query seems like a bit of a performance problem, especially because the 
parsing itself can be different for different databases.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From bill at celestial.net  Tue Apr 25 01:12:32 2006
From: bill at celestial.net (Bill Campbell)
Date: Mon, 24 Apr 2006 16:12:32 -0700
Subject: [DB-SIG] how to know the connexion is closed
In-Reply-To: <17485.8823.318107.617005@gargle.gargle.HOWL>
References: <e2du3m$83o$1@sea.gmane.org>
	<17485.8823.318107.617005@gargle.gargle.HOWL>
Message-ID: <20060424231232.GA76508@alexis.mi.celestial.com>

On Mon, Apr 24, 2006, Dieter Maurer wrote:
>William Dode wrote at 2006-4-22 18:53 +0000:
>>I search for a generic way (for psycopg, mysqldb, adodbapi) to
>>know that the connexion is closed
>
>I fear there is no generic way....

One could use something like:

try: cur = conn.cursor()
except: cur = None

I would think that attempting to get a cursor from a closed
connection would always raise an error.

Bill
--
INTERNET:   bill at Celestial.COM  Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/  PO Box 820; 6641 E. Mercer Way
FAX:            (206) 232-9186  Mercer Island, WA 98040-0820; (206) 236-1676

``Intellectually, teachers fall between education theorists and bright
cocker spaniels. (Probably closer to the education theorists. The AKC has
been doing wonders with spaniels.) If you think I'm kidding look at the
GREs for education majors, whose scores are the lowest of all fields, and
remember that these are the smart ones.'' -- http://www.FredOnEverything.net

From wilk-ml at flibuste.net  Tue Apr 25 09:24:56 2006
From: wilk-ml at flibuste.net (William Dode)
Date: Tue, 25 Apr 2006 07:24:56 +0000 (UTC)
Subject: [DB-SIG] how to know the connexion is closed
References: <e2du3m$83o$1@sea.gmane.org>
	<17485.8823.318107.617005@gargle.gargle.HOWL>
Message-ID: <e2kis8$kbc$1@sea.gmane.org>

On 24-04-2006, Dieter Maurer wrote:
> William Dode wrote at 2006-4-22 18:53 +0000:
>>I search for a generic way (for psycopg, mysqldb, adodbapi) to
>>know that the connexion is closed
>
> I fear there is no generic way....
>

ok... so, perhaps a solution for postgresql with psycopg ?


-- 
William Dod? - http://flibuste.net


From chris at cogdon.org  Tue Apr 25 09:38:03 2006
From: chris at cogdon.org (Chris Cogdon)
Date: Tue, 25 Apr 2006 00:38:03 -0700
Subject: [DB-SIG] how to know the connexion is closed
In-Reply-To: <e2kis8$kbc$1@sea.gmane.org>
References: <e2du3m$83o$1@sea.gmane.org>
	<17485.8823.318107.617005@gargle.gargle.HOWL>
	<e2kis8$kbc$1@sea.gmane.org>
Message-ID: <E0870464-69DD-42D8-8B76-FFB0DA07F186@cogdon.org>


On Apr 25, 2006, at 12:24 AM, William Dode wrote:

> On 24-04-2006, Dieter Maurer wrote:
>> William Dode wrote at 2006-4-22 18:53 +0000:
>>> I search for a generic way (for psycopg, mysqldb, adodbapi) to
>>> know that the connexion is closed
>>
>> I fear there is no generic way....
>>
>
> ok... so, perhaps a solution for postgresql with psycopg ?

How about trying a statement that doesn't do anything, and if it  
fails with an exception, then it was closed? :)



-- 
    ("`-/")_.-'"``-._        Chris Cogdon <chris at cogdon.org>
     . . `; -._    )-;-,_`)
    (v_,)'  _  )`-.\  ``-'
   _.- _..-_/ / ((.'
((,.-'   ((,/   fL


From wilk-ml at flibuste.net  Tue Apr 25 11:00:17 2006
From: wilk-ml at flibuste.net (William Dode)
Date: Tue, 25 Apr 2006 09:00:17 +0000 (UTC)
Subject: [DB-SIG] how to know the connexion is closed
References: <e2du3m$83o$1@sea.gmane.org>
	<17485.8823.318107.617005@gargle.gargle.HOWL>
	<20060424231232.GA76508@alexis.mi.celestial.com>
Message-ID: <e2kof1$7m2$1@sea.gmane.org>

On 24-04-2006, Bill Campbell wrote:
> On Mon, Apr 24, 2006, Dieter Maurer wrote:
>>William Dode wrote at 2006-4-22 18:53 +0000:
>>>I search for a generic way (for psycopg, mysqldb, adodbapi) to
>>>know that the connexion is closed
>>
>>I fear there is no generic way....
>
> One could use something like:
>
> try: cur = conn.cursor()
> except: cur = None
>
> I would think that attempting to get a cursor from a closed
> connection would always raise an error.

Not with MySQLdb :
>>> cx = MySQLdb.connect(db='scrabble')
>>> cx.close()
>>> cx.cursor()
<MySQLdb.cursors.Cursor object at 0xb7dc1d8c>
(and a query fail with InterfaceError)

with sqlite it raise ProgrammingError
>>> cx = sqlite.connect(':memory:')
>>> cx.close()
>>> cx.cursor()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 552, in cursor
    self._checkNotClosed("cursor")
  File "/usr/lib/python2.3/site-packages/sqlite/main.py", line 482, in _checkNotClosed
    raise _sqlite.ProgrammingError, \
_sqlite.ProgrammingError: cursor failed - Connection is closed.

with psycopg it raise InterfaceError
>>> import psycopg
>>> cx = psycopg.connect('dbname=pytodo')
>>> cx.close()
>>> cx.cursor()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
psycopg.InterfaceError: already closed


Same problem if i try to close a connection already closed
MySQLdb and sqlite raise ProgrammingError
psycopg raise InterfaceError

Don't you think it should be a spec in dbapi ?

-- 
William Dod? - http://flibuste.net


From mal at egenix.com  Tue Apr 25 12:15:52 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 25 Apr 2006 12:15:52 +0200
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <444D4F03.6070609@colorstudy.com>
References: <804e5c70604141000t95cd9b0odb786fed293f748a@mail.gmail.com>	<20060415120141.GA12186@yapgi.hopto.org>	<1145139880.8861.9.camel@localhost.localdomain>	<44421F81.4080702@halfcooked.com>	<Pine.LNX.4.64.0604161508390.3514@lx3.local>	<444363CA.2030304@halfcooked.com>	<Pine.LNX.4.64.0604171448250.7805@lx3.local>	<804e5c70604180846s564ae677wc17064944f3eb553@mail.gmail.com>	<804e5c70604181100i414b0cf6w1da5a578f99380bd@mail.gmail.com>	<4445EA8D.6040806@halfcooked.com>	<44462BEA.6080006@egenix.com>	<1145450923.7406.27.camel@dot.uniqsys.com>	<44464745.5020409@egenix.com>	<44464D2C.9020601@ghaering.de>	<44465191.5090600@egenix.com>	<44465961.1090904@ghaering.de>	<1145463224.7406.83.camel@dot.uniqsys.com>	<4446654F.3020704@colorstudy.com>	<44468325.3050007@egenix.com>	<17479.52784.244921.297972@gargle.gargle.HOWL>	<444D1932.1020503@egenix.com>	<444D499E.4030409@ingres.com>
	<444D4F03.6070609@colorstudy.com>
Message-ID: <444DF6D8.7040608@egenix.com>

Ian Bicking wrote:
> Chris Clark wrote:
>> In reference to finding question marks in SQL queries and ignoring 
>> string literals
>>
>> M.-A. Lemburg wrote:
>>> Does anyone know a good implementation of such a search&replace
>>> mechanism ?
>>>  
>> I've not seen any existing code to deal with this BUT there are a couple 
>> of places to check:
>>
>>     * SnakeSQL <http://www.pythonweb.org/projects/snakesql/>
>>     * gadfly <http://gadfly.sourceforge.net/>
>>
>>
>> as they are pure Python database engines and so have some SQL parsing 
>> code in them already that maybe usable.
>>
>> But you can always cheat and iterate through the SQL string and just 
>> count (incrementing and decrementing) single quotes.
>> You need to remember to deal with the special case of embedded single 
>> quotes (by keeping a mini back buffer) like this:
>>
>>     'it''s Monty Python'
> 
> Postgres and MySQL allow 'it\'s Monty Python'.  Parsing SQL for every 
> query seems like a bit of a performance problem, especially because the 
> parsing itself can be different for different databases.

I don't think that's much of a problem: each database module will
have to implement whatever parsing is needed. In most cases,
this will only have to be some counting of quotes to avoid
replacing question marks inside SQL literals.

You can fast track this by searching for a quote first: in many
cases you won't need to bother with the more complicated SQL
literal aware search&replace at all, if there are no literals
in the SQL statement.

Perhaps we could collect a few quote quoting rules for the various
backends.

So far we have:

* '' - standard SQL quoted quote
* \' - C style quoted quote

We probably also need to look at different ways to write a SQL
literal. Using single quotes is standard SQL, but there might
also be some backends which use or support double quotes.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 25 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From andy47 at halfcooked.com  Sun Apr 30 04:52:38 2006
From: andy47 at halfcooked.com (Andy Todd)
Date: Sun, 30 Apr 2006 12:52:38 +1000
Subject: [DB-SIG] paramstyles (mysql string length?)
In-Reply-To: <AF2DBDAE7523734B8E8FDC3EAE6BCCE70330C223@dewdfe23.wdf.sap.corp>
References: <AF2DBDAE7523734B8E8FDC3EAE6BCCE70330C223@dewdfe23.wdf.sap.corp>
Message-ID: <44542676.2060300@halfcooked.com>

Dittmar, Daniel wrote:
>> On Wed, 2006-04-19 at 08:24, M.-A. Lemburg wrote:
> 
> But how should a module support multiple notations?
> - by tokenizing any SQL statement to check which paramstyle was used
> - by having an option on the connection or the cursor
> - by having an additional method which requires qmark placeholders (this
> one would be backwards compatible)
> 
> 
> Daniel
> 

As paramstyle is currently an attribute of the module perhaps it could 
be inherited by the connection. That way the user (if the module allows) 
could set the paramstyle on their connection to one of the available 
options. If the module only supports a single paramstyle then changing 
the attribute on the connection would have no effect.

Regards,
Andy
-- 
--------------------------------------------------------------------------------
 From the desk of Andrew J Todd esq - http://www.halfcooked.com/