From fog at initd.org Thu Jan 1 16:09:29 2004 From: fog at initd.org (Federico Di Gregorio) Date: Thu Jan 1 16:09:29 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <07AF3C77A0FBD311A99F00508B65203903272BB4@sebastian.peppermillcas.com> References: <07AF3C77A0FBD311A99F00508B65203903272BB4@sebastian.peppermillcas.com> Message-ID: <1072991369.1282.284.camel@localhost> Il mer, 2003-12-31 alle 18:14, Vernon Cole ha scritto: [snip] > At that time, Jon Franz remarked: " Unfortunately, I think you've fallen > victim to some of the hubris present in the SIG. Many of the people here > seem perfectly fine with the status quo. Perhaps they worked on DB-API > specifications, and thus feel protective of it? it can be. i *only* worked at the last revision of the dbapi when some parts where clarified and there were some small additions. and i don't feel protective. :) maybe people that have contributed much more than me feel that forcing tens of developers to upgrade adapters to a new api is a bad thing. my oppinion is that the part of the dbapi related to typecasts is flacky. it does not even mandate that it should be possible to re-insert an object coming from the db (for example an mx.DateTime wrapper). i don't care if the dbapi does not specify what kind of object should represent a date, but i care when it does not even tell what the properties of such an object are. federico -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org We are all dust, Saqi, so play the lute We are all wind, Saqi, so bring wine. -- Omar Khayam -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040101/adcb640d/attachment.bin From greg at msn.com Fri Jan 2 17:17:01 2004 From: greg at msn.com (Patty Hagan) Date: Fri Jan 2 01:21:57 2004 Subject: [DB-SIG] Ladies beautiful breasts safely-- bordello cavemen gainful cevcif snuck Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040102/7940090b/attachment.html From mal at egenix.com Fri Jan 2 07:22:35 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 2 07:22:41 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <1072991369.1282.284.camel@localhost> References: <07AF3C77A0FBD311A99F00508B65203903272BB4@sebastian.peppermillcas.com> <1072991369.1282.284.camel@localhost> Message-ID: <3FF5628B.2000607@egenix.com> Federico Di Gregorio wrote: > Il mer, 2003-12-31 alle 18:14, Vernon Cole ha scritto: > [snip] > >> At that time, Jon Franz remarked: " Unfortunately, I think you've fallen >>victim to some of the hubris present in the SIG. Many of the people here >>seem perfectly fine with the status quo. Perhaps they worked on DB-API >>specifications, and thus feel protective of it? > > it can be. i *only* worked at the last revision of the dbapi when some > parts where clarified and there were some small additions. and i don't > feel protective. :) maybe people that have contributed much more than me > feel that forcing tens of developers to upgrade adapters to a new api is > a bad thing. Since I've always tried to keep up the spirit of the DB API spec ever since I was editor of the 2.0 version, you probably feel that I'm protective of it. It is true that I've invested a considerable amount of time into discussing the various issues raised with 1.0 on this list and after 2.0 was released amended it various times (without changing the version number) in fully backwards compatible ways (e.g. by adding standardized extensions that we discussed on this list some time ago). However, that is *not* the reason of why I think making the DB API more restrictive is a bad thing. The spirit of the DB API spec has always been to give developers a guideline in creating new database modules while not making the implementation too burdensome. As a result we have quite a few database modules out there which implement the spec, including some of the extensions. They also often come with a multitude of new developments that are permitted by the spec (e.g. customized row objects that behave like a sequence). By making the spec more restrictive we would not only make life of the developers harder, but also break backwards compatibility of most database modules that work perfectly OK today. My aim has always been to see what database developers come up with and then standardize those new ideas in a way that makes life easier for the user. The database extensions are a good example of this practice and I have strong belief that this method of evolving the standard has proven to be much more of a success than trying to break compatibility every year or so by trying to stay bleeding edge all the time. Many of the things people have asked about in the past are really not intended to be implemented at the level the DB API is working at. Object relational wrappers, ADO style access, etc. can easily be programmed on top of the DB API and indeed have been implemented quite successfully for a number of database modules out there. If someone feels we need a standard for these wrappers, I'd suggest to first look at the question whether there will ever be more than just one or two such wrappers written. I've been in the database business for quite some time now and to be honest, every single application has had slightly different requirements for the database abstraction layer - no single API would have captured all cases. Of course, that's just my personal opinion. If people like the existing OR-wrappers, they should simply use them. > my oppinion is that the part of the dbapi related to typecasts is > flacky. Probably because it only defines them for the input side and not the output side :-) It is certainly a weak point in the spec and we should do something about it for the next revision. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 02 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fog at initd.org Fri Jan 2 08:51:30 2004 From: fog at initd.org (Federico Di Gregorio) Date: Fri Jan 2 08:51:30 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <3FF5628B.2000607@egenix.com> References: <07AF3C77A0FBD311A99F00508B65203903272BB4@sebastian.peppermillcas.com> <1072991369.1282.284.camel@localhost> <3FF5628B.2000607@egenix.com> Message-ID: <1073051490.1260.18.camel@localhost> Il ven, 2004-01-02 alle 13:22, M.-A. Lemburg ha scritto: [snip] > However, that is *not* the reason of why I think making the > DB API more restrictive is a bad thing. The spirit of the DB > API spec has always been to give developers a guideline in > creating new database modules while not making the implementation > too burdensome. As a result we have quite a few database modules > out there which implement the spec, including some of the extensions. > They also often come with a multitude of new developments > that are permitted by the spec (e.g. customized row objects > that behave like a sequence). this is exactly what i mean (and i wholeheartly agree with everything i deleted): the DBAPI is very good because does not say "results must be tuples" but "results must be accessible as tuples". unfortunately this is not true for the whole dbapi. > By making the spec more restrictive we would not only make > life of the developers harder, but also break backwards > compatibility of most database modules that work perfectly > OK today. completely agreed. > Many of the things people have asked about in the past > are really not intended to be implemented at the level > the DB API is working at. Object relational wrappers, > ADO style access, etc. can easily be programmed on top > of the DB API and indeed have been implemented quite > successfully for a number of database modules out there. and agreed again. > > my oppinion is that the part of the dbapi related to typecasts is > > flacky. > > Probably because it only defines them for the input side > and not the output side :-) > > It is certainly a weak point in the spec and we should > do something about it for the next revision. exactly. let's start with a first very simple proposal that, imho, will improve the dbapi a lot: objects returned in result sets should be directly useable as bound variables, without needing a call to wrapping functions/constructors. e.g., if datefield is of (any) date type and curs is a cursor: curs.execute("INSERT into test VALUES (%s)", (module.DateFromTicks(x),)) curs.execute("SELECT datefiled from test") d = curs.fetchone()[0] curs.execute("INSERT into test VALUES(%s)", (d,)) -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org We are all dust, Saqi, so play the lute We are all wind, Saqi, so bring wine. -- Omar Khayam -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040102/7c86d7fd/attachment.bin From jacobs at penguin.theopalgroup.com Fri Jan 2 09:29:30 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Fri Jan 2 09:29:34 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <1073051490.1260.18.camel@localhost> Message-ID: On Fri, 2 Jan 2004, Federico Di Gregorio wrote: > exactly. let's start with a first very simple proposal that, imho, will > improve the dbapi a lot: > > objects returned in result sets should be directly useable as bound > variables, without needing a call to wrapping functions/constructors. > e.g., if datefield is of (any) date type and curs is a cursor: > > curs.execute("INSERT into test VALUES (%s)", (module.DateFromTicks(x),)) > curs.execute("SELECT datefiled from test") > d = curs.fetchone()[0] > curs.execute("INSERT into test VALUES(%s)", (d,)) This proposal does not address the needs of those that: 1) need to take data from one DB-API driver to another driver or backend database. 2) have inherently ambiguous DB->Python type mappings 3) like the fact that DB-API adapters return builtin Python types in most cases (if only for efficiency reasons since they have their own type or OR mapping systems built upon DB-API). Thus, this proposal should be a use-case for a type mapping extension for input and output variables, but is not general enough to be a stand-alone, mandated feature. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From fog at initd.org Fri Jan 2 09:37:35 2004 From: fog at initd.org (Federico Di Gregorio) Date: Fri Jan 2 09:37:28 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: References: Message-ID: <1073054254.1260.27.camel@localhost> Il ven, 2004-01-02 alle 15:29, Kevin Jacobs ha scritto: [snip] > 1) need to take data from one DB-API driver to another driver or backend > database. as already said by a few people on this list the DBAPI is not a cross-database-backend tool. it is there to provide a common API, not to guarantee database independence. > 2) have inherently ambiguous DB->Python type mappings the whole DBAPI is _based_ on ambiguous mappings. i agree with marc on that: it does not make sense to force a unique set of type mapping because it would be too a burden on the module developers. what we need is, imho, a way to tell that a STRING from a database can be safetly mapped to a STRING on another backend. and so on. mm.. this seems "protocols"[1] stuff to me. > 3) like the fact that DB-API adapters return builtin Python types in most > cases (if only for efficiency reasons since they have their own type > or OR mapping systems built upon DB-API). i don't understand what you mean here (not a native english speaker, sorry, i know this is my fault.) federico [1] like in PyProtocols, google for it. -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org Nessuno dice che non si possa sognare in dettaglio, essere realistici e magari realizzarlo pure, il sogno. -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040102/05a320f5/attachment.bin From jacobs at penguin.theopalgroup.com Fri Jan 2 10:00:28 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Fri Jan 2 10:00:32 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <1073054254.1260.27.camel@localhost> Message-ID: On Fri, 2 Jan 2004, Federico Di Gregorio wrote: > Il ven, 2004-01-02 alle 15:29, Kevin Jacobs ha scritto: > [snip] > > 1) need to take data from one DB-API driver to another driver or backend > > database. > > as already said by a few people on this list the DBAPI is not a > cross-database-backend tool. it is there to provide a common API, not to > guarantee database independence. I agree, though the problems related to multiple backends still exist when dealing with a single DB-API backend. It has everything to do with adding features with the wrong granularity -- i.e., features that require the use of blessed types in order to be used as bound variables. At their essence, the last I saw of them, all of the implementations of this proposal specify the backend type and thus the literal encoding of Python objects. The problem is that this information is part of the object type (e.g., a __sqlstr__ or __sqlrepr__ magic method). This is easy when all data come from the DB-API driver and are passed back to that driver, though things become much more invasive when data come from user interfaces, or other sources that do not have the magic type annotations. > > 2) have inherently ambiguous DB->Python type mappings > > the whole DBAPI is _based_ on ambiguous mappings. i agree with marc on > that: it does not make sense to force a unique set of type mapping > because it would be too a burden on the module developers. > > what we need is, imho, a way to tell that a STRING from a database can > be safetly mapped to a STRING on another backend. and so on. mm.. this > seems "protocols"[1] stuff to me. You miss the point. Your literal round-tripping proposal does not address the needs of those with data elements that may have _multiple_ literal forms. i.e., the type is not enough to specify the literal encoding when used as a bound variable. > > 3) like the fact that DB-API adapters return builtin Python types in most > > cases (if only for efficiency reasons since they have their own type > > or OR mapping systems built upon DB-API). > > i don't understand what you mean here (not a native english speaker, > sorry, i know this is my fault.) Put simply: I want the simplest, most primitive, most light-weight types returned in rows. This way I can apply my own type mapping layer without sacrificing efficiency for any superfluous and/or flawed functionality. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From fog at initd.org Fri Jan 2 10:29:39 2004 From: fog at initd.org (Federico Di Gregorio) Date: Fri Jan 2 10:29:29 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: References: Message-ID: <1073057379.1260.35.camel@localhost> Il ven, 2004-01-02 alle 16:00, Kevin Jacobs ha scritto: [big snip] > This is easy when all data come from the DB-API driver and are passed back > to that driver, though things become much more invasive when data come from > user interfaces, or other sources that do not have the magic type > annotations. agreed. but adding the magic type information is not a task for the database driver, imho. your example of data coming froma gui just reinforce my idea of a protocols-driver layer over the DBAPI and under the OR mappers or applications. [another snip] > You miss the point. Your literal round-tripping proposal does not address > the needs of those with data elements that may have _multiple_ literal > forms. i.e., the type is not enough to specify the literal encoding when > used as a bound variable. do such types exists? really, i can't imagine a (postgresql) data type that can't do the round trip. [snip] > Put simply: I want the simplest, most primitive, most light-weight types > returned in rows. This way I can apply my own type mapping layer without > sacrificing efficiency for any superfluous and/or flawed functionality. ok. i am currently rewriting (finishing, that is) psycopg's type system. can you please tell me _exactly_ how your ideal type-system should work? i will be glad to give it a try if it is clearly better than current one. federico -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org One key. One input. One enter. All right. -- An american consultant (then the system crashed and took down the *entire* network) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040102/0acfa579/attachment-0001.bin From guido at python.org Fri Jan 2 10:35:54 2004 From: guido at python.org (Guido van Rossum) Date: Fri Jan 2 10:35:59 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: Your message of "Fri, 02 Jan 2004 13:22:35 +0100." <3FF5628B.2000607@egenix.com> References: <07AF3C77A0FBD311A99F00508B65203903272BB4@sebastian.peppermillcas.com> <1072991369.1282.284.camel@localhost> <3FF5628B.2000607@egenix.com> Message-ID: <200401021535.i02FZsF09365@c-24-5-183-134.client.comcast.net> [Vernon Cole] > >>At that time, Jon Franz remarked: " Unfortunately, I think you've > >>fallen victim to some of the hubris present in the SIG. Many of > >>the people here seem perfectly fine with the status quo. Perhaps > >>they worked on DB-API specifications, and thus feel protective of > >>it? [Federico Di Gregorio] > > it can be. i *only* worked at the last revision of the dbapi when > > some parts where clarified and there were some small > > additions. and i don't feel protective. :) maybe people that have > > contributed much more than me feel that forcing tens of developers > > to upgrade adapters to a new api is a bad thing. [MAL] > Since I've always tried to keep up the spirit of the DB API > spec ever since I was editor of the 2.0 version, you probably > feel that I'm protective of it. > > It is true that I've invested > a considerable amount of time into discussing the various > issues raised with 1.0 on this list and after 2.0 was released > amended it various times (without changing the version number) > in fully backwards compatible ways (e.g. by adding standardized > extensions that we discussed on this list some time ago). > > However, that is *not* the reason of why I think making the > DB API more restrictive is a bad thing. The spirit of the DB > API spec has always been to give developers a guideline in > creating new database modules while not making the implementation > too burdensome. As a result we have quite a few database modules > out there which implement the spec, including some of the extensions. > They also often come with a multitude of new developments > that are permitted by the spec (e.g. customized row objects > that behave like a sequence). > > By making the spec more restrictive we would not only make > life of the developers harder, but also break backwards > compatibility of most database modules that work perfectly > OK today. For Python itself, making the life of the developers harder is usually considered okay if it makes the life of the users easier. I'm curious if the life of the users isn't made *considerably* harder by some gratuitous differences between DB-API implementations, when a user needs to switch database vendors or (a slightly different case!) needs to support multiple vendors simultaneously. Yes, you can write all your code using the facilities to inquire DB-API features, but in practice nobody does that until they are forced to consider multiple versions and realize just how different they are. In a few months on this list I've seen many folks post about this. (Of course, you can argue that the database vendors are a major source of the differences -- the differences in SQL syntax are indeed staggering! But I'm not sure if that gives DB-API developers an excuse -- otherwise we might as well have no standard at all.) > My aim has always been to see what database developers > come up with and then standardize those new ideas in a > way that makes life easier for the user. The database > extensions are a good example of this practice and I > have strong belief that this method of evolving the > standard has proven to be much more of a success than > trying to break compatibility every year or so by trying > to stay bleeding edge all the time. > > Many of the things people have asked about in the past > are really not intended to be implemented at the level > the DB API is working at. Object relational wrappers, > ADO style access, etc. can easily be programmed on top > of the DB API and indeed have been implemented quite > successfully for a number of database modules out there. But the creation of such wrappers in a vendor-agnostic way is harder because of the wide variation at the DB-API level. > If someone feels we need a standard for these wrappers, > I'd suggest to first look at the question whether there > will ever be more than just one or two such wrappers > written. I've been in the database business for quite > some time now and to be honest, every single application > has had slightly different requirements for the database > abstraction layer - no single API would have captured > all cases. Of course, that's just my personal opinion. > If people like the existing OR-wrappers, they should > simply use them. I agree, in practice the needed higher-level abstractions are incredibly application-specific, and generic abstraction modules are of limited value to "real" applications. But this argues for *more* standardization between DB-API modules: everyone who develops a serious application that eventually gets to curse the gratuitous differences -- between vendor SQL as well as between DB-API modules. We can't fix the vendor's SQL, but we *can* try to agree on more similarity between DB-API modules. It's clear to me now that this would have to be DB-API 3.0, and that it will take some years before a 3.0-compliant module is available for most db vendors -- that's okay with me. Standardization takes a long time. For my part, I'd be happy to have a public C API added to the datetime module starting with Python 2.4 (we could even backport it to Python 2.3.4). I'd appreciate some help, but if this is the *only* think keeping datetime from eventually becoming the de-facto standard for representing database date and time values, I'd do it myself. > > my oppinion is that the part of the dbapi related to typecasts is > > flacky. > > Probably because it only defines them for the input side > and not the output side :-) > > It is certainly a weak point in the spec and we should > do something about it for the next revision. Great! I don't know this part, but it seems there's consensus that a new version should be developed. Let's get started! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Fri Jan 2 10:53:31 2004 From: guido at python.org (Guido van Rossum) Date: Fri Jan 2 10:53:36 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: Your message of "Fri, 02 Jan 2004 09:29:30 EST." References: Message-ID: <200401021553.i02FrVj09461@c-24-5-183-134.client.comcast.net> > > exactly. let's start with a first very simple proposal that, imho, will > > improve the dbapi a lot: > > > > objects returned in result sets should be directly useable as bound > > variables, without needing a call to wrapping functions/constructors. > > e.g., if datefield is of (any) date type and curs is a cursor: > > > > curs.execute("INSERT into test VALUES (%s)", (module.DateFromTicks(x),)) > > curs.execute("SELECT datefiled from test") > > d = curs.fetchone()[0] > > curs.execute("INSERT into test VALUES(%s)", (d,)) > > This proposal does not address the needs of those that: > > 1) need to take data from one DB-API driver to another driver or backend > database. > 2) have inherently ambiguous DB->Python type mappings > 3) like the fact that DB-API adapters return builtin Python types in most > cases (if only for efficiency reasons since they have their own type > or OR mapping systems built upon DB-API). Right, but some of these requirements seem to follow from the fact that there *are* so many differences between DB-API modules. Doesn't the addition of a standard, builtin datetime type take away much of (2) and hence (1), while improving (3)? > Thus, this proposal should be a use-case for a type mapping extension for > input and output variables, but is not general enough to be a stand-alone, > mandated feature. I imagine it also depends on the smarts of the database. In MySQL, I seem to recall that string values of a certain format are acceptable as date input, so if a DB-API MySQL wrapper returned strings (with the right format) for date values, that would work just fine. But this may not work for all db vendors. --Guido van Rossum (home page: http://www.python.org/~guido/) From jacobs at penguin.theopalgroup.com Fri Jan 2 11:55:29 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Fri Jan 2 11:55:36 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <200401021553.i02FrVj09461@c-24-5-183-134.client.comcast.net> Message-ID: On Fri, 2 Jan 2004, Guido van Rossum wrote: > > This proposal does not address the needs of those that: > > > > 1) need to take data from one DB-API driver to another driver or backend > > database. > > 2) have inherently ambiguous DB->Python type mappings > > 3) like the fact that DB-API adapters return builtin Python types in most > > cases (if only for efficiency reasons since they have their own type > > or OR mapping systems built upon DB-API). > > Right, but some of these requirements seem to follow from the fact > that there *are* so many differences between DB-API modules. Doesn't > the addition of a standard, builtin datetime type take away much of > (2) and hence (1), while improving (3)? Actually, I'm in favor of the inclusion of the new datetime objects into DB-API. It is ubiquitous, light-weight, has timezone support, and is capable of representing most of the date and time values I care about. With the inclusion of precision information, it would be a equivalent to most advanced SQL date/time representations. My above concerns are directed toward adopting a Python-type based method of escaping SQL literals based on adding a new magic method (like __sqlrepr__ or __sqlstr__). I assumed (incorrectly) that this was what Frederico was suggesting with his proposal; a topic that has come up many times before and one the sources of the "DB-API snob" perception. > > Thus, this proposal should be a use-case for a type mapping extension for > > input and output variables, but is not general enough to be a stand-alone, > > mandated feature. > > I imagine it also depends on the smarts of the database. In MySQL, I > seem to recall that string values of a certain format are acceptable > as date input, so if a DB-API MySQL wrapper returned strings (with the > right format) for date values, that would work just fine. But this > may not work for all db vendors. True -- for example, the problem of proving support for bound variables in drivers for backends that support a sophisticated SQL dialect, but not explicit support for binding (e.g., PostgreSQL before version 7.4). To compute the best represent ion of various Python data as SQL literals may require an SQL dialect parser and schema introspection. This is why mandating a one-size-fits all type mapping solution that naively guarantees round-trip data representations is not possible via a simple extension to DB-API, or at least I've yet to see a proposal to do it properly. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From randall at tnr.cc Fri Jan 2 13:05:23 2004 From: randall at tnr.cc (Randall Smith) Date: Fri Jan 2 13:08:14 2004 Subject: [DB-SIG] DB-API wrapper with db_rows Message-ID: <3FF5B2E3.3050004@tnr.cc> From reading this list, I discovered Kevin Jacob's db_row which I took to quickly. I wrote a wrapper for the DB-API that ads db_row functionality plus some simple insert and update methods that have a similar interface to db_row. This should work with any DB-API 2 module. Using a wrapper rather than inheritance allows it to work with modules written in C and all DB-API 2 modules without module-specific code. Here is an example of how it is used: ********************************************************** import rdb #the wrapper module #notice that I specifiy the db module to use cn = rdb.connect('pgdb', database='test') #same as always cs = cn.cursor() query = 'select id, name, email from customer' cs.execute(query) #note the o (objects) on fetchall results = cs.fetchallo() #note also that the db-api methods are still available #so we could still say results = fetchall() #the magic from db_row (thanks to Kevin) for row in results: id = row.fields.id name = row.fields.name email = row.fields.email # do some stuff #end of db_row part ********************** #methods I added for insert/update******** #create an input instance by specifying table input = cs.input('customer') #define field values input.fields.name = 'Charlie' input.fields.email = 'charlie@nowhere.com' #insert data - may optionally return a row id input.insert() #there is also an update method to the input object. *********************************************************** I may try to make it into a single db module for all databases that hide differences like the (qmark,numeric,named,format,pyformat) paramstyles so that a programmer doesn't have to think about these things when switching db modules, say from mxODBC to pgdb to psycopg. Does anyone know if I have duplicated someone elses effort (Not that there is much effort on my part)? I haven't yet posted my code, but if you would like a copy, just email me. From mal at egenix.com Fri Jan 2 14:03:55 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Fri Jan 2 14:04:00 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <200401021535.i02FZsF09365@c-24-5-183-134.client.comcast.net> References: <07AF3C77A0FBD311A99F00508B65203903272BB4@sebastian.peppermillcas.com> <1072991369.1282.284.camel@localhost> <3FF5628B.2000607@egenix.com> <200401021535.i02FZsF09365@c-24-5-183-134.client.comcast.net> Message-ID: <3FF5C09B.3040204@egenix.com> Guido van Rossum wrote: >>Since I've always tried to keep up the spirit of the DB API >>spec ever since I was editor of the 2.0 version, you probably >>feel that I'm protective of it. >> >>It is true that I've invested >>a considerable amount of time into discussing the various >>issues raised with 1.0 on this list and after 2.0 was released >>amended it various times (without changing the version number) >>in fully backwards compatible ways (e.g. by adding standardized >>extensions that we discussed on this list some time ago). >> >>However, that is *not* the reason of why I think making the >>DB API more restrictive is a bad thing. The spirit of the DB >>API spec has always been to give developers a guideline in >>creating new database modules while not making the implementation >>too burdensome. As a result we have quite a few database modules >>out there which implement the spec, including some of the extensions. >>They also often come with a multitude of new developments >>that are permitted by the spec (e.g. customized row objects >>that behave like a sequence). >> >>By making the spec more restrictive we would not only make >>life of the developers harder, but also break backwards >>compatibility of most database modules that work perfectly >>OK today. > > For Python itself, making the life of the developers harder is > usually considered okay if it makes the life of the users easier. Not so in the DB-SIG world :-) We believe that it is more important to have a multitude of database modules for various different databases than to have just a few for database backends that provide enough features to actually implement all the requirements. We also think that supporting multiple Python versions is a good thing and that more specific design decisions (e.g. the mapping between database and Python data types) should be left to the database module implementor. And last not least, it has always been the aim of the DB API to provide enough freedom of choice for developers to add functionality that complies to the standard, but extends it in various different useful ways. > I'm curious if the life of the users isn't made *considerably* harder > by some gratuitous differences between DB-API implementations, when > a user needs to switch database vendors or (a slightly different > case!) needs to support multiple vendors simultaneously. That hard part in supporting multiple backends lies in the differences at the SQL level (different syntax, data types, restrictions, etc.), not the database interfacing level. > Yes, you can write all your code using the facilities to inquire > DB-API features, but in practice nobody does that until they are > forced to consider multiple versions and realize just how different > they are. In a few months on this list I've seen many folks post > about this. > > (Of course, you can argue that the database vendors are a major source > of the differences -- the differences in SQL syntax are indeed > staggering! But I'm not sure if that gives DB-API developers an > excuse -- otherwise we might as well have no standard at all.) Hmm, that contradicts the success of the DB API specifications 1.0 and 2.0 which have enabled Python developers to write good working code for more than 7 years. >>My aim has always been to see what database developers >>come up with and then standardize those new ideas in a >>way that makes life easier for the user. The database >>extensions are a good example of this practice and I >>have strong belief that this method of evolving the >>standard has proven to be much more of a success than >>trying to break compatibility every year or so by trying >>to stay bleeding edge all the time. >> >>Many of the things people have asked about in the past >>are really not intended to be implemented at the level >>the DB API is working at. Object relational wrappers, >>ADO style access, etc. can easily be programmed on top >>of the DB API and indeed have been implemented quite >>successfully for a number of database modules out there. > > But the creation of such wrappers in a vendor-agnostic way is harder > because of the wide variation at the DB-API level. Have you tried any of them ? Adding support for new DB-API compatible drivers is usually just a matter of overriding a few driver specific access or connect methods. >>If someone feels we need a standard for these wrappers, >>I'd suggest to first look at the question whether there >>will ever be more than just one or two such wrappers >>written. I've been in the database business for quite >>some time now and to be honest, every single application >>has had slightly different requirements for the database >>abstraction layer - no single API would have captured >>all cases. Of course, that's just my personal opinion. >>If people like the existing OR-wrappers, they should >>simply use them. > > I agree, in practice the needed higher-level abstractions are > incredibly application-specific, and generic abstraction modules are > of limited value to "real" applications. > > But this argues for *more* standardization between DB-API modules: > everyone who develops a serious application that eventually gets to > curse the gratuitous differences -- between vendor SQL as well as > between DB-API modules. > > We can't fix the vendor's SQL, but we *can* try to agree on more > similarity between DB-API modules. Can you give some examples of what you find so incredibly disturbing about the current set of database modules ? Where are the differences that you are talking about and what makes you think that making a specification less flexible would result in better database modules ? > It's clear to me now that this > would have to be DB-API 3.0, and that it will take some years before a > 3.0-compliant module is available for most db vendors -- that's okay > with me. Standardization takes a long time. Right. It took us 7 years so far and we have learned a lot in that time. > For my part, I'd be happy to have a public C API added to the datetime > module starting with Python 2.4 (we could even backport it to Python > 2.3.4). I'd appreciate some help, but if this is the *only* think > keeping datetime from eventually becoming the de-facto standard for > representing database date and time values, I'd do it myself. Note that the DB API specification allows database module authors to use datetime types to represent date and time coming from the database and accepting it on input. There's no need to change the DB API to have database authors use datetime, e.g. as configurable option. The reason that there isn't much support for it is that most database modules are written in C and without an easy to use C API it's not easy to convince authors to provide such an option (I've tried to do that myself for mxODBC but then refrained from adding support simply because the overhead of calling out to Python would have slowed down result set access considerably). Another downside is that datetime types don't support int/float slots providing Unix ticks values. Adding these would make it a lot easier to at least use datetime types as input to .execute() since many database modules accept integers/floats and interpret them as Unix ticks values. >>>my oppinion is that the part of the dbapi related to typecasts is >>>flacky. >> >>Probably because it only defines them for the input side >>and not the output side :-) >> >>It is certainly a weak point in the spec and we should >>do something about it for the next revision. > > Great! I don't know this part, but it seems there's consensus that a > new version should be developed. Let's get started! Not a new version.... a new revision will do nicely. We don't need revolution, just evolution. Happy New Year, BTW :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 02 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fog at initd.org Fri Jan 2 14:39:01 2004 From: fog at initd.org (Federico Di Gregorio) Date: Fri Jan 2 14:39:28 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: References: Message-ID: <1073072341.1078.5.camel@localhost> Il ven, 2004-01-02 alle 17:55, Kevin Jacobs ha scritto: [snip] > My above concerns are directed toward adopting a Python-type based method of > escaping SQL literals based on adding a new magic method (like __sqlrepr__ > or __sqlstr__). I assumed (incorrectly) that this was what Frederico was > suggesting with his proposal; a topic that has come up many times before and > one the sources of the "DB-API snob" perception. i did not. i think __sqlstr__ (name it as you like) is very usefull but it is not the solution for cross-driver compatibility. (that was my *wrong* idea about 6 month ago.) [snip] > compute the best represent ion of various Python data as SQL literals may > require an SQL dialect parser and schema introspection. This is why > mandating a one-size-fits all type mapping solution that naively guarantees > round-trip data representations is not possible via a simple extension to > DB-API, or at least I've yet to see a proposal to do it properly. sorry, but i don't agree here. please, show me a postgresql type that does not support round trip. i think i can write a wrapper object that does the round trip correctly for every postgresql type. :) federico -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. -- D.E.Knuth -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040102/4cd115b8/attachment.bin From jacobs at penguin.theopalgroup.com Fri Jan 2 16:44:11 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Fri Jan 2 16:44:16 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <1073072341.1078.5.camel@localhost> Message-ID: On Fri, 2 Jan 2004, Federico Di Gregorio wrote: > Il ven, 2004-01-02 alle 17:55, Kevin Jacobs ha scritto: > [snip] > > My above concerns are directed toward adopting a Python-type based method of > > escaping SQL literals based on adding a new magic method (like __sqlrepr__ > > or __sqlstr__). I assumed (incorrectly) that this was what Frederico was > > suggesting with his proposal; a topic that has come up many times before and > > one the sources of the "DB-API snob" perception. > > i did not. i think __sqlstr__ (name it as you like) is very usefull but > it is not the solution for cross-driver compatibility. (that was my > *wrong* idea about 6 month ago.) We agree -- I said that I _incorrectly_ assumed this was what you were bringing up again. > [snip] > > compute the best represent ion of various Python data as SQL literals may > > require an SQL dialect parser and schema introspection. This is why > > mandating a one-size-fits all type mapping solution that naively guarantees > > round-trip data representations is not possible via a simple extension to > > DB-API, or at least I've yet to see a proposal to do it properly. > > sorry, but i don't agree here. please, show me a postgresql type that > does not support round trip. i think i can write a wrapper object that > does the round trip correctly for every postgresql type. :) I don't disagree that you can write code to round-trip any datum from fetch to bound argument. However, the real trick is doing that in such a way that the datum is useful in between -- i.e., if all we cared about was naive round-trip-ability, then we'd pass around pre-escaped literal strings everywhere. However, I can SELECT an integer, add 1.5 to it, and be very annoyed when the literal representation is or is not truncated/rounded back to an integer without regard for the destination field type. This is where the result context is so critically important. To this, add the previously mentioned problem of data that doesn't come from any database backend, and you're right back where you started before. Problems like this also become insidious when one tries to maintain normal Python semantics on objects being mapped to and from SQL types. i.e., it is perfectly reasonable to expect that 0/1 can be used interchangeably with False/True as bound arguments, but the converse in SQL is decidedly not true. Thus, to know how to properly encode the resulting literal requires that one know the SQL expression type. So this entire debate boils down to one thing: Do you want to map SQL types to and from native classes, or do you need something more specialized? I am in favor of letting the DB-API user decide which perspective benefits their application best, rather than mandating anything simplistic or invasive that any sophisticated user will want to bypass anyway. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From jfranz at neurokode.com Sat Jan 3 01:27:54 2004 From: jfranz at neurokode.com (Jon Franz) Date: Sat Jan 3 01:28:03 2004 Subject: [DB-SIG] Re: DB-API wrapper with db_rows Message-ID: <00ed01c3d1c2$b91f1b30$c701a8c0@letterbox1> >From reading this list, I discovered Kevin Jacob's db_row which I took <> >Does anyone know if I have duplicated someone elses effort (Not that >there is much effort on my part)? I haven't yet posted my code, but if >you would like a copy, just email me. Well, it overlaps with PDO, including the version we're just now testing (which has input functionality - though in PDO land its an 'EditableResultSet' and allows for editing of existing records). But that's in testing, and isn't released yet :) * shift into thinking-out-loud mode * As a side question - PDO currently uses results['foo'].value | results.fields['foo'].value style syntax - we could (very) easily add results.fields.foo.value - but _should_ we do it? It would take the number of ways to access a column to 3, which seems somewhat redundant, but it does save 4 characters per column access. Of course, a column name may be unusable for this access method without mangling - I'm thinking of access and spaces within a column name. Hrm. Thoughts? * end off-topic brainstorming junk * Take a look at PDO - you're welcome to join the project if this sort of thing interests you. If you don't like it, you can always just keep working on your own. No harm, no foul either way. http://pdo.neurokode.com - also - http://sourceforge.net/projects/pythondbo/ ~Jon Franz NeuroKode Labs, LLC From jfranz at neurokode.com Sat Jan 3 04:11:31 2004 From: jfranz at neurokode.com (Jon Franz) Date: Sat Jan 3 04:11:47 2004 Subject: [DB-SIG] Re: WHat's the status of DB modules and datetime.py supp ort? Message-ID: <00f301c3d1d9$94897290$c701a8c0@letterbox1> (Since I was quoted, I might as well jump in) my two cents on the current debates: - moving towards/requiring the new datetime objects in a future version of the db-api sounds very good to me. Its standard python - Roundtrip objects just make sense. Perhaps the names and interfaces of the type-classes should be defined, and not the implementation. Module writers can then choose to make such objects via c, or python, or a mixture - whichever they are more comfortable with. As long as it walks like a BLOB and talks like a BLOB, who cares how it is implemented internally? This is a basic pythonism, yes? This even ties into a conversation from last month where the type_code defined within the .description of a cursor can vary wildly between modules - see http://mail.python.org/pipermail/db-sig/2003-December/003795.html Perhaps, instead, due to speed issues, memory issues, and implementation issues of keeping the results in a python-object form, the types defined as allowable for such a field (type_code) should (at the least): a) Be defined within the standard b) be represented within the field by a reference to a class within the module that is a 'wrapper' for that type. This type-class should provide a standard interface (varying based upon the underlying type), limited of course by the precision of the underlying DB. The interfaces should be defined within the standard. - These classes should allow for instance creation from an output column of that type. - These type classes should be the same as the types used for insertion, or at least provide interfaces so that they _work_ for insertion and hide the details from the user. If actual results are _not_ going to be returned as objects suitable for insertion (memory concerns, overhead concerns, ease of use of values via native python types), then the above would at least allow for easy conversion of a value into insertable form without the type of the column needing to be explicitly known, as such: type_code = 1 # see pep 249 curs.execute("INSERT into test VALUES (%s)", (module.DateFromTicks(x),)) curs.execute("SELECT datefiled from test") d = curs.fetchone()[0] curs.execute("INSERT into test VALUES(%s)", (curs.description[0][1](d),)) Of course, that's rather hard to read, so perhaps a utility function to utilize said .description field should be made, thus we get: curs.execute("INSERT into test VALUES(%s)", (curs.convertforinput([0], d),)) The name of the function could be better. - differences in modules currently suck. It goes beyond just how to connect and the SQL dialects. It may be that some of these differences are easy to smooth out and remove; if so, then why not? Also, optional extensions are useless (IMHO) - Users fear to use them because they might not be there if they switch modules. Either it should be in, or it shouldn't be. Implementations of standards can have bugs - the optional extensions muddy the water even before such bugs are found. Mark said: "Not a new version.... a new revision will do nicely. We don't need revolution, just evolution." To be quite literal with the terms being used: Every once in while evolution makes giant leaps forward (anaerobic to aerobic, single-cell to multicellular) - these leaps are revolutionary, but they are part of the process. I'm not advocating ripping things down and starting from scratch - I don't think anyone is - but I think the movement forward many people want (or at least I want) goes beyond simple revisions such as datetime versus mxDateTime; I think the basic module-developer versus user balance needs to be addressed. I tend to agree with the Python-way - as Guido stated: "making the life of the developers harder is usually considered okay if it makes the life of the users easier" I don't think we should make life too hard for them, however! A middle-ground must be found. I think we'll see a short term 2.1 as a stopgap, and a 3.0 with more drastic changes - but I could be very wrong. ~Jon Franz NeuroKode Labs, LLC From nc-kizilkat2 at netcologne.de Fri Jan 2 08:58:08 2004 From: nc-kizilkat2 at netcologne.de (Atnen Kizlkaya) Date: Sat Jan 3 08:58:31 2004 Subject: [DB-SIG] Sibel aus =?iso-8859-1?q?G=F6ttingen?= Uni Message-ID: <000601c3d138$75cf27b0$0662a8d5@atnenr1na6jgoq> hallo sibel ben de senin gibi birini arayorum handy nr.g?nderir misin. slm -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040102/f74cf6f8/attachment-0001.html From jacobs at penguin.theopalgroup.com Sat Jan 3 09:14:29 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Sat Jan 3 09:14:38 2004 Subject: [DB-SIG] Re: DB-API wrapper with db_rows In-Reply-To: <00ed01c3d1c2$b91f1b30$c701a8c0@letterbox1> Message-ID: On Sat, 3 Jan 2004, Jon Franz wrote: > >From reading this list, I discovered Kevin Jacob's db_row which I took > <> > >Does anyone know if I have duplicated someone elses effort (Not that > >there is much effort on my part)? I haven't yet posted my code, but if > >you would like a copy, just email me. > > Well, it overlaps with PDO, including the version we're just now > testing (which has input functionality - though in PDO land its an > 'EditableResultSet' and allows for editing of existing records). Great! Variety and choice are good. Just remember, db_row solves a fundamentally different (and simpler) problem than PDO and has completely different design trade-offs. Specifically: 1) db_rows must act very much like tuples and dicts, for backward compatibility with DB-API conforming apps and drivers. 2) db_rows must allow field access by index, by item, and by attribute syntax. 3) db_rows must be light-weight and fast. E.g., a result set of 200,000 result tuples may take 13MB of memory, the same result as a list of dictionaries takes about 117MB of memory, while a list of db_rows takes just over 14MB of memory. Performance (using the optional C extension) is very competitive with tuples and much better than dicts. > Take a look at PDO - you're welcome to join the project if this sort > of thing interests you. If you don't like it, you can always just keep > working on your own. No harm, no foul either way. Thanks, but I'm not much of a fan of ADO-style interfaces. However, I have looked at a few versions of PDO and think it is great for those that are. We have a long way to go before the current generation of interfaces mature to the point where there will be natural convergence. Until then the key is for the various implementers to keep an open mind, track what is happening with other projects, and share ideas so that we're not stuck continually re-inventing the same wheels. Best regards, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From jfranz at neurokode.com Sat Jan 3 16:16:40 2004 From: jfranz at neurokode.com (Jon Franz) Date: Sat Jan 3 16:16:49 2004 Subject: [DB-SIG] Re: DB-API wrapper with db_rows References: Message-ID: <010301c3d23e$e1dedbb0$c701a8c0@letterbox1> [ Kevin Jacobs wrote ] > 3) db_rows must be light-weight and fast. E.g., a result set of 200,000 > result tuples may take 13MB of memory, the same result as a list of > dictionaries takes about 117MB of memory, while a list of db_rows takes > just over 14MB of memory. Performance (using the optional C extension) > is very competitive with tuples and much better than dicts. One thing of note: PDO only provides an dictionary interface, it does not create dictionaries in memory - the actual memory usage of a PDO resultset is very close to the original sequence of tuples - and the overhead is fairly constant on a per-column level (not per-row) basis. Performance is very good, but of course not as fast as tuples. Even per-column metadata, such as type_id and size, are all stored once, not per row. PDO uses interface tricks, not actual dictionaries (except for the internal column-name-to-index lookup, of course). In your example, I'm pretty sure PDO would be between 13MB and 14MB in size. In other words, PDO is also lightweight and fast, thus point #3 actually reveals a commonality between PDO and db_rows, not a difference. > Thanks, but I'm not much of a fan of ADO-style interfaces. Just curiosity, but what about ADO/JDBC/delphi-data-object style interfaces puts you off? I always find myself abhorring things like DBD/DBI and many of the chosen methods of doing things with the DB-API, and longing for a higher-level interface - so hearing from the opposite viewpoint may do me some good. Bryan Gudorf and myself realized awhile ago that our wants/needs are sometimes just semantic, and even the additional features we wanted could be handled by a wrapper of the DB-API - hence PDO's existence. > However, I have > looked at a few versions of PDO and think it is great for those that are. > We have a long way to go before the current generation of interfaces mature > to the point where there will be natural convergence. Until then the key is > for the various implementers to keep an open mind, track what is happening > with other projects, and share ideas so that we're not stuck continually > re-inventing the same wheels. I guess you looked at the documentation and not the code? Perhaps we should modify the documentation to explicitly note that only a dictionary interface is provided and that the actual implementation is lightweight and fast. I completely agree on the need to keep an open mind and track other projects. ~Jon Franz NeuroKode Labs, LLC From mcfletch at rogers.com Sat Jan 3 16:57:03 2004 From: mcfletch at rogers.com (Mike C. Fletcher) Date: Sat Jan 3 16:57:07 2004 Subject: [DB-SIG] Re: DB-API wrapper with db_rows In-Reply-To: <00ed01c3d1c2$b91f1b30$c701a8c0@letterbox1> References: <00ed01c3d1c2$b91f1b30$c701a8c0@letterbox1> Message-ID: <3FF73AAF.6060202@rogers.com> Jon Franz wrote: >>From reading this list, I discovered Kevin Jacob's db_row which I took ><> > > >>Does anyone know if I have duplicated someone elses effort (Not that >>there is much effort on my part)? I haven't yet posted my code, but if >>you would like a copy, just email me. >> >> Similar functionality in PyTable RDBMS DBRow class (btw, yes, I know about the Pytables project, that's a different package, just an unfortunately similar name). It's a heavy wrapper (4 dictionaries/row when using update/insert functionality) focused on very flexible operations driven by database schema descriptions (e.g. setup of database from schema, insert/update/refresh/delete rows, plugging in arbitrary classes for row/field implementations from the schema, etceteras). It works on MySQL and PostgreSQL at the moment. It doesn't try to change paramstyles. There's code to read in basic schemas from the database(s) as well. PyTable rows could be much lighter-weight. I generally only need to actually *produce* a few thousand of them for any given web-view, so haven't felt the need as of yet to optimise them. They store the original DB value, the wrapped/processed value (i.e. a Password object for a chkpass field, or an OID object for a varchar field that says that's the correct type), and any newly-set values. In practice the code only tends to use the "most recent" value, so I could readily compress down to a single array of values instead of the dictionaries (and use arrays regardless). It's probably safe to say that there's lots of these wrapper modules with very similar and overlapping functionality. Kevin's db_row seems very interesting, but I need flexible update support based on property changes to the row objects. Similarly, the lazyresultset module from PyTable probably would be useful to someone (it's not dependent on anything else in PyTable). The logic in PyTable to do fairly intelligent insert/update/refresh/delete queries might be useful as well some day. But to be really useful, we'd need to have some sort of standard for the wrapper modules in which to plug in all these pieces so that people can choose which vectors/aspects are important to them when choosing how to compose their own wrappers. >* shift into thinking-out-loud mode * >As a side question - PDO currently uses >results['foo'].value | results.fields['foo'].value >style syntax - we could (very) easily add >results.fields.foo.value > - but _should_ we do it? It would take the number of ways to >access a column to 3, which seems somewhat redundant, but it >does save 4 characters per column access. Of course, a column >name may be unusable for this access method without mangling - >I'm thinking of access and spaces within a column name. >Hrm. >Thoughts? >* end off-topic brainstorming junk * > > I started off just providing [ key ] and .key access, eventually had to add .get( key ) and .setdefault( key ) , haven't found a need for .fields[key] yet myself... some of the internal code uses the equivalent, but nothing external to the package. Enjoy all, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From fog at initd.org Mon Jan 5 05:50:09 2004 From: fog at initd.org (Federico Di Gregorio) Date: Mon Jan 5 05:51:31 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: References: Message-ID: <1073299809.1020.23.camel@localhost> Il ven, 2004-01-02 alle 22:44, Kevin Jacobs ha scritto: [snip] > I don't disagree that you can write code to round-trip any datum from fetch > to bound argument. However, the real trick is doing that in such a way that > the datum is useful in between -- i.e., if all we cared about was naive > round-trip-ability, then we'd pass around pre-escaped literal strings obviously. > everywhere. However, I can SELECT an integer, add 1.5 to it, and be very > annoyed when the literal representation is or is not truncated/rounded back > to an integer without regard for the destination field type. This is where unfortunately the right solution, even for a case simple as yours involves some kind of knowledge about the column types. that's why we have the type constructors like DateFromTicks in the modules. the only reason why we don't have IntFromFloat is that this functionality is built-in in python, e.g., "int(...)". here is a sketch of a propostal: 1/ let's extend the dbapi by adding some usefull things (as the round-trip, imho, but there are other extensions that can be usefull for higher-level module implementors, like some more type singletons) that does not depend on knowledge of column types. 2/ let's extend the dbapi with a standard way to extract type-information (and other kind of meta-information from the database). i *know* people think the dbapi is not the place for this; we can start a new PEP or anything else but i think we need a standard way to access meta-information right now. 3/ with the type information would be quite easy to write a (protocols-based?) utility module that automatically convert the types so that they can be safetly used cross-db. quoting is to be left to the DBAPI proper, we only need to convert to a type that the specific DBAPI implementation can understand. something like: # pcurs is a psycopg cursor # mcurs is a mysql cursor pcurs.execute("SELECT datefield FROM test") pdate = psycopg_curs.fetchone()[0] mdate = adapt(pdate, mysqlinterfaces.IDATE) mcurs.execute("INSERT INTO test (datefield) VALUES (%s)", (mdate,)) don't look at the formalism (adapt), look at the underlying idea. if we know enough about types, we can provide conversion functions from one module to the others quite easily. maybe we don't even need point (2) to write (3). [snip] > Do you want to map SQL types to and from native classes, or do you need > something more specialized? > > I am in favor of letting the DB-API user decide which perspective benefits > their application best, rather than mandating anything simplistic or > invasive that any sophisticated user will want to bypass anyway. agreed. that's why i built into psycopg a user-customizable type-system from the start :) [and sincerely, i would like to see the type-registration system standarized in DBAPI-3.0. i know other modules provide the same functionality...] federico -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org Those who do not study Lisp are doomed to reimplement it. Poorly. -- from Karl M. Hegbloom .signature -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040105/25d992e8/attachment.bin From jacobs at penguin.theopalgroup.com Mon Jan 5 07:01:42 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Mon Jan 5 07:01:48 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <1073299809.1020.23.camel@localhost> Message-ID: On Mon, 5 Jan 2004, Federico Di Gregorio wrote: > Il ven, 2004-01-02 alle 22:44, Kevin Jacobs ha scritto: > [snip] > 1/ let's extend the dbapi by adding some usefull things (as the > round-trip, imho, but there are other extensions that can be usefull for > higher-level module implementors, like some more type singletons) that > does not depend on knowledge of column types. Can you outline exactly how you intend on doing this? It is likely that my concerns can be addressed, though it is difficult to know until it is 100% clear what you are proposing. > 2/ let's extend the dbapi with a standard way to extract > type-information (and other kind of meta-information from the database). > i *know* people think the dbapi is not the place for this; we can start > a new PEP or anything else but i think we need a standard way to access > meta-information right now. Actually, there is a simpler things that can be done -- to start, let's expand the existing DB-API type API. e.g.: a) Add TIME, DATE, TIMESTAMP, DATETIMETZ type classes since DATETIME is far too generic. b) Similarly, add MONEY, INTEGER, FLOAT, DECIMAL/NUMERIC type classes to begin to disambiguate numeric types. c) Add VARCHAR, CHAR, and TEXT type classes to begin to disambiguate non-LOB string types. d) Add a BOOLEAN type class since NUMBER is wholly inadequate for describing boolean literals. e) Add an interface for mapping from type_id's to type classes (i.e., the reverse of the mapping provided by type objects). > 3/ with the type information would be quite easy to write a > (protocols-based?) utility module that automatically convert the types > so that they can be safetly used cross-db. quoting is to be left to the > DBAPI proper, we only need to convert to a type that the specific DBAPI > implementation can understand. something like: > > # pcurs is a psycopg cursor > # mcurs is a mysql cursor > > pcurs.execute("SELECT datefield FROM test") > pdate = psycopg_curs.fetchone()[0] > mdate = adapt(pdate, mysqlinterfaces.IDATE) > mcurs.execute("INSERT INTO test (datefield) VALUES (%s)", (mdate,)) > > don't look at the formalism (adapt), look at the underlying idea. if we > know enough about types, we can provide conversion functions from one > module to the others quite easily. maybe we don't even need point (2) to > write (3). Instead of adapt(), why not call it toSQLliteral()? Isn't that the real operation that execute needs a protocol to perform? > [snip] > > Do you want to map SQL types to and from native classes, or do you need > > something more specialized? > > > > I am in favor of letting the DB-API user decide which perspective benefits > > their application best, rather than mandating anything simplistic or > > invasive that any sophisticated user will want to bypass anyway. > > agreed. that's why i built into psycopg a user-customizable type-system > from the start :) [and sincerely, i would like to see the > type-registration system standarized in DBAPI-3.0. i know other modules > provide the same functionality...] As a happy user of your type extensions, I am grateful. However, it isn't very difficult to add such behavior to drivers that do not have builtin support, as I have done with about a dozen others. The runtime performance hit tends to be fairly insignificant as compared to query overhead. Thanks, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From jfranz at neurokode.com Mon Jan 5 09:18:58 2004 From: jfranz at neurokode.com (Jon Franz) Date: Mon Jan 5 09:19:09 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? Message-ID: <003c01c3d396$dc3fa6e0$8401a8c0@letterbox1> >e) Add an interface for mapping from type_id's to type classes (i.e., the > reverse of the mapping provided by type objects). Wouldn't the system I sugested saturday be just that? Perhaps my message was too politcal for the actual technical sugestions to get noticed - or the weekend timing. >From said message: (verbose, I know) "Perhaps, instead, due to speed issues, memory issues, and implementation issues of keeping the results in a python-object form, the types defined as allowable for such a field (type_code) should (at the least): a) Be defined within the standard b) be represented within the field by a reference to a class within the module that is a 'wrapper' for that type. This type-class should provide a standard interface (varying based upon the underlying type), limited of course by the precision of the underlying DB. The interfaces should be defined within the standard. - These classes should allow for instance creation from an output column of that type. - These type classes should be the same as the types used for insertion, or at least provide interfaces so that they _work_ for insertion and hide the details from the user. If actual results are _not_ going to be returned as objects suitable for insertion (memory concerns, overhead concerns, ease of use of values via native python types), then the above would at least allow for easy conversion of a value into insertable form without the type of the column needing to be explicitly known, as such: type_code = 1 # see pep 249 curs.execute("INSERT into test VALUES (%s)", (module.DateFromTicks(x),)) curs.execute("SELECT datefiled from test") d = curs.fetchone()[0] curs.execute("INSERT into test VALUES(%s)", (curs.description[0][1](d),)) Of course, that's rather hard to read, so perhaps a utility function to utilize said .description field should be made, thus we get: curs.execute("INSERT into test VALUES(%s)", (curs.convertforinput([0], d),)) " Of course, taking it further and making it so that columns of the same type from different DBs can be passed around and cross inserted sounds nice. +1 for toSQLliteral() From jacobs at penguin.theopalgroup.com Mon Jan 5 09:37:11 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Mon Jan 5 09:37:21 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <003c01c3d396$dc3fa6e0$8401a8c0@letterbox1> Message-ID: On Mon, 5 Jan 2004, Jon Franz wrote: > >e) Add an interface for mapping from type_id's to type classes (i.e., the > > reverse of the mapping provided by type objects). > > Wouldn't the system I sugested saturday be just that? > Perhaps my message was too politcal for the actual technical sugestions to > get noticed - or the weekend timing. Sorry -- I've been on the road and have not had the bandwidth or time to read everything from this weekend as carefully as I'd like. However, what I am proposing is much more modest than yours. My proposal doesn't restrict the contents of type_codes or mandate that all type codes have explicit Python-object representations and type constructors. I'm fine with type_codes being backend specific cookies, since I prefer the precision of knowing the native type code to the readability of a DB-API "type class" in this case. So I'm not really against the goals of your proposal, per se. I'd just prefer to start off simple, gain more experience with how these enhancements play out in real code (of which I have more than enough of), and only then contemplate adding more layers of functionality (evolution, not revolution). Better (more precise) type objects are a logical next step, followed by generic type mapping interface, followed by more complex things like you and Frederico are proposing (standardized type constructors and exposed literal escaping APIs). How does this sound? -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From fog at initd.org Mon Jan 5 09:45:48 2004 From: fog at initd.org (Federico Di Gregorio) Date: Mon Jan 5 09:45:28 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: References: Message-ID: <1073313948.1020.59.camel@localhost> Il lun, 2004-01-05 alle 15:37, Kevin Jacobs ha scritto: [snip] > So I'm not really against the goals of your proposal, per se. I'd just > prefer to start off simple, gain more experience with how these enhancements > play out in real code (of which I have more than enough of), and only then > contemplate adding more layers of functionality (evolution, not revolution). > Better (more precise) type objects are a logical next step, followed by > generic type mapping interface, followed by more complex things like > you and Frederico are proposing (standardized type constructors and exposed > literal escaping APIs). > > How does this sound? fine to me. i don't have enough time right now, but i'll try to elaborate the first part of that (more types) in DBAPI format and post to this list for revision and inclusion in DBAPI-2.1 (if everybody agrees, off curse). -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org Bhoe, bhe, bhe. Sono brutto e cattivo. Brutto lama! -- Cuzco -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040105/c94017d2/attachment.bin From jacobs at penguin.theopalgroup.com Mon Jan 5 09:48:25 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Mon Jan 5 09:48:30 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <1073313948.1020.59.camel@localhost> Message-ID: On Mon, 5 Jan 2004, Federico Di Gregorio wrote: > Il lun, 2004-01-05 alle 15:37, Kevin Jacobs ha scritto: > [snip] > > So I'm not really against the goals of your proposal, per se. I'd just > > prefer to start off simple, gain more experience with how these enhancements > > play out in real code (of which I have more than enough of), and only then > > contemplate adding more layers of functionality (evolution, not revolution). > > Better (more precise) type objects are a logical next step, followed by > > generic type mapping interface, followed by more complex things like > > you and Frederico are proposing (standardized type constructors and exposed > > literal escaping APIs). > > > > How does this sound? > > fine to me. i don't have enough time right now, but i'll try to > elaborate the first part of that (more types) in DBAPI format and post > to this list for revision and inclusion in DBAPI-2.1 (if everybody > agrees, off curse). Great! Please take a look at my other e-mail from this morning where I list a few of the extra types that I'd like to see. Any feedback on that starting-point would be very useful. Thanks, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From jfranz at neurokode.com Mon Jan 5 10:09:29 2004 From: jfranz at neurokode.com (Jon Franz) Date: Mon Jan 5 10:09:42 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? References: Message-ID: <006501c3d39d$eb63d360$8401a8c0@letterbox1> > Sorry -- I've been on the road and have not had the bandwidth or time to > read everything from this weekend as carefully as I'd like. No worries. > However, what I am proposing is much more modest than yours. My proposal > doesn't restrict the contents of type_codes or mandate that all type codes > have explicit Python-object representations and type constructors. I'm fine > with type_codes being backend specific cookies, since I prefer the precision > of knowing the native type code to the readability of a DB-API "type class" > in this case. > > So I'm not really against the goals of your proposal, per se. I'd just > prefer to start off simple, gain more experience with how these enhancements > play out in real code (of which I have more than enough of), and only then > contemplate adding more layers of functionality (evolution, not revolution). > Better (more precise) type objects are a logical next step, followed by > generic type mapping interface, followed by more complex things like > you and Frederico are proposing (standardized type constructors and exposed > literal escaping APIs). > > How does this sound? It sounds pretty good - but if you remove the cross database aspect Fredrico (and myself) would like, you could mandate the type_codes and interfaces and let the implementations be DB specific. In this way, you keep the module author's lives (reletivaly) simple while adding benefit for the users. I know, I know, its down the line in what you're talking about - but I just want to keep the goal in sight. I think merging the type_code from output with the extended input types (you had a good list), even if interfaces are not defined to Do anything with the information, would be another good step. That way, you could move from DB to DB and the type_ids would at least be the same (for similar columns), reducing the burdon on the users. ~Jon Franz NeuroKode Labs, LLC > > -Kevin > > -- > -- > Kevin Jacobs > The OPAL Group - Enterprise Systems Architect > Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com > Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ > From jacobs at penguin.theopalgroup.com Mon Jan 5 10:23:32 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Mon Jan 5 10:23:40 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <006501c3d39d$eb63d360$8401a8c0@letterbox1> Message-ID: On Mon, 5 Jan 2004, Jon Franz wrote: > > > > So I'm not really against the goals of your proposal, per se. I'd just > > prefer to start off simple, gain more experience with how these > > enhancements play out in real code (of which I have more than enough > > of), and only then contemplate adding more layers of functionality > > (evolution, not revolution). Better (more precise) type objects are a > > logical next step, followed by generic type mapping interface, followed > > by more complex things like you and Frederico are proposing > > (standardized type constructors and exposed literal escaping APIs). > > > > How does this sound? > > It sounds pretty good - but if you remove the cross database aspect > Fredrico (and myself) would like, you could mandate the type_codes and > interfaces and let the implementations be DB specific. I'm not sure I follow you here. I'm proposing an extension of the two existing levels of type information provided by DB-API: 1) type_code, which typically represents some backend specific value/cookie that identifies the native storage type. 2) type objects: e.g., STRING, BINARY, NUMBER, etc. These objects currently include functionality to match type_codes to type objects. >From native type_codes, it is fairly easy to write a function to obtain a type objects. It is not easy, nor commonly useful, to map a type object to a type code, since the mapping is usually not unique. I like having native type_codes, since that is (usually) totally unambiguous and useful for my own type mapping extensions. > In this way, you keep the module author's lives (reletivaly) simple while > adding benefit for the users. I know, I know, its down the line in what > you're talking about - but I just want to keep the goal in sight. I think > merging the type_code from output with the extended input types (you had a > good list), even if interfaces are not defined to Do anything with the > information, would be another good step. That way, you could move from DB > to DB and the type_ids would at least be the same (for similar columns), > reducing the burdon on the users. In my view, what you want can be accomplished by a helper routine. So instead of using the code_code in the description directly as a type constructor, one could call a function, say, getTypeConstructor(type_code). Or, if you really want syntactic sugar, one could define a type_code object that includes __call__ method that "does the right thing". Unfortunately, "the right thing" is frequently not so simple. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From mal at egenix.com Mon Jan 5 12:48:35 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Mon Jan 5 12:48:26 2004 Subject: [DB-SIG] Type code mappings In-Reply-To: References: Message-ID: <3FF9A373.40806@egenix.com> [Switching thread subject] Since there seems interest in discussing the introduction of better type code mapping in the DB API, let's start looking at it... Here's what we have (type_code refers to the .description entry; type object to what the DB API 2.0 spec defines as such): The DB API 2.0 currently only deals with the problem of letting the database module figure out an intended binding type of a parameter by providing factory functions which return objects with the right type signature for the intended type, e.g. you can wrap a binary data string into Binary(string) to let the module bind that particular column as binary data instead of text data. On the output side, the support for type mappings is almost non-existent. The DB API 2.0 only provides means to tell which group of types a particular output column is member of, e.g. whether a string return value refers to binary data (type_code == BINARY) or text data (type_code == STRING). This is a list of requirements that the new spec should provide: a. factory functions to map input objects to objects known and supported by the database module (DB API 2.0 already defines a few of these, but they should be extended to include more fine grained support for numbers, etc.) b. per-cursor based support for output type mapping, to e.g. have the database module return DATETIME columns as Python datetime objects, or convert DECIMAL columns to mx.Number rational numbers, etc. c. a standard output type mapping that allow round-trip safety, so that passing the output from a query as input for an INSERT will not lose information d. a way to map type_code information to factory functions (note: not type object to factory function, since type objects often cover more than one type_code) The bazar is open :-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 05 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fog at initd.org Mon Jan 5 17:11:55 2004 From: fog at initd.org (Federico Di Gregorio) Date: Mon Jan 5 17:11:30 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FF9A373.40806@egenix.com> References: <3FF9A373.40806@egenix.com> Message-ID: <1073340715.1254.20.camel@localhost> Il lun, 2004-01-05 alle 18:48, M.-A. Lemburg ha scritto: [snip] > The bazar is open :-) let's start with a _very_ small change, i.e., augmenting the number of type objects. i think we can do that in an absolutely backward compatible way. instead of having a flat set of type objects let's order them by placing the old ones at the first level of a tree and the others on the second level, as leaves: STRING |__CHAR |__VARCHAR |__TEXT BINARY NUMBER |__INTEGER |__LONGINTEGER |__FLOAT |__COMPLEX DATETIME |__TIME |__DATE |__TIMESTAMP |__INTERVAL ROWID and require that is a type object matches a type code, the same type code is matched by the parent(s) too. this allows for expanding the types without requiring any modification of old code, because if, for example, the type code for an int4 matches INTEGER it will always (as in DBAPI-2.0) also match NUMBER. the list is just preliminary and open to suggestions. i suppose we can have more string types (unicode anyone?), different binaries, times with tz information and such. the trick here is to let the modules implement extra type objects (like geometic types in postgresql) but mandate a wide enough base to satisfy everybody. -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org God is real. Unless declared integer. -- Anonymous FORTRAN programmer -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040105/36f7c730/attachment.bin From mal at egenix.com Tue Jan 6 05:51:42 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 6 05:51:45 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <1073340715.1254.20.camel@localhost> References: <3FF9A373.40806@egenix.com> <1073340715.1254.20.camel@localhost> Message-ID: <3FFA933E.7090603@egenix.com> Federico Di Gregorio wrote: > Il lun, 2004-01-05 alle 18:48, M.-A. Lemburg ha scritto: > [snip] > >>The bazar is open :-) > > > let's start with a _very_ small change, i.e., augmenting the number of > type objects. i think we can do that in an absolutely backward > compatible way. instead of having a flat set of type objects let's order > them by placing the old ones at the first level of a tree and the others > on the second level, as leaves: > > STRING > |__CHAR > |__VARCHAR > |__TEXT > BINARY > NUMBER > |__INTEGER > |__LONGINTEGER > |__FLOAT > |__COMPLEX > DATETIME > |__TIME > |__DATE > |__TIMESTAMP > |__INTERVAL > ROWID > > and require that is a type object matches a type code, the same type > code is matched by the parent(s) too. this allows for expanding the > types without requiring any modification of old code, because if, for > example, the type code for an int4 matches INTEGER it will always (as in > DBAPI-2.0) also match NUMBER. Good idea. > the list is just preliminary and open to suggestions. i suppose we can > have more string types (unicode anyone?), different binaries, times with > tz information and such. In mxODBC I'm passing through the SQL type code integers for the columns in .description. This allows for all supported types being reported. Currently these are: BIGINT BINARY BIT CHAR DATE DECIMAL DOUBLE FLOAT INTEGER LONGVARBINARY LONGVARCHAR NUMERIC REAL SMALLINT TIME TIMESTAMP VARBINARY VARCHAR WCHAR WCHAR_LONGVARCHAR WCHAR_VARCHAR plus a few more exotic ones that are ODBC cruft. The WCHAR ones are Unicode string columns. Most of these are SQL92 names for the types, perhaps we should stick to them ?! > the trick here is to let the modules implement extra type objects (like > geometic types in postgresql) but mandate a wide enough base to satisfy > everybody. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fog at initd.org Tue Jan 6 07:09:06 2004 From: fog at initd.org (Federico Di Gregorio) Date: Tue Jan 6 07:09:32 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FFA933E.7090603@egenix.com> References: <3FF9A373.40806@egenix.com> <1073340715.1254.20.camel@localhost> <3FFA933E.7090603@egenix.com> Message-ID: <1073390946.1254.122.camel@localhost> Il mar, 2004-01-06 alle 11:51, M.-A. Lemburg ha scritto: [snip] > plus a few more exotic ones that are ODBC cruft. The WCHAR ones > are Unicode string columns. > > Most of these are SQL92 names for the types, perhaps we should > stick to them ?! i think sticking to SQL92 names would be the right thing. here is a reorganized tree. note that i don't have an SQL reference at hand so some of the type names can be wrong (just copied marc's). STRING |__CHAR |__VARCHAR |__LONGVARCHAR |__TEXT |__WCHAR |__WVARCHAR |__WLONGVARCHAR BINARY |__VARBINARY |__LONGVARBINARY NUMBER |__BIT |__INTEGER |__SMALLINT |__BIGINT |__FLOAT |__DOUBLE |__REAL |__DECIMAL |__MONEY |__COMPLEX |__BOOLEAN DATETIME |__DATE |__TIME |__TIMESTAMP |__INTERVAL |__TZTIME |__TZTIMESTAMP ROWID some open questions: 1/ CHAR is under STRING or under NUMBER? 2/ there are real differences between VARCHAR and LONGVARCHAR? (i never found a db that has this difference and we don't want to include types just because they are defined in SQL92 right?) 3/ same for VARBINARY and LONGVARBINARY 4/ are MONEY and DECIMAL the same thing? (both are fixed-point, right?) 5/ in spite of SQL92 i renamed WCHAR_XXX -> WXXX, the other name is terrible :) 6/ where do we put BOOLEAN for compatibility under NUMBER; but IMHO it should have a top-level place in its own. 7/ is TEXT just LONGVARCHAR? -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org The number of the beast: vi vi vi. -- Delexa Jones -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040106/1e302dda/attachment.bin From mal at egenix.com Tue Jan 6 07:56:05 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Tue Jan 6 07:56:07 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <1073390946.1254.122.camel@localhost> References: <3FF9A373.40806@egenix.com> <1073340715.1254.20.camel@localhost> <3FFA933E.7090603@egenix.com> <1073390946.1254.122.camel@localhost> Message-ID: <3FFAB065.1020505@egenix.com> Federico Di Gregorio wrote: > Il mar, 2004-01-06 alle 11:51, M.-A. Lemburg ha scritto: > [snip] > >>plus a few more exotic ones that are ODBC cruft. The WCHAR ones >>are Unicode string columns. >> >>Most of these are SQL92 names for the types, perhaps we should >>stick to them ?! > > > i think sticking to SQL92 names would be the right thing. here is a > reorganized tree. note that i don't have an SQL reference at hand so > some of the type names can be wrong (just copied marc's). > > STRING > |__CHAR > |__VARCHAR > |__LONGVARCHAR > |__TEXT > |__WCHAR > |__WVARCHAR > |__WLONGVARCHAR > BINARY > |__VARBINARY > |__LONGVARBINARY > NUMBER > |__BIT > |__INTEGER > |__SMALLINT > |__BIGINT > |__FLOAT > |__DOUBLE > |__REAL > |__DECIMAL > |__MONEY > |__COMPLEX > |__BOOLEAN > DATETIME > |__DATE > |__TIME > |__TIMESTAMP > |__INTERVAL > |__TZTIME > |__TZTIMESTAMP > ROWID > > some open questions: > > 1/ CHAR is under STRING or under NUMBER? STRING > 2/ there are real differences between VARCHAR and LONGVARCHAR? > (i never found a db that has this difference and we don't want > to include types just because they are defined in SQL92 right?) The maximum length of the string makes the difference. VARCHAR is often restricted to about 4000 characters, while LONGVARCHAR extends to 2GB of character data. > 3/ same for VARBINARY and LONGVARBINARY Dito. > 4/ are MONEY and DECIMAL the same thing? (both are fixed-point, > right?) Not sure. AFAIK, MONEY is a MS (or maybe Sybase) invention. It is not defined in SQL92. I suppose that MONEY also carries currency information ?! > 5/ in spite of SQL92 i renamed WCHAR_XXX -> WXXX, the other name > is terrible :) True. > 6/ where do we put BOOLEAN for compatibility under NUMBER; but > IMHO it should have a top-level place in its own. I'd put it under NUMBER. ODBC doesn't have boolean, just BIT and that's grouped under numbers as well (even though SQL92 puts it in the same bin as strings). > 7/ is TEXT just LONGVARCHAR? Yes. Dito for BLOB and LONGVARBINARY. Some databases don't make a difference between binary and text data, others do and complain. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fog at initd.org Thu Jan 8 05:26:44 2004 From: fog at initd.org (Federico Di Gregorio) Date: Thu Jan 8 05:27:30 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FFAB065.1020505@egenix.com> References: <3FF9A373.40806@egenix.com> <1073340715.1254.20.camel@localhost> <3FFA933E.7090603@egenix.com> <1073390946.1254.122.camel@localhost> <3FFAB065.1020505@egenix.com> Message-ID: <1073557604.988.23.camel@localhost> another list, amended with marc's comments. if everybody agrees on this list i'll go on and write the new paragraph for the PEP document. changes: 1/ removed TEXT in favor of LONGVARCHAR; 2/ removed MONEY (DECIMAL is the name for fixed point), backends providing currency information will define a sub-type of decimal, right? 3/ BOOLEAN is under NUMBER (this will probably give some problems to postgresql and other db using 't' and 'f' but there is really no simple solution and Python *do* use 1 and 0 for True and False (at least untill 2.3) i tought about using three or four levels: STRING |__VARCHAR |__LONGVARCHAR but i really don't see any improvement. the two-level hierarchy is there for _backward compatibility_, not for organizing the tree. does this make sense? here is the list: STRING |__CHAR |__VARCHAR |__LONGVARCHAR |__TEXT |__WCHAR |__WVARCHAR |__WLONGVARCHAR BINARY |__VARBINARY |__LONGVARBINARY NUMBER |__BIT |__INTEGER |__SMALLINT |__BIGINT |__FLOAT |__DOUBLE |__REAL |__DECIMAL |__COMPLEX |__BOOLEAN DATETIME |__DATE |__TIME |__TIMESTAMP |__INTERVAL |__TZTIME |__TZTIMESTAMP ROWID -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org Gli esseri umani, a volte, sono destinati, per il solo fatto di esistere, a fare del male a qualcuno. -- Haruki Murakami -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040108/0a451074/attachment.bin From mal at egenix.com Thu Jan 8 05:58:43 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 8 05:59:00 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <1073557604.988.23.camel@localhost> References: <3FF9A373.40806@egenix.com> <1073340715.1254.20.camel@localhost> <3FFA933E.7090603@egenix.com> <1073390946.1254.122.camel@localhost> <3FFAB065.1020505@egenix.com> <1073557604.988.23.camel@localhost> Message-ID: <3FFD37E3.9000301@egenix.com> Federico Di Gregorio wrote: > another list, amended with marc's comments. if everybody agrees on this > list i'll go on and write the new paragraph for the PEP document. > > changes: > > 1/ removed TEXT in favor of LONGVARCHAR; > > 2/ removed MONEY (DECIMAL is the name for fixed point), backends > providing currency information will define a sub-type of decimal, > right? > > 3/ BOOLEAN is under NUMBER (this will probably give some problems to > postgresql and other db using 't' and 'f' but there is really no > simple solution and Python *do* use 1 and 0 for True and False (at > least untill 2.3) > > i tought about using three or four levels: > > STRING > |__VARCHAR > |__LONGVARCHAR > > but i really don't see any improvement. the two-level hierarchy is there > for _backward compatibility_, not for organizing the tree. does this > make sense? Not really: LONGVARCHAR is a completely different beast than VARCHAR. It is stored differently, usually has different properties in the database and switching from VARCHAR to LONGVARCHAR is not always easy. > here is the list: > > STRING > |__CHAR > |__VARCHAR > |__LONGVARCHAR > |__TEXT > |__WCHAR > |__WVARCHAR > |__WLONGVARCHAR > BINARY > |__VARBINARY > |__LONGVARBINARY > NUMBER > |__BIT > |__INTEGER > |__SMALLINT > |__BIGINT > |__FLOAT > |__DOUBLE > |__REAL > |__DECIMAL > |__COMPLEX > |__BOOLEAN > DATETIME > |__DATE > |__TIME > |__TIMESTAMP > |__INTERVAL > |__TZTIME > |__TZTIMESTAMP > ROWID A question: while the inheritence scheme makes sense for the implementation of the type objects, is it necessary to have them decend this way ? I'm asking because mxODBC has for many years exported symbols under these new names mapping to the underlying SQL integer constants. Changing these to type objects will break code. Now, the DB API only asks that these type objects must compare equal to the entries in the .description field. If e.g. CHAR is an integer, it would still compare equal to the .description field, but these would be no inheritence from STRING. OTOH, STRING has always been a special type object, so comparing STRING and CHAR will result in a true value in mxODBC. If CHAR is implemented as type object this would no longer be the case. Perhaps we ought to be more careful with adding globals to the module scope in the DB API spec. Possible work-arounds are e.g. STRING_CHAR, STRING_VARCHAR, etc. What do you think ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From jacobs at penguin.theopalgroup.com Thu Jan 8 06:23:56 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Thu Jan 8 06:24:04 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <1073557604.988.23.camel@localhost> Message-ID: On Thu, 8 Jan 2004, Federico Di Gregorio wrote: > another list, amended with marc's comments. if everybody agrees on this > list i'll go on and write the new paragraph for the PEP document. > > changes: > > 1/ removed TEXT in favor of LONGVARCHAR; Just call it TEXT -- it really isn't a VARCHAR in most regards. > 2/ removed MONEY (DECIMAL is the name for fixed point), backends > providing currency information will define a sub-type of decimal, > right? I'd keep it for now, even if it is vestigial. There are a lot of other backends that honor the difference. > 3/ BOOLEAN is under NUMBER (this will probably give some problems to > postgresql and other db using 't' and 'f' but there is really no > simple solution and Python *do* use 1 and 0 for True and False (at > least untill 2.3) SQL 92 and 99 are very clear that booleans and integers are not interchangeable. The Python semantics should not be the driving factor here. > i tought about using three or four levels: > > STRING > |__VARCHAR > |__LONGVARCHAR > > but i really don't see any improvement. the two-level hierarchy is there > for _backward compatibility_, not for organizing the tree. does this > make sense? The more compelling reason for a deeper hierarchy is so that type code matching can take place at several levels. i.e., some applications are satisfied to know that a datum is a string, while others will wish to decode the type information more deeply, but maybe not to the most derived type. e.g., many apps may be satisfied to know that a datum is a DECIMAL type, but not care enough to check if it is a MONEY type. > here is the list: > > STRING > |__CHAR > |__VARCHAR > |__LONGVARCHAR > |__TEXT > |__WCHAR > |__WVARCHAR > |__WLONGVARCHAR I'm somewhat sceptical about the WCHAR versions of STRING types. Python has native support for unicode strings, so why maintain this artificial distinction? The better solution would be to augment the column description format to include character encoding information. > BINARY > |__VARBINARY > |__LONGVARBINARY > NUMBER > |__BIT > |__INTEGER > |__SMALLINT > |__BIGINT > |__FLOAT > |__DOUBLE > |__REAL > |__DECIMAL > |__COMPLEX > |__BOOLEAN > DATETIME > |__DATE > |__TIME > |__TIMESTAMP > |__INTERVAL > |__TZTIME > |__TZTIMESTAMP Recent SQL standards put the TZ at the end, last I checked. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From fog at initd.org Thu Jan 8 06:25:53 2004 From: fog at initd.org (Federico Di Gregorio) Date: Thu Jan 8 06:25:46 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FFD37E3.9000301@egenix.com> References: <3FF9A373.40806@egenix.com> <1073340715.1254.20.camel@localhost> <3FFA933E.7090603@egenix.com> <1073390946.1254.122.camel@localhost> <3FFAB065.1020505@egenix.com> <1073557604.988.23.camel@localhost> <3FFD37E3.9000301@egenix.com> Message-ID: <1073561152.988.51.camel@localhost> Il gio, 2004-01-08 alle 11:58, M.-A. Lemburg ha scritto: [snip] > > but i really don't see any improvement. the two-level hierarchy is there > > for _backward compatibility_, not for organizing the tree. does this > > make sense? > > Not really: LONGVARCHAR is a completely different beast than > VARCHAR. It is stored differently, usually has different properties > in the database and switching from VARCHAR to LONGVARCHAR is > not always easy. agreed. [list snipped, see previous mail :) ] > A question: while the inheritence scheme makes sense for the > implementation of the type objects, is it necessary to have > them decend this way ? > > I'm asking because mxODBC has for many years exported > symbols under these new names mapping to the underlying > SQL integer constants. Changing these to type objects will > break code. > > Now, the DB API only asks that these type objects must compare > equal to the entries in the .description field. If e.g. CHAR > is an integer, it would still compare equal to the .description > field, but these would be no inheritence from STRING. maybe i used the wrong rapresentation (inheritance tree). by inheritance i just mean that if the type_code x from the .description is compared with STRING and CHAR the following holds true: x == CHAR implies x == STRING x == STRING does not implies x == CHAR this is just to make sure that old code comparing x with STRING still works and that new code has more types available. if mxODBC exports CHAR as an integer constant, the two equations above are _still true_. now, by not requiring a specific kind of object for object types but just a generic "object type", i think you're safe by having two kind of object types: type(STRING) and simple int. or maybe, my two other proposals are better, see below. > OTOH, STRING has always been a special type object, so > comparing STRING and CHAR will result in a true value > in mxODBC. If CHAR is implemented as type object this would > no longer be the case. or we can ask for this *to be* the case :) [it makes sense to me, even if it complicates the coding of the type objects a little bit] > Perhaps we ought to be more careful with adding globals > to the module scope in the DB API spec. Possible work-arounds > are e.g. STRING_CHAR, STRING_VARCHAR, etc. > > What do you think ? i think we should have requested, a long time ago, developers to not pollute the module namespace but use a sub-module. here's another little proposal: 1/ all the module-level objects specified by DBAPI-2.0 should be duplicated inside a sub-module, named 'dbapi' (or a dictionary?); i.e., mxODBC.STRING -> mxODBC.dbapi.STRING mxODBC.connect -> mxODBC.dbapi.connect 2/ the old DBAPI-2.0 names should not be removed from the top-level module, but new objects are added only to the new 'dbapi' module/dictionary. this effectively add a safe namespace for future DBAPI additions. -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org Qu'est ce que la folie? Juste un sentiment de libert? si fort qu'on en oublie ce qui nous rattache au monde... -- J. de Loctra -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040108/63182078/attachment.bin From fog at initd.org Thu Jan 8 06:35:49 2004 From: fog at initd.org (Federico Di Gregorio) Date: Thu Jan 8 06:35:56 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: References: Message-ID: <1073561748.988.62.camel@localhost> Il gio, 2004-01-08 alle 12:23, Kevin Jacobs ha scritto: > On Thu, 8 Jan 2004, Federico Di Gregorio wrote: > > > another list, amended with marc's comments. if everybody agrees on this > > list i'll go on and write the new paragraph for the PEP document. > > > > changes: > > > > 1/ removed TEXT in favor of LONGVARCHAR; > > Just call it TEXT -- it really isn't a VARCHAR in most regards. if no backend has both and honore the difference it is just a naming convention. i would better like to have semi-standard LONGVARCHAR and LONGBINARY than TEXT, BLOB, GLOB, etc. > > 2/ removed MONEY (DECIMAL is the name for fixed point), backends > > providing currency information will define a sub-type of decimal, > > right? > > I'd keep it for now, even if it is vestigial. There are a lot of other > backends that honor the difference. did not know. i'll put MONEY back. > > 3/ BOOLEAN is under NUMBER (this will probably give some problems to > > postgresql and other db using 't' and 'f' but there is really no > > simple solution and Python *do* use 1 and 0 for True and False (at > > least untill 2.3) > > SQL 92 and 99 are very clear that booleans and integers are not > interchangeable. The Python semantics should not be the driving factor > here. did not know about that too. we need a poll here, to see where every module puts booleans right now. if there are enough differences we can just have a new top-level class (and maybe let individual modules keep the old meaning too; i see no problems with type_code x comparing true both to BOOLEAN and NUMERIC). > > i tought about using three or four levels: > > > > STRING > > |__VARCHAR > > |__LONGVARCHAR > > > > but i really don't see any improvement. the two-level hierarchy is there > > for _backward compatibility_, not for organizing the tree. does this > > make sense? > > The more compelling reason for a deeper hierarchy is so that type code > matching can take place at several levels. i.e., some applications are > satisfied to know that a datum is a string, while others will wish to decode > the type information more deeply, but maybe not to the most derived type. > e.g., many apps may be satisfied to know that a datum is a DECIMAL type, but > not care enough to check if it is a MONEY type. this is good example because MONEY is clearly more specific than DECIMAL. but i dubt i can say that TEXT is more specific than VARCHAR or that VARCHAR is more specific than text. mm.. if by "more specific" we mean "can be casted to without loss of information" then a VARCHAR is more specific... > > here is the list: > > > > STRING > > |__CHAR > > |__VARCHAR > > |__LONGVARCHAR > > |__TEXT > > |__WCHAR > > |__WVARCHAR > > |__WLONGVARCHAR > > I'm somewhat sceptical about the WCHAR versions of STRING types. Python has > native support for unicode strings, so why maintain this artificial > distinction? The better solution would be to augment the column description > format to include character encoding information. i wholehartly support the additiong of an encoding field to .description. > > BINARY > > |__VARBINARY > > |__LONGVARBINARY > > NUMBER > > |__BIT > > |__INTEGER > > |__SMALLINT > > |__BIGINT > > |__FLOAT > > |__DOUBLE > > |__REAL > > |__DECIMAL > > |__COMPLEX > > |__BOOLEAN > > DATETIME > > |__DATE > > |__TIME > > |__TIMESTAMP > > |__INTERVAL > > |__TZTIME > > |__TZTIMESTAMP > > Recent SQL standards put the TZ at the end, last I checked. so let's follow the standard. :) i'll wait for some more comments and send a new tree soon. federico p.s. do you people like list+cc postings? i can easily avoid... :) -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org La felicit? ? una tazza di cioccolata calda. Sempre. -- Io -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040108/5b032a2c/attachment.bin From mal at egenix.com Thu Jan 8 06:54:28 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 8 06:54:42 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: References: Message-ID: <3FFD44F4.2000604@egenix.com> Kevin Jacobs wrote: > On Thu, 8 Jan 2004, Federico Di Gregorio wrote: >> 3/ BOOLEAN is under NUMBER (this will probably give some problems to >> postgresql and other db using 't' and 'f' but there is really no >> simple solution and Python *do* use 1 and 0 for True and False (at >> least untill 2.3) > > SQL 92 and 99 are very clear that booleans and integers are not > interchangeable. The Python semantics should not be the driving factor > here. The question here is what you want to do with the inheritence information. Most (if not all) database modules return booleans/bits as 1/0 or True/False, so NUMBER would make sense if you're interested in what the application will see. SQL would make them a subclass of STRING. >> STRING >> |__CHAR >> |__VARCHAR >> |__LONGVARCHAR >> |__TEXT >> |__WCHAR >> |__WVARCHAR >> |__WLONGVARCHAR > > I'm somewhat sceptical about the WCHAR versions of STRING types. Python has > native support for unicode strings, so why maintain this artificial > distinction? The better solution would be to augment the column description > format to include character encoding information. Unicode and strings *are* different in Python and also handled differently in databases, so the distinction makes a lot of sense, e.g. strings are often subject to an encoding specified by the database, while Unicode does not have these deficiencies. Add fields to .description is problematic. Applications tend to use tuple unpacking to access the tuples in that list and adding fields would break this. In general, using tuples for things can may be extended in the future is a bad idea. For DB API 3.0 we should probably switch to a list of ColumnDescription objects instead. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fog at initd.org Thu Jan 8 07:00:59 2004 From: fog at initd.org (Federico Di Gregorio) Date: Thu Jan 8 07:01:46 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FFD44F4.2000604@egenix.com> References: <3FFD44F4.2000604@egenix.com> Message-ID: <1073563259.988.68.camel@localhost> Il gio, 2004-01-08 alle 12:54, M.-A. Lemburg ha scritto: [snip] > Add fields to .description is problematic. Applications tend > to use tuple unpacking to access the tuples in that list and > adding fields would break this. mm.. you're right. but encoding information is something that i would like to see in DBAPI 2.1 or 3.0. btw, are we talking about 2.1 or 3.0? in my head the "more types" discussion is 2.1 but if we add all the other stuff related to types i think we go a great deal toward 3.0. i ask because i think we can't have any incompatible changes in 2.1 but maybe a little backward incompatibility in 2.0->3.0 would not be that bad. federico -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org La felicit? ? una tazza di cioccolata calda. Sempre. -- Io -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040108/6fdbf2d7/attachment-0001.bin From jacobs at penguin.theopalgroup.com Thu Jan 8 07:07:38 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Thu Jan 8 07:07:50 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FFD44F4.2000604@egenix.com> Message-ID: On Thu, 8 Jan 2004, M.-A. Lemburg wrote: > Kevin Jacobs wrote: > > On Thu, 8 Jan 2004, Federico Di Gregorio wrote: > >> 3/ BOOLEAN is under NUMBER (this will probably give some problems to > >> postgresql and other db using 't' and 'f' but there is really no > >> simple solution and Python *do* use 1 and 0 for True and False (at > >> least untill 2.3) > > > > SQL 92 and 99 are very clear that booleans and integers are not > > interchangeable. The Python semantics should not be the driving factor > > here. > > The question here is what you want to do with the inheritence > information. Most (if not all) database modules return booleans/bits > as 1/0 or True/False, so NUMBER would make sense if you're interested > in what the application will see. > > SQL would make them a subclass of STRING. I've in the process of moving to a different city, so I don't have my SQL99 and SQL200x drafts handy, but SQL seems to be moving rapidly in the direction of having boolean types distinct from both STRING and NUMBER. e.g., the "official" boolean literals are TRUE and FALSE, though various backends implement varying degrees of backward compatibility with other representations like 't' and 'f' for PostgreSQL. However, use of BOOLEAN columns can only increase as more database vendors are enhancing their products to be more standards compliant. > > I'm somewhat sceptical about the WCHAR versions of STRING types. Python has > > native support for unicode strings, so why maintain this artificial > > distinction? The better solution would be to augment the column description > > format to include character encoding information. > > Unicode and strings *are* different in Python and also handled > differently in databases, so the distinction makes a lot of sense, > e.g. strings are often subject to an encoding specified by the > database, while Unicode does not have these deficiencies. I agree, so long as we add language that stipulates that CHAR and VARCHAR are always returned as undecoded str types with encoding information available (unless we go farther and add support for string output encodings too). Several "unicode-enabled" drivers, if I remember correctly, will return unicode strings automagically if the default data encoding is not ASCII. > Add fields to .description is problematic. Applications tend > to use tuple unpacking to access the tuples in that list and > adding fields would break this. > > In general, using tuples for things can may be extended in > the future is a bad idea. For DB API 3.0 we should probably > switch to a list of ColumnDescription objects instead. In the mean time, maybe we should support an extended_description dictionary? A list seems unnecessarily -- well... -- linear. -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From mal at egenix.com Thu Jan 8 08:00:23 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 8 08:00:40 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: References: Message-ID: <3FFD5467.1080209@egenix.com> Kevin Jacobs wrote: > On Thu, 8 Jan 2004, M.-A. Lemburg wrote: > >>Kevin Jacobs wrote: >> >>>On Thu, 8 Jan 2004, Federico Di Gregorio wrote: >>> >>>> 3/ BOOLEAN is under NUMBER (this will probably give some problems to >>>> postgresql and other db using 't' and 'f' but there is really no >>>> simple solution and Python *do* use 1 and 0 for True and False (at >>>> least untill 2.3) >>> >>>SQL 92 and 99 are very clear that booleans and integers are not >>>interchangeable. The Python semantics should not be the driving factor >>>here. >> >>The question here is what you want to do with the inheritence >>information. Most (if not all) database modules return booleans/bits >>as 1/0 or True/False, so NUMBER would make sense if you're interested >>in what the application will see. >> >>SQL would make them a subclass of STRING. > > > I've in the process of moving to a different city, so I don't have my SQL99 > and SQL200x drafts handy, Here's a useful URL: http://www.wiscorp.com/SQLStandards.html > but SQL seems to be moving rapidly in the > direction of having boolean types distinct from both STRING and NUMBER. > e.g., the "official" boolean literals are TRUE and FALSE, though various > backends implement varying degrees of backward compatibility with other > representations like 't' and 'f' for PostgreSQL. However, use of BOOLEAN > columns can only increase as more database vendors are enhancing their > products to be more standards compliant. In that case, perhaps BOOLEAN should be at top-level ?! >>>I'm somewhat sceptical about the WCHAR versions of STRING types. Python has >>>native support for unicode strings, so why maintain this artificial >>>distinction? The better solution would be to augment the column description >>>format to include character encoding information. >> >>Unicode and strings *are* different in Python and also handled >>differently in databases, so the distinction makes a lot of sense, >>e.g. strings are often subject to an encoding specified by the >>database, while Unicode does not have these deficiencies. > > I agree, so long as we add language that stipulates that CHAR and VARCHAR > are always returned as undecoded str types with encoding information > available (unless we go farther and add support for string output encodings > too). Sounds good. > Several "unicode-enabled" drivers, if I remember correctly, will > return unicode strings automagically if the default data encoding is not > ASCII. mxODBC let's you set a flag to one of these values: * always return 8-bit strings (in the database encoding) * allow the database to decide whether or not to send Unicode * always return Unicode (with 8-bit strings decoded according to an encoding you set on the connection; unfortunately, there's no easy way to read the database default encoding) >>Add fields to .description is problematic. Applications tend >>to use tuple unpacking to access the tuples in that list and >>adding fields would break this. >> >>In general, using tuples for things can may be extended in >>the future is a bad idea. For DB API 3.0 we should probably >>switch to a list of ColumnDescription objects instead. > > In the mean time, maybe we should support an extended_description > dictionary? A list seems unnecessarily -- well... -- linear. Another possibility would be to add a method which returns whatever information you need depending on the parameters (this is how ODBC works most of the time and they happily add new query parameter constants in each new revision). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From mal at egenix.com Thu Jan 8 08:06:20 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 8 08:06:36 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <1073563259.988.68.camel@localhost> References: <3FFD44F4.2000604@egenix.com> <1073563259.988.68.camel@localhost> Message-ID: <3FFD55CC.6030409@egenix.com> Federico Di Gregorio wrote: > Il gio, 2004-01-08 alle 12:54, M.-A. Lemburg ha scritto: > [snip] > >>Add fields to .description is problematic. Applications tend >>to use tuple unpacking to access the tuples in that list and >>adding fields would break this. > > mm.. you're right. but encoding information is something that i would > like to see in DBAPI 2.1 or 3.0. Depends on what you mean with "encoding information" :-) Typically you have a database encoding that the database uses to store text data. The encoding then applies to all data and thus all columns in the result set, so you would put such information on the connection as .encoding attribute. Then again, databases may not make it easy for the interface to tell which encoding it uses, so you will probably also make that setting writeable (this is what mxODBC 2.1 does). > btw, are we talking about 2.1 or 3.0? in my head the "more types" > discussion is 2.1 but if we add all the other stuff related to types i > think we go a great deal toward 3.0. i ask because i think we can't have > any incompatible changes in 2.1 but maybe a little backward > incompatibility in 2.0->3.0 would not be that bad. I think we should focus on 2.1 first and perhaps add notes on what could happen in 3.0. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 06 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From fog at initd.org Thu Jan 8 14:38:23 2004 From: fog at initd.org (Federico Di Gregorio) Date: Thu Jan 8 14:39:31 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FFD5467.1080209@egenix.com> References: <3FFD5467.1080209@egenix.com> Message-ID: <1073590703.5609.32.camel@localhost> Il gio, 2004-01-08 alle 14:00, M.-A. Lemburg ha scritto: [snip] > > products to be more standards compliant. > > In that case, perhaps BOOLEAN should be at top-level ?! i vote for making it top-level. [snip] > mxODBC let's you set a flag to one of these values: > * always return 8-bit strings (in the database encoding) > * allow the database to decide whether or not to send Unicode > * always return Unicode (with 8-bit strings decoded according > to an encoding you set on the connection; unfortunately, > there's no easy way to read the database default encoding) sounds good and i would like to see something like that standarized in the DBAPI. currently psycopg uses the encoding parameter of the connection to set the "client_encoding" variable but stops there. -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org God is real. Unless declared integer. -- Anonymous FORTRAN programmer -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040108/df968964/attachment.bin From anthony at computronix.com Thu Jan 8 17:22:43 2004 From: anthony at computronix.com (Anthony Tuininga) Date: Thu Jan 8 17:22:48 2004 Subject: [DB-SIG] Time() and TimeFromTicks() constructors Message-ID: <1073600563.28215.199.camel@chl0263.edmonton.computronix.com> It has recently come to my attention that the presence of Time() and TimeFromTicks() in cx_Oracle is misleading at best since Oracle does not support the concept of a time only column. The DB API document does not indicate that these constructors are optional so I was wondering what the consensus of this list would be on what ought to be done. 1) Remove the constructors. 2) Raise an exception when these constructors are used, indicating that Oracle doesn't support the concept. I lean towards number 2 myself but would like some feedback from others on the list. In both cases the documentation will be updated to indicate why these constructors do not work (or are not present). It would also be useful if the DB API document would be modified appropriately since (apparently) not all databases support the concept of time only columns. There might be others that I am not aware of. -- Anthony Tuininga anthony@computronix.com Computronix Distinctive Software. Real People. Suite 200, 10216 - 124 Street NW Edmonton, AB, Canada T5N 4A3 Phone: (780) 454-3700 Fax: (780) 454-3838 http://www.computronix.com From fog at initd.org Thu Jan 8 17:25:40 2004 From: fog at initd.org (Federico Di Gregorio) Date: Thu Jan 8 17:25:29 2004 Subject: [DB-SIG] Time() and TimeFromTicks() constructors In-Reply-To: <1073600563.28215.199.camel@chl0263.edmonton.computronix.com> References: <1073600563.28215.199.camel@chl0263.edmonton.computronix.com> Message-ID: <1073600740.5609.96.camel@localhost> Il gio, 2004-01-08 alle 23:22, Anthony Tuininga ha scritto: [snip] > 2) Raise an exception when these constructors are used, indicating that > Oracle doesn't support the concept. > > I lean towards number 2 myself but would like some feedback from others > on the list. In both cases the documentation will be updated to indicate > why these constructors do not work (or are not present). I'd vote for (2) because it is was the DBAPI mandates for other not-supported-by-all-db methods (like .callproc). ciao, -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org All programmers are optimists. -- Frederick P. Brooks, Jr. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040108/0a0168c5/attachment.bin From mal at egenix.com Thu Jan 8 17:36:07 2004 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Jan 8 17:36:08 2004 Subject: [DB-SIG] Time() and TimeFromTicks() constructors In-Reply-To: <1073600740.5609.96.camel@localhost> References: <1073600563.28215.199.camel@chl0263.edmonton.computronix.com> <1073600740.5609.96.camel@localhost> Message-ID: <3FFDDB57.3060909@egenix.com> Federico Di Gregorio wrote: > Il gio, 2004-01-08 alle 23:22, Anthony Tuininga ha scritto: > [snip] > >>2) Raise an exception when these constructors are used, indicating that >>Oracle doesn't support the concept. >> >>I lean towards number 2 myself but would like some feedback from others >>on the list. In both cases the documentation will be updated to indicate >>why these constructors do not work (or are not present). > > I'd vote for (2) because it is was the DBAPI mandates for other > not-supported-by-all-db methods (like .callproc). +1 The DB API defines the NotSupportedError for this purpose. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 08 2004) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From helena_b2001 at yahoo.com Thu Jan 8 19:19:42 2004 From: helena_b2001 at yahoo.com (helena bhaska) Date: Thu Jan 8 19:19:46 2004 Subject: [DB-SIG] InternalError Message-ID: <20040109001942.19232.qmail@web20411.mail.yahoo.com> Hi, I am pretty new to Python database programming, and I am trying to write a network application, where the client sends some data to the server, and the server saves the data into the database. Client and Server communicate through sockets, and the server throws the following error when trying to connect to the database: Exception happened during processing of request from ('127.0.0.1', 34516) Traceback (most recent call last): File "/usr/local/lib/python2.3/SocketServer.py", line 463, in process_request_thread self.finish_request(request, client_address) File "/usr/local/lib/python2.3/SocketServer.py", line 254, in finish_request self.RequestHandlerClass(request, client_address, self) File "/usr/local/lib/python2.3/SocketServer.py", line 521, in __init__ self.handle() File "test_server.py", line 16, in handle db = MySQLdb.Connect(host="localhost", user="root", passwd="password", db="Purple") File "/usr/local/lib/python2.3/site-packages/MySQLdb/__init__.py", line 63, in Connect return apply(Connection, args, kwargs) File "/usr/local/lib/python2.3/site-packages/MySQLdb/connections.py", line 115, in __init__ self._make_connection(args, kwargs2) File "/usr/local/lib/python2.3/site-packages/MySQLdb/connections.py", line 41, in _make_connection apply(super(ConnectionBase, self).__init__, args, kwargs) InternalError: (-1, 'connection is closed') here are the chunks of code that attempt to do that: client code: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) host = 'localhost' port = 8081 s.connect((host, port)) netstring.writens(s, string_to_send) netstring.writens(s, '') s.close() server code: class PurpleRequestHandler(SocketServer.BaseRequestHandler): def handle(self): while 1: rq = netstring.readns(self.request) if rq == "": break dt = DateTime.now() db = MySQLdb.Connect(host="localhost", user="root", passwd="password", db="Test") c = db.cursor() c.execute("""INSERT INTO Commands (Time, Command) VALUES (%s, %s)""", (str(dt), rq)) c.close() db.close() purpleServer = SocketServer.ThreadingTCPServer(('', 8081), PurpleRequestHandler) purpleServer.serve_forever() The interesting part is that if my server runs on apache and I connect to it through urllib.urlopen(), the data is saved into the database just fine. Is my MySQL db not configured correctly or something? Thanks for any pointers! __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus From chris at cogdon.org Thu Jan 8 19:46:10 2004 From: chris at cogdon.org (Chris Cogdon) Date: Thu Jan 8 19:46:23 2004 Subject: [DB-SIG] InternalError In-Reply-To: <20040109001942.19232.qmail@web20411.mail.yahoo.com> References: <20040109001942.19232.qmail@web20411.mail.yahoo.com> Message-ID: <37955EC0-423D-11D8-91FB-000A95E3823E@cogdon.org> On Jan 8, 2004, at 16:19, helena bhaska wrote: > class > PurpleRequestHandler(SocketServer.BaseRequestHandler): > def handle(self): > while 1: > rq = netstring.readns(self.request) > if rq == "": > break > dt = DateTime.now() > db = MySQLdb.Connect(host="localhost", > user="root", passwd="password", db="Test") > c = db.cursor() > c.execute("""INSERT INTO Commands (Time, > Command) VALUES (%s, %s)""", (str(dt), rq)) > c.close() > db.close() > purpleServer = SocketServer.ThreadingTCPServer(('', > 8081), PurpleRequestHandler) > purpleServer.serve_forever() > > The interesting part is that if my server runs on > apache and I connect to it through urllib.urlopen(), > the data is saved into the database just fine. Is my > MySQL db not configured correctly or something? > Thanks for any pointers! It's very odd that it's working via a apache CGI, but not through the python server handlers. To test it out on my own machine, I wrote this quick thing: import SocketServer, MySQLdb class Handler ( SocketServer.StreamRequestHandler ): def handle ( self ): while 1: t = self.rfile.readline () if t == '': break db = MySQLdb.Connect ( host='localhost', user='root', passwd='xxx', db='xxx' ) db.close () print "read command:", t server = SocketServer.ThreadingTCPServer ( ('',8081), Handler ) server.serve_forever () ( I used the StreamRequestHandler because I don't have netstring :) And then telnetted to port 8081 and typed a command. The command came up just fine, and no error message from the MySQL Connect. You might want to try and eliminate problems in the client, first, by simply telnetting to your port 8081 and typing commands in netstring format. Eg: "5:Hello," (omit quotes) If THAT works, then that's really odd. Narrow it down to this: test1.py: import MySQLdb db = MySQLdb.Connect ( host='localhost', user='root', passwd='xxx', db='xxx' ) Run that... if that fails, then you've got problems in the MySQLdb installation. If that works, try my script above, if that fails, then ... well, I dont know :/ One thing to ask... are the parameters to Connect for your SocketServer version EXACTLY the same as the parameters for your apache CGI version? -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From helena_b2001 at yahoo.com Thu Jan 8 20:05:42 2004 From: helena_b2001 at yahoo.com (helena bhaska) Date: Thu Jan 8 20:05:46 2004 Subject: [DB-SIG] InternalError In-Reply-To: <37955EC0-423D-11D8-91FB-000A95E3823E@cogdon.org> Message-ID: <20040109010542.11188.qmail@web20420.mail.yahoo.com> Thanks, yes, in fact running test1.py as you suggested fails with a Segmentation Fault...I'm going to try to reinstall MySQLdb. --- Chris Cogdon wrote: > > On Jan 8, 2004, at 16:19, helena bhaska wrote: > > > class > > > PurpleRequestHandler(SocketServer.BaseRequestHandler): > > def handle(self): > > while 1: > > rq = netstring.readns(self.request) > > if rq == "": > > break > > dt = DateTime.now() > > db = MySQLdb.Connect(host="localhost", > > user="root", passwd="password", db="Test") > > c = db.cursor() > > c.execute("""INSERT INTO Commands > (Time, > > Command) VALUES (%s, %s)""", (str(dt), rq)) > > c.close() > > db.close() > > purpleServer = > SocketServer.ThreadingTCPServer(('', > > 8081), PurpleRequestHandler) > > purpleServer.serve_forever() > > > > The interesting part is that if my server runs on > > apache and I connect to it through > urllib.urlopen(), > > the data is saved into the database just fine. Is > my > > MySQL db not configured correctly or something? > > Thanks for any pointers! > > It's very odd that it's working via a apache CGI, > but not through the > python server handlers. To test it out on my own > machine, I wrote this > quick thing: > > import SocketServer, MySQLdb > > class Handler ( SocketServer.StreamRequestHandler ): > def handle ( self ): > while 1: > t = self.rfile.readline () > if t == '': break > db = MySQLdb.Connect ( > host='localhost', user='root', > passwd='xxx', db='xxx' ) > db.close () > print "read command:", t > > server = SocketServer.ThreadingTCPServer ( > ('',8081), Handler ) > server.serve_forever () > > ( I used the StreamRequestHandler because I don't > have netstring :) > > And then telnetted to port 8081 and typed a command. > The command came > up just fine, and no error message from the MySQL > Connect. > > You might want to try and eliminate problems in the > client, first, by > simply telnetting to your port 8081 and typing > commands in netstring > format. Eg: > > "5:Hello," (omit quotes) > > If THAT works, then that's really odd. Narrow it > down to this: > > test1.py: > > import MySQLdb > db = MySQLdb.Connect ( host='localhost', > user='root', passwd='xxx', > db='xxx' ) > > Run that... if that fails, then you've got problems > in the MySQLdb > installation. If that works, try my script above, if > that fails, then > ... well, I dont know :/ > > > One thing to ask... are the parameters to Connect > for your SocketServer > version EXACTLY the same as the parameters for your > apache CGI version? > > > -- > ("`-/")_.-'"``-._ Chris Cogdon > > . . `; -._ )-;-,_`) > (v_,)' _ )`-.\ ``-' > _.- _..-_/ / ((.' > ((,.-' ((,/ fL > __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus From helena_b2001 at yahoo.com Thu Jan 8 21:04:47 2004 From: helena_b2001 at yahoo.com (helena bhaska) Date: Thu Jan 8 21:04:51 2004 Subject: [DB-SIG] InternalError In-Reply-To: <37955EC0-423D-11D8-91FB-000A95E3823E@cogdon.org> Message-ID: <20040109020447.23022.qmail@web20420.mail.yahoo.com> Hi, so I fixed my weird problem by reinstalling MySQLdb (note to self: should try that first before running for help :)), but I am still curious why it worked via apache! Does anyone have any ideas? Thanks! Helena --- Chris Cogdon wrote: > > On Jan 8, 2004, at 16:19, helena bhaska wrote: > > > class > > > PurpleRequestHandler(SocketServer.BaseRequestHandler): > > def handle(self): > > while 1: > > rq = netstring.readns(self.request) > > if rq == "": > > break > > dt = DateTime.now() > > db = MySQLdb.Connect(host="localhost", > > user="root", passwd="password", db="Test") > > c = db.cursor() > > c.execute("""INSERT INTO Commands > (Time, > > Command) VALUES (%s, %s)""", (str(dt), rq)) > > c.close() > > db.close() > > purpleServer = > SocketServer.ThreadingTCPServer(('', > > 8081), PurpleRequestHandler) > > purpleServer.serve_forever() > > > > The interesting part is that if my server runs on > > apache and I connect to it through > urllib.urlopen(), > > the data is saved into the database just fine. Is > my > > MySQL db not configured correctly or something? > > Thanks for any pointers! > > It's very odd that it's working via a apache CGI, > but not through the > python server handlers. To test it out on my own > machine, I wrote this > quick thing: > > import SocketServer, MySQLdb > > class Handler ( SocketServer.StreamRequestHandler ): > def handle ( self ): > while 1: > t = self.rfile.readline () > if t == '': break > db = MySQLdb.Connect ( > host='localhost', user='root', > passwd='xxx', db='xxx' ) > db.close () > print "read command:", t > > server = SocketServer.ThreadingTCPServer ( > ('',8081), Handler ) > server.serve_forever () > > ( I used the StreamRequestHandler because I don't > have netstring :) > > And then telnetted to port 8081 and typed a command. > The command came > up just fine, and no error message from the MySQL > Connect. > > You might want to try and eliminate problems in the > client, first, by > simply telnetting to your port 8081 and typing > commands in netstring > format. Eg: > > "5:Hello," (omit quotes) > > If THAT works, then that's really odd. Narrow it > down to this: > > test1.py: > > import MySQLdb > db = MySQLdb.Connect ( host='localhost', > user='root', passwd='xxx', > db='xxx' ) > > Run that... if that fails, then you've got problems > in the MySQLdb > installation. If that works, try my script above, if > that fails, then > ... well, I dont know :/ > > > One thing to ask... are the parameters to Connect > for your SocketServer > version EXACTLY the same as the parameters for your > apache CGI version? > > > -- > ("`-/")_.-'"``-._ Chris Cogdon > > . . `; -._ )-;-,_`) > (v_,)' _ )`-.\ ``-' > _.- _..-_/ / ((.' > ((,.-' ((,/ fL > __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus From chris at cogdon.org Thu Jan 8 21:08:37 2004 From: chris at cogdon.org (Chris Cogdon) Date: Thu Jan 8 21:08:44 2004 Subject: [DB-SIG] InternalError In-Reply-To: <20040109020447.23022.qmail@web20420.mail.yahoo.com> References: <20040109020447.23022.qmail@web20420.mail.yahoo.com> Message-ID: On Jan 8, 2004, at 18:04, helena bhaska wrote: > Hi, > so I fixed my weird problem by reinstalling MySQLdb > (note to self: should try that first before running > for help :)), but I am still curious why it worked via > apache! Does anyone have any ideas? My guess is that there was an inconsistency between the various mysql client libraries, and a recompile fixed it. Perhaps the 'via apache' version might not work anymore :) -- ("`-/")_.-'"``-._ Chris Cogdon . . `; -._ )-;-,_`) (v_,)' _ )`-.\ ``-' _.- _..-_/ / ((.' ((,.-' ((,/ fL From andy47 at halfcooked.com Fri Jan 9 15:34:24 2004 From: andy47 at halfcooked.com (Andy Todd) Date: Fri Jan 9 15:34:55 2004 Subject: [DB-SIG] Type code mappings: expanding the type objects In-Reply-To: <3FFD55CC.6030409@egenix.com> References: <3FFD44F4.2000604@egenix.com> <1073563259.988.68.camel@localhost> <3FFD55CC.6030409@egenix.com> Message-ID: <3FFF1050.2090105@halfcooked.com> M.-A. Lemburg wrote: > Federico Di Gregorio wrote: > >> Il gio, 2004-01-08 alle 12:54, M.-A. Lemburg ha scritto: >> [snip] >> >>> Add fields to .description is problematic. Applications tend >>> to use tuple unpacking to access the tuples in that list and >>> adding fields would break this. >> >> >> mm.. you're right. but encoding information is something that i would >> like to see in DBAPI 2.1 or 3.0. > > > Depends on what you mean with "encoding information" :-) > > Typically you have a database encoding that the database uses > to store text data. The encoding then applies to all data and > thus all columns in the result set, so you would put such > information on the connection as .encoding attribute. > Just a word of warning, some databases (specifically SQL Server) allow you to specify the encoding at a more granular level than the database. In SQL Server it can be defined at the column, table or database (schema) level. Whilst it could be an attribute of the connection this may not be specific enough. Regards, Andy -- -------------------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com/ From beverle at fushigiyugi.com Sat Jan 10 17:22:36 2004 From: beverle at fushigiyugi.com (Jonathon) Date: Sat Jan 10 17:22:41 2004 Subject: [DB-SIG] Your own ebay business DA4DC9336810806F0EBF5B74897B2F95 Message-ID: <200401102222.i0AMMJvZ052677@mxzilla6.xs4all.nl> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040110/f8262fb1/attachment.html From arjen.dijkstra at hccnet.nl Sun Jan 11 10:19:11 2004 From: arjen.dijkstra at hccnet.nl (arjen.dijkstra@hccnet.nl) Date: Sun Jan 11 10:19:19 2004 Subject: [DB-SIG] Oracle to Mysql (dates) Help please Message-ID: <200401111519.i0BFJBNC002972@smtp.hccnet.nl> Hi all, I'm posting to this sig for the first time, i hope I'm doing this right. I'm trying to export a view tables from a Oracle database to a Mysql database. I create insert statements (they look alright), but it all goes wrong when I try to execute them in Mysql, because the dates must have quotes on each side. I just don't know how make the dates right. Well I'll just show you the code and some insert statements it generates. Could anyone please help me? Thanks, Arjen ####Code#### import cx_Oracle tabellen=["machine"] con_oracle=cx_Oracle.connect("bla/bla") c_oracle=con_oracle.cursor() import MySQLdb my=MySQLdb.Connect("localhost", db="bla") my_mysql=my.cursor() for tabel in tabellen: print tabel c_oracle.execute("select * from %s" % tabel) a_oracle=c_oracle.fetchone() #file=open("%s.sql" % tabel, 'w') while a_oracle != None: b=str(a_oracle) ins="insert into %s values %s;\n" % (tabel, b) #file.write(ins) my_mysql.execute(ins) #print ins a_oracle=c_oracle.fetchone() file.close() con_oracle.close() my.close() ##insert statement### insert into machine values ('230KM', ' ', '230KM', 1980-01-01 00:00:00, 2035-01-01 00:00:00, 1, 100, 'asap', 'NO', 0, 0, 'corrugator', 2003-12-04 06:00:00, 1970-01-01 01:00:00, ' ', 'normal', 0.0, 0.0, 7, ' ', ' ', 'normal', ' ', ' ', 'A', 2003-12-04 09:42:14, 82766); From prethorian at asistencia.org Mon Jan 12 12:10:47 2004 From: prethorian at asistencia.org (Textiles L. Tarim) Date: Mon Jan 12 00:45:06 2004 Subject: [DB-SIG] (no subject) Message-ID: <6.0.0.22.1.20040112171047.39a5e687@asistencia.org> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040112/f322eb32/attachment.html From andy47 at halfcooked.com Mon Jan 12 05:01:22 2004 From: andy47 at halfcooked.com (Andy Todd) Date: Mon Jan 12 05:06:08 2004 Subject: [DB-SIG] Oracle to Mysql (dates) Help please In-Reply-To: <200401111519.i0BFJBNC002972@smtp.hccnet.nl> References: <200401111519.i0BFJBNC002972@smtp.hccnet.nl> Message-ID: <40027072.3030108@halfcooked.com> arjen.dijkstra@hccnet.nl wrote: > Hi all, > > I'm posting to this sig for the first time, i hope I'm doing this right. > You are asking your questions in the right place. > I'm trying to export a view tables from a Oracle database to a Mysql > > database. I create insert statements (they look alright), but it all goes > > wrong when I try to execute them in Mysql, because the dates must have > > quotes on each side. > > I just don't know how make the dates right. > > Well I'll just show you the code and some insert statements it generates. > > Could anyone please help me? > > > > Thanks, > Arjen > > > [snip sample code] > The simple answer is because you are converting all of your results to strings when selecting them from Oracle you have to convert them to the appropriate types before you insert them into MySQL. In particular date and time objects need to be handled (and converted) with care. Its a tricky subject and one not easily covered in a short email. The essence is that databases are quite strict on what they accept from client programs to store in their tables. If you specified that a column is a positive integer with no more than 2 digits then the database will reject anything over 99 and below 0. Pythons types and objects, on the other hand, are generally a lot more magnanimous. To aid the developer of programs which access databases the DB-API and the people who write the DB modules provide a means for adapting the values you have in your Python program to the correct types for insertion into the database. For instance, when inserting a date into an Oracle database using cx_Oracle you use the function Date, e.g.; >>> today=cx_Oracle.Date(2004,1,12) And then use the result in your insert, e.g. >>> c_oracle.execute("INSERT INTO my_table (date_column) VALUES (:today)", {'today':today}) To start to get to grips with these issues, your best bet is to have a quick read of the DB-API ( http://www.python.org/peps/pep-0249.html ) and then the documentation of the various modules you are using. Regards, Andy -- -------------------------------------------------------------------------------- From the desk of Andrew J Todd esq - http://www.halfcooked.com/ From jfranz at neurokode.com Mon Jan 12 05:24:41 2004 From: jfranz at neurokode.com (Jon Franz) Date: Mon Jan 12 05:24:47 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? References: Message-ID: <01cb01c3d8f6$4abdc110$8401a8c0@letterbox1> > In my view, what you want can be accomplished by a helper routine. So > instead of using the code_code in the description directly as a type > constructor, one could call a function, say, getTypeConstructor(type_code). > Or, if you really want syntactic sugar, one could define a type_code object > that includes __call__ method that "does the right thing". Unfortunately, > "the right thing" is frequently not so simple. Sorry for the delay in responding - my notebook died last week and it has taken me awhile to catch up with the backlog. Instead of a helper routine, how about a code to type mapping? Currently you can manually do a bunch of if-elif statements to compare the code against the types, but a simple mapping would make things much cleaner - less code, and easy to understand - and flexible enough to handle new types that later revisions of the API might mandate. Call it the TypesByCode mapping (unless someone else thinks of something better) It will be DB specific, but it should be very easy to implement, and wouldn't cause any problems with the new type heirarchy being discussed. I would think returning the most specific type possible would be useful in some cases, meanwhile at other times returning the higher-level type would be useful, so perhaps the values should contain a tuple - index 0 gets the general, top level type reference, and index 1 gets the most specific reference. Sometimes these would be the same. Thus, you' could do something like this: # specific and general type of column at index 2 mycode = module.cursor.description[2][1] # pep 249 myType = module.TypesByCode[mycode][1] # specific type myGeneralType = module.TypesByCode[mycode][0] This would keep native type_codes, be backwards compatible, and yet allow for some useful functionality. Ideas/Thoughts? PS: another small suggestion. I think it would be useful for the API to define required constants for the modules to expose, namely for the different fields in description and their indexes. It seems trivial, and is, but I imagine it would be useful for many people to avoid opening the PEP every time they needed the index for a field in .description. Similarly, if the TypesByCode mapping were added, I'd suggest two new constants for the indexes of the general and specific types within the returned tuple. From fog at initd.org Mon Jan 12 06:27:27 2004 From: fog at initd.org (Federico Di Gregorio) Date: Mon Jan 12 06:27:27 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <01cb01c3d8f6$4abdc110$8401a8c0@letterbox1> References: <01cb01c3d8f6$4abdc110$8401a8c0@letterbox1> Message-ID: <1073906845.1026.47.camel@localhost> Il lun, 2004-01-12 alle 11:24, Jon Franz ha scritto: [?nip] > PS: another small suggestion. I think it would be useful for the API to > define required constants for the modules to expose, namely for the > different fields in description and their indexes. It seems trivial, and > is, > but I imagine it would be useful for many people to avoid opening the > PEP every time they needed the index for a field in .description. still thinking about the mappings suggestion. but this is, imho, a nice addition. i _always_ forget the right index into the description sequence and having the indices as constants in the module would be a nice addition. -- Federico Di Gregorio http://people.initd.org/fog Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org Alla faccia della trasparenza! Ma che si fa, lo si prende e lo si mena? (In pubblico, per trasparenza.) -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/db-sig/attachments/20040112/b542fb1a/attachment.bin From pythontutor at venix.com Mon Jan 12 09:41:55 2004 From: pythontutor at venix.com (Lloyd Kvam) Date: Mon Jan 12 09:42:12 2004 Subject: [DB-SIG] Oracle to Mysql (dates) Help please In-Reply-To: <200401111519.i0BFJBNC002972@smtp.hccnet.nl> References: <200401111519.i0BFJBNC002972@smtp.hccnet.nl> Message-ID: <4002B233.6050604@venix.com> To pass the date as a string, use 'yyyy-mm-dd'. Today would be: '2004-01-12' If you actually need date and time, the string format is: 'yyyy-mm-dd hh:mm:ss' arjen.dijkstra@hccnet.nl wrote: > Hi all, > > > > I'm posting to this sig for the first time, i hope I'm doing this right. > > > > > > I'm trying to export a view tables from a Oracle database to a Mysql > > database. I create insert statements (they look alright), but it all goes > > wrong when I try to execute them in Mysql, because the dates must have > > quotes on each side. > > I just don't know how make the dates right. > > Well I'll just show you the code and some insert statements it generates. > > Could anyone please help me? > > > > Thanks, > > > > Arjen > > > > ####Code#### > > > > import cx_Oracle > > tabellen=["machine"] > > con_oracle=cx_Oracle.connect("bla/bla") > > c_oracle=con_oracle.cursor() > > > > import MySQLdb > > my=MySQLdb.Connect("localhost", db="bla") > > my_mysql=my.cursor() > > for tabel in tabellen: > > print tabel > > c_oracle.execute("select * from %s" % tabel) > > a_oracle=c_oracle.fetchone() > > #file=open("%s.sql" % tabel, 'w') > > while a_oracle != None: > > b=str(a_oracle) > > ins="insert into %s values %s;\n" % (tabel, b) > > #file.write(ins) > > my_mysql.execute(ins) > > #print ins > > > > a_oracle=c_oracle.fetchone() > > file.close() > > > > con_oracle.close() > > > > my.close() > > > > > > ##insert statement### > > > > insert into machine values ('230KM', ' ', '230KM', 1980-01-01 00:00:00, > > 2035-01-01 00:00:00, 1, 100, 'asap', 'NO', 0, 0, 'corrugator', 2003-12-04 > > 06:00:00, 1970-01-01 01:00:00, ' ', 'normal', 0.0, 0.0, 7, ' ', ' ', > > 'normal', ' ', ' ', 'A', 2003-12-04 09:42:14, 82766); > > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-653-8139 fax: 801-459-9582 From jacobs at penguin.theopalgroup.com Mon Jan 12 12:16:32 2004 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Mon Jan 12 12:16:38 2004 Subject: [DB-SIG] WHat's the status of DB modules and datetime.py supp ort? In-Reply-To: <1073906845.1026.47.camel@localhost> Message-ID: On Mon, 12 Jan 2004, Federico Di Gregorio wrote: > Il lun, 2004-01-12 alle 11:24, Jon Franz ha scritto: > [?nip] > > PS: another small suggestion. I think it would be useful for the API to > > define required constants for the modules to expose, namely for the > > different fields in description and their indexes. It seems trivial, and > > is, > > but I imagine it would be useful for many people to avoid opening the > > PEP every time they needed the index for a field in .description. > > still thinking about the mappings suggestion. but this is, imho, a nice > addition. i _always_ forget the right index into the description > sequence and having the indices as constants in the module would be a > nice addition. I have an adapter that uses my db_row product to represent .descriptions in a way where fields can be accessed by name. -Kevin From jfranz at neurokode.com Mon Jan 12 16:31:44 2004 From: jfranz at neurokode.com (Jon Franz) Date: Mon Jan 12 16:34:24 2004 Subject: [DB-SIG] .description constants, was: What's the status of DB modules and datetime.py support? Message-ID: <002a01c3d953$7a5b90c0$8401a8c0@letterbox1> >> > PS: another small suggestion. I think it would be useful for the API to >> > define required constants for the modules to expose, namely for the >> > different fields in description and their indexes. It seems trivial, and > > is, >> > but I imagine it would be useful for many people to avoid opening the >> > PEP every time they needed the index for a field in .description. >> >> still thinking about the mappings suggestion. but this is, imho, a nice >> addition. i _always_ forget the right index into the description >> sequence and having the indices as constants in the module would be a >> nice addition. > >I have an adapter that uses my db_row product to represent .descriptions in >a way where fields can be accessed by name. > >-Kevin PDO does this as well - but the idea that addin modules and wrappers do this is not the point - the sugestion is to add the constants to the original modules themselves and to the definition of the DBAPI itself. ~Jon Franz NeuroKode Labs, LLC PS: yes, I am a jerk at times. From brandishes at juno.com Thu Jan 15 05:21:27 2004 From: brandishes at juno.com (Brandishes) Date: Thu Jan 15 05:22:00 2004 Subject: [DB-SIG] Fwd[4]: Via-Grra. Allmoosstt ffrree. 0nIy T0d@y (db-sig@python.org). In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040115/53a8ccd2/attachment.html From w965yvxl at broadcastindia.com Sat Jan 17 13:26:16 2004 From: w965yvxl at broadcastindia.com (Guadalupe Gibbons) Date: Sun Jan 18 12:59:38 2004 Subject: [DB-SIG] Learn to make a fortune on eBay! Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040117/ebc7c06f/attachment.html From britney21procuring at yahoo.com Mon Jan 19 03:04:06 2004 From: britney21procuring at yahoo.com (Distutils-sig-request) Date: Mon Jan 19 05:02:42 2004 Subject: [DB-SIG] Fwd: Want to have S E X, 20 times a day ? Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040119/5e2130bd/attachment.html From manus.hand at sapereon.com Mon Jan 19 20:06:08 2004 From: manus.hand at sapereon.com (Manus Hand) Date: Mon Jan 19 20:05:36 2004 Subject: [DB-SIG] cx_Oracle doesn't like me Message-ID: <013b01c3def1$977fb700$0300a8c0@developers.bullfrogtech.com> On Linux: >>> import cx_Oracle >>> cx_Oracle.connect('user', pwd', 'hostStr') Traceback (most recent call last): File "", line 1, in ? RuntimeError: Unable to acquire Oracle environment handle The 'user', 'pwd', and 'hostStr' are all correct, as verified thru sqlplus. The only thing I found about this error online was that $ORACLE_HOME and $LD_LIBRARY_PATH must match. Both of them had no value when I echoed them. So I set them both to: /apps/oracle/product/9.2/lib (which is where my libclntsh.so.9.0 is - I softlinked it into /usr/lib) and exported both environment variables. Then I went back into the Python interpreter and got the same error (pasted above). Help? Thanks, and PLEASE REPLY TO ME AT MY ADDRESS - I AM NOT SUBSCRIBED TO THE LIST. Manus Hand -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040119/ff48d551/attachment.html From anthony at computronix.com Mon Jan 19 22:20:21 2004 From: anthony at computronix.com (Anthony Tuininga) Date: Mon Jan 19 22:23:21 2004 Subject: [DB-SIG] cx_Oracle doesn't like me In-Reply-To: <013b01c3def1$977fb700$0300a8c0@developers.bullfrogtech.com> References: <013b01c3def1$977fb700$0300a8c0@developers.bullfrogtech.com> Message-ID: <1074568821.15545.1.camel@bluerose.ed.shawcable.net> You should state what Oracle client you are using, what Python version you are using and what cx_Oracle version you are using. That might help clarify things. :-) On Mon, 2004-01-19 at 18:06, Manus Hand wrote: > On Linux: > > > > >>> import cx_Oracle > > >>> cx_Oracle.connect('user', pwd', 'hostStr') > > Traceback (most recent call last): > > File "", line 1, in ? > > RuntimeError: Unable to acquire Oracle environment handle > > > > The 'user', 'pwd', and 'hostStr' are all correct, as verified thru > sqlplus. > > > > The only thing I found about this error online was that $ORACLE_HOME > and $LD_LIBRARY_PATH must match. > > Both of them had no value when I echoed them. So I set them both to: > > > > /apps/oracle/product/9.2/lib (which is where my libclntsh.so.9.0 is ? > I softlinked it into /usr/lib) > > > > and exported both environment variables. Then I went back into the > Python interpreter and got > > the same error (pasted above). > > > > Help? > > > > Thanks, and PLEASE REPLY TO ME AT MY ADDRESS ? I AM NOT SUBSCRIBED TO > THE LIST. > > > > Manus Hand > > > > > > ______________________________________________________________________ > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig -- Anthony Tuininga From clean_actfives at wildmail.com Tue Jan 20 06:47:11 2004 From: clean_actfives at wildmail.com (Cc-mode-announce-request) Date: Tue Jan 20 09:29:43 2004 Subject: [DB-SIG] Don't get busted ! ! . . . . . . . . . looseness Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040120/9693d093/attachment.html From arkamir at softhome.net Wed Jan 21 19:07:48 2004 From: arkamir at softhome.net (Conrad Koziol) Date: Wed Jan 21 19:26:10 2004 Subject: [DB-SIG] Select question Message-ID: <1074730068.5065.6.camel@conradpc> This isnt really about python but I was hoping you could help me with a tricky select question. The problem I have is that I need to select a distinct item from column A, but it has to be the item with the greatest value in C. So if column A holds all the birthdays of joes and bobs, I need to select the joe and bob with the closest birthday(youngest). I am using MySQL 4.0.15. Thanks a lot!! Conrad From mcfletch at rogers.com Wed Jan 21 23:47:31 2004 From: mcfletch at rogers.com (Mike C. Fletcher) Date: Wed Jan 21 23:47:35 2004 Subject: [DB-SIG] Select question In-Reply-To: <1074730068.5065.6.camel@conradpc> References: <1074730068.5065.6.camel@conradpc> Message-ID: <400F55E3.9000102@rogers.com> SELECT DISTINCT ( a ), * FROM t ORDER BY c DESC; DISTINCT chooses the first row from the result set with equal distinct As, which, with the ordering of the set by C descending, should be the value with the maximal C value (first row of the items duplicating A for each A). I'm going from postgresql background, but I think everything there is standard SQL operation. That's a very weird problem description, btw. Why the heck does someone need the nearest "Joe"? I can think of a few applications with intellisense-like mechanisms, but doesn't really seem like a natural approach. Oh well, HTH, Mike Conrad Koziol wrote: >This isnt really about python but I was hoping you could help me with a >tricky select question. The problem I have is that I need to select a >distinct item from column A, but it has to be the item with the greatest >value in C. So if column A holds all the birthdays of joes and bobs, I >need to select the joe and bob with the closest birthday(youngest). I am >using MySQL 4.0.15. > >Thanks a lot!! > Conrad > > _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/ From bouataa at hotmail.com Thu Jan 22 04:38:20 2004 From: bouataa at hotmail.com (samira M'Bata) Date: Thu Jan 22 04:38:24 2004 Subject: [DB-SIG] Mysqldb python variable in request Message-ID: Hi! i hope someone could help me. I d like to make an sql statement with python, in which the column names will be parameter of a python function. Bevore I tried to define my definition, i tried this code: target= 'DataType' constraint= 'Location' #DataType and Location are two columns of my Table constraint_value= 'stuttgart' cursor.execute(""" select %s from Xml2Mysql where %s=%s """,(target, constraint, constraint_value)) I think the problem comes from the fact that my columns names are not as a string type defined. Should I change something in my table definition? _________________________________________________________________ Add photos to your messages with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail From msanchez at grupoburke.com Thu Jan 22 04:55:07 2004 From: msanchez at grupoburke.com (=?ISO-8859-1?Q?Marcos_S=E1nchez_Provencio?=) Date: Thu Jan 22 04:54:12 2004 Subject: [DB-SIG] Mysqldb python variable in request In-Reply-To: References: Message-ID: <400F9DFB.6090901@grupoburke.com> I am not an expert in mysql, but i believe column names can not be a parameter. You should create the sql with the column names, and leave only the value as a parameter. cursor.execute(' select ' +target+ ' from Xml2Mysql where ' +constraint+ '=%s ',(constraint_value,)) samira M'Bata wrote: > Hi! > i hope someone could help me. > I d like to make an sql statement with python, in which the column > names will be parameter of a python function. > Bevore I tried to define my definition, i tried this code: > > target= 'DataType' > constraint= 'Location' > #DataType and Location are two columns of my Table > constraint_value= 'stuttgart' > cursor.execute(""" select %s from Xml2Mysql where %s=%s """,(target, > constraint, constraint_value)) > > I think the problem comes from the fact that my columns names are not > as a string type defined. > > Should I change something in my table definition? > > _________________________________________________________________ From wnvcole at peppermillcas.com Thu Jan 22 11:47:39 2004 From: wnvcole at peppermillcas.com (Vernon Cole) Date: Thu Jan 22 11:47:54 2004 Subject: [DB-SIG] Mysqldb python variable in request Message-ID: <07AF3C77A0FBD311A99F00508B65203903272BC0@sebastian.peppermillcas.com> There is some confusion here between supplying parameters to the SQL server, which is done differently between different servers, and a python string substitution. The python method always works, but is less efficient for repetative searches. This snippet from one of my programs does something like you want: Sel = "select AuditDate from BB_PeriodEnd where (Casino_Code = '%s' and Period_ID = 5 and AuditDate = '%s')" % (Casino_Code, AuditDate) pdEnd.execute(Sel) Note that all substitution is done in python, so I can check the syntax using a python print statement. Also note that there are single quotes around the %s constructs. These single quotes are passed to the SQL server, which needs to see a string literal at that point. Python is filling in the value for the literal. In your case, try something like: cursor.execute(""" select %s from Xml2Mysql where %s='%s' """ % (target, constraint, constraint_value)) -----Original Message----- From: samira M'Bata [mailto:bouataa@hotmail.com] Sent: Thursday, January 22, 2004 2:38 AM To: db-sig@python.org Subject: [DB-SIG] Mysqldb python variable in request Hi! i hope someone could help me. I d like to make an sql statement with python, in which the column names will be parameter of a python function. Bevore I tried to define my definition, i tried this code: target= 'DataType' constraint= 'Location' #DataType and Location are two columns of my Table constraint_value= 'stuttgart' cursor.execute(""" select %s from Xml2Mysql where %s=%s """,(target, constraint, constraint_value)) I think the problem comes from the fact that my columns names are not as a string type defined. Should I change something in my table definition? _________________________________________________________________ Add photos to your messages with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail _______________________________________________ DB-SIG maillist - DB-SIG@python.org http://mail.python.org/mailman/listinfo/db-sig From helena_b2001 at yahoo.com Thu Jan 22 17:55:37 2004 From: helena_b2001 at yahoo.com (helena bhaska) Date: Thu Jan 22 17:55:41 2004 Subject: [DB-SIG] document contains no data Message-ID: <20040122225537.65183.qmail@web20414.mail.yahoo.com> Hi everybody, I am trying to connect to a database and read something from it in a program that runs under apache. Here's my code: db = MySQLdb.Connect(host="host", user="usr", passwd="pswd", db="db") c = db.cursor() c.execute("""SELECT * FROM SomeTable""") results = c.fetchall() The execute statement causes the internet browser to spit out a dialog box with a message "document contains no data" If I run these statements from python command line, it works fine. First of all, I would like what that message about no data means... The apache error log gives me the following: [Thu Jan 22 17:49:58 2004] [notice] child pid 9186 exit signal Segmentation fault (11) - which obviously is bad, but I again don't know what this means. Thanks for any pointers. Helena __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ From msanchez at grupoburke.com Fri Jan 23 05:05:44 2004 From: msanchez at grupoburke.com (=?ISO-8859-1?Q?Marcos_S=E1nchez_Provencio?=) Date: Fri Jan 23 04:56:00 2004 Subject: [DB-SIG] document contains no data In-Reply-To: <20040122225537.65183.qmail@web20414.mail.yahoo.com> References: <20040122225537.65183.qmail@web20414.mail.yahoo.com> Message-ID: <4010F1F8.8090508@grupoburke.com> Are you using cgi? Be sure to include the correct headers in your cgi. Check also that the environment for the cgi (path, etc.) is the same as for the user you are testing from. You may try to create a minimal cgi first. http://www.devshed.com/c/a/Python/Writing-CGI-Programs-in-Python/2/ helena bhaska escribi?: > Hi everybody, > I am trying to connect to a database and read > something from it in a program that runs under apache. > Here's my code: > db = MySQLdb.Connect(host="host", user="usr", > passwd="pswd", db="db") > c = db.cursor() > c.execute("""SELECT * FROM SomeTable""") > results = c.fetchall() > > The execute statement causes the internet browser to > spit out a dialog box with a message "document > contains no data" > > If I run these statements from python command line, it > works fine. > > First of all, I would like what that message about no > data means... > The apache error log gives me the following: > [Thu Jan 22 17:49:58 2004] [notice] child pid 9186 > exit signal Segmentation fault (11) - which obviously > is bad, but I again don't know what this means. > > Thanks for any pointers. > Helena > > > __________________________________ > Do you Yahoo!? > Yahoo! SiteBuilder - Free web site building tool. Try it! > http://webhosting.yahoo.com/ps/sb/ > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig From ods at strana.ru Fri Jan 23 09:55:10 2004 From: ods at strana.ru (Denis S. Otkidach) Date: Fri Jan 23 09:55:34 2004 Subject: [DB-SIG] Mysqldb python variable in request In-Reply-To: <07AF3C77A0FBD311A99F00508B65203903272BC0@sebastian.peppermillcas.com> Message-ID: On Thu, 22 Jan 2004, Vernon Cole wrote: VC> Note that all substitution is done in python, so I can check VC> the syntax VC> using a python print statement. Also note that there are VC> single quotes VC> around the %s constructs. These single quotes are passed to VC> the SQL server, VC> which needs to see a string literal at that point. Python is VC> filling in the VC> value for the literal. In your case, try something like: VC> VC> cursor.execute(""" select %s from Xml2Mysql where %s='%s' VC> """ % (target, VC> constraint, constraint_value)) It's a bad advice, that in some cases can lead to security holes in the program. What if some string is from untrusted source and contains a "'" character? Let me provide an example. If your program provides some service for many users and allows the user to remove his own data with the following SQL statement: """delete from important_data where user='%s'""" % user then I can try user name "my name' OR '1" and condition in your statement will be always true! -- Denis S. Otkidach http://www.python.ru/ [ru] From helena_b2001 at yahoo.com Fri Jan 23 12:33:16 2004 From: helena_b2001 at yahoo.com (helena bhaska) Date: Fri Jan 23 12:33:26 2004 Subject: [DB-SIG] document contains no data In-Reply-To: <4010F1F8.8090508@grupoburke.com> Message-ID: <20040123173316.74015.qmail@web20412.mail.yahoo.com> Thanks, hm, I don't think that it's a code problem, since I wrote it on a different machine, where it worked perfectly. Then I transferred it to a different machine, where I'm running into this problem. I thought I set it up exactly the same way as the other machine, but apparently not. Do you know what that dialog box error means? Thanks. --- Marcos_Sánchez_Provencio wrote: > Are you using cgi? > > Be sure to include the correct headers in your cgi. > Check also that the > environment for the cgi (path, etc.) is the same as > for the user you are > testing from. You may try to create a minimal cgi > first. > > http://www.devshed.com/c/a/Python/Writing-CGI-Programs-in-Python/2/ > > helena bhaska escribió: > > Hi everybody, > > I am trying to connect to a database and read > > something from it in a program that runs under > apache. > > Here's my code: > > db = MySQLdb.Connect(host="host", user="usr", > > passwd="pswd", db="db") > > c = db.cursor() > > c.execute("""SELECT * FROM SomeTable""") > > results = c.fetchall() > > > > The execute statement causes the internet browser > to > > spit out a dialog box with a message "document > > contains no data" > > > > If I run these statements from python command > line, it > > works fine. > > > > First of all, I would like what that message about > no > > data means... > > The apache error log gives me the following: > > [Thu Jan 22 17:49:58 2004] [notice] child pid 9186 > > exit signal Segmentation fault (11) - which > obviously > > is bad, but I again don't know what this means. > > > > Thanks for any pointers. > > Helena > > > > > > __________________________________ > > Do you Yahoo!? > > Yahoo! SiteBuilder - Free web site building tool. > Try it! > > http://webhosting.yahoo.com/ps/sb/ > > > > _______________________________________________ > > DB-SIG maillist - DB-SIG@python.org > > http://mail.python.org/mailman/listinfo/db-sig > > __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free web site building tool. Try it! http://webhosting.yahoo.com/ps/sb/ From pythontutor at venix.com Fri Jan 23 20:13:20 2004 From: pythontutor at venix.com (Lloyd Kvam) Date: Fri Jan 23 20:13:36 2004 Subject: [DB-SIG] document contains no data In-Reply-To: <20040123173316.74015.qmail@web20412.mail.yahoo.com> References: <20040123173316.74015.qmail@web20412.mail.yahoo.com> Message-ID: <4011C6B0.2020107@venix.com> apache does not run your script by "saying" python it simply executes your script. Your script must be saved with execute permission chmod +x Also the very FIRST line of the script must specify that you want python to run the script. This line will look something like: #!/usr/bin/python #!/usr/local/bin/python is also quite common. It MUST match your server configuration. Also a script MUST print a line describing what kind of output is being produced. (You could be sending a picture or sound file for instance.) This is done by printing: print "Content-type: text/html\r\n\r\n" Your case may actually be text/text. The folks that administer the server should be able to help you with these issues. helena bhaska wrote: > Thanks, > hm, I don't think that it's a code problem, since I > wrote it on a different machine, where it worked > perfectly. Then I transferred it to a different > machine, where I'm running into this problem. I > thought I set it up exactly the same way as the other > machine, but apparently not. Do you know what that > dialog box error means? > Thanks. > --- Marcos_S?nchez_Provencio > wrote: > >>Are you using cgi? >> >>Be sure to include the correct headers in your cgi. >>Check also that the >>environment for the cgi (path, etc.) is the same as >>for the user you are >>testing from. You may try to create a minimal cgi >>first. >> >> > > http://www.devshed.com/c/a/Python/Writing-CGI-Programs-in-Python/2/ > >>helena bhaska escribi?: >> >>>Hi everybody, >>>I am trying to connect to a database and read >>>something from it in a program that runs under >> >>apache. >> >>>Here's my code: >>>db = MySQLdb.Connect(host="host", user="usr", >>>passwd="pswd", db="db") >>>c = db.cursor() >>>c.execute("""SELECT * FROM SomeTable""") >>>results = c.fetchall() >>> >>>The execute statement causes the internet browser >> >>to >> >>>spit out a dialog box with a message "document >>>contains no data" >>> >>>If I run these statements from python command >> >>line, it >> >>>works fine. >>> >>>First of all, I would like what that message about >> >>no >> >>>data means... >>>The apache error log gives me the following: >>>[Thu Jan 22 17:49:58 2004] [notice] child pid 9186 >>>exit signal Segmentation fault (11) - which >> >>obviously >> >>>is bad, but I again don't know what this means. >>> >>>Thanks for any pointers. >>>Helena >>> >>> >>>__________________________________ >>>Do you Yahoo!? >>>Yahoo! SiteBuilder - Free web site building tool. >> >>Try it! >> >>>http://webhosting.yahoo.com/ps/sb/ >>> >>>_______________________________________________ >>>DB-SIG maillist - DB-SIG@python.org >>>http://mail.python.org/mailman/listinfo/db-sig >> >> > > > __________________________________ > Do you Yahoo!? > Yahoo! SiteBuilder - Free web site building tool. Try it! > http://webhosting.yahoo.com/ps/sb/ > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig > -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-653-8139 fax: 801-459-9582 From andy at dustman.net Sat Jan 24 11:43:33 2004 From: andy at dustman.net (Andy Dustman) Date: Sat Jan 24 11:45:19 2004 Subject: [DB-SIG] Mysqldb python variable in request In-Reply-To: References: Message-ID: <1074962613.10191.16.camel@chef.neosouth.net> On Thu, 2004-01-22 at 04:38, samira M'Bata wrote: > target= 'DataType' > constraint= 'Location' > #DataType and Location are two columns of my Table > constraint_value= 'stuttgart' > cursor.execute(""" select %s from Xml2Mysql where %s=%s """,(target, > constraint, constraint_value)) > > I think the problem comes from the fact that my columns names are not as a > string type defined. > > Should I change something in my table definition? Simply put, table column names cannot be passed as parameters using the DB-API; this is not particularly a MySQLdb issue. Your code should look something like this. target= 'DataType' constraint= 'Location' #DataType and Location are two columns of my Table # note doubled % for parameter placeholder base_query = """ select %s from Xml2Mysql where %s=%%s """ real_query = base_query % (target, constraint) constraint_value= 'stuttgart' cursor.execute(real_query, (constraint_value,)) As someone else has pointed out, if you are taking table or column names from user-supplied data, you'd better check them very, very carefully. A malicious user could execute arbitrary SQL code this way. -- Andy Dustman PGP: 0x930B8AB6 @ .net http://dustman.net/andy Freedom isn't free. It's sold to the highest bidder. From randall at tnr.cc Sat Jan 24 14:05:54 2004 From: randall at tnr.cc (Randall Smith) Date: Sat Jan 24 14:07:13 2004 Subject: [DB-SIG] threads sharing connection with cx_Oracle Message-ID: <4012C212.9000504@tnr.cc> Are there any known issues with threads sharing a connection with cx_Oracle? Environment: cx_Oracle for Oracle 8i and Python2.3, Oracle 8.1.5 The top level program creates a connection. It then creates threads and passes that connection in to each thread as an argument so that all threads are sharing the connection. Each thread creates a cursor as needed and issues only SELECT statements. After about 10 threads have been created the next thread throws an Oracle Exception with a message something like 'End of string line ...'. Sorry about the vagueness on the error messgae, I'm at home and don't have access to it now. I set up the same environment using psycopg/Postgresql and didn't have any problem. So I think it is specific to Oracle or cx_Oracle. I can get the exact error message on Tuesday, but I'm sending this message now in hopes that someone is already familiar with this problem. Randall Smith From pf_moore at yahoo.co.uk Sat Jan 24 17:43:34 2004 From: pf_moore at yahoo.co.uk (Paul Moore) Date: Sat Jan 24 17:43:56 2004 Subject: [DB-SIG] Re: threads sharing connection with cx_Oracle References: <4012C212.9000504@tnr.cc> Message-ID: Randall Smith writes: > Are there any known issues with threads sharing a connection with > cx_Oracle? You need to do cx_Oracle.OPT_Threading = 1 at the start of your script in order to enable thread-safe operations. It's not the default, because it has a small performance penalty. Check the documentation for more details. (Apologies if you know this, and have done it already...) Paul. -- This signature intentionally left blank From anthony at computronix.com Sun Jan 25 00:05:57 2004 From: anthony at computronix.com (Anthony Tuininga) Date: Sun Jan 25 00:10:16 2004 Subject: [DB-SIG] threads sharing connection with cx_Oracle In-Reply-To: <4012C212.9000504@tnr.cc> References: <4012C212.9000504@tnr.cc> Message-ID: <1075007157.9078.2.camel@bluerose.ed.shawcable.net> You do need to ensure that the OCI threading option is turned on. Specifically, you need to set the following module level flag: cx_Oracle.OPT_Threading = 1 If you don't, you will find the program very unstable. :-) Otherwise, things are rock solid in our experience. We have run stress tests on applications with 30 threads for several hours at a time and not noticed any problems. On Sat, 2004-01-24 at 12:05, Randall Smith wrote: > Are there any known issues with threads sharing a connection with cx_Oracle? > > Environment: cx_Oracle for Oracle 8i and Python2.3, Oracle 8.1.5 > > The top level program creates a connection. It then creates threads and > passes that connection in to each thread as an argument so that all > threads are sharing the connection. Each thread creates a cursor as > needed and issues only SELECT statements. After about 10 threads have > been created the next thread throws an Oracle Exception with a message > something like 'End of string line ...'. Sorry about the vagueness on > the error messgae, I'm at home and don't have access to it now. > > I set up the same environment using psycopg/Postgresql and didn't have > any problem. So I think it is specific to Oracle or cx_Oracle. > > I can get the exact error message on Tuesday, but I'm sending this > message now in hopes that someone is already familiar with this problem. > > Randall Smith > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig -- Anthony Tuininga From vfsqnrqufegjx at msn.com Sun Jan 25 23:23:14 2004 From: vfsqnrqufegjx at msn.com (Forrest Mckenzie) Date: Sun Jan 25 23:29:18 2004 Subject: [DB-SIG] gimbal algebraic Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040126/7e802b8b/attachment.html From laboring at 263.net.cn Mon Jan 26 02:22:26 2004 From: laboring at 263.net.cn (Laboring) Date: Mon Jan 26 02:22:33 2004 Subject: [DB-SIG] Db-sig, sweet tr@nny & hot strlp to you In-Reply-To: <39I2HGCA1GFB5FCF@python.org> References: <39I2HGCA1GFB5FCF@python.org> Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040126/2ab67079/attachment.html From DrWEB-DAEMON at vladregion.ru Wed Jan 28 10:00:23 2004 From: DrWEB-DAEMON at vladregion.ru (DrWeb-DAEMON) Date: Wed Jan 28 09:58:31 2004 Subject: [DB-SIG] Undelivered mail: Status Message-ID: <200401281500.i0SF0NBF015388@vladregion.ru> Dear User, The message with following attributes has not been delivered. because contains an infected object. Sender = db-sig@python.org Recipients = promo@vladregion.ru Subject = Status Message-ID = i0SF04bk015343 Antivirus filter report: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.HU3sDE - archive MAIL drweb.tmp.HU3sDE/[text:plain] - Ok drweb.tmp.HU3sDE/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- The original message was stored in archive record named: file was not created In order to receive the original message, please send request to , referring to the archive record name given above. --- Antivirus service provided by DrWeb Daemon (http://www.drweb32.com) Please send your comments to DialogueScience, Inc. (http://www.dials.ru/english, support@dials.ru) -------------- next part -------------- ????????? ???????????, ?????????, ????????? ???? promo@vladregion.ru ???????????? ? ?? ???? ??????????. ????? ????????????? ???????: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.HU3sDE - archive MAIL drweb.tmp.HU3sDE/[text:plain] - Ok drweb.tmp.HU3sDE/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- ????????? ????????? ? ????????? ??? ??????: file was not created ??? ?? ???????? ??? ????????? ?????????? ? ?????????????? ?? ?????? , ?????? ???, ??? ??????? ????????? ???? ?????????. --- ???????????? ?????? DrWeb Daemon (http://www.drweb.ru) ?????????? ???? ????????? ? DialogueScience, Inc. (http://www.dials.ru, support@dials.ru) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/rfc822-headers Size: 839 bytes Desc: not available Url : http://mail.python.org/pipermail/db-sig/attachments/20040128/90d74600/attachment.bin From DrWEB-DAEMON at vladregion.ru Wed Jan 28 10:00:46 2004 From: DrWEB-DAEMON at vladregion.ru (DrWeb-DAEMON) Date: Wed Jan 28 09:58:51 2004 Subject: [DB-SIG] Undelivered mail: Status Message-ID: <200401281500.i0SF0ktP015435@vladregion.ru> Dear User, The message with following attributes has not been delivered. because contains an infected object. Sender = db-sig@python.org Recipients = promo@vladregion.ru Subject = Status Message-ID = i0SF0Wbk015408 Antivirus filter report: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.N48guc - archive MAIL drweb.tmp.N48guc/[text:plain] - Ok drweb.tmp.N48guc/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- The original message was stored in archive record named: file was not created In order to receive the original message, please send request to , referring to the archive record name given above. --- Antivirus service provided by DrWeb Daemon (http://www.drweb32.com) Please send your comments to DialogueScience, Inc. (http://www.dials.ru/english, support@dials.ru) -------------- next part -------------- ????????? ???????????, ?????????, ????????? ???? promo@vladregion.ru ???????????? ? ?? ???? ??????????. ????? ????????????? ???????: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.N48guc - archive MAIL drweb.tmp.N48guc/[text:plain] - Ok drweb.tmp.N48guc/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- ????????? ????????? ? ????????? ??? ??????: file was not created ??? ?? ???????? ??? ????????? ?????????? ? ?????????????? ?? ?????? , ?????? ???, ??? ??????? ????????? ???? ?????????. --- ???????????? ?????? DrWeb Daemon (http://www.drweb.ru) ?????????? ???? ????????? ? DialogueScience, Inc. (http://www.dials.ru, support@dials.ru) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/rfc822-headers Size: 839 bytes Desc: not available Url : http://mail.python.org/pipermail/db-sig/attachments/20040128/24330526/attachment.bin From DrWEB-DAEMON at vladregion.ru Wed Jan 28 10:01:11 2004 From: DrWEB-DAEMON at vladregion.ru (DrWeb-DAEMON) Date: Wed Jan 28 09:59:16 2004 Subject: [DB-SIG] Undelivered mail: Status Message-ID: <200401281501.i0SF1BvL015507@vladregion.ru> Dear User, The message with following attributes has not been delivered. because contains an infected object. Sender = db-sig@python.org Recipients = promo@vladregion.ru Subject = Status Message-ID = i0SF0qbk015449 Antivirus filter report: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.kv7WNm - archive MAIL drweb.tmp.kv7WNm/[text:plain] - Ok drweb.tmp.kv7WNm/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- The original message was stored in archive record named: file was not created In order to receive the original message, please send request to , referring to the archive record name given above. --- Antivirus service provided by DrWeb Daemon (http://www.drweb32.com) Please send your comments to DialogueScience, Inc. (http://www.dials.ru/english, support@dials.ru) -------------- next part -------------- ????????? ???????????, ?????????, ????????? ???? promo@vladregion.ru ???????????? ? ?? ???? ??????????. ????? ????????????? ???????: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.kv7WNm - archive MAIL drweb.tmp.kv7WNm/[text:plain] - Ok drweb.tmp.kv7WNm/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- ????????? ????????? ? ????????? ??? ??????: file was not created ??? ?? ???????? ??? ????????? ?????????? ? ?????????????? ?? ?????? , ?????? ???, ??? ??????? ????????? ???? ?????????. --- ???????????? ?????? DrWeb Daemon (http://www.drweb.ru) ?????????? ???? ????????? ? DialogueScience, Inc. (http://www.dials.ru, support@dials.ru) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/rfc822-headers Size: 839 bytes Desc: not available Url : http://mail.python.org/pipermail/db-sig/attachments/20040128/1ec0743d/attachment.bin From DrWEB-DAEMON at vladregion.ru Wed Jan 28 10:01:37 2004 From: DrWEB-DAEMON at vladregion.ru (DrWeb-DAEMON) Date: Wed Jan 28 09:59:43 2004 Subject: [DB-SIG] Undelivered mail: Status Message-ID: <200401281501.i0SF1bJv015571@vladregion.ru> Dear User, The message with following attributes has not been delivered. because contains an infected object. Sender = db-sig@python.org Recipients = promo@vladregion.ru Subject = Status Message-ID = i0SF1Ibk015523 Antivirus filter report: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.NoBVA4 - archive MAIL drweb.tmp.NoBVA4/[text:plain] - Ok drweb.tmp.NoBVA4/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- The original message was stored in archive record named: file was not created In order to receive the original message, please send request to , referring to the archive record name given above. --- Antivirus service provided by DrWeb Daemon (http://www.drweb32.com) Please send your comments to DialogueScience, Inc. (http://www.dials.ru/english, support@dials.ru) -------------- next part -------------- ????????? ???????????, ?????????, ????????? ???? promo@vladregion.ru ???????????? ? ?? ???? ??????????. ????? ????????????? ???????: --- DrWeb report --- = DrWeb detailed report: drweb.tmp.NoBVA4 - archive MAIL drweb.tmp.NoBVA4/[text:plain] - Ok drweb.tmp.NoBVA4/text.zip infected with Win32.HLLM.MyDoom.32768 = Following virus(es) has been found: infected with Win32.HLLM.MyDoom.32768 = Scanning statistic: --- DrWeb report --- ????????? ????????? ? ????????? ??? ??????: file was not created ??? ?? ???????? ??? ????????? ?????????? ? ?????????????? ?? ?????? , ?????? ???, ??? ??????? ????????? ???? ?????????. --- ???????????? ?????? DrWeb Daemon (http://www.drweb.ru) ?????????? ???? ????????? ? DialogueScience, Inc. (http://www.dials.ru, support@dials.ru) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/rfc822-headers Size: 839 bytes Desc: not available Url : http://mail.python.org/pipermail/db-sig/attachments/20040128/aa9490af/attachment.bin From kinquiry at kennedytech.com Fri Jan 30 02:55:36 2004 From: kinquiry at kennedytech.com (kinquiry@kennedytech.com) Date: Fri Jan 30 03:09:02 2004 Subject: [DB-SIG] Help w/ bugs in dbiDate win32 Message-ID: Not sure if I'm posting to the right group. Below describes situations with dbiDate and odbc modules under win32 that will crash python. >From all that I have been able to gather, the win32all odbc.py module uses the dbi.dbiDate object whenever a result column in SQL references an ODBC timestamp type field. I've played with this object on various win32 machines, with various versions of python 2.X , and have experienced the following (which I think is due to unimplemented or incorrectly implemented python base object functions in the underlying C++ code in odbc.cpp and dbi.cpp): When instantiating the dbiDate with a float in the constructor (which I think is how it's supposed to be used), everything works OK..... $ python ActivePython 2.2.2 Build 224 (ActiveState Corp.) based on Python 2.2.2 (#37, Nov 26 2002, 10:24:37) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import dbi >>> import time >>> x = dbi.dbiDate(time.time()) >>> x >>> float(x) 1075446574.7260001 >>> str(x) 'Fri Jan 30 02:09:34 2004' >>> long(x) 1075446574L >>> int(x) 1075446574 >>> x <> 0 1 >>> x == 0 0 >>> x == None 0 >>> However, the dbiDate object will let you instantiate it with something other than a float in the constructor (like None). I wouldn't do this in my code, but I think there might be some cases when odbc.pyd does this, or something similar. For example: $ python ActivePython 2.2.2 Build 224 (ActiveState Corp.) based on Python 2.2.2 (#37, Nov 26 2002, 10:24:37) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import dbi >>> x = dbi.dbiDate(None) >>> float(x) WILL CRASH PYTHON. I might be barking up the wrong tree, but I suspect that the above condition is similar to what happens in the following REAL WORLD code snippet: $ python ActivePython 2.2.2 Build 224 (ActiveState Corp.) based on Python 2.2.2 (#37, Nov 26 2002, 10:24:37) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import odbc >>> cur = odbc.odbc('beaulieu').cursor() # MySQL database on win32 >>> cur.execute('use newbeaulieu') 0 >>> cur.execute(""" ... select date_format(PRDOB8,'%M %D %Y') , PRDOB8 ... from cvt_1_prpmss where import_sequence = 342056 ... """ ... ) 0 >>> res = cur.fetchall() >>> res [('November 23rd 1985', )] >>> testDate = res[0][1] >>> >>> cur.execute(""" ... create table test_table ( dt datetime ) ... """ ... ) 0 >>> cur.execute('insert into test_table ( dt ) values ( ? )', ( testDate , ) ) 1 >>> cur.execute(""" ... select date_format(PRDOB8,'%M %D %Y') , PRDOB8 ... from cvt_1_prpmss where import_sequence = 1 ... """ ... ) 0 >>> res = cur.fetchall() >>> res [('April 19th 1954', )] >>> testDate = res[0][1] >>> >>> cur.execute('insert into test_table ( dt ) values ( ? )', (testDate, ) ) The last line crashed python with an invalid memory access error. I think it's because the date was before the epoch, which probably isn't supported by the C libraries that dbiDate uses. This seems a logical conclusion, since the code for the 2nd 'insert' was exactly the same as the first, but it used an older date as a parameter. From what I can gather, once an old date is put into a dbiDate instance, many of the methods on the dbiDate overwrite memory and will eventually crash python. I'm at a standstill with this now, because I'm not a good C programmer, and I don't have visual studio. There doesn't seem to be too much code involved in the two .cpp modules that underly the dbi and odbc python modules in win32all. Ideally, the odbc module should be given a function or a flag to check that would cause it to pass the date value back as a float ( or whatever the underlying odbc API returns ) so that application programmers can deal with it. Or, even better, dbiDate could be replaced with an object that works for dates older than 1970. Does anyone have any ideas? Any help would be greatly appreciated. Best Regards, John Kennedy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20040130/8560c4b1/attachment.html From julie at schof.Colorado.EDU Fri Jan 30 05:51:22 2004 From: julie at schof.Colorado.EDU (Julie Fredlund) Date: Fri Jan 30 05:51:48 2004 Subject: [DB-SIG] AUTOMATIC REPLY Message-ID: <200401301051.i0UApMeF003946@schof.Colorado.EDU> The user julie no longer has an active account on schof. The mail you sent will be received by julie in the event that the account is re-activated. Another means of communication is advised. Important requests for more information on the julie account can be sent to postmaster@schof. From msanchez at grupoburke.com Fri Jan 30 06:07:29 2004 From: msanchez at grupoburke.com (=?ISO-8859-1?Q?Marcos_S=E1nchez_Provencio?=) Date: Fri Jan 30 05:57:26 2004 Subject: [DB-SIG] Help w/ bugs in dbiDate win32 In-Reply-To: References: Message-ID: <401A3AF1.8060005@grupoburke.com> This module is really obsolete. I suggest that you use mx.ODBC http://www.egenix.com/files/python/eGenix-mx-Extensions.html#mxODBC or adodbapi http://adodbapi.sourceforge.net/ kinquiry@kennedytech.com escribi?: > > Not sure if I'm posting to the right group. > > Below describes situations with dbiDate and odbc modules under win32 > that will crash python. > > From all that I have been able to gather, the win32all odbc.py module > uses the dbi.dbiDate object whenever a result column in SQL references > an ODBC timestamp type field. > > I've played with this object on various win32 machines, with various > versions of python 2.X , and have experienced the following (which I > think is due to unimplemented or incorrectly implemented python base > object functions in the underlying C++ code in odbc.cpp and dbi.cpp): > > When instantiating the dbiDate with a float in the constructor (which I > think is how it's supposed to be used), everything works OK..... > > $ python > ActivePython 2.2.2 Build 224 (ActiveState Corp.) based on > Python 2.2.2 (#37, Nov 26 2002, 10:24:37) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import dbi > >>> import time > >>> x = dbi.dbiDate(time.time()) > >>> x > > >>> float(x) > 1075446574.7260001 > >>> str(x) > 'Fri Jan 30 02:09:34 2004' > >>> long(x) > 1075446574L > >>> int(x) > 1075446574 > >>> x <> 0 > 1 > >>> x == 0 > 0 > >>> x == None > 0 > >>> > > However, the dbiDate object will let you instantiate it with something > other than a float in the constructor (like None). I wouldn't do this > in my code, but I think there might be some cases when odbc.pyd does > this, or something similar. For example: > > $ python > ActivePython 2.2.2 Build 224 (ActiveState Corp.) based on > Python 2.2.2 (#37, Nov 26 2002, 10:24:37) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import dbi > >>> x = dbi.dbiDate(None) > >>> float(x) > > WILL CRASH PYTHON. > > I might be barking up the wrong tree, but I suspect that the above > condition is similar to what happens in the following REAL WORLD code > snippet: > > $ python > ActivePython 2.2.2 Build 224 (ActiveState Corp.) based on > Python 2.2.2 (#37, Nov 26 2002, 10:24:37) [MSC 32 bit (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > >>> import odbc > >>> cur = odbc.odbc('beaulieu').cursor() # MySQL database on win32 > >>> cur.execute('use newbeaulieu') > 0 > >>> cur.execute(""" > ... select date_format(PRDOB8,'%M %D %Y') , PRDOB8 > ... from cvt_1_prpmss where import_sequence = 342056 > ... """ > ... ) > 0 > >>> res = cur.fetchall() > >>> res > [('November 23rd 1985', )] > >>> testDate = res[0][1] > >>> > >>> cur.execute(""" > ... create table test_table ( dt datetime ) > ... """ > ... ) > 0 > >>> cur.execute('insert into test_table ( dt ) values ( ? )', ( > testDate , ) ) > 1 > >>> cur.execute(""" > ... select date_format(PRDOB8,'%M %D %Y') , PRDOB8 > ... from cvt_1_prpmss where import_sequence = 1 > ... """ > ... ) > 0 > >>> res = cur.fetchall() > >>> res > [('April 19th 1954', )] > >>> testDate = res[0][1] > >>> > >>> cur.execute('insert into test_table ( dt ) values ( ? )', > (testDate, ) ) > > The last line crashed python with an invalid memory access error. I > think it's because the date was before the epoch, which probably isn't > supported by the C libraries that dbiDate uses. This seems a logical > conclusion, since the code for the 2nd 'insert' was exactly the same as > the first, but it used an older date as a parameter. From what I can > gather, once an old date is put into a dbiDate instance, many of the > methods on the dbiDate overwrite memory and will eventually crash python. > > I'm at a standstill with this now, because I'm not a good C programmer, > and I don't have visual studio. There doesn't seem to be too much code > involved in the two .cpp modules that underly the dbi and odbc python > modules in win32all. > > Ideally, the odbc module should be given a function or a flag to check > that would cause it to pass the date value back as a float ( or whatever > the underlying odbc API returns ) so that application programmers can > deal with it. Or, even better, dbiDate could be replaced with an object > that works for dates older than 1970. > > Does anyone have any ideas? Any help would be greatly appreciated. > > Best Regards, > John Kennedy >