From nemeth@iqsoft.hu Sat Nov 6 11:22:42 1999 From: nemeth@iqsoft.hu (Németh Miklós) Date: Sat, 06 Nov 1999 12:22:42 +0100 Subject: [DB-SIG] Oracle DB driver Message-ID: <38240F82.A04063DE@iqsoft.hu> Dear DB-SIG Pythonians, I am quite new on this list, and also quite new in Python. However, I am an old chap in OO (Java, C++) and database (and even in ODBMS) application programming. I think my beloved language will be Python for the next years. I am also employed by several corporate projects as chief technology officer. So, I started evaluating Python and the Python platform (tools necessary for creating mission critical business applications). Python as a language seems to be perfect: easy to learn, pycode compilation, multithreading, OO, easy extensibility, etc. But what about database access support, especially to Oracle? I found that there is a Python Database API Specification 2.0 which seems to be OK. I would have some remarks and suggestions, but more on this later. However, both Oracle drivers I found do not support this specification. I had a look at Thomas J. Cullington's oramodule, but I think it is quite old and unmaintained. The other is Christopher Petrilli's DCOracle which supports Python DB-API 1.0, its implementation was very complex (he used SWIG). And even DCOracle does not support calling PL/SQL procedures having array (PL/SQL table) arguments, which is very important for some large scale Oracle applications. So I decided to write a third Oracle driver compatible with Python DB-API 2.0 and supporting all reasonable services available via OCI. I am an experienced OCI programmer, but quite new in writing Python extensions. I'd like to have your support and help in cases where I have problems. Any comments are welcome. Miklos Nemeth IQSOFT From hniksic@iskon.hr Sat Nov 6 15:56:18 1999 From: hniksic@iskon.hr (Hrvoje Niksic) Date: 06 Nov 1999 16:56:18 +0100 Subject: [DB-SIG] Questions about the DB API Message-ID: <9t9so2jipgd.fsf@mraz.iskon.hr> I am not very experienced with database programming, and some things in the DB API seem strange. Perhaps there are easy explanations for those, but they're not obvious. So: * Why is a cursor required to execute an SQL query? Supporting database cursors is of course nice, but other DB API's (e.g. Perl's) allow executing SQL statements directly from connection objects. It sometimes seems silly to have to create a cursor object for the sole purpose of executing something. * Why is there no `prepare' support? Some database API's support "preparing" a statement before sending it. Why doesn't Python's DB API include such a thing? * Is there any specification on what you can do with Python cursors? For instance, can I send more than one query to the same cursor, or need I create a new one? If I use a new query and want to retrieve the (new) data, will old query's data be ignored? * Supporting Python's iteration would be nice. It would be nice if there were a ready cursor method, other than .fetchall(), which I could use in conjunction with `for'. Code could look like this: cursor.exec("select blah blah blah...") for (customerid, address) in cursor.fetch_iterate(): ... Currently you can replace fetch_iterate() with fetchall(), but it will get all the data at once, which can be (needlessly) memory consuming if you only need one (customerid, address) pair at a time. You can also use fetchone(), but the code is not as nice: while 1: tuple = cursor.fetchone() if tuple is None: break customerid, address = tuple From gstein@lyra.org Sat Nov 6 22:48:36 1999 From: gstein@lyra.org (Greg Stein) Date: Sat, 6 Nov 1999 14:48:36 -0800 (PST) Subject: [DB-SIG] Questions about the DB API In-Reply-To: <9t9so2jipgd.fsf@mraz.iskon.hr> Message-ID: On 6 Nov 1999, Hrvoje Niksic wrote: > * Why is a cursor required to execute an SQL query? > > Supporting database cursors is of course nice, but other DB API's > (e.g. Perl's) allow executing SQL statements directly from connection > objects. It sometimes seems silly to have to create a cursor object > for the sole purpose of executing something. An older version of the API did exactly that -- allow execution directly from the connection object. It was not broadly implemented (i.e. against the spec). When 2.0 came around, we decided that creating a cursor is not a hardship on the programmer and simplified the API to allowing execution only thru cursors. > * Why is there no `prepare' support? > > Some database API's support "preparing" a statement before sending > it. Why doesn't Python's DB API include such a thing? There is, although it is not explicit, and it isn't termed a "prepare". Take a look at the execute() method. It states that if you pass the same string *object* (not value!) to the execute() method a second time, then the underlying system can take advantage of a prepared cursor from the last call to execute(). Several DBAPI modules take advantage of this -- it is a big win on sequences of queries with different params and on multiple inserts. > * Is there any specification on what you can do with Python cursors? > > For instance, can I send more than one query to the same cursor, or > need I create a new one? If I use a new query and want to retrieve > the (new) data, will old query's data be ignored? Hrm. Guess the spec doesn't make this explicit... Yes, you can definitely send more than one query to a cursor; the old query's contents will be tossed. > * Supporting Python's iteration would be nice. > > It would be nice if there were a ready cursor method, other than > .fetchall(), which I could use in conjunction with `for'. Code could It would be very easy to write a small wrapper object to do this. The DBAPI is intended to impose as little effort on the module writers as possible. The multiple fetch methods are almost a bit much to request, but they exist so that different optimizations can be done (e.g. fetchmany() exists so that a module writer can take advantage of array fetches). The current trend is actually to do a very thin C layer and then write the DBAPI module in Python. Anyhow: the point is that if you'd like to use iterators, then write yourself a small class to do the fetchone() calls and expose that using Python's iteration semantics (__getitem__; IndexError). Cheers, -g -- Greg Stein, http://www.lyra.org/ From hniksic@iskon.hr Sat Nov 6 23:03:54 1999 From: hniksic@iskon.hr (Hrvoje Niksic) Date: 07 Nov 1999 00:03:54 +0100 Subject: [DB-SIG] Questions about the DB API In-Reply-To: Greg Stein's message of "Sat, 6 Nov 1999 14:48:36 -0800 (PST)" References: Message-ID: <9t9wvrvfcit.fsf@mraz.iskon.hr> Greg Stein writes: > An older version of the API did exactly that -- allow execution > directly from the connection object. It was not broadly implemented > (i.e. against the spec). When 2.0 came around, we decided that > creating a cursor is not a hardship on the programmer and simplified > the API to allowing execution only thru cursors. Hmm. OK. One reason why I asked this is that cursors have a defined semantics in database API's, and the Perl people in the company always look at me strangely when I create a cursor for the sole reason of executing a query. > > * Why is there no `prepare' support? > > > > Some database API's support "preparing" a statement before sending > > it. Why doesn't Python's DB API include such a thing? > > There is, although it is not explicit, and it isn't termed a > "prepare". Take a look at the execute() method. It states that if > you pass the same string *object* (not value!) to the execute() > method a second time, then the underlying system can take advantage > of a prepared cursor from the last call to execute(). Indeed, you are right. It's a bit subtle, but I like the idea. > > * Is there any specification on what you can do with Python cursors? > > > > For instance, can I send more than one query to the same cursor, or > > need I create a new one? If I use a new query and want to retrieve > > the (new) data, will old query's data be ignored? > > Hrm. Guess the spec doesn't make this explicit... > > Yes, you can definitely send more than one query to a cursor; the > old query's contents will be tossed. Okay. FWIW, the spec says: These objects represent a database cursor, which is used to manage the context of a fetch operation. Given "/a/ fetch operation", one could conclude that only one is allowed. A revision of the specification should perhaps make the intention clearer. > > * Supporting Python's iteration would be nice. > > > > It would be nice if there were a ready cursor method, other than > > .fetchall(), which I could use in conjunction with `for'. Code could > > It would be very easy to write a small wrapper object to do this. I agree; it's just that it would be nice if the API guaranteed the user to have one, so that every user needn't reimplement it. But if the goals of the DBAPI definition are not compatible with this desire, I can understand that. Thanks a lot for your responses. From tbryan@server.python.net Sun Nov 7 05:28:26 1999 From: tbryan@server.python.net (Tom Bryan) Date: Sun, 7 Nov 1999 00:28:26 -0500 (EST) Subject: [DB-SIG] Questions about the DB API In-Reply-To: <9t9wvrvfcit.fsf@mraz.iskon.hr> Message-ID: On 7 Nov 1999, Hrvoje Niksic wrote: > Greg Stein writes: > > > An older version of the API did exactly that -- allow execution > > directly from the connection object. It was not broadly implemented > > (i.e. against the spec). When 2.0 came around, we decided that > > creating a cursor is not a hardship on the programmer and simplified > > the API to allowing execution only thru cursors. > > Hmm. OK. One reason why I asked this is that cursors have a defined > semantics in database API's, and the Perl people in the company always > look at me strangely when I create a cursor for the sole reason of > executing a query. You might want to write your own small module to wrap the DB API. It could import the appropriate DB module for you and provide a function for transparently executing single SQL statements such as 'DROP TABLE perl_hacks'. You could also add your fetch_iterate to such a module. ---Tom From mal@lemburg.com Sun Nov 7 21:26:09 1999 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 07 Nov 1999 22:26:09 +0100 Subject: [DB-SIG] Oracle DB driver References: <38240F82.A04063DE@iqsoft.hu> Message-ID: <3825EE71.B04EAB8@lemburg.com> "Németh Miklós" wrote: > ... > But what about database access support, especially to Oracle? > I found that there is a Python Database API Specification 2.0 which > seems to be OK. I would have some remarks and suggestions, but more on > this later. mxODBC is moving towards full 2.0 compliance. It allows you to access Oracle via ODBC from Unix and Windows provided you have the appropriate ODBC drivers installed. > ... > So I decided to write a third Oracle driver compatible with Python > DB-API 2.0 and supporting all reasonable services available via OCI. I > am an experienced OCI programmer, but quite new in writing Python > extensions. I'd like to have your support and help in cases where I have > problems. Why not take the DCOracle code and continue from there ? -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 55 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Sun Nov 7 21:46:00 1999 From: mal@lemburg.com (M.-A. Lemburg) Date: Sun, 07 Nov 1999 22:46:00 +0100 Subject: [DB-SIG] Questions about the DB API References: Message-ID: <3825F318.2A1C52C2@lemburg.com> Greg Stein wrote: > > > * Why is there no `prepare' support? > > > > Some database API's support "preparing" a statement before sending > > it. Why doesn't Python's DB API include such a thing? > > There is, although it is not explicit, and it isn't termed a "prepare". > Take a look at the execute() method. It states that if you pass the same > string *object* (not value!) to the execute() method a second time, then > the underlying system can take advantage of a prepared cursor from the > last call to execute(). Several DBAPI modules take advantage of this -- it > is a big win on sequences of queries with different params and on multiple > inserts. I think we should add such a method to the next revision of the spec though: in interactive systems or systems that create queries at runtime it is often nice to have a way to check the syntax of a query without actually executing it. .prepare() could be used for just this purpose. I've implemented a .prepare() method in mxODBC and think that the integration with the .fetchXXX() and .executeXXX methods fits rather well: prepare(operation) [cursor method] Prepare a database operation (query or command) statement for later execution and set cursor.statement. To execute a prepared statement, pass cursor.statement to the .executeXXX() methods. Return values are not defined. command [cursor read-only attribute] Provides access to the current prepared SQL command available on the cursor. This is set by .prepare(), all catalog methods, .execute() and .executemany(). Note how cursor.command makes the cached command string available for explicit use by the programs using the mechanism. Interfaces which do not use the caching mechanism could just leave the attribute undefined. I intend to use this feature as speedup technique for a OODB layer by pooling cursors with already prepared queries. -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 55 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From jim@digicool.com Mon Nov 8 13:06:31 1999 From: jim@digicool.com (Jim Fulton) Date: Mon, 08 Nov 1999 08:06:31 -0500 Subject: [DB-SIG] Oracle DB driver References: <38240F82.A04063DE@iqsoft.hu> <3825EE71.B04EAB8@lemburg.com> Message-ID: <3826CAD7.819C0F58@digicool.com> "M.-A. Lemburg" wrote: > (snip) > > ... > > So I decided to write a third Oracle driver compatible with Python > > DB-API 2.0 and supporting all reasonable services available via OCI. I > > am an experienced OCI programmer, but quite new in writing Python > > extensions. I'd like to have your support and help in cases where I have > > problems. > > Why not take the DCOracle code and continue from there ? I was going to suggest the same thing when I realized that if *we* were going to do major work on DCOracle, we'd probably start from scratch because we'd like to see a pure Oracle 8 OCI module. (OCI 8 dosn't look at all like OCI 7 and DCOracle, for various resons, uses OCI 7 with a smattering of OCI 8 to support blobs.) Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From petrilli@amber.org Mon Nov 8 15:12:05 1999 From: petrilli@amber.org (Christopher Petrilli) Date: Mon, 8 Nov 1999 10:12:05 -0500 Subject: [DB-SIG] Questions about the DB API In-Reply-To: <3825F318.2A1C52C2@lemburg.com>; from mal@lemburg.com on Sun, Nov 07, 1999 at 10:46:00PM +0100 References: <3825F318.2A1C52C2@lemburg.com> Message-ID: <19991108101205.A12351@trump.amber.org> M.-A. Lemburg [mal@lemburg.com] wrote: > > I think we should add such a method to the next revision of the > spec though: in interactive systems or systems that create queries > at runtime it is often nice to have a way to check the syntax of > a query without actually executing it. .prepare() could be used > for just this purpose. This is also what DCOracle does... It creates new statement handles in Oracle explictely... we never tried to do it implicitely. And this is also how it's spelled :-) Chris -- | Christopher Petrilli | petrilli@amber.org From david@hotjobs2000.com Tue Nov 9 02:50:58 1999 From: david@hotjobs2000.com (David Winsen) Date: Mon, 8 Nov 1999 18:50:58 -0800 Subject: [DB-SIG] WEB DATABASE PROGRAMMER POSITION AVAILABLE Message-ID: <000801bf2a5d$40f0a0e0$2a2565d8@pacbell.net> This is a multi-part message in MIME format. ------=_NextPart_000_0005_01BF2A1A.323DF220 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable =20 WEB DATABASE PROGRAMMER POSITION AVAILABLE =20 URGENT MESSAGE! =20 From: David Winsen - Senior Consultant - High Technology Executive = Search=20 =20 We have an out-dated copy of your resume in our database or have viewed = your credentials on the internet. HTES is an established national = Executive Search and Consulting Firm who has been serving the High Tech = Industries for over 25 years. =20 =20 We have been confidentially retained by a Los Angeles, Ca. based Adult = Internet Fulfillment/Billing Company. Salary is $50k-$100k DOE.=20 We are confidentially pre-screening top candidates for the following = position: Web Database Programmer Description =20 =20 Candidates will need to be extremely detail oriented and have a = solid work ethic. Will be developing cutting edge software and = e-commerce applications. They are an established and ever growing = Internet Fulfillment/Billing Company. They offer a casual & unique work = environment unlike any other, full benefits, & room for growth and = advancement. =20 =20 =20 =20 Requirements =20 =20 Candidates will need experience in Perl, Python, PHP, Javascript, = UNIX, Linux or FreeBSD, MySQL a plus. BS Computer Science or equivalent. = At least 3 years of Web experience is a plus. =20 =20 If you are interested, please E-mail me in MS Word 95-98 a recent copy = of your resume and a cover letter with your specific information, = including your recent compensation package to Position-for: Web Database = Programmer =20 My personal E-mail is david@hotjobs2000.com or fax your resume to (310) = 855-0840. If you have any questions about the position(s), please call = me at (310) 855-0406 and I will discuss them in detail. =20 We also have developed an interactive Website that you can view over = 6000 national openings www.hotjobs2000.com. This system is effective, = easy to use and new positions are posted daily. We encourage you to use = it and nominate yourself for other positions you feel you are qualified = for. We are looking forward to working with you now and in the future. ------=_NextPart_000_0005_01BF2A1A.323DF220 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
 

WEB DATABASE PROGRAMMER POSITION AVAILABLE

 

URGENT=20 MESSAGE!

 

From:  David Winsen - Senior = Consultant - High=20 Technology Executive Search

 

We=20 have an out-dated copy of your resume in our database or have viewed = your=20 credentials on the internet.  = HTES=20 is an established national Executive Search and Consulting Firm who has = been=20 serving the High Tech Industries for over 25 years. 

 

We=20 have been confidentially retained by a Los Angeles, Ca. based Adult = Internet=20 Fulfillment/Billing Company. Salary is $50k-$100k DOE. =

We=20 are confidentially pre-screening top candidates for the following = position: Web Database = Programmer

 

 

Description

Candidates=20 will need to be extremely detail oriented and have a solid work = ethic.=20 Will be developing cutting edge software and e-commerce = applications. They=20 are an established and ever growing Internet Fulfillment/Billing = Company.=20 They offer a casual & unique work environment unlike any = other, full=20 benefits, & room for growth and advancement.

Requirements

Candidates=20 will need experience in Perl, Python, PHP, Javascript, UNIX, Linux = or=20 FreeBSD, MySQL a plus. BS Computer Science or equivalent. At least = 3 years=20 of Web experience is a plus.

 

If=20 you are interested, please E-mail me in MS Word 95-98 a recent copy of = your=20 resume and a cover letter with your specific information, including your = recent=20 compensation package to Position-for:=20 Web Database Programmer

 

My=20 personal E-mail is david@hotjobs2000.com=20 or fax your resume to (310) 855-0840. If you have any questions about = the=20 position(s), please call me at (310) 855-0406 and I will discuss them in = detail.

 

We=20 also have developed an interactive Website that you can view over 6000 = national=20 openings www.hotjobs2000.com.  This system is effective, easy = to use=20 and new positions are posted daily. =20 We encourage you to use it and nominate yourself for other = positions you=20 feel you are qualified for.  = We are=20 looking forward to working with you now and in the future.

------=_NextPart_000_0005_01BF2A1A.323DF220-- From nemeth@iqsoft.hu Mon Nov 8 23:32:55 1999 From: nemeth@iqsoft.hu (Németh Miklós) Date: Tue, 09 Nov 1999 00:32:55 +0100 Subject: [DB-SIG] Oracle DB driver References: <38240F82.A04063DE@iqsoft.hu> <3825EE71.B04EAB8@lemburg.com> <3826CAD7.819C0F58@digicool.com> Message-ID: <38275DA7.80384D59@iqsoft.hu> Jim Fulton wrote: > "M.-A. Lemburg" wrote: > > > (snip) > > > ... > > > So I decided to write a third Oracle driver compatible with Python > > > DB-API 2.0 and supporting all reasonable services available via OCI. I > > > am an experienced OCI programmer, but quite new in writing Python > > > extensions. I'd like to have your support and help in cases where I have > > > problems. > > > > Why not take the DCOracle code and continue from there ? > > I was going to suggest the same thing when I realized that if > *we* were going to do major work on DCOracle, we'd probably start > from scratch because we'd like to see a pure Oracle 8 OCI module. > (OCI 8 dosn't look at all like OCI 7 and DCOracle, > for various resons, uses OCI 7 with a smattering of > OCI 8 to support blobs.) Does This mean you'd like to have a DCOracle7 and a DCOracle8 ? NM From jim@digicool.com Tue Nov 9 12:52:21 1999 From: jim@digicool.com (Jim Fulton) Date: Tue, 09 Nov 1999 07:52:21 -0500 Subject: [DB-SIG] Oracle DB driver References: <38240F82.A04063DE@iqsoft.hu> <3825EE71.B04EAB8@lemburg.com> <3826CAD7.819C0F58@digicool.com> <38275DA7.80384D59@iqsoft.hu> Message-ID: <38281905.43F54EFC@digicool.com> "Németh Miklós" wrote: > > Jim Fulton wrote: > > > "M.-A. Lemburg" wrote: > > > > > (snip) > > > > ... > > > > So I decided to write a third Oracle driver compatible with Python > > > > DB-API 2.0 and supporting all reasonable services available via OCI. I > > > > am an experienced OCI programmer, but quite new in writing Python > > > > extensions. I'd like to have your support and help in cases where I have > > > > problems. > > > > > > Why not take the DCOracle code and continue from there ? > > > > I was going to suggest the same thing when I realized that if > > *we* were going to do major work on DCOracle, we'd probably start > > from scratch because we'd like to see a pure Oracle 8 OCI module. > > (OCI 8 dosn't look at all like OCI 7 and DCOracle, > > for various resons, uses OCI 7 with a smattering of > > OCI 8 to support blobs.) > > Does This mean you'd like to have a DCOracle7 and a DCOracle8 ? Yes, or a NMOracle8 or Whatever. :) Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats. From jadiel@thevision.net Sat Nov 13 02:17:15 1999 From: jadiel@thevision.net (Jose de Leon) Date: Fri, 12 Nov 1999 18:17:15 -0800 Subject: [DB-SIG] Please some clarification of DB Spec 2.0 Message-ID: <010401bf2d7d$39a8f800$3d00a8c0@invsn.com> Hello! I apologize if I ask questions asked many times before. I'd search the archives but my connetion to the python site is VERY slow. First: a) I'm new to python. b) I'm new to the Python DB Spec 2.0. I'm creating a database module interface. But before I code it python. I'm trying to code it in C++, a language I know much better. I then intend to use SWIG to convert the C++ code to python. Possibly by creating a python shadow wrapper. One of the items I need clarification on is "sequence". What exactly is a sequence and how can I relate that to a C++ datatype? From what I can gather from "Programming Python" by O'Reilly is that it is a string with delimiters. Or it could be an array? The specific specification from the DB Spec 2.0 that confuses me is the description for a the "description" method of a Cursor object. It explains: description This read-only attribute is a sequence of 7-item sequences....etc. Likewise, the description for "fetchone()" method also confuses me on how to return the data. Should the data be returned in an array? Thanks, Jose de Leon From mal@lemburg.com Sat Nov 13 08:38:18 1999 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 13 Nov 1999 09:38:18 +0100 Subject: [DB-SIG] Please some clarification of DB Spec 2.0 References: <010401bf2d7d$39a8f800$3d00a8c0@invsn.com> Message-ID: <382D237A.E7C30073@lemburg.com> Jose de Leon wrote: > > Hello! > > I apologize if I ask questions asked many times before. I'd search the > archives but my connetion to the python site is VERY slow. > > First: > a) I'm new to python. > b) I'm new to the Python DB Spec 2.0. > > I'm creating a database module interface. But before I code it python. I'm > trying to code it in C++, a language I know much better. I then intend to > use SWIG to convert the C++ code to python. Possibly by creating a python > shadow wrapper. > > One of the items I need clarification on is "sequence". > > What exactly is a sequence and how can I relate that to a C++ datatype? > >From what I can gather from "Programming Python" by O'Reilly is that it is a > string with delimiters. Or it could be an array? Its a conglomeration of elements which can be indexed using integers. Python lists and tuples are examples for sequences. C arrays also fit this definition. Basically, they have to provide a length and a method which allows indexing all elements starting from index 0 to length - 1. > The specific specification from the DB Spec 2.0 that confuses me is the > description for a the "description" method of a Cursor object. > > It explains: > > description > This read-only attribute is a sequence of 7-item sequences....etc. This normal refers to a tuple of tuples, one 7-tuple (a tuple of length 7) for each column in the result set. > Likewise, the description for "fetchone()" method also confuses me on how to > return the data. Should the data be returned in an array? It should return a sequence, such as a tuple corresponding to a row in the result set, with each element representing a column entry. Hope that helps, -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 48 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From hniksic@iskon.hr Wed Nov 17 17:34:21 1999 From: hniksic@iskon.hr (Hrvoje Niksic) Date: 17 Nov 1999 18:34:21 +0100 Subject: [DB-SIG] [summary] Connecting to MS SQL from Python Message-ID: <9t9u2ml9g4i.fsf@mraz.iskon.hr> Eight weeks ago, I asked about the possibilities of connecting to MS SQL server from within Python. I received several responses, and I promised to post a summary of my experiences. As promised, here is the summary of my experiences after investigating the possible solutions. To remind you, I needed to access MS SQL from Python applications on Linux. In my company, we currently use Michael Peppler's Sybperl (CTlib) for this purpose. The Perl solution is free, so the Python one should be free, or at least inexpensive, to stand a chance of acceptance. To cut a long story short, I must sadly report that I have NOT been able to find a free solution that offers the needed functionality. If someone can offer additional insight, I'll be more than helpful, but I admit I'm beginning to lose hope. The long story follows, in the form of a listing of various possibilities, in no particular order: * Wrap Sybase's CTlib into Python. Peter Godman's Sybase module is supposed to do that, but it appears unmaintained and in a really bad shape. Besides, it's compliant with the old version of the DB API. For someone knowledgable with databases or with Python internals, it would probably be easy to write his own wrapper, or pick up where Peter left off. But I am new to the world of databases, and know next to nothing about Python C internals, and for me it's a daunting task. * Find an ODBC driver for MS SQL, and use the excellent mxODBC module written by M.-A. Lemburg. It turned out that finding a Linux ODBC driver for MS SQL is not an easy thing to do. Which brings us to these four attempts: * EasySoft ODBC-ODBC bridge (OOB). It's a nice product, but it's still in beta, and it will cost 800$ when finished (160$ for a personal version). If I could persuade my boss to buy it, I'd certainly be using it. * OpenLink ODBC drivers for Linux (the limited version that comes free). This sounded promising, but when i tried them, they proved very convoluted. They install ODBC libraries and a request broker, as well as the agents for various databases. The MS SQL agent uses (sic!) CTlib. So what you end up doing as having the application use OpenLink libraries which contact the OpenLink request broker, which talks to the agent, which uses CTlib to connect to the database. When I couldn't get it to work after long and frustrating hours of reading manuals and tweaking conffiles, I gave up on OpenLink. * Merant (formerly InterSolv) ODBC drivers for MS SQL. These are supposed to be available for Linux, but I couldn't find them on their site. It remains a mystery, and I intend to try again soon. * FreeTDS ODBC drivers. The FreeTDS project () is dedicated to writing full support for TDS (Tabular Data Stream, protocol used by Sybase and MS SQL), including CTlib, DB-Lib, ODBC, and JDBC. However, their work is far from complete; in fact, the work on ODBC has hardly begun. In a year or so, it may be usable, but not now. * Write a proxy running on NT that reads serialized Python ODBC requests from clients, and issues the appropriate mxODBC calls locally, then writes serialized results back to the client. The problem with this is twofold: first, I'd need physical access to the NT machine, a compiler on Windows, Python on Windows, etc., and I have none of it. Second, the performance of such a setup could prove to be a serious problem. Python is no speed champion, and the overhead of all the serializations could prove fatal. The OOB people at EasySoft did a lot of work to optimize their proxy, and they use C. :-( From mclay@nist.gov Thu Nov 18 00:23:24 1999 From: mclay@nist.gov (Michael McLay) Date: Wed, 17 Nov 1999 19:23:24 -0500 (EST) Subject: [DB-SIG] [summary] Connecting to MS SQL from Python In-Reply-To: <9t9u2ml9g4i.fsf@mraz.iskon.hr> References: <9t9u2ml9g4i.fsf@mraz.iskon.hr> Message-ID: <14387.17776.919474.464701@fermi.eeel.nist.gov> Hrvoje Niksic writes: > * Find an ODBC driver for MS SQL, and use the excellent mxODBC module > written by M.-A. Lemburg. It turned out that finding a Linux ODBC > driver for MS SQL is not an easy thing to do. Which brings us to > these four attempts: ... [list of options investigated]... Your list did not include FreeODBC http://users.ids.net/~bjepson/FreeODBC/ The following was quoted from that page: mySQL MyODBC is an ODBC driver for mySQL. MyODBC compiles and works with iODBC 2.50.2 or later, and can be obtained from http://www.tcx.se/download.html. > > The problem with this is twofold: first, I'd need physical access to > the NT machine, a compiler on Windows, Python on Windows, etc., and > I have none of it. Second, the performance of such a setup could > prove to be a serious problem. Python is no speed champion, and the > overhead of all the serializations could prove fatal. The OOB > people at EasySoft did a lot of work to optimize their proxy, and > they use C. :-( I haven't tried to compile mxODBC to work with the FreeODBC library interface. Has anyone else tried using it? From Anthony Baxter Wed Nov 17 22:32:25 1999 From: Anthony Baxter (Anthony Baxter) Date: Thu, 18 Nov 1999 09:32:25 +1100 Subject: [DB-SIG] [summary] Connecting to MS SQL from Python In-Reply-To: Message from Hrvoje Niksic of "17 Nov 1999 18:34:21 BST." <9t9u2ml9g4i.fsf@mraz.iskon.hr> Message-ID: <199911172232.JAA10693@mbuna.arbhome.com.au> >>> Hrvoje Niksic wrote > Eight weeks ago, I asked about the possibilities of connecting to MS > SQL server from within Python. I received several responses, and I > promised to post a summary of my experiences. > * Wrap Sybase's CTlib into Python. Peter Godman's Sybase module is > supposed to do that, but it appears unmaintained and in a really bad > shape. Besides, it's compliant with the old version of the DB API. > > For someone knowledgable with databases or with Python internals, it > would probably be easy to write his own wrapper, or pick up where > Peter left off. But I am new to the world of databases, and know > next to nothing about Python C internals, and for me it's a daunting > task. Brian Hooper's existing ZSybaseDA works for us under Zope1 without too many troubles. That's for talking from a Redhat Linux 5.2 box with Sybase ASE 11.0 to an NT4sp5 box with SQL Server 6.5sp3 There's also the soon-to-be-released new ZSybaseDA, which a little bird tells me will also work when run against SQL Server. I think Paul said last Friday it would be released "within a week", but hey, those sales-types always over-promise :) Anthony -- Anthony Baxter It's never too late to have a happy childhood. From Anthony Baxter Wed Nov 17 22:51:18 1999 From: Anthony Baxter (Anthony Baxter) Date: Thu, 18 Nov 1999 09:51:18 +1100 Subject: [DB-SIG] [summary] Connecting to MS SQL from Python In-Reply-To: Message from Michael McLay of "Wed, 17 Nov 1999 19:23:24 CDT." <14387.17776.919474.464701@fermi.eeel.nist.gov> Message-ID: <199911172251.JAA10769@mbuna.arbhome.com.au> > Hrvoje Niksic writes: > > * Find an ODBC driver for MS SQL, and use the excellent mxODBC module > > written by M.-A. Lemburg. It turned out that finding a Linux ODBC > > driver for MS SQL is not an easy thing to do. Which brings us to > > these four attempts: There's one other option I forgot - FreeTDS. This would be instead of the Sybase libs, but still needs the Sybase DA. Check www.freshmeat.net for the FreeTDS webpage. Anthony From hniksic@iskon.hr Thu Nov 18 08:13:27 1999 From: hniksic@iskon.hr (Hrvoje Niksic) Date: 18 Nov 1999 09:13:27 +0100 Subject: [DB-SIG] [summary] Connecting to MS SQL from Python In-Reply-To: Michael McLay's message of "Wed, 17 Nov 1999 19:23:24 -0500 (EST)" References: <9t9u2ml9g4i.fsf@mraz.iskon.hr> <14387.17776.919474.464701@fermi.eeel.nist.gov> Message-ID: <9t91z9o9pzs.fsf@mraz.iskon.hr> Michael McLay writes: > mySQL > > MyODBC is an ODBC driver for mySQL. MyODBC compiles and works with > iODBC 2.50.2 or later, and can be obtained from > http://www.tcx.se/download.html. That's all nice, but I need all that for _MS_ SQL, not MySQL. From hniksic@iskon.hr Thu Nov 18 08:13:58 1999 From: hniksic@iskon.hr (Hrvoje Niksic) Date: 18 Nov 1999 09:13:58 +0100 Subject: [DB-SIG] [summary] Connecting to MS SQL from Python In-Reply-To: Anthony Baxter's message of "Thu, 18 Nov 1999 09:51:18 +1100" References: <199911172251.JAA10769@mbuna.arbhome.com.au> Message-ID: <9t9u2mk8beh.fsf@mraz.iskon.hr> Anthony Baxter writes: > > Hrvoje Niksic writes: > > > * Find an ODBC driver for MS SQL, and use the excellent mxODBC module > > > written by M.-A. Lemburg. It turned out that finding a Linux ODBC > > > driver for MS SQL is not an easy thing to do. Which brings us to > > > these four attempts: > > There's one other option I forgot - FreeTDS. I listed FreeTDS in my message. From mal@lemburg.com Thu Nov 18 09:21:31 1999 From: mal@lemburg.com (M.-A. Lemburg) Date: Thu, 18 Nov 1999 10:21:31 +0100 Subject: [DB-SIG] [summary] Connecting to MS SQL from Python References: <9t9u2ml9g4i.fsf@mraz.iskon.hr> Message-ID: <3833C51B.E9B1EDC4@lemburg.com> Hrvoje Niksic wrote about interfacing from Linux -> MS SQL Server: > > * Write a proxy running on NT that reads serialized Python ODBC > requests from clients, and issues the appropriate mxODBC calls > locally, then writes serialized results back to the client. > > The problem with this is twofold: first, I'd need physical access to > the NT machine, a compiler on Windows, Python on Windows, etc., and > I have none of it. Second, the performance of such a setup could > prove to be a serious problem. Python is no speed champion, and the > overhead of all the serializations could prove fatal. The OOB > people at EasySoft did a lot of work to optimize their proxy, and > they use C. :-( One of the next major releases will include such a proxy. Note that you don't need a C compiler for it to work: the mxODBC package includes a precompiled binary for Windows. Performance will probably not be as good as with the EasySoft bridge, but I'll do my best to avoid too much overhead. Note that a proxy solution has more benefits: you can write low level for the server side and only transfer data at a very high level to the client side, e.g. by building complete data structure and then sending them over the line via cPickle and sockets. -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 43 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From petrilli@amber.org Thu Nov 18 16:08:40 1999 From: petrilli@amber.org (Christopher Petrilli) Date: Thu, 18 Nov 1999 11:08:40 -0500 Subject: [DB-SIG] [summary] Connecting to MS SQL from Python In-Reply-To: <199911172232.JAA10693@mbuna.arbhome.com.au>; from anthony@interlink.com.au on Thu, Nov 18, 1999 at 09:32:25AM +1100 References: <199911172232.JAA10693@mbuna.arbhome.com.au> Message-ID: <19991118110840.A16614@trump.amber.org> Anthony Baxter [anthony@interlink.com.au] wrote: > There's also the soon-to-be-released new ZSybaseDA, which a little bird > tells me will also work when run against SQL Server. Yup, Chris McDonough has been working on testing it against SQLServer 6.5, and hopefully 7 at some point. So far it looks ok... > I think Paul said last Friday it would be released "within a week", but > hey, those sales-types always over-promise :) I think you should be seeing it RSN. :-) Chris -- | Christopher Petrilli | petrilli@amber.org