From ccurvey at earthlink.net Thu Mar 3 17:22:09 2005 From: ccurvey at earthlink.net (Chris Curvey) Date: Thu Mar 3 17:16:03 2005 Subject: [DB-SIG] ADO database connection seems to be closing Message-ID: <422739B1.30505@earthlink.net> I'm trying to get to an MS-SQL datbase via the win32com.client library and the ADO library, but I seem to lose the context of the connection whenever I execute a statement. For example, the first "assert" below is OK, but the second one is not. I'm also losing the value of @@identity. Is this a known issue? I was hoping for a DB-API 2.0 interface to MS-SQL, but I can't get my clients to pay for an mxODBC license. import unittest from win32com.client import Dispatch class TransactionTest(unittest.TestCase): def testConnection(self): conn = Dispatch('ADODB.Connection') conn.Open("Driver={SQL Server};Server=mxdev;UID=chris_dev;PWD=chris;DATABASE=tempdb") # start a transaction conn.execute("begin tran") result = conn.execute("select @@trancount") while not result.EOF: # this test passes assert result.Fields.Item(0).Value == 1 result.MoveNext() result = conn.execute("select @@trancount") while not result.EOF: #this test fails assert result.Fields.Item(0).Value == 1 result.MoveNext() ######################################################################## if __name__ == "__main__": unittest.main() From fumanchu at amor.org Thu Mar 3 17:37:37 2005 From: fumanchu at amor.org (Robert Brewer) Date: Thu Mar 3 17:41:40 2005 Subject: [DB-SIG] ADO database connection seems to be closing Message-ID: <3A81C87DC164034AA4E2DDFE11D258E3398507@exchange.hqamor.amorhq.net> Chris Curvey wrote: > I'm trying to get to an MS-SQL datbase via the > win32com.client library > and the ADO library, but I seem to lose the context of the connection > whenever I execute a statement. For example, the first > "assert" below > is OK, but the second one is not. I'm also losing the value > of @@identity. > > Is this a known issue? I was hoping for a DB-API 2.0 interface to > MS-SQL, but I can't get my clients to pay for an mxODBC license. > > import unittest > from win32com.client import Dispatch > > class TransactionTest(unittest.TestCase): > def testConnection(self): > conn = Dispatch('ADODB.Connection') > conn.Open("Driver={SQL > Server};Server=mxdev;UID=chris_dev;PWD=chris;DATABASE=tempdb") > > # start a transaction > conn.execute("begin tran") > > result = conn.execute("select @@trancount") > > while not result.EOF: > # this test passes > assert result.Fields.Item(0).Value == 1 > result.MoveNext() > > result = conn.execute("select @@trancount") > while not result.EOF: > #this test fails > assert result.Fields.Item(0).Value == 1 > result.MoveNext() > > > ############################################################## > ########## > if __name__ == "__main__": > unittest.main() Two ideas: 1. Have you tried result.Close() before the second conn.execute()? 2. I use a new ADODB.Recordset object, which works for me: res = win32com.client.Dispatch(r'ADODB.Recordset') res.Open(query, conn, adOpenForwardOnly, adLockReadOnly) data = [] if not(res.BOF and res.EOF): data = res.GetRows() # Convert cols x rows -> rows x cols data = zip(*data) res.Close() Hope that helps! Robert Brewer MIS Amor Ministries fumanchu@amor.org From ccurvey at earthlink.net Thu Mar 3 18:26:57 2005 From: ccurvey at earthlink.net (Chris Curvey) Date: Thu Mar 3 18:21:23 2005 Subject: [DB-SIG] ADO database connection seems to be closing In-Reply-To: <3A81C87DC164034AA4E2DDFE11D258E3398507@exchange.hqamor.amorhq.net> References: <3A81C87DC164034AA4E2DDFE11D258E3398507@exchange.hqamor.amorhq.net> Message-ID: <422748E1.8040507@earthlink.net> Robert Brewer wrote: >Chris Curvey wrote: > > >>I'm trying to get to an MS-SQL datbase via the >>win32com.client library >>and the ADO library, but I seem to lose the context of the connection >>whenever I execute a statement. For example, the first >>"assert" below >>is OK, but the second one is not. I'm also losing the value >>of @@identity. >> >>Is this a known issue? I was hoping for a DB-API 2.0 interface to >>MS-SQL, but I can't get my clients to pay for an mxODBC license. >> >>import unittest >>from win32com.client import Dispatch >> >>class TransactionTest(unittest.TestCase): >> def testConnection(self): >> conn = Dispatch('ADODB.Connection') >> conn.Open("Driver={SQL >>Server};Server=mxdev;UID=chris_dev;PWD=chris;DATABASE=tempdb") >> >> # start a transaction >> conn.execute("begin tran") >> >> result = conn.execute("select @@trancount") >> >> while not result.EOF: >> # this test passes >> assert result.Fields.Item(0).Value == 1 >> result.MoveNext() >> >> result = conn.execute("select @@trancount") >> while not result.EOF: >> #this test fails >> assert result.Fields.Item(0).Value == 1 >> result.MoveNext() >> >> >>############################################################## >>########## >>if __name__ == "__main__": >> unittest.main() >> >> > >Two ideas: > >1. Have you tried result.Close() before the second conn.execute()? > >2. I use a new ADODB.Recordset object, which works for me: > >res = win32com.client.Dispatch(r'ADODB.Recordset') >res.Open(query, conn, adOpenForwardOnly, adLockReadOnly) >data = [] >if not(res.BOF and res.EOF): > data = res.GetRows() > # Convert cols x rows -> rows x cols > data = zip(*data) >res.Close() > > >Hope that helps! > > >Robert Brewer >MIS >Amor Ministries >fumanchu@amor.org > > Yup, adding "result.Close()" made it all better! thanks! From marcos at burke.ath.cx Thu Mar 3 18:35:26 2005 From: marcos at burke.ath.cx (Marcos =?ISO-8859-1?Q?S=E1nchez?= Provencio) Date: Thu Mar 3 18:35:39 2005 Subject: [DB-SIG] [OT] ADO database connection seems to be closing In-Reply-To: <422739B1.30505@earthlink.net> References: <422739B1.30505@earthlink.net> Message-ID: <1109871326.4519.6.camel@localhost.localdomain> You are using ADO to access SQL using an ODBC bridge, ?is this what you really want? You may access SQL Server without the ODBC bridge. "Provider=sqloledb;Data Source=Aron1;Initial Catalog=pubs;User Id=sa;Password=asdasd;" versus "Driver={SQL Server};Server=Aron1;Database=pubs;Uid=sa;Pwd=asdasd;" [from www.connectrionstrings.com] It is probably faster, although I have no empirical experience. El jue, 03-03-2005 a las 11:22 -0500, Chris Curvey escribi?: > I'm trying to get to an MS-SQL datbase via the win32com.client library > and the ADO library, but I seem to lose the context of the connection > whenever I execute a statement. For example, the first "assert" below > is OK, but the second one is not. I'm also losing the value of @@identity. > > Is this a known issue? I was hoping for a DB-API 2.0 interface to > MS-SQL, but I can't get my clients to pay for an mxODBC license. > > import unittest > from win32com.client import Dispatch > > class TransactionTest(unittest.TestCase): > def testConnection(self): > conn = Dispatch('ADODB.Connection') > conn.Open("Driver={SQL > Server};Server=mxdev;UID=chris_dev;PWD=chris;DATABASE=tempdb") > > # start a transaction > conn.execute("begin tran") > > result = conn.execute("select @@trancount") > > while not result.EOF: > # this test passes > assert result.Fields.Item(0).Value == 1 > result.MoveNext() > > result = conn.execute("select @@trancount") > while not result.EOF: > #this test fails > assert result.Fields.Item(0).Value == 1 > result.MoveNext() > > > ######################################################################## > if __name__ == "__main__": > unittest.main() > > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig From jlh at yvn.com Mon Mar 14 05:47:15 2005 From: jlh at yvn.com (jim) Date: Mon Mar 14 05:46:03 2005 Subject: [DB-SIG] xdb-xbase bindings for python Message-ID: <200503140447.15400.jlh@yvn.com> Sirs: I have bindings for the xbase-2.0.0 c++ library with the sip files and a python wrapper module. I would be willing to share if there is any interest. I would also appreciate any suggestions for how to present this offer if this isn't the correct forum/format. Jim Hurlburt Yakima, Wa From jannee at brikks.com Mon Mar 14 18:33:30 2005 From: jannee at brikks.com (=?iso-8859-1?Q?Jan_Ekstr=F6m?=) Date: Mon Mar 14 19:00:13 2005 Subject: [DB-SIG] (no subject) Message-ID: <000601c528bb$f02b1d80$25e7e953@D7M46F1J> -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20050314/204622c8/attachment.htm From davidrushby at yahoo.com Mon Mar 14 21:28:27 2005 From: davidrushby at yahoo.com (David Rushby) Date: Mon Mar 14 21:28:32 2005 Subject: [DB-SIG] xdb-xbase bindings for python In-Reply-To: 6667 Message-ID: <20050314202828.6997.qmail@web30001.mail.mud.yahoo.com> --- jim wrote: > I have bindings for the xbase-2.0.0 c++ library with the sip files > and a python wrapper module. > > I would be willing to share if there is any interest. I'm interested, although I won't be able to devote any time to that interest for several months, so don't expect immediate feedback from me. I suggest that you either: a) (preferably) establish a SourceForge project or b) put a tarball link on a web page Based on personal experience, I advise you not to get too hung up initially on issues such as whether the module builds out of the box on 36 platforms and whether it covers every last corner of the underlying C++ library. Those issues can be ironed out in due time. With regard to early documentation, it'd be helpful to explain how the module compares to other python-xbase modules and what your ultimate goals are. This'll help programmers who are browsing for a "Python xbase module" to decide whether yours is worth investing time in. __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From jlh at yvn.com Tue Mar 15 03:08:17 2005 From: jlh at yvn.com (jim) Date: Tue Mar 15 03:06:58 2005 Subject: [DB-SIG] xdb-xbase bindings for python In-Reply-To: <20050314202828.6997.qmail@web30001.mail.mud.yahoo.com> References: <20050314202828.6997.qmail@web30001.mail.mud.yahoo.com> Message-ID: <200503150208.17372.jlh@yvn.com> David: Thanks for the reply. > --- jim wrote: > > I have bindings for the xbase-2.0.0 c++ library with the sip files > > and a python wrapper module. > > > > I would be willing to share if there is any interest. > > I suggest that you either: > a) (preferably) establish a SourceForge project I'll look into sourceforge. Other than creating a project there, do you have any suggestions about how to publicise this in an approved and useful fashion? Not to gain fame and glory, but to let others who need it, find it. > Based on personal experience, I advise you not to get too hung up > initially on issues such as whether the module builds out of the box on > 36 platforms and whether it covers every last corner of the underlying > C++ library. Those issues can be ironed out in due time. My plan exactly. I am building it for my own use at work. So, things that annoy me will get dealt with. Those that don't will wait till someone whines about it. At the moment, I can build it on linux-gentoo and win2k borland 5.5. Again, until someone else whines about it, I won't bother with anything else. > With regard to early documentation, it'd be helpful to explain how the > module compares to other python-xbase modules and what your ultimate > goals are. My goal is to have a way to seamlessly work with the Clipper5.2 production data software/system at work with python. All data are currently in clipper xbase dbf files. Clipper was as good as it got in '95 when I started work there, and still is better than most, but has been abandoned by Computer Associates and starts to not work in a serious fashion when you get to Win2K. Works Ok with 98 but is still a dos program at heart. However, I have ~500k lines of code that work and work pretty well and don't want to even think about an abrupt switchover -- thence a gradual transition to python, from there eventually to some sort of SQL database instead of xbase. So far as I can find, there are *NO* other xbase-python modules. mxOdbc/windows ODBC sort of works with clipper dbf files if you don't want indexes. Other than that, I don't know of anything, and I have looked pretty hard. Now, I suppose that you will tell me about some project that does everything I need if I had just found it ( Hope, Hope, Hope?) but not expect. For the moment, documentation is implicit in a python unittest module that I have been using to debug. This will grow as I find bugs and need tests to excorcise them. That and a README that this email is the start of. > This'll help programmers who are browsing for a "Python > xbase module" to decide whether yours is worth investing time in. > Any further suggestions regarding the above thoughts would be most welcome. Jim From fve at phgroup.com Tue Mar 15 04:17:14 2005 From: fve at phgroup.com (Frederic Vander Elst) Date: Tue Mar 15 11:05:39 2005 Subject: [DB-SIG] xdb-xbase bindings for python In-Reply-To: <200503150208.17372.jlh@yvn.com> References: <20050314202828.6997.qmail@web30001.mail.mud.yahoo.com> <200503150208.17372.jlh@yvn.com> Message-ID: <423653BA.6000206@phgroup.com> Hello Jim, hello David, I have had this problem and found 2 solutions 1. somewhere there is a read-only interface to .DBF's (vaults of parnassus I think). I think minor tweaking would make it read-write. indexes are not supported, but it's about 150 lines, so, basic, basic ! 2. installing visual foxpro with the odbc drivers lets you use mxODBC (and the native python odbc). The Vfox Odbc driver does understand compound index files (.CDX) and uses them properly to access records by index (i.e. c.execute("select * where id = ?id", ('xx',)) uses the cdx index if possible, not a table scan). I can't remember for sure if CDX indexes appeared after clipper 5.2, but I think they were there already. This allows you to use the original dbf format (which excel also can read, the one with no nulls, signature 0x03 in the header), and also the newer vfox format (which allows nulls, but neither excel nor clipper can read nor write, and has signature 0x83 in the first byte). I have found that I could write code that could read and write either a DBF or the same table on db2/oracle/postgres, pretty much with no change of code, just by opening a different ODBC dsn; that has helped me transition some DBF-dependent code to python and then to sql servers. Of course you have to go for the lower common standard of sql if you want interoperability. I've written a table module that allows tables to be treated like a mutable list of objects (and across vfox/db2/oracle/dbf), but it's not pretty (one of my first coding attemps): t = table(dsn, "select * from blah", pk=(id,id2,id3,etc)) for r in t: print r.id, r.name r.name = r.name.upper() will print and uppercase the name column in the table, committing at the del of r. Please get in touch if you'd like to discuss. -frederic vander elst jim wrote: >David: >Thanks for the reply. > > > >>--- jim wrote: >> >> >>>I have bindings for the xbase-2.0.0 c++ library with the sip files >>>and a python wrapper module. >>> >>>I would be willing to share if there is any interest. >>> >>> >>I suggest that you either: >> a) (preferably) establish a SourceForge project >> >> > >I'll look into sourceforge. Other than creating a project there, do you have >any suggestions about how to publicise this in an approved and useful >fashion? Not to gain fame and glory, but to let others who need it, find it. > > > >>Based on personal experience, I advise you not to get too hung up >>initially on issues such as whether the module builds out of the box on >>36 platforms and whether it covers every last corner of the underlying >>C++ library. Those issues can be ironed out in due time. >> >> > >My plan exactly. I am building it for my own use at work. So, things that >annoy me will get dealt with. Those that don't will wait till someone whines >about it. > >At the moment, I can build it on linux-gentoo and win2k borland 5.5. Again, >until someone else whines about it, I won't bother with anything else. > > > >>With regard to early documentation, it'd be helpful to explain how the >>module compares to other python-xbase modules and what your ultimate >>goals are. >> >> > >My goal is to have a way to seamlessly work with the Clipper5.2 production >data software/system at work with python. All data are currently in clipper >xbase dbf files. > >Clipper was as good as it got in '95 when I started work there, and still is >better than most, but has been abandoned by Computer Associates and starts to >not work in a serious fashion when you get to Win2K. Works Ok with 98 but is >still a dos program at heart. > >However, I have ~500k lines of code that work and work pretty well and don't >want to even think about an abrupt switchover -- thence a gradual transition >to python, from there eventually to some sort of SQL database instead of >xbase. > >So far as I can find, there are *NO* other xbase-python modules. >mxOdbc/windows ODBC sort of works with clipper dbf files if you don't want >indexes. Other than that, I don't know of anything, and I have looked pretty >hard. > >Now, I suppose that you will tell me about some project that does everything I >need if I had just found it ( Hope, Hope, Hope?) but not expect. > >For the moment, documentation is implicit in a python unittest module that I >have been using to debug. This will grow as I find bugs and need tests to >excorcise them. That and a README that this email is the start of. > > > >>This'll help programmers who are browsing for a "Python >>xbase module" to decide whether yours is worth investing time in. >> >> >> > >Any further suggestions regarding the above thoughts would be most welcome. > >Jim >_______________________________________________ >DB-SIG maillist - DB-SIG@python.org >http://mail.python.org/mailman/listinfo/db-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/db-sig/attachments/20050315/baff7f19/attachment.htm From Chris.Clark at ca.com Tue Mar 15 20:21:48 2005 From: Chris.Clark at ca.com (Clark, Chris M) Date: Tue Mar 15 20:21:51 2005 Subject: [DB-SIG] xdb-xbase bindings for python Message-ID: <5F0EF692A76C0F4D970C00F5D5B3445F7EC568@usilms28.ca.com> > -----Original Message----- > From: db-sig-bounces@python.org > [mailto:db-sig-bounces@python.org] On Behalf Of jim > Sent: Monday, March 14, 2005 6:08 PM > To: David Rushby > Cc: db-sig@python.org > Subject: Re: [DB-SIG] xdb-xbase bindings for python > ... > I'll look into sourceforge. Other than creating a project > there, do you have any suggestions about how to publicise > this in an approved and useful fashion? Not to gain fame and > glory, but to let others who need it, find it. > My 2 cents, this is what I do and it seems to work out OK: Sourceforge is a great place to host. I don't like CVS that much but it is better than nothing and sf.net is much more than free source code control (mirroring for one). As for publicity, I usually make use of the web page hosting at sourceforge and then use http://www.google.com/addurl/ to add the page to a search engine (as most people use Google). Similar advice for yahoo, etc. There are places that claim to submit to multiple sites but I've never used them ( http://www.evrsoft.com/fastsubmit/ ) If the module is partially DBI compliant try and get it listed on http://python.org/topics/database/modules.html (see the "Email Us" link on the left of that page). If it isn't slightly compliant get it on http://python.org/topics/database/other-db.html Finally, register the project on http://freshmeat.net I find this covers all the places I would think to search; I'm sure there are other things that could be done but it will get you started. RE clipper, have you seen http://www.harbour-project.org/ , http://www.xharbour.org/index.asp?page=about/index and http://www.itk.ru/english/clip/aboutclip.shtml I have to confess I've not used any of them. The Clipper community is still going strong. Chris From djc at object-craft.com.au Sat Mar 19 04:59:45 2005 From: djc at object-craft.com.au (Dave Cole) Date: Sat Mar 19 04:58:52 2005 Subject: [DB-SIG] Sybase module 0.37pre1 released Message-ID: <423BA3B1.50408@object-craft.com.au> WHAT IS IT: The Sybase module provides a Python interface to the Sybase relational database system. It supports all of the Python Database API, version 2.0 with extensions. NOTES: This release contains a number of small bugfixes and patches received from users. I have been unable to find the source of the memory leak reported here: http://www.object-craft.com.au/pipermail/python-sybase/2004-December/000346.html The test program I wrote follows: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - import sys import Sybase db = Sybase.connect(..., auto_commit=True) db.execute(''' if exists (select name from sysobjects where name = "sp_test_leak") begin drop procedure sp_test_leak end ''') db.execute(''' create procedure sp_test_leak @arg int as select @arg ''') for i in range(200): for j in range(1000): c = db.cursor() c.callproc('sp_test_leak', {'@arg': 12345 }) sys.stdout.write('%3d\r' % i) sys.stdout.flush() sys.stdout.write('\n') - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If someone is able to modify this and come up with a leaking result I am interested in working on the fix. You can build for FreeTDS like this: python setup.py build_ext -D HAVE_FREETDS -U WANT_BULKCOPY python setup.py install The module is available here: http://www.object-craft.com.au/projects/sybase/download/sybase-0.37pre1.tar.gz The module home page is here: http://www.object-craft.com.au/projects/sybase/ CHANGES SINCE 0.36: * Cursor output parameters now work when parameters are passed as a sequence. * Output parameters now work for FreeTDS 0.62.4. 1> create procedure sp_test_output 2> @num int, @result int output 3> as 4> select @result = @num 5> go params = c.callproc('sp_test_output', {'@num': 12345, '@result': Sybase.OUTPUT(1)}) print params['@result'] * The CS_STATUS_RESULT result set is now consumed internally in the Cursor and does not appear in the result sets consumed by the fetch and nextset methods. The return value from the stored procedure is available in the .return_status member of the Cursor. It will only contain a meaningful value once all of the row result sets have been consumed. Note that this does not work with FreeTDS 0.62.4. The return_status seems to always be 0. Research shows that the problem is probably in the CT emulation layer as tsql displays the correct value, but sqsh displays 0. * Output hook patch from Ty Sarna has been applied. * Applied patch from Andre Sedinin to improve error handling. * Improved detection of SYBASE_OCS. - Dave -- http://www.object-craft.com.au From djc at object-craft.com.au Mon Mar 21 00:15:36 2005 From: djc at object-craft.com.au (Dave Cole) Date: Mon Mar 21 00:14:34 2005 Subject: [DB-SIG] Sybase module 0.37pre2 released Message-ID: <423E0418.6080405@object-craft.com.au> WHAT IS IT: The Sybase module provides a Python interface to the Sybase relational database system. It supports all of the Python Database API, version 2.0 with extensions. NOTES: This release contains a number of small bugfixes and patches received from users. I have been unable to find the source of the memory leak reported here: http://www.object-craft.com.au/pipermail/python-sybase/2004-December/000346.html The test program I wrote follows: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - import sys import Sybase db = Sybase.connect(..., auto_commit=True) db.execute(''' if exists (select name from sysobjects where name = "sp_test_leak") begin drop procedure sp_test_leak end ''') db.execute(''' create procedure sp_test_leak @arg int as select @arg ''') for i in range(200): for j in range(1000): c = db.cursor() c.callproc('sp_test_leak', {'@arg': 12345 }) sys.stdout.write('%3d\r' % i) sys.stdout.flush() sys.stdout.write('\n') - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - If someone is able to modify this and come up with a leaking result I am interested in working on the fix. You can build for FreeTDS like this: python setup.py build_ext -D HAVE_FREETDS -U WANT_BULKCOPY python setup.py install The module is available here: http://www.object-craft.com.au/projects/sybase/download/sybase-0.37pre1.tar.gz The module home page is here: http://www.object-craft.com.au/projects/sybase/ CHANGES SINCE 0.36: * Cursor state initialisation fix from Skip Montanaro. * Callback declaration fix on Windows from Vadim Beloborodov. * Cursor output parameters now work when parameters are passed as a sequence. * Output parameters now work for FreeTDS 0.62.4. 1> create procedure sp_test_output 2> @num int, @result int output 3> as 4> select @result = @num 5> go params = c.callproc('sp_test_output', {'@num': 12345, '@result': Sybase.OUTPUT(1)}) print params['@result'] * The CS_STATUS_RESULT result set is now consumed internally in the Cursor and does not appear in the result sets consumed by the fetch and nextset methods. The return value from the stored procedure is available in the .return_status member of the Cursor. It will only contain a meaningful value once all of the row result sets have been consumed. Note that this does not work with FreeTDS 0.62.4. The return_status seems to always be 0. Research shows that the problem is probably in the CT emulation layer as tsql displays the correct value, but sqsh displays 0. * Output hook patch from Ty Sarna has been applied. * Applied patch from Andre Sedinin to improve error handling. * Improved detection of SYBASE_OCS. - Dave -- http://www.object-craft.com.au From djc at object-craft.com.au Mon Mar 21 00:55:07 2005 From: djc at object-craft.com.au (Dave Cole) Date: Mon Mar 21 00:54:09 2005 Subject: [DB-SIG] Sybase module 0.37pre2 released In-Reply-To: <423E0418.6080405@object-craft.com.au> References: <423E0418.6080405@object-craft.com.au> Message-ID: <423E0D5B.70004@object-craft.com.au> Dave Cole wrote: > The module is available here: > > > http://www.object-craft.com.au/projects/sybase/download/sybase-0.37pre1.tar.gz Ooops. Make that: http://www.object-craft.com.au/projects/sybase/download/sybase-0.37pre2.tar.gz - Dave -- http://www.object-craft.com.au From skip at pobox.com Sat Mar 19 23:59:34 2005 From: skip at pobox.com (Skip Montanaro) Date: Tue Mar 22 06:20:55 2005 Subject: [DB-SIG] Re: [python-sybase] Sybase module 0.37pre1 released In-Reply-To: <423BA3B1.50408@object-craft.com.au> References: <423BA3B1.50408@object-craft.com.au> Message-ID: <16956.44758.566302.13875@montanaro.dyndns.org> Dave> This release contains a number of small bugfixes and patches Dave> received from users. ... I don't see this change: *** /Users/skip/src/sybase-0.36/Sybase.py Sun Apr 27 05:54:35 2003 --- /Users/skip/tmp/Sybase.py Sat Mar 19 16:46:17 2005 *************** *** 436,441 **** --- 471,477 ---- def start(self, arraysize): self._arraysize = arraysize + self._set_state(_LAZY_FETCHING) status = self._cmd.ct_send() if status != CS_SUCCEED: self._raise_error(Error, 'ct_send') You created that in response to some problems one of my users was having. I also don't see anything related to Python's builtin datetime module. I can whip up a patch based upon my local changes if you like. mx.DateTime is nice, but overkill for most things and represents yet another thing that needs to be installed. Finally, way back when I first started using the Sybase module I added a converters arg to the Connection class. I realize that's not necessarily everybody's idea of the best way to implement type converters, but it allows me to pretty transparently use whatever date/time module I have available. (Some users at work have old code that expects times to be floats in epoch seconds as returned by time.time(). This allows them to easily use that with minimal changes to their code.) -- Skip Montanaro skip@pobox.com From python at venix.com Sat Mar 26 19:17:36 2005 From: python at venix.com (Lloyd Kvam) Date: Sat Mar 26 19:17:39 2005 Subject: [DB-SIG] Pycon2005 and database divide (ODB vs relational DB) Message-ID: <1111861055.4268.57.camel@laptop.venix.com> I was not present at the Pycon presentation that triggered some heated discussion about the relative merits of an Object Data Base vs a Relational Data Base, but I did see some followup discussions and there were some emails. http://mail.python.org/pipermail/pycon2005-attendees/2005-March/000054.html http://mail.python.org/pipermail/pycon2005-attendees/2005-March/000061.html Personally, I use relational databases because of the set theory that underpins them and because that's what I know (old dogs do old tricks). The discussions that I've seen on this list generally deal with the nuts and bolts of using relational databases. I've viewed Zope's ability to take a bundle of html and programming logic and save it as a persistent URL in ZODB as a cool trick, but not terribly relevant to the way I write applications. (Yes, I know that is selling Zope short, but I think it is still true enough.) I have persistent program logic (.py files) and persistent data in SQL databases plus configuration files and presentation templates that help tie the pieces together. Has the world changed? Is it now easy to apply set theory predicates (or some alternative theory) to ODB data? Should "normal" applications be managing "objects" rather than "data"? I'm posting this to db-sig since that seems like the best forum for a discussion. -- Lloyd Kvam Venix Corp From pobrien at orbtech.com Sat Mar 26 23:27:44 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Sat Mar 26 23:28:01 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <1111861055.4268.57.camel@laptop.venix.com> References: <1111861055.4268.57.camel@laptop.venix.com> Message-ID: <4245E1E0.5060403@orbtech.com> Lloyd Kvam wrote: [snip] > > Has the world changed? Is it now easy to apply set theory predicates > (or some alternative theory) to ODB data? Should "normal" applications > be managing "objects" rather than "data"? Yes. Start here --> http://orbtech.com/blog/schevo/pycon2005 -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From peter at monicol.co.uk Sun Mar 27 00:30:18 2005 From: peter at monicol.co.uk (Peter Mott) Date: Sun Mar 27 00:32:25 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <4245E1E0.5060403@orbtech.com> Message-ID: I always understood that the weakness of Object Databases was that they provided no general ad hoc query language like SQL (or QUEL) though there was no a priori reason why they couldn't. The way relational databases were grounded in 1st order predicate logic (FOPC) seemed to give them an advantage which Object Databases despite announcements and languages to the contrary never quite manageed to overcome. I do know that the Relational Model is based on the logic of Frege (1879), the predicate logic, while the object model harks back to the IS_A hierarchies underpinning the logic of Aristotle (300 BC). I checked out the orbtech URL but found nothing about languages, logic or even taxonomies. So give us more evidence that the world has changed! It has been long announced but has not happened. Frege rules. Peter > -----Original Message----- > From: db-sig-bounces@python.org [mailto:db-sig-bounces@python.org] On > Behalf Of Patrick K. O'Brien > Sent: 26 March 2005 22:28 > To: Lloyd Kvam > Cc: 'db-sig@python.org'; pycon2005-attendees@python.org > Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide > (ODB vs relational DB) > > Lloyd Kvam wrote: > [snip] > > > > Has the world changed? Is it now easy to apply set theory predicates > > (or some alternative theory) to ODB data? Should "normal" applications > > be managing "objects" rather than "data"? > > Yes. Start here --> http://orbtech.com/blog/schevo/pycon2005 > > -- > Patrick K. O'Brien > Orbtech http://www.orbtech.com > Schevo http://www.schevo.org > Pypersyst http://www.pypersyst.org > > _______________________________________________ > DB-SIG maillist - DB-SIG@python.org > http://mail.python.org/mailman/listinfo/db-sig From pobrien at orbtech.com Sun Mar 27 00:41:57 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Sun Mar 27 00:42:09 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: References: Message-ID: <4245F345.4090804@orbtech.com> Peter Mott wrote: > I always understood that the weakness of Object Databases was that they > provided no general ad hoc query language like SQL (or QUEL) though there > was no a priori reason why they couldn't. The way relational databases were > grounded in 1st order predicate logic (FOPC) seemed to give them an > advantage which Object Databases despite announcements and languages to the > contrary never quite manageed to overcome. I do know that the Relational > Model is based on the logic of Frege (1879), the predicate logic, while the > object model harks back to the IS_A hierarchies underpinning the logic of > Aristotle (300 BC). I checked out the orbtech URL but found nothing about > languages, logic or even taxonomies. So give us more evidence that the world > has changed! It has been long announced but has not happened. Frege rules. The proof is in the pudding. There is much misinformation in the world and especially in the academic publications on this topic. I don't have the time or interest in correcting that. I want to build tools that work. Other people can follow up with the theory about why they work. I've already read too many theoretical papers that said what I've already done with Schevo couldn't be done. Never piss off an Irishman by telling him something can't be done. He might just go and do it anyway. ;-) -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From pobrien at orbtech.com Sun Mar 27 00:44:51 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Sun Mar 27 00:45:02 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: References: Message-ID: <4245F3F3.4000209@orbtech.com> Peter Mott wrote: > I always understood that the weakness of Object Databases was that they > provided no general ad hoc query language like SQL (or QUEL) though there > was no a priori reason why they couldn't. The way relational databases were > grounded in 1st order predicate logic (FOPC) seemed to give them an > advantage which Object Databases despite announcements and languages to the > contrary never quite manageed to overcome. I do know that the Relational > Model is based on the logic of Frege (1879), the predicate logic, while the > object model harks back to the IS_A hierarchies underpinning the logic of > Aristotle (300 BC). I checked out the orbtech URL but found nothing about > languages, logic or even taxonomies. So give us more evidence that the world > has changed! It has been long announced but has not happened. Frege rules. P.S. Aristotle is overrated. I've always been more of a General Semantics (Korzybski) kind of guy myself. But then, philosophy is also overrated. So I've become more of a "do your own thing" kind of guy. -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From python at venix.com Sun Mar 27 01:12:39 2005 From: python at venix.com (Lloyd Kvam) Date: Sun Mar 27 01:12:43 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Re: The Database Divide (another country heard from :) In-Reply-To: <4245EC79.5020205@comcast.net> References: <032520051347.27680.4244165A000AAC3C00006C2022070029530E9D0E030E0CD203D202080106@comcast.net> <42443263.8080701@orbtech.com> <20050325155022.GA28851@rogue.amk.ca> <42457717.6080103@orbtech.com> <4245EC79.5020205@comcast.net> Message-ID: <1111882359.4268.137.camel@laptop.venix.com> I'm simply passing this on to db-sig since they were not part of the recipient list. On Sat, 2005-03-26 at 18:12, Stephen Waterbury wrote: > Patrick K. O'Brien wrote: > > ... I thought I'd just follow up with a > > link to a companion paper on Schevo that covers more than I was able to > > in my presentation and includes lots of pretty screen shots and more > > detailed explanations of Schevo concepts: > > > > http://schevo.org/documentation/reference/current/ > > Thanks very much for this, Patrick! I was at PyCon but unfortunately > missed your talk and (maybe not so unfortunately ;) all the relational > vs. odb hooha, except for the thread on pycon2005-attendees, which > has been quite interesting. > > Schevo (is that pronounced "skeevo", with a hard "ch", as in > "schema"?) is interesting, but for me the "REST API for Schevo" > is even more interesting! > > I have been busy implementing an O-R API on top of PostgreSQL, > and it's similar to Schevo's. Like your current Schevo work, > I'm using Twisted's web module -- but for xml-rpc. I plan to > implement PB and/or "JUCE" (sp?) service(s) over the same > underlying logical interface layer RSN. I use the Twisted > adbapi interface to PostgreSQL, but nothing else from > twisted.enterprise. > > I'm going to see if I can implement a Schevo-REST interface > for my repository app. I may even be able to reuse some of > your Python code -- I use ElementTree for all my XML work. > (Thanks, Fredrik! ... for the thousandth time. :) > > IMO Schevo-REST is a good candidate for a standard REST > "Repository API". (I use "repository" to mean a generalized > persistence service, which could have any kind of backend.) > > Such a standard could enable user interface reuse, but also > inter-repository communications and some forms of federation > among possibly heterogeneous repositories. > > The latter in particular would likely be of interest to your > "real-world" Navy customers, among others. DoD customers > typically have zillions of legacy databases (well, 2 or 3 at > least!) that they need to glue together in various ways and > this could provide an elegant wrapping/gluing technique. > The same applies for any large manufacturing outfit -- a.k.a. > "OEMs". NASA included. ;) > > In preface to the next paragraph, I'll emphasize that I am > *not* a big semantic web (SW) nor UML maven. I'm interested in > using SW techniques within controlled domains for integration and > interoperability and (down the road) inferencing and "AI"-type > capabilities. As to UML, some of my customers want to use UML > tools and extend them in some ... uhhh ... interesting ways. > > All through your docs of Schevo are concepts of domain classes, > relationships, and such. These well-established concepts occur > in several industry standards. I am developing interfaces to > import/export OWL/RDF, UML/XMI, and STEP (ISO 10303) for my app, > mainly to enable interoperability with other tools and apps that > implement them. > > One of my points in bringing that up is that Schevo's REST API > document resonates with some high-level SW concepts and RDF/OWL. > I take a naive and minimalist approach to such standards (partly > because I am trying to deal with so many, so I can't afford > to get lost in the details/warts), and IMO the documentation of > a standard REST repository API could actually benefit by > referring to some SW concepts. > > A simple mapping of some Schevo REST API elements to > OWL/SW/XSD: > > domain ........... 'owl:ontology' (~ schema, in db parlance) > domain name ...... ~ xsd:ns (or prefix, like 'owl', above) > collection ....... a local population of instances of a Class > supplier ......... a Class instance > action/execute ... (no concept in SW ... thank goodness! but > OWL-S [services] is coming ... urg ;) > > For one thing, I like the idea of giving a domain > ontology a URI and a prefix. A globally unique identifier > enables intelligent publishing (the URI doesn't have to be > a URL, but it's nice if it is ;), discovery, importing, etc. > > As Tim Peters famously observed: "Namespaces are one > honking great idea -- let's do more of those!" > > I'm working on an OWL import/export interface for my app's > "meta-repository", as one way of publishing its ontology, > importing/exporting and integrating with related ontologies, > etc. I'll try to make it as independent of the rest of my > app as possible, in case anyone is interested in using it. > > A Python ontology module might also be useful. I have some > of the classes, but they are a bit too tightly coupled to > my app right now -- I need to de-couple them more anyway, so > if anyone thinks it would be useful, I could release it. I'm > using RDFLib for the import/export (an exception to my use of > ElementTree for XML -- the RDF/XML spec makes me nauseous ;). > > Interesting that your example uses the "oid" attribute for > "suppliers" in exactly the same way that STEP (ISO 10303) > identifies (and cross-references) instances within a STEP > file (ISO 10303-21): integers -- a simple, locally unique > identifier. > > BTW, as an indication of just how similar our app domains are, > the "organization" class in my app's core ontology mirrors the > structure of "Commercial And Government Entity" (CAGE), which > is a concept I'd bet money you use in your Navy logistics-related > app. ;) If you're interested, I'll send you the ontology, which > I'll be releasing publicly Real Soon Now, anyway. > > Sorry it got so long! Let me know if any of this sounds > interesting. > > Cheers, > Steve > > _______________________________________________ > Pycon2005-attendees mailing list > Pycon2005-attendees@python.org > http://mail.python.org/mailman/listinfo/pycon2005-attendees -- Lloyd Kvam Venix Corp. 1 Court Street, Suite 378 Lebanon, NH 03766-1358 voice: 603-653-8139 fax: 320-210-3409 -- Lloyd Kvam Venix Corp From ed at leafe.com Sun Mar 27 04:46:33 2005 From: ed at leafe.com (Ed Leafe) Date: Sun Mar 27 09:43:20 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <4245F345.4090804@orbtech.com> References: <4245F345.4090804@orbtech.com> Message-ID: On Mar 26, 2005, at 6:41 PM, Patrick K. O'Brien wrote: > Never piss off an Irishman by telling him something can't be done. He > might just go and do it anyway. ;-) Bah! You'll never be able to send me a million dollars! ;-) ___/ / __/ / ____/ Ed Leafe http://leafe.com/ http://dabodev.com/ From john.m.camara at comcast.net Sun Mar 27 05:25:30 2005 From: john.m.camara at comcast.net (john.m.camara@comcast.net) Date: Sun Mar 27 09:43:21 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) Message-ID: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> Do all databases need ad hoc query languages? Oh, I can already feel the heat is on its way before I even get the words typed out. IMHO, saying or believing a database needs an ad hoc query language is equivalent to saying or believing a language needs static types. All Relational Databases lovers should now take a couple of slooooow deep breaths and cool down. There are quite a few object oriented applications that require only querying over a relatively few fields and thus does not need the flexibility that SQL provides. If such an application uses a Relational Database it will come with some costs. - Query are slower - SQL Queries require complex parsing before data is retrieved - Results need to be transformed to objects (takes more code and processor time) - Refactoring the Application is harder and more error prone as changes need to be done in many places (object model, relational mapping, database, etc). Just like the good old procedural days when you had to remember to make changes scattered through out the code base. - Storage requirements may be higher if the tables become sparsely populated. - If tables are sparsely populated the code for the application will likely have many if/else conditions. - Good object oriented design is likely to be compromised to make working with a relational database easier. - Applications with high volumes of data may have to resort to back dooring the data into the database. If you every find yourself in this situation you definitely choose the wrong type of database. As I stated in previous posts, I'm not against Relational Databases. There is no one type of database that is a silver bullet. They all have their strengths and weaknesses and for the for-seeable future I will continue using the three types. In general I use Relational when I need an ad hoc query language, Embedded when I need to save complex state information, and object orient for the remaining. In some applications I may use 2 or even all 3 types. In some applications where it may be ideal to use 2 or more types I some times make a compromise and use one type if one particular type meets the majority of the application needs well. I also make the compromise for applications that have very simple database needs. John > Peter Mott wrote: > > I always understood that the weakness of Object Databases was that they > > provided no general ad hoc query language like SQL (or QUEL) though there > > was no a priori reason why they couldn't. The way relational databases were > > grounded in 1st order predicate logic (FOPC) seemed to give them an > > advantage which Object Databases despite announcements and languages to the > > contrary never quite manageed to overcome. I do know that the Relational > > Model is based on the logic of Frege (1879), the predicate logic, while the > > object model harks back to the IS_A hierarchies underpinning the logic of > > Aristotle (300 BC). I checked out the orbtech URL but found nothing about > > languages, logic or even taxonomies. So give us more evidence that the world > > has changed! It has been long announced but has not happened. Frege rules. > > The proof is in the pudding. There is much misinformation in the world > and especially in the academic publications on this topic. I don't have > the time or interest in correcting that. I want to build tools that > work. Other people can follow up with the theory about why they work. > I've already read too many theoretical papers that said what I've > already done with Schevo couldn't be done. Never piss off an Irishman > by telling him something can't be done. He might just go and do it > anyway. ;-) > > -- > Patrick K. O'Brien > Orbtech http://www.orbtech.com > Schevo http://www.schevo.org > Pypersyst http://www.pypersyst.org > > _______________________________________________ > Pycon2005-attendees mailing list > Pycon2005-attendees@python.org > http://mail.python.org/mailman/listinfo/pycon2005-attendees From bob at redivi.com Sun Mar 27 06:04:52 2005 From: bob at redivi.com (Bob Ippolito) Date: Sun Mar 27 09:43:21 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> References: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> Message-ID: <1C600F60-24D0-4615-ABF0-7B7E4ACB6426@redivi.com> On Mar 26, 2005, at 10:25 PM, john.m.camara@comcast.net wrote: > Do all databases need ad hoc query languages? > > Oh, I can already feel the heat is on its way before I even get the > words typed out. > > IMHO, saying or believing a database needs an ad hoc query language > is equivalent to saying or believing a language needs static types. > > All Relational Databases lovers should now take a couple of > slooooow deep breaths and cool down. > > There are quite a few object oriented applications that require > only querying over a relatively few fields and thus does not need > the flexibility that SQL provides. If such an application uses a > Relational Database it will come with some costs. > > - Query are slower - SQL Queries require complex parsing before > data is retrieved > - Results need to be transformed to objects (takes more code and > processor time) > - Refactoring the Application is harder and more error prone as > changes need to be done in many places (object model, relational > mapping, database, etc). Just like the good old procedural days > when you had to remember to make changes scattered through out the > code base. > - Storage requirements may be higher if the tables become sparsely > populated. > - If tables are sparsely populated the code for the application > will likely have many if/else conditions. > - Good object oriented design is likely to be compromised to make > working with a relational database easier. > - Applications with high volumes of data may have to resort to back > dooring the data into the database. If you every find yourself in > this situation you definitely choose the wrong type of database. > > As I stated in previous posts, I'm not against Relational > Databases. There is no one type of database that is a silver > bullet. They all have their strengths and weaknesses and for the > for-seeable future I will continue using the three types. In > general I use Relational when I need an ad hoc query language, > Embedded when I need to save complex state information, and object > orient for the remaining. In some applications I may use 2 or even > all 3 types. In some applications where it may be ideal to use 2 > or more types I some times make a compromise and use one type if > one particular type meets the majority of the application needs > well. I also make the compromise for applications that have very > simple database needs. MonetDB is worth looking at if you're interested in a hybrid model that has the relational database API when you want it, and something lower level and more efficient when you don't. It also supports both the embedded and client/server model, and has some pretty impressive technology behind the scenes. I haven't done more than play around with it, so I can't personally vouch for it in a production environment, but I sure want to at some point. (They also support Python out of the box) -bob From ianb at colorstudy.com Sun Mar 27 10:43:43 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun Mar 27 10:43:27 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> References: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> Message-ID: <4246723F.7000506@colorstudy.com> john.m.camara@comcast.net wrote: > Do all databases need ad hoc query languages? A single language, no, but I do think they all need good support for ad hoc queries. If I have to change my persistent data model to efficiently and consistently query the objects (or tuples) in a new way, then that model is too fragile. As for languages -- most ad hoc queries could be expressed as list comprehension (maybe evil nested list comprehensions), but that's hopelessly inefficient so it wouldn't really do, which is really to say that the imperative approach to queries doesn't work. Python doesn't have great support for creating queries without executing them -- you can kind of do it, and it's not terribly hard, but it's kind of a hack of the Python syntax and a bit confusing. This probably leads to a "language" -- but in another programming language (like Lisp) a query might not seem so unnatural and might not even be seen as a separate language at all. > Oh, I can already feel the heat is on its way before I even get the > words typed out. > > IMHO, saying or believing a database needs an ad hoc query language > is equivalent to saying or believing a language needs static types. Perhaps so, but that argument goes both ways. People use static typing as a technique to control complexity (and static proponents say that dynamic typing is only appropriate for simple projects). But static typing is one technique among many to control complexity, and if we can leverage dynamic typing to control complexity in other ways then it could be a better way to approach problems. Ad hoc queries are a kind of future protection, so that as the way you use data changes you can facilitate that efficiently without changing your data model. Maybe you can manage a way to change your data model with less cost than in a relational database... but that's a pretty hard problem. But to argue that this kind of future protection isn't necessary for all projects is only to say that object databases are only useful for small projects or disposable data... which probably isn't your argument. > All Relational Databases lovers should now take a couple of slooooow > deep breaths and cool down. > > There are quite a few object oriented applications that require only > querying over a relatively few fields and thus does not need the > flexibility that SQL provides. If such an application uses a > Relational Database it will come with some costs. > > - Query are slower Sometimes, though of course other queries are faster because of the abstract and well optimized query engine. > - SQL Queries require complex parsing before data > is retrieved A flaw of SQL, not an RDBMS. > - Results need to be transformed to objects (takes more > code and processor time) True, but this will be true of any language-neutral database. > - Refactoring the Application is harder and > more error prone as changes need to be done in many places (object > model, relational mapping, database, etc). Just like the good old > procedural days when you had to remember to make changes scattered > through out the code base. Hmm... this I don't understand. Ad hoc queries should *isolate* changes to the places where you use specific queries. They are not in themselves a method of abstraction, but you can still achieve that abstraction on top of those queries. Efficiently implementing a new type of query without the relational mechanisms itself can lead to many more significant changes to your code. > - Good object oriented design is likely to be compromised > to make working with a relational database easier. Eh... maybe that's a feature to me ;) I'm not a fan of OO design, insofar as design being good because it is OO. Typically OO has a place in a good design, but I seldom think it's right to start with ideas like classes and inheritance when considering design. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From pobrien at orbtech.com Sun Mar 27 15:53:12 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Sun Mar 27 15:53:25 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: References: <4245F345.4090804@orbtech.com> Message-ID: <4246BAC8.308@orbtech.com> Ed Leafe wrote: > On Mar 26, 2005, at 6:41 PM, Patrick K. O'Brien wrote: > >> Never piss off an Irishman by telling him something can't be done. He >> might just go and do it anyway. ;-) > > > Bah! You'll never be able to send me a million dollars! Grrr. Don't... make... me... angry!!! ;-) -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From pobrien at orbtech.com Sun Mar 27 16:00:48 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Sun Mar 27 16:01:01 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <1C600F60-24D0-4615-ABF0-7B7E4ACB6426@redivi.com> References: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> <1C600F60-24D0-4615-ABF0-7B7E4ACB6426@redivi.com> Message-ID: <4246BC90.4080209@orbtech.com> Bob Ippolito wrote: > > MonetDB is worth looking at if you're > interested in a hybrid model that has the relational database API when > you want it, and something lower level and more efficient when you > don't. It also supports both the embedded and client/server model, and > has some pretty impressive technology behind the scenes. > > I haven't done more than play around with it, so I can't personally > vouch for it in a production environment, but I sure want to at some > point. > > (They also support Python out of the box) Bob, could you say a little more about what you like about MonetDB and what kind of support they have for Python? If you could compare it to any other DBMS or ODBMS that would be great too. I'm just curious to hear more from someone who has actually tried it. Thanks. -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From pobrien at orbtech.com Sun Mar 27 16:24:48 2005 From: pobrien at orbtech.com (Patrick K. O'Brien) Date: Sun Mar 27 16:25:02 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <4246723F.7000506@colorstudy.com> References: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> <4246723F.7000506@colorstudy.com> Message-ID: <4246C230.5050906@orbtech.com> Ian Bicking wrote: > john.m.camara@comcast.net wrote: > >> Do all databases need ad hoc query languages? > > > A single language, no, but I do think they all need good support for ad > hoc queries. If I have to change my persistent data model to > efficiently and consistently query the objects (or tuples) in a new way, > then that model is too fragile. One of the real conceptual challenges here is that queries in the relational model are quite different from queries of objects. In relational, you have "dumb" data that you can pick apart and recombine to form new sets of "dumb" data. So a flexible language like SQL makes sense. With objects, I find it awkward to think about queries this way. Schevo does have built-in methods that make it easy to find subsets of objects, and those methods have some simple optimizations they apply based on behind-the-scenes indexes that Schevo maintains. And Schevo automatically maintains bi-directional relationships between all parent-child relationships and exposes access to this information in a very simple and consistent API. The combination of these two features along with Python list comprehensions turns out to be extremely powerful and flexible and works great for application code. If I had to support completely ad-hoc end-user querying then I would have someone write an ODBC driver so that a Schevo database looked like a relational database to any ODBC client tool, like Access or Crystal Reports. Creating this ODBC driver is beyond my skill level as it will involve a lot of nasty C code, but conceptually I'm confident it could be done, since we can already easily export SQL from any Schevo database. -- Patrick K. O'Brien Orbtech http://www.orbtech.com Schevo http://www.schevo.org Pypersyst http://www.pypersyst.org From bob at redivi.com Sun Mar 27 20:18:01 2005 From: bob at redivi.com (Bob Ippolito) Date: Mon Mar 28 00:46:40 2005 Subject: [DB-SIG] Re: [Pycon2005-attendees] Pycon2005 and database divide (ODB vs relational DB) In-Reply-To: <4246BC90.4080209@orbtech.com> References: <032720050325.6010.424627AA0007C3240000177A22058860140E9D0E030E0CD203D202080106@comcast.net> <1C600F60-24D0-4615-ABF0-7B7E4ACB6426@redivi.com> <4246BC90.4080209@orbtech.com> Message-ID: On Mar 27, 2005, at 9:00 AM, Patrick K. O'Brien wrote: > Bob Ippolito wrote: >> >> MonetDB is worth looking at if you're >> interested in a hybrid model that has the relational database API >> when >> you want it, and something lower level and more efficient when you >> don't. It also supports both the embedded and client/server model, >> and >> has some pretty impressive technology behind the scenes. >> >> I haven't done more than play around with it, so I can't personally >> vouch for it in a production environment, but I sure want to at some >> point. >> >> (They also support Python out of the box) > > Bob, could you say a little more about what you like about MonetDB and > what kind of support they have for Python? If you could compare it to > any other DBMS or ODBMS that would be great too. I'm just curious to > hear more from someone who has actually tried it. Thanks. I don't feel that I've used it enough yet to be able to give a really intelligent comparison at this time, but they do have a lot of information on their site, and I suggest reading that for now. -bob From carsten at uniqsys.com Tue Mar 29 00:10:45 2005 From: carsten at uniqsys.com (Carsten Haese) Date: Tue Mar 29 00:11:06 2005 Subject: [DB-SIG] Looking for Stephen Turner, maintainer of informixdb Message-ID: <1112047844.31465.108.camel@dot.uniqsys.com> Hello everybody: I have discovered that the functionality for connecting Python to an Informix database is currently in a frustrating state of neglect. The link to Kinfxdb is dead, and informixdb doesn't build on Python 2. I couldn't find any usable workarounds to the build problem, so I worked out successfully how to build informixdb using distutils. Now I'd like to share the result with the community, but the maintainer appears to have gone missing. My emails to sjturner@ix.netcom.com and stepturn@mediaone.net have bounced back undeliverable, so now I'm posting to the lists trying to locate Stephen. If anybody has any pointers for locating Stephen Turner, please let me know. If Stephen can't be located, I'd be willing to take over the project, but I'd prefer the torch be given to me rather than me just taking it. Thanks, -- Carsten Haese - Software Engineer | Phone: (419) 861-3331 Unique Systems, Inc. | FAX: (419) 861-3340 1446 Reynolds Rd, Suite 313 | Maumee, OH 43537 | mailto:carsten@uniqsys.com From brunson at brunson.com Wed Mar 30 20:04:50 2005 From: brunson at brunson.com (Eric Brunson) Date: Wed Mar 30 20:05:11 2005 Subject: [DB-SIG] Looking for Stephen Turner, maintainer of informixdb In-Reply-To: <1112047844.31465.108.camel@dot.uniqsys.com> References: <1112047844.31465.108.camel@dot.uniqsys.com> Message-ID: <424AEA42.5060305@brunson.com> Read the license. If he's released it under GPL or BSD, then you could, in all good faith, release a fork of the code until he surfaces. Carsten Haese wrote: >Hello everybody: > >I have discovered that the functionality for connecting Python to an >Informix database is currently in a frustrating state of neglect. The >link to Kinfxdb is dead, and informixdb doesn't build on Python 2. I >couldn't find any usable workarounds to the build problem, so I worked >out successfully how to build informixdb using distutils. > >Now I'd like to share the result with the community, but the maintainer >appears to have gone missing. My emails to sjturner@ix.netcom.com and >stepturn@mediaone.net have bounced back undeliverable, so now I'm >posting to the lists trying to locate Stephen. > >If anybody has any pointers for locating Stephen Turner, please let me >know. If Stephen can't be located, I'd be willing to take over the >project, but I'd prefer the torch be given to me rather than me just >taking it. > >Thanks, > > > From ianb at colorstudy.com Wed Mar 30 21:41:25 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed Mar 30 21:42:16 2005 Subject: [DB-SIG] URI syntax for databases Message-ID: <424B00E5.7080700@colorstudy.com> What do people think about standardizing on a URI syntax for connecting to databases? It makes it much easier to configure databases from my experience, and helps unify the different signatures of connect(). In SQLObject we've been using: module_name://user:password@hostname:port/database_name?other_parameters except database names instead of module names, like firebird: instead of kinterbasdb:, and pgsql: instead of psycopg:, but maybe that could change. Or modules could provide aliases... though that would make it harder, since it's harder to detect aliases. Maybe the URI could be translated to: mod = __import__(uri.split(':')[0]) conn = mod.uri_connection(uri) Some databases are different, like sqlite:/path_to_db_file, but they pretty much all fit into the pattern. SQLite in-memory databases are weird, since you connect to ':memory:', and there's all the Windows-path-as-URI issues, but this is exactly the sort of thing where a small spec could give consistency. Also parameters end up all being strings, so you have to coerce them if you want booleans or some other value -- usually this is fairly easy to figure out, but it should be moved to the URI translation. It also occurs to me that it would be nice to parse the connection string without actually making a connection -- i.e., produce (factory, args, kwargs), like (psycopg.connect, ('dsn=...',), {}). My particular desire here is I want a database backend for a web sessions, and that database has to be configurable, and a URI is the nicest way to configure it. But I want to write it to the DB-API, not SQLObject, so pushing that standard up the stack would be nice. I could factor this out of SQLObject right now, but ultimately it would be better if database drivers included the parsing code in their own packages. Thoughts? -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From mal at egenix.com Wed Mar 30 22:34:08 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Mar 30 22:34:14 2005 Subject: [DB-SIG] URI syntax for databases In-Reply-To: <424B00E5.7080700@colorstudy.com> References: <424B00E5.7080700@colorstudy.com> Message-ID: <424B0D40.6010703@egenix.com> Ian Bicking wrote: > What do people think about standardizing on a URI syntax for connecting > to databases? It makes it much easier to configure databases from my > experience, and helps unify the different signatures of connect(). In > SQLObject we've been using: > > module_name://user:password@hostname:port/database_name?other_parameters > > except database names instead of module names, like firebird: instead of > kinterbasdb:, and pgsql: instead of psycopg:, but maybe that could > change. Or modules could provide aliases... though that would make it > harder, since it's harder to detect aliases. Maybe the URI could be > translated to: > > mod = __import__(uri.split(':')[0]) > conn = mod.uri_connection(uri) > > Some databases are different, like sqlite:/path_to_db_file, but they > pretty much all fit into the pattern. SQLite in-memory databases are > weird, since you connect to ':memory:', and there's all the > Windows-path-as-URI issues, but this is exactly the sort of thing where > a small spec could give consistency. > > Also parameters end up all being strings, so you have to coerce them if > you want booleans or some other value -- usually this is fairly easy to > figure out, but it should be moved to the URI translation. > > It also occurs to me that it would be nice to parse the connection > string without actually making a connection -- i.e., produce (factory, > args, kwargs), like (psycopg.connect, ('dsn=...',), {}). > > > My particular desire here is I want a database backend for a web > sessions, and that database has to be configurable, and a URI is the > nicest way to configure it. But I want to write it to the DB-API, not > SQLObject, so pushing that standard up the stack would be nice. I could > factor this out of SQLObject right now, but ultimately it would be > better if database drivers included the parsing code in their own packages. > > Thoughts? This doesn't strike me as very Pythonic. The DB API spec specifies a set of keyword parameters that cover most usage scenarios already: As a guideline the connection constructor parameters should be implemented as keyword parameters for more intuitive use and follow this order of parameters: dsn Data source name as string user User name as string (optional) password Password as string (optional) host Hostname (optional) database Database name (optional) E.g. a connect could look like this: connect(dsn='myhost:MYDB',user='guido',password='234$') Your syntax seems to be more geared torwards an abstract interface to databases, which is - as you say - one level above the DB API spec. It should be rather simple to write a factory function which takes your syntax and then imports the right module, translates the parameters and connects to the database. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 30 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ianb at colorstudy.com Wed Mar 30 22:40:38 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed Mar 30 22:41:54 2005 Subject: [DB-SIG] URI syntax for databases In-Reply-To: <424B0D40.6010703@egenix.com> References: <424B00E5.7080700@colorstudy.com> <424B0D40.6010703@egenix.com> Message-ID: <424B0EC6.4090203@colorstudy.com> M.-A. Lemburg wrote: > This doesn't strike me as very Pythonic. The DB API spec specifies a > set of keyword parameters that cover most usage scenarios already: > > As a guideline the connection constructor parameters should be > implemented as keyword parameters for more intuitive use and > follow this order of parameters: > > dsn Data source name as string > user User name as string (optional) > password Password as string (optional) > host Hostname (optional) > database Database name (optional) > > E.g. a connect could look like this: > > connect(dsn='myhost:MYDB',user='guido',password='234$') > > Your syntax seems to be more geared torwards an abstract > interface to databases, which is - as you say - one level > above the DB API spec. > > It should be rather simple to write a factory function which > takes your syntax and then imports the right module, translates > the parameters and connects to the database. This isn't meant to replace the current connect function, just augment it. I would expect that implementations would specifically translate URIs into a connect invocation. However, it would be useful if this was distributed with drivers, since there's no a general way to map URIs to connections for all databases. If it's not part of drivers, then I'll just make a separate library to do this so other people can use it, but it won't (at least initially) support drivers I don't use (which happens to include mxODBC among others). I have also noticed that connect functions are rather poorly documented for many drivers, so my implementation might not be accurate. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From mal at egenix.com Wed Mar 30 22:50:10 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Wed Mar 30 22:50:20 2005 Subject: [DB-SIG] URI syntax for databases In-Reply-To: <424B0EC6.4090203@colorstudy.com> References: <424B00E5.7080700@colorstudy.com> <424B0D40.6010703@egenix.com> <424B0EC6.4090203@colorstudy.com> Message-ID: <424B1102.1040409@egenix.com> Ian Bicking wrote: > M.-A. Lemburg wrote: > >> This doesn't strike me as very Pythonic. The DB API spec specifies a >> set of keyword parameters that cover most usage scenarios already: >> >> As a guideline the connection constructor parameters should be >> implemented as keyword parameters for more intuitive use and >> follow this order of parameters: >> >> dsn Data source name as string >> user User name as string (optional) >> password Password as string (optional) >> host Hostname (optional) >> database Database name (optional) >> >> E.g. a connect could look like this: >> >> connect(dsn='myhost:MYDB',user='guido',password='234$') >> >> Your syntax seems to be more geared torwards an abstract >> interface to databases, which is - as you say - one level >> above the DB API spec. >> >> It should be rather simple to write a factory function which >> takes your syntax and then imports the right module, translates >> the parameters and connects to the database. > > > This isn't meant to replace the current connect function, just augment > it. I would expect that implementations would specifically translate > URIs into a connect invocation. However, it would be useful if this was > distributed with drivers, since there's no a general way to map URIs to > connections for all databases. If it's not part of drivers, then I'll > just make a separate library to do this so other people can use it, but > it won't (at least initially) support drivers I don't use (which happens > to include mxODBC among others). > > I have also noticed that connect functions are rather poorly documented > for many drivers, so my implementation might not be accurate. What we could add an optional keyword argument uri="..." which database could then interpret according to your suggestion. This would be backwards compatible with the existing DB API. However, I don't see why the scheme name should be the name of the database module... I'd opt for "dbapi2:" as scheme - after all, that's what the protocol scheme is all about ;-) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 30 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From ianb at colorstudy.com Wed Mar 30 23:34:06 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed Mar 30 23:35:24 2005 Subject: [DB-SIG] URI syntax for databases In-Reply-To: <424B1102.1040409@egenix.com> References: <424B00E5.7080700@colorstudy.com> <424B0D40.6010703@egenix.com> <424B0EC6.4090203@colorstudy.com> <424B1102.1040409@egenix.com> Message-ID: <424B1B4E.8030607@colorstudy.com> M.-A. Lemburg wrote: > What we could add an optional keyword argument uri="..." which > database could then interpret according to your suggestion. I suppose that would be fine -- I thought it would be easier to implement it on top of connect instead of as an extension to that function. > This would be backwards compatible with the existing DB API. It's not incompatible if we add an entirely separate function, like uri_connect (or connect_uri or whatever). I don't want to mess with stuff that's already there, just add something new. > However, I don't see why the scheme name should be the name > of the database module... I'd opt for "dbapi2:" as scheme - > after all, that's what the protocol scheme is all about ;-) Well, we have to start by finding the correct module to get the connect (or connect_uri or whatever) function from. The most obvious way is to use the scheme as the module lookup, and let the module's function do the rest of the parsing. The resulting URIs look fairly URI-ish as well. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org From mal at egenix.com Thu Mar 31 10:22:29 2005 From: mal at egenix.com (M.-A. Lemburg) Date: Thu Mar 31 10:22:32 2005 Subject: [DB-SIG] URI syntax for databases In-Reply-To: <424B1B4E.8030607@colorstudy.com> References: <424B00E5.7080700@colorstudy.com> <424B0D40.6010703@egenix.com> <424B0EC6.4090203@colorstudy.com> <424B1102.1040409@egenix.com> <424B1B4E.8030607@colorstudy.com> Message-ID: <424BB345.4040009@egenix.com> Ian Bicking wrote: > M.-A. Lemburg wrote: > >> What we could add an optional keyword argument uri="..." which >> database could then interpret according to your suggestion. > > > I suppose that would be fine -- I thought it would be easier to > implement it on top of connect instead of as an extension to that function. > >> This would be backwards compatible with the existing DB API. > > > It's not incompatible if we add an entirely separate function, like > uri_connect (or connect_uri or whatever). I don't want to mess with > stuff that's already there, just add something new. > >> However, I don't see why the scheme name should be the name >> of the database module... I'd opt for "dbapi2:" as scheme - >> after all, that's what the protocol scheme is all about ;-) > > > Well, we have to start by finding the correct module to get the connect > (or connect_uri or whatever) function from. The most obvious way is to > use the scheme as the module lookup, and let the module's function do > the rest of the parsing. The resulting URIs look fairly URI-ish as well. Yes, but I thought you wanted a change to the DB API spec which specifies the interface for the modules themselves, not an abstraction ?! In the end, I think it's more efficient to implement such a function in an abstraction layer which is maintained independently from the DB-API database modules. Otherwise, you'd have to wait for all modules to implement the change and that can take a few years. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 31 2005) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! :::: From daniel.dittmar at sap.com Thu Mar 31 11:22:47 2005 From: daniel.dittmar at sap.com (Dittmar, Daniel) Date: Thu Mar 31 11:22:51 2005 Subject: [DB-SIG] URI syntax for databases Message-ID: >In the end, I think it's more efficient to implement such >a function in an abstraction layer which is maintained >independently from the DB-API database modules. Otherwise, >you'd have to wait for all modules to implement the change >and that can take a few years. Perhaps the abstraction layer should look for .uri_connect. If this function cannot be found, then a database specific translation of uri to connect () is used. Otherwise, changes to the driver specific connect would also require changes to the abstraction layer connect. An alternative would be to always use uri_connect. In addition, a set of modules is provided that implements this uri translation for older drivers. The user would then have to choose between the real module or the translation module. Daniel -- Daniel Dittmar SAP Labs Berlin daniel.dittmar@sap.com From ianb at colorstudy.com Thu Mar 31 22:48:54 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu Mar 31 22:49:45 2005 Subject: [DB-SIG] URI syntax for databases In-Reply-To: References: Message-ID: <424C6236.3060700@colorstudy.com> Dittmar, Daniel wrote: >>In the end, I think it's more efficient to implement such >>a function in an abstraction layer which is maintained >>independently from the DB-API database modules. Otherwise, >>you'd have to wait for all modules to implement the change >>and that can take a few years. > > > Perhaps the abstraction layer should look for .uri_connect. If > this function cannot be found, then a database specific translation of > uri to connect () is used. Otherwise, changes to the driver specific > connect would also require changes to the abstraction layer connect. This is probably what I'd end up doing -- obviously I can't change the code that's out there, and I want to use this relatively recently. Mostly I'm just hoping for a convention that other's can follow, until this transitional module is no longer necessary. > An alternative would be to always use uri_connect. In addition, a set of > modules is provided that implements this uri translation for older > drivers. The user would then have to choose between the real module or > the translation module. That seems awkward, but I suppose doable. That translating module would also have to include the rest of the required functions if it was to be a DB-API module. Monkey-patching in a uri_connect function seems easier, if we're going down that route. Hmm... except you can't monkey patch something in lazily -- i.e., wait until the user actually imports MySQLdb and then insert the function. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org