From mal at egenix.com Mon May 2 13:39:27 2011 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 02 May 2011 13:39:27 +0200 Subject: [DB-SIG] About LOB In-Reply-To: References: <4DB84CD5.1090807@ingres.com> Message-ID: <4DBE97EF.9040104@egenix.com> Kwon, Chan Young wrote: > Hi, > > Let me describe more... > 1. My suggestion of LOB object is kind of Locator object. It does not contain whole massive data. It just passes small piece of data when its methods are called. > 2. buffer object is not enough to handle various kinds of LOB types. For example, character LOB, binary LOB and unicode character LOB. > > In my company, this LOB class is already used as extension of DBAPI spec 2.0. > All colleagues like LOB class because it is very simple and easy to use. > So I sent this for your information. There are various ways large binary/text objects are handled by databases at the API level. Some databases provide APIs which allow reading/writing such data in chunks, others provide file descriptors which can be use used for this, yet others maintain the files outside the database and provide "locator" objects for these. Since all of these approaches use a file-like interface in one way or another, perhaps we could agree on a common DB-API extension that defines the minimum interface of such objects and a constructor that turns an existing file into an object which can then be passed to the database (much like the Binary() constructor we have for binary data). Do you have documentation for the LOB object you are using available on the web ? Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 02 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2011-06-20: EuroPython 2011, Florence, Italy 49 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ > Best wishes, > Chanyoung Kwon > > SAP R&D Center Korea > > > > -----Original Message----- > From: Chris Clark [mailto:Chris.Clark at ingres.com] > Sent: Thursday, April 28, 2011 2:05 AM > To: Kwon, Chan Young > Cc: db-sig at python.org > Subject: Re: [DB-SIG] About LOB > > Kwon, Chan Young wrote: >> I am implementing new DB API for new Database according to DB API 2.0. (as C-Extension) >> There is no specification about LOB in DB API 2.0 >> Especially, I need LOB object for piecewise LOB writing/reading after query execution. >> So I defined my own LOB object. >> I tried to make simple and easy interface for LOB. >> > > > BLarge OBject's (BLOBs) are alluded to in pep 249 BUT they really are > not detailed so I think you have spotted a weakness in the spec. CLOBs > (Character Large OBjects) are not covered at all. Locators are not > discussed but implied by references to bufferobjects. > > For BLOBs as input bind parameters check out the Binary() constructor > (case sensitive) at http://www.python.org/dev/peps/pep-0249/. > For BLOB results look for bufferobject in > http://www.python.org/dev/peps/pep-0249/ > > The spec (as I read it) really expects LOBs to be handled as one massive > block. However implementators can choose to implement locators behind > the scene so that data isn't sent across the wire unless the buffer > object is read from. > > So, model your LOB access methods on bufferobjects and you should be all > set! Marc-Andre reads/responds to the list regularly so take note of any > suggestions he has :-) > > Hope that helps get you started. > > Chris > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig From Chris.Clark at ingres.com Mon May 2 17:15:56 2011 From: Chris.Clark at ingres.com (Chris Clark) Date: Mon, 02 May 2011 08:15:56 -0700 Subject: [DB-SIG] About LOB In-Reply-To: <4DBE97EF.9040104@egenix.com> References: <4DB84CD5.1090807@ingres.com> <4DBE97EF.9040104@egenix.com> Message-ID: <4DBECAAC.5050600@ingres.com> M.-A. Lemburg wrote: > Kwon, Chan Young wrote: > >> Hi, >> >> Let me describe more... >> 1. My suggestion of LOB object is kind of Locator object. It does not contain whole massive data. It just passes small piece of data when its methods are called. >> 2. buffer object is not enough to handle various kinds of LOB types. For example, character LOB, binary LOB and unicode character LOB. >> >> In my company, this LOB class is already used as extension of DBAPI spec 2.0. >> All colleagues like LOB class because it is very simple and easy to use. >> So I sent this for your information. >> > > There are various ways large binary/text objects are handled by > databases at the API level. > > Some databases provide APIs which allow reading/writing such > data in chunks, others provide file descriptors which can be use > used for this, yet others maintain the files outside the database > and provide "locator" objects for these. > > Since all of these approaches use a file-like interface in one > way or another, perhaps we could agree on a common DB-API extension > that defines the minimum interface of such objects and a constructor > that turns an existing file into an object which can then be passed > to the database (much like the Binary() constructor we have for > binary data). > > Do you have documentation for the LOB object you are using available > on the web ? > Excellent idea, here are Java Docs for the JDBC clob/blob locator classes: http://download.oracle.com/javase/1.4.2/docs/api/java/sql/Clob.html http://download.oracle.com/javase/1.4.2/docs/api/java/sql/Blob.html If we add this, I think we need clearly separate classes for the 2 types. In the same way Python 2.x has str/unicode and Python 3.x as byte/str. Chris From chan.young.kwon at sap.com Tue May 3 08:45:16 2011 From: chan.young.kwon at sap.com (Kwon, Chan Young) Date: Tue, 3 May 2011 08:45:16 +0200 Subject: [DB-SIG] About LOB In-Reply-To: <4DBE97EF.9040104@egenix.com> References: <4DB84CD5.1090807@ingres.com> <4DBE97EF.9040104@egenix.com> Message-ID: Hi Lemburg, There is no documentation on the web open to external access. Our new DB project is little bit confidential. I can just send you a copy of wiki page inside my company's intranet. My LOB is also file-like object. So it has just two interfaces. One is read() The other is write() Different from JDBC, this python LOB object does not need different functions for UNICODE,STRING and BINARY. Inside read/write function, there must be some routine for various input data type. In my humble opinion, LOB object?s extension should be like this. * LOB o read(data[,position]) o write(data[,position]) o find(data[,position]) Here is my DB API?s documentation inside intranet of my company. API Module Interface Attribute Description connect(parameters...) Constructor for creating a connection to the database.Returns a Connection Object. It takes a number of parameters which are database dependent. -Example * dbapi.connect(address='localhost', port=30415, user='system', password='manager') ? common usage LOB() Return LOB type object. Date(year,month,day) Return datetime type object holding a date value. Time(hour,minute,second,millisecond=0) Return datetime type object holding a time value. Timestamp(year,month,day,hour,minute,second,millisecond=0) Return datetime type object holding a date+time value. DateFromTicks(ticks) Return datetime type object holding a date value. TimeFromTicks(ticks) Return datetime type object holding a time value. TimestampFromTicks(ticks) Return datetime type object holding a date+time value. Binary(string) Return an buffer type object holding a binary string value. Exceptions Attribute Description Warning Exception raised for important warnings. Error Exception that is the base class of all other error exceptions. -error information is saved as tuple * errobject[0] ? contains error code * errobject[1] ? contains error message InterfaceError Exception raised for errors that are related to the database interface rather than the database itself. DatabaseError Exception raised for errors that are related to the database. DataError Exception raised for errors that are due to problems with the processed data like division by zero, numeric value out of range, etc. OperationError Exception raised for errors that are related to the database's operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc. IntegrityError Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails. InternalError Exception raised when the database encounters an internal error, e.g. the cursor is not valid anymore, the transaction is out of sync, etc. ProgrammingError Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. NotSupportedError Exception raised in case a method or database API was used which is not supported by the database, e.g. requesting a .rollback() on a connection that does not support transaction or has transactions turned off. Connection Object Attribute Description close() Close the cursor now (rather than whenever _del_ is called). The cursor will be unusable from this point forward; an Error (or subclass) exception will be raised if any operation is attempted with the cursor. commit() Commit any pending transactions to the database. rollback() Rollback any pending transactions. cursor() Return a new Cursor object using the connection. setautocommit(auto=True) Set auto-commit mode. getautocommit() Get auto-commit mode. cancel() Cancel the running database request that is executed on the connection. isconnected() Return True if the connection is valid(connected to DB). setclientinfo(key, value=None) Set client info. If the value is None, the key is removed. getclientinfo(key=None) Get client info. If the key is None, All key:value sets are returned. Cursor Object Attribute Description description Sequence of column's information; The information contains 7 items : (name, type_code, display_size, internal_size, precision, scale, null_ok). rowcount Updated column count. callproc(procname[,parameters]) Call a stored database procedure with the given name. close() Close the cursor now. nextset() Skip to the next result set, closing current result set. (for multiple resultsets) execute(operation[,parameters]) Prepare and execute a database operation (query or command). -Steps included 1. prepare operation(statement) 2. bind paramters 3. execute prepared-statement executemany(operation,seq_of_parameters) Prepare a database operation (query or command) and then execute it against all parameter sequences or mappings found in the sequence seq_of_parameters. fetchone(uselob=False) Fetch the next row of a query result set, returning a single sequence, or None when no more data is available. fetchmany([size=cursor.arraysize]) Fetch the next set of rows of a query result, returning a sequence of sequences (e.g. a list of tuples). An empty sequence is returned when no more rows are available. fetchall() Fetch all (remaining) rows of a query result, returning them as a sequence of sequences (e.g. a list of tuples). Note that the cursor's arraysize attribute can affect the performance of this operation. LOB Object Attribute Description read(size[,position]) Return a portion (or all) of the data in the LOB object write(data) Write the data to the LOB object close() Close the LOB object Best wishes, Chanyoung Kwon SAP R&D Center Korea -----Original Message----- From: M.-A. Lemburg [mailto:mal at egenix.com] Sent: Monday, May 02, 2011 8:39 PM To: Kwon, Chan Young Cc: Chris Clark; db-sig at python.org Subject: Re: [DB-SIG] About LOB Kwon, Chan Young wrote: > Hi, > > Let me describe more... > 1. My suggestion of LOB object is kind of Locator object. It does not contain whole massive data. It just passes small piece of data when its methods are called. > 2. buffer object is not enough to handle various kinds of LOB types. For example, character LOB, binary LOB and unicode character LOB. > > In my company, this LOB class is already used as extension of DBAPI spec 2.0. > All colleagues like LOB class because it is very simple and easy to use. > So I sent this for your information. There are various ways large binary/text objects are handled by databases at the API level. Some databases provide APIs which allow reading/writing such data in chunks, others provide file descriptors which can be use used for this, yet others maintain the files outside the database and provide "locator" objects for these. Since all of these approaches use a file-like interface in one way or another, perhaps we could agree on a common DB-API extension that defines the minimum interface of such objects and a constructor that turns an existing file into an object which can then be passed to the database (much like the Binary() constructor we have for binary data). Do you have documentation for the LOB object you are using available on the web ? Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 02 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2011-06-20: EuroPython 2011, Florence, Italy 49 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ > Best wishes, > Chanyoung Kwon > > SAP R&D Center Korea > > > > -----Original Message----- > From: Chris Clark [mailto:Chris.Clark at ingres.com] > Sent: Thursday, April 28, 2011 2:05 AM > To: Kwon, Chan Young > Cc: db-sig at python.org > Subject: Re: [DB-SIG] About LOB > > Kwon, Chan Young wrote: >> I am implementing new DB API for new Database according to DB API 2.0. (as C-Extension) >> There is no specification about LOB in DB API 2.0 >> Especially, I need LOB object for piecewise LOB writing/reading after query execution. >> So I defined my own LOB object. >> I tried to make simple and easy interface for LOB. >> > > > BLarge OBject's (BLOBs) are alluded to in pep 249 BUT they really are > not detailed so I think you have spotted a weakness in the spec. CLOBs > (Character Large OBjects) are not covered at all. Locators are not > discussed but implied by references to bufferobjects. > > For BLOBs as input bind parameters check out the Binary() constructor > (case sensitive) at http://www.python.org/dev/peps/pep-0249/. > For BLOB results look for bufferobject in > http://www.python.org/dev/peps/pep-0249/ > > The spec (as I read it) really expects LOBs to be handled as one massive > block. However implementators can choose to implement locators behind > the scene so that data isn't sent across the wire unless the buffer > object is read from. > > So, model your LOB access methods on bufferobjects and you should be all > set! Marc-Andre reads/responds to the list regularly so take note of any > suggestions he has :-) > > Hope that helps get you started. > > Chris > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig From chan.young.kwon at sap.com Tue May 3 09:09:29 2011 From: chan.young.kwon at sap.com (Kwon, Chan Young) Date: Tue, 3 May 2011 09:09:29 +0200 Subject: [DB-SIG] About LOB References: <4DB84CD5.1090807@ingres.com> <4DBE97EF.9040104@egenix.com> Message-ID: * LOB o read(size[,position]) o write(data[,position]) o find(data[,position]) Best wishes, Chanyoung Kwon SAP R&D Center Korea _____________________________________________ From: Kwon, Chan Young Sent: Tuesday, May 03, 2011 3:45 PM To: 'M.-A. Lemburg' Cc: Chris Clark; db-sig at python.org Subject: RE: [DB-SIG] About LOB Hi Lemburg, There is no documentation on the web open to external access. Our new DB project is little bit confidential. I can just send you a copy of wiki page inside my company's intranet. My LOB is also file-like object. So it has just two interfaces. One is read() The other is write() Different from JDBC, this python LOB object does not need different functions for UNICODE,STRING and BINARY. Inside read/write function, there must be some routine for various input data type. In my humble opinion, LOB object?s extension should be like this. * LOB o read(data[,position]) o write(data[,position]) o find(data[,position]) Here is my DB API?s documentation inside intranet of my company. API Module Interface Attribute Description connect(parameters...) Constructor for creating a connection to the database.Returns a Connection Object. It takes a number of parameters which are database dependent. -Example * dbapi.connect(address='localhost', port=30415, user='system', password='manager') ? common usage LOB() Return LOB type object. Date(year,month,day) Return datetime type object holding a date value. Time(hour,minute,second,millisecond=0) Return datetime type object holding a time value. Timestamp(year,month,day,hour,minute,second,millisecond=0) Return datetime type object holding a date+time value. DateFromTicks(ticks) Return datetime type object holding a date value. TimeFromTicks(ticks) Return datetime type object holding a time value. TimestampFromTicks(ticks) Return datetime type object holding a date+time value. Binary(string) Return an buffer type object holding a binary string value. Exceptions Attribute Description Warning Exception raised for important warnings. Error Exception that is the base class of all other error exceptions. -error information is saved as tuple * errobject[0] ? contains error code * errobject[1] ? contains error message InterfaceError Exception raised for errors that are related to the database interface rather than the database itself. DatabaseError Exception raised for errors that are related to the database. DataError Exception raised for errors that are due to problems with the processed data like division by zero, numeric value out of range, etc. OperationError Exception raised for errors that are related to the database's operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc. IntegrityError Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails. InternalError Exception raised when the database encounters an internal error, e.g. the cursor is not valid anymore, the transaction is out of sync, etc. ProgrammingError Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. NotSupportedError Exception raised in case a method or database API was used which is not supported by the database, e.g. requesting a .rollback() on a connection that does not support transaction or has transactions turned off. Connection Object Attribute Description close() Close the cursor now (rather than whenever _del_ is called). The cursor will be unusable from this point forward; an Error (or subclass) exception will be raised if any operation is attempted with the cursor. commit() Commit any pending transactions to the database. rollback() Rollback any pending transactions. cursor() Return a new Cursor object using the connection. setautocommit(auto=True) Set auto-commit mode. getautocommit() Get auto-commit mode. cancel() Cancel the running database request that is executed on the connection. isconnected() Return True if the connection is valid(connected to DB). setclientinfo(key, value=None) Set client info. If the value is None, the key is removed. getclientinfo(key=None) Get client info. If the key is None, All key:value sets are returned. Cursor Object Attribute Description description Sequence of column's information; The information contains 7 items : (name, type_code, display_size, internal_size, precision, scale, null_ok). rowcount Updated column count. callproc(procname[,parameters]) Call a stored database procedure with the given name. close() Close the cursor now. nextset() Skip to the next result set, closing current result set. (for multiple resultsets) execute(operation[,parameters]) Prepare and execute a database operation (query or command). -Steps included 1. prepare operation(statement) 2. bind paramters 3. execute prepared-statement executemany(operation,seq_of_parameters) Prepare a database operation (query or command) and then execute it against all parameter sequences or mappings found in the sequence seq_of_parameters. fetchone(uselob=False) Fetch the next row of a query result set, returning a single sequence, or None when no more data is available. fetchmany([size=cursor.arraysize]) Fetch the next set of rows of a query result, returning a sequence of sequences (e.g. a list of tuples). An empty sequence is returned when no more rows are available. fetchall() Fetch all (remaining) rows of a query result, returning them as a sequence of sequences (e.g. a list of tuples). Note that the cursor's arraysize attribute can affect the performance of this operation. LOB Object Attribute Description read(size[,position]) Return a portion (or all) of the data in the LOB object write(data) Write the data to the LOB object close() Close the LOB object Best wishes, Chanyoung Kwon SAP R&D Center Korea -----Original Message----- From: M.-A. Lemburg [mailto:mal at egenix.com] Sent: Monday, May 02, 2011 8:39 PM To: Kwon, Chan Young Cc: Chris Clark; db-sig at python.org Subject: Re: [DB-SIG] About LOB Kwon, Chan Young wrote: > Hi, > > Let me describe more... > 1. My suggestion of LOB object is kind of Locator object. It does not contain whole massive data. It just passes small piece of data when its methods are called. > 2. buffer object is not enough to handle various kinds of LOB types. For example, character LOB, binary LOB and unicode character LOB. > > In my company, this LOB class is already used as extension of DBAPI spec 2.0. > All colleagues like LOB class because it is very simple and easy to use. > So I sent this for your information. There are various ways large binary/text objects are handled by databases at the API level. Some databases provide APIs which allow reading/writing such data in chunks, others provide file descriptors which can be use used for this, yet others maintain the files outside the database and provide "locator" objects for these. Since all of these approaches use a file-like interface in one way or another, perhaps we could agree on a common DB-API extension that defines the minimum interface of such objects and a constructor that turns an existing file into an object which can then be passed to the database (much like the Binary() constructor we have for binary data). Do you have documentation for the LOB object you are using available on the web ? Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 02 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2011-06-20: EuroPython 2011, Florence, Italy 49 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ > Best wishes, > Chanyoung Kwon > > SAP R&D Center Korea > > > > -----Original Message----- > From: Chris Clark [mailto:Chris.Clark at ingres.com] > Sent: Thursday, April 28, 2011 2:05 AM > To: Kwon, Chan Young > Cc: db-sig at python.org > Subject: Re: [DB-SIG] About LOB > > Kwon, Chan Young wrote: >> I am implementing new DB API for new Database according to DB API 2.0. (as C-Extension) >> There is no specification about LOB in DB API 2.0 >> Especially, I need LOB object for piecewise LOB writing/reading after query execution. >> So I defined my own LOB object. >> I tried to make simple and easy interface for LOB. >> > > > BLarge OBject's (BLOBs) are alluded to in pep 249 BUT they really are > not detailed so I think you have spotted a weakness in the spec. CLOBs > (Character Large OBjects) are not covered at all. Locators are not > discussed but implied by references to bufferobjects. > > For BLOBs as input bind parameters check out the Binary() constructor > (case sensitive) at http://www.python.org/dev/peps/pep-0249/. > For BLOB results look for bufferobject in > http://www.python.org/dev/peps/pep-0249/ > > The spec (as I read it) really expects LOBs to be handled as one massive > block. However implementators can choose to implement locators behind > the scene so that data isn't sent across the wire unless the buffer > object is read from. > > So, model your LOB access methods on bufferobjects and you should be all > set! Marc-Andre reads/responds to the list regularly so take note of any > suggestions he has :-) > > Hope that helps get you started. > > Chris > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig From chan.young.kwon at sap.com Tue May 3 09:11:01 2011 From: chan.young.kwon at sap.com (Kwon, Chan Young) Date: Tue, 3 May 2011 09:11:01 +0200 Subject: [DB-SIG] About LOB In-Reply-To: References: <4DB84CD5.1090807@ingres.com> <4DBE97EF.9040104@egenix.com> Message-ID: Source code sample def test_LobInsertWithPieceWiseLOB(self): """Piece-wise LOB INSERT""" old_mode = self.conn.getautocommit() self.conn.setautocommit(False) cur = self.conn.cursor() try: cur.execute("DROP TABLE PIECEWISE_LOB_TEST") except dbapi.Error, err: pass cur.execute("CREATE ROW TABLE PIECEWISE_LOB_TEST (key int, blob BLOB, clob CLOB, nclob NCLOB)") blob = dbapi.LOB() try: blob.read() except dbapi.Error, err: print err clob = dbapi.LOB() nclob = dbapi.LOB() cur.execute("INSERT INTO PIECEWISE_LOB_TEST VALUES (?,?,?,?)", (1, blob, clob, nclob)) blob.write(data = "blob"*1024) blob.write("2blob"*512) blob.write(u"UNICODE"*32) blob.close() clob.write("clob"*1024) clob.write(buffer("binary"*32)) clob.write(u"UNICODE"*32) clob.write(None) clob.close() chanyoung = codecs.utf_8_decode("\xEC\xB0\xAC")[0] + u"YOUNG" nclob.write(u"CHANYOUNG's UNICODE") nclob.write(chanyoung*256) nclob.close() self.conn.commit() cur.execute('select key, blob, clob, nclob from PIECEWISE_LOB_TEST') r = cur.fetchone(True) # uselob == True for c in r: if isinstance(c,dbapi.LOB) : c.read(0) c.read() c.read(999999999, 1) c.read(size=10,position=1) c.read(size=4,position=1) while True: data = c.read(1000) if data is None: break else: #print data, pass else: #print c pass cur.execute('select nclob from PIECEWISE_LOB_TEST') row = cur.fetchone() self.assertEqual(u"CHANYOUNG's UNICODE" + chanyoung*256, row[0]) try: cur.execute("DROP TABLE PIECEWISE_LOB_TEST") except dbapi.Error, err: Best wishes, Chanyoung Kwon SAP R&D Center Korea _____________________________________________ From: Kwon, Chan Young Sent: Tuesday, May 03, 2011 4:09 PM To: Kwon, Chan Young; M.-A. Lemburg Cc: Chris Clark; db-sig at python.org Subject: RE: [DB-SIG] About LOB * LOB o read(size[,position]) o write(data[,position]) o find(data[,position]) Best wishes, Chanyoung Kwon SAP R&D Center Korea _____________________________________________ From: Kwon, Chan Young Sent: Tuesday, May 03, 2011 3:45 PM To: 'M.-A. Lemburg' Cc: Chris Clark; db-sig at python.org Subject: RE: [DB-SIG] About LOB Hi Lemburg, There is no documentation on the web open to external access. Our new DB project is little bit confidential. I can just send you a copy of wiki page inside my company's intranet. My LOB is also file-like object. So it has just two interfaces. One is read() The other is write() Different from JDBC, this python LOB object does not need different functions for UNICODE,STRING and BINARY. Inside read/write function, there must be some routine for various input data type. In my humble opinion, LOB object?s extension should be like this. * LOB o read(data[,position]) o write(data[,position]) o find(data[,position]) Here is my DB API?s documentation inside intranet of my company. API Module Interface Attribute Description connect(parameters...) Constructor for creating a connection to the database.Returns a Connection Object. It takes a number of parameters which are database dependent. -Example * dbapi.connect(address='localhost', port=30415, user='system', password='manager') ? common usage LOB() Return LOB type object. Date(year,month,day) Return datetime type object holding a date value. Time(hour,minute,second,millisecond=0) Return datetime type object holding a time value. Timestamp(year,month,day,hour,minute,second,millisecond=0) Return datetime type object holding a date+time value. DateFromTicks(ticks) Return datetime type object holding a date value. TimeFromTicks(ticks) Return datetime type object holding a time value. TimestampFromTicks(ticks) Return datetime type object holding a date+time value. Binary(string) Return an buffer type object holding a binary string value. Exceptions Attribute Description Warning Exception raised for important warnings. Error Exception that is the base class of all other error exceptions. -error information is saved as tuple * errobject[0] ? contains error code * errobject[1] ? contains error message InterfaceError Exception raised for errors that are related to the database interface rather than the database itself. DatabaseError Exception raised for errors that are related to the database. DataError Exception raised for errors that are due to problems with the processed data like division by zero, numeric value out of range, etc. OperationError Exception raised for errors that are related to the database's operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc. IntegrityError Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails. InternalError Exception raised when the database encounters an internal error, e.g. the cursor is not valid anymore, the transaction is out of sync, etc. ProgrammingError Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc. NotSupportedError Exception raised in case a method or database API was used which is not supported by the database, e.g. requesting a .rollback() on a connection that does not support transaction or has transactions turned off. Connection Object Attribute Description close() Close the cursor now (rather than whenever _del_ is called). The cursor will be unusable from this point forward; an Error (or subclass) exception will be raised if any operation is attempted with the cursor. commit() Commit any pending transactions to the database. rollback() Rollback any pending transactions. cursor() Return a new Cursor object using the connection. setautocommit(auto=True) Set auto-commit mode. getautocommit() Get auto-commit mode. cancel() Cancel the running database request that is executed on the connection. isconnected() Return True if the connection is valid(connected to DB). setclientinfo(key, value=None) Set client info. If the value is None, the key is removed. getclientinfo(key=None) Get client info. If the key is None, All key:value sets are returned. Cursor Object Attribute Description description Sequence of column's information; The information contains 7 items : (name, type_code, display_size, internal_size, precision, scale, null_ok). rowcount Updated column count. callproc(procname[,parameters]) Call a stored database procedure with the given name. close() Close the cursor now. nextset() Skip to the next result set, closing current result set. (for multiple resultsets) execute(operation[,parameters]) Prepare and execute a database operation (query or command). -Steps included 1. prepare operation(statement) 2. bind paramters 3. execute prepared-statement executemany(operation,seq_of_parameters) Prepare a database operation (query or command) and then execute it against all parameter sequences or mappings found in the sequence seq_of_parameters. fetchone(uselob=False) Fetch the next row of a query result set, returning a single sequence, or None when no more data is available. fetchmany([size=cursor.arraysize]) Fetch the next set of rows of a query result, returning a sequence of sequences (e.g. a list of tuples). An empty sequence is returned when no more rows are available. fetchall() Fetch all (remaining) rows of a query result, returning them as a sequence of sequences (e.g. a list of tuples). Note that the cursor's arraysize attribute can affect the performance of this operation. LOB Object Attribute Description read(size[,position]) Return a portion (or all) of the data in the LOB object write(data) Write the data to the LOB object close() Close the LOB object Best wishes, Chanyoung Kwon SAP R&D Center Korea -----Original Message----- From: M.-A. Lemburg [mailto:mal at egenix.com] Sent: Monday, May 02, 2011 8:39 PM To: Kwon, Chan Young Cc: Chris Clark; db-sig at python.org Subject: Re: [DB-SIG] About LOB Kwon, Chan Young wrote: > Hi, > > Let me describe more... > 1. My suggestion of LOB object is kind of Locator object. It does not contain whole massive data. It just passes small piece of data when its methods are called. > 2. buffer object is not enough to handle various kinds of LOB types. For example, character LOB, binary LOB and unicode character LOB. > > In my company, this LOB class is already used as extension of DBAPI spec 2.0. > All colleagues like LOB class because it is very simple and easy to use. > So I sent this for your information. There are various ways large binary/text objects are handled by databases at the API level. Some databases provide APIs which allow reading/writing such data in chunks, others provide file descriptors which can be use used for this, yet others maintain the files outside the database and provide "locator" objects for these. Since all of these approaches use a file-like interface in one way or another, perhaps we could agree on a common DB-API extension that defines the minimum interface of such objects and a constructor that turns an existing file into an object which can then be passed to the database (much like the Binary() constructor we have for binary data). Do you have documentation for the LOB object you are using available on the web ? Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 02 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2011-06-20: EuroPython 2011, Florence, Italy 49 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ > Best wishes, > Chanyoung Kwon > > SAP R&D Center Korea > > > > -----Original Message----- > From: Chris Clark [mailto:Chris.Clark at ingres.com] > Sent: Thursday, April 28, 2011 2:05 AM > To: Kwon, Chan Young > Cc: db-sig at python.org > Subject: Re: [DB-SIG] About LOB > > Kwon, Chan Young wrote: >> I am implementing new DB API for new Database according to DB API 2.0. (as C-Extension) >> There is no specification about LOB in DB API 2.0 >> Especially, I need LOB object for piecewise LOB writing/reading after query execution. >> So I defined my own LOB object. >> I tried to make simple and easy interface for LOB. >> > > > BLarge OBject's (BLOBs) are alluded to in pep 249 BUT they really are > not detailed so I think you have spotted a weakness in the spec. CLOBs > (Character Large OBjects) are not covered at all. Locators are not > discussed but implied by references to bufferobjects. > > For BLOBs as input bind parameters check out the Binary() constructor > (case sensitive) at http://www.python.org/dev/peps/pep-0249/. > For BLOB results look for bufferobject in > http://www.python.org/dev/peps/pep-0249/ > > The spec (as I read it) really expects LOBs to be handled as one massive > block. However implementators can choose to implement locators behind > the scene so that data isn't sent across the wire unless the buffer > object is read from. > > So, model your LOB access methods on bufferobjects and you should be all > set! Marc-Andre reads/responds to the list regularly so take note of any > suggestions he has :-) > > Hope that helps get you started. > > Chris > > _______________________________________________ > DB-SIG maillist - DB-SIG at python.org > http://mail.python.org/mailman/listinfo/db-sig From info at egenix.com Wed May 25 13:24:46 2011 From: info at egenix.com (eGenix Team: M.-A. Lemburg) Date: Wed, 25 May 2011 13:24:46 +0200 Subject: [DB-SIG] ANN: eGenix mxODBC - Python ODBC Database Interface 3.1.1 Message-ID: <4DDCE6FE.3040305@egenix.com> ________________________________________________________________________ ANNOUNCING eGenix.com mxODBC - Python ODBC Database Interface Version 3.1.1 mxODBC is our commercially supported Python extension providing ODBC database connectivity to Python applications on Windows, Mac OS X, Unix and BSD platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-3.1.1-GA.html ________________________________________________________________________ INTRODUCTION mxODBC provides an easy-to-use, high-performance, reliable and robust Python interface to ODBC compatible databases such as MS SQL Server, MS Access, Oracle Database, IBM DB2 and Informix , Sybase ASE and Sybase Anywhere, MySQL, PostgreSQL, SAP MaxDB and many more: http://www.egenix.com/products/python/mxODBC/ The "eGenix mxODBC - Python ODBC Database Interface" product is a commercial extension to our open-source eGenix mx Base Distribution: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ NEWS mxODBC 3.1 Update ----------------- The mxODBC 3.1.1 patch level release adds compatibility to our new eGenix.com mx Base Distribution 3.2.0, which includes Windows x64 support. If you are currently using the combinations mxODBC 3.1.0 + mx Base 3.1.3, please consider upgrading to our latest releases mxODBC 3.1.1 + mx Base 3.2.0. Licenses for mxODBC 3.1.0 remain valid for mxODBC 3.1.1 as well. Release Highlights ------------------ * We've added Python 2.7 support and builds for all platforms. * mxODBC 3.1 adds native support for the Windows 64-bit platforms as well as the Mac OS X 10.6 (Snow Leopard) 64-bit builds of Python. * mxODBC now fully supports the Oracle Instant Client ODBC driver. * We have updated the support for the latest IBM DB2 9.7 ODBC drivers and enhanced compatibility of mxODBC with the MS SQL Server Native Client ODBC driver on Windows and the Sybase ASE 15 ODBC drivers on Unix. * mxODBC 3.1 adds support for large-scale data warehouse databases Netezza and Teradata. * In addition to the Windows, Mac OS X, iODBC and unixODBC ODBC driver managers, we now also include support for the DataDirect ODBC manager. * The 64-bit support on Unix platforms was updated to support the new unixODBC 2.3.0 version. * We've improved the documentation on how to connect to various popular databases and now include many tips & tricks for each database/driver. * The Python 2.7 memoryview object is now supported as binary data container. * We have simplified handling of database warnings using a new customizable .warningformat attribute. * The catalog methods now accept both Unicode and 8-bit strings as parameters. * You can now select whether to use ANSI (8-bit) or Unicode ODBC APIs in the ODBC drivers, removing unnecessary data conversions and enhancing ODBC driver compatibility. For the full set of changes please check the mxODBC change log: http://www.egenix.com/products/python/mxODBC/changelog.html Feature Highlights ------------------ * Python Database API 2.0 Compliance: the mxODBC API is fully Python DB-API 2.0 compatible and implements a large number of powerful extensions. * Support for all popular ODBC Drivers: mxODBC includes adjustments and work-arounds to support MS SQL Server Native Client, MS SQL Server ODBC Driver, FreeTDS ODBC Driver, Oracle Instant Client ODBC Driver, IBM DB2 ODBC Driver, Sybase ASE ODBC Driver, Netezza ODBC Driver, Teradata ODBC Driver, PostgreSQL ODBC Driver, MySQL ODBC Driver, .MaxDB ODBC Driver as well as the ODBC driver sets of EasySoft, DataDirect, OpenLink, Actual Technologies. * Support for all popular ODBC Driver Managers: mxODBC comes with subpackages for the native Windows and Mac OS X ODBC managers, as well as the ODBC managers unixODBC, iODBC and DataDirect, which are commonly used on Unix systems. * Stable, robust and reliable:the mxODBC API has been in active production use for more than 10 years. * Full Python Support: mxODBC works with Python 2.4, 2.5, 2.6 and 2.7. * Full 64-bit Support: mxODBC runs on the following 64-bit platforms: Windows, Linux, FreeBSD and Mac OS X. For the full set of features mxODBC has to offer, please see: http://www.egenix.com/products/python/mxODBC/#Features New mxODBC Editions ------------------- Due to popular demand, we have extended the set of available mxODBC editions and included a new low-cost standard edition. mxODBC is now available in thesethree editions: * The low-cost Standard Edition which provides data connectivity to a selected set of database backends. * The Professional Edition, which gives full access to all mxODBC features. * The Product Development Edition, which allows including mxODBC in applications you develop. At the same time we have simplified our license terms to clarify the situation on multi-core and virtual machines. In most cases, you no longer need to purchase more than one license per processor or virtual machine, scaling down the overall license costs significantly compared to earlier mxODBC releases. For a complete overview of the new editions, please see the product page. http://www.egenix.com/products/python/mxODBC/#mxODBCEditions ________________________________________________________________________ DOWNLOADS The download archives and instructions for installing the package can be found at: http://www.egenix.com/products/python/mxODBC/ In order to use the eGenix mxODBC package you will first need to install the eGenix mx Base package: http://www.egenix.com/products/python/mxBase/ ________________________________________________________________________ UPGRADING Users are encouraged to upgrade to this latest mxODBC release to benefit from the new features and updated ODBC driver support. We have taken special care, not to introduce backwards incompatible changes, making the upgrade experience as smooth as possible. Customers who have purchased mxODBC 3.1 license can continue to use their licenses with this patch level release. Customers who have purchased mxODBC 2.0, 2.1 or 3.0 licenses can upgrade their licenses using the mxODBC Professional Edition Upgrade License. If you want to try the new release before purchace, you can request 30-day evaluation licenses by visiting our web-site http://www.egenix.com/products/python/mxODBC/#Evaluation or by writing to sales at egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need. _______________________________________________________________________ SUPPORT Commercial support for this product is available from eGenix.com. Please see http://www.egenix.com/services/support/ for details about our support offerings. _______________________________________________________________________ INFORMATION About Python (http://www.python.org/): Python is an object-oriented Open Source programming language which runs on all modern platforms. By integrating ease-of-use, clarity in coding, enterprise application connectivity and rapid application design, Python establishes an ideal programming platform for today's IT challenges. About eGenix (http://www.egenix.com/): eGenix is a software project, consulting and product company focusing on expert services and professional quality products for companies, Python users and developers. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 25 2011) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2011-05-23: Released eGenix mx Base 3.2.0 http://python.egenix.com/ 2011-05-25: Released mxODBC 3.1.1 http://python.egenix.com/ 2011-06-20: EuroPython 2011, Florence, Italy 26 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From phd at phdru.name Mon May 30 13:20:10 2011 From: phd at phdru.name (Oleg Broytman) Date: Mon, 30 May 2011 15:20:10 +0400 Subject: [DB-SIG] SQLObject 1.0.1 Message-ID: <20110530112010.GD12650@iskra.aviel.ru> Hello! I'm pleased to announce version 1.0.1, a bugfix release of branch 1.0 of SQLObject. What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite, Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB). Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Archives: http://news.gmane.org/gmane.comp.python.sqlobject Download: http://cheeseshop.python.org/pypi/SQLObject/1.0.1 News and changes: http://sqlobject.org/News.html What's New ========== * A syntax incompatibility was fixed in SQLiteConnection that prevented SQLObject to be used with Python 2.4. For a more complete list, please see the news: http://sqlobject.org/News.html Oleg. -- Oleg Broytman http://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN.